Skip to content

Extending

Adding a provider

A provider is a package of its own, in its own repository and distribution, such as aiollm_openai (aiollm-org/aiollm_openai) or the echo provider of the test kit, aiollm.testing.echo; it is built from these parts:

  1. api.py: its IAPI, usually BaseAPI("https://api.provider.com", "v1"), and PROVIDER: Final = Provider("<name>"), the one place the package names its provider. Provider is an open value, a str made from the name: the library itself declares only Provider.GENERIC, every provider package (in the library or outside it) its own, and two packages must not share a name, since files and errors tell providers apart by it. Compare providers with ==, never with is: a Provider copied, pickled or read from a field is another object with the same value.
  2. resolver.py: a BaseErrorResolver subclass with provider = PROVIDER, the provider's error_statuses table (the status each in-band code or type stands for), and get_error_code or classify_status for its error codes, such as which 429s are quota errors and which statuses are transient.
  3. client.py: a module-level SessionFactory(AiohttpSession, auth_factory=..., api=..., error_resolver=...) and a BaseProviderClient subclass that sets provider = PROVIDER, session_factory, api_key_env, auth_headers (the headers the provider takes the key in) and every shared resource as a class attribute made by bind_resource(ResourceClass[...], AdapterClass), such as generation = bind_resource(GenerationResource[Response, ResponseStreamEvent], ExampleGenerationAdapter). An adapter with options gets adapter_factory=, a function (often a method of the client) that builds it from the client: the client's __init__ keeps the options and then calls super().__init__(), which builds every adapter before the session, so a bad option fails without one, and every resource right after it. Type checkers see the attribute as the resource; a factory that returns another class, or a subclass of another provider, raises TypeError. Its __init__ types **session_kwargs. A generation adapter also has with_options(options, /) -> Self (IGenerationAdapter), which GenerationResource calls for a call's provider_options: the provider's TypedDict of the fields its method has and the shared request does not (types/options.py, every key a method field of the same type, which a test checks), checked with checked_options and merged into the method's fields of the copy it returns; the client binds GenerationResource[Raw, Event, Options]. An adapter of a provider with no such fields takes NoProviderOptions (closed: only {} matches it), still checks what it gets with checked_options(options, NoProviderOptions, self.provider), so a key passed through an untyped dict is refused instead of dropped, and then returns itself. The copy with_options makes may be shallow, since an adapter and its conversions hold no mutable state that outlives one method call: concurrent calls share them. The base class reads the token, the ready session or the environment variable, builds the session and warns about a given session that does not fit. The client, the factory's resolver and every adapter must name the same provider: a mismatch raises TypeError when the class is defined.
  4. types/, methods/ and adapters/ subpackages with its first endpoints: response models, method classes, and adapters that map them to the shared types (an adapter sets provider = PROVIDER; a concrete adapter without it raises TypeError); its resources build AdapterUseCases from those adapters. An optional field of a shared request that was not given is MISSING: pass it to the method as it is, and test it with is MISSING, never by truth (bool(MISSING) is True). Write the method's fields after the provider's request schema (see Optional fields), and add the endpoint to the wire-contract tests, which check them.

The package __init__.py re-exports the client, which makes it the one __init__ allowed to import a transport.

How resources are bound

A client's resources are not built in its own code, so a reader of the client sees only the bind_resource(...) lines. What happens behind them:

  • Where they come from. BaseProviderClient looks for bind_resource values among the class attributes of the client and of every base class (walking the MRO from object down), at class definition for the provider check and again in __init__. A subclass replaces an inherited resource with another bind_resource under the same name, or removes it by assigning any other value to that name.
  • When they are built. BaseProviderClient.__init__ first calls each binding's adapter_factory (or the adapter class without arguments), then builds the session, then each resource on it, and sets it as an attribute of the instance. So a client keeps the options its factories read (as AnthropicClient keeps default_max_tokens) before it calls super().__init__(); a factory that reads an option set later fails with AttributeError.
  • What the class attribute is. On an instance, client.generation is the resource. On the class itself, OpenAIClient.generation stays the ResourceBinding that bind_resource returned, although type checkers see it as the resource: like dataclasses.field, it is typed as what it becomes. A descriptor would make the class attribute honest, but pyright does not apply a descriptor's __get__ when it checks a client against a protocol, so every IGenerationClient argument in user code would fail. Read resources from an instance, never from the class.
  • How a subclass changes one. It declares the resource again under the same name with another bind_resource, such as one with its own adapter_factory; it does not override __init__ for that. The new binding replaces the inherited one, and its adapter's provider is checked like any other.

What a provider may take from the core

A provider package stands on a fixed set of names of the core, its extension API, listed by module in aiollm/testing/extension_api.json; a test fails when a provider imports a name outside it, and when a listed name is gone, and aiollm.testing.extension.check_extension_api runs the same check on a package of your own. These are the parts the steps above name: Provider, BaseAPI and the auth classes, BaseErrorResolver and the error classes it builds, BaseAPIMethod, BaseStreamMethod and the parsers, SessionFactory and AiohttpSession (imported from aiollm.session.aiohttp, the one place a provider binds its transport), BaseProviderClient and bind_resource, the base adapters and Stop, the checks an adapter runs on a shared request (checked_options, merge_options, request_errors, check_conversation and check_files, exported from aiollm.adapters), GenerationResource and FilesResource, and the shared request and response types. The core removes or changes a name of the list only as a breaking change, and a provider that needs a new one adds it to the list in the same change, so the contract grows in plain sight.

The core never imports a provider (the providers depend on the core, never the other way round), and the core's infrastructure (HTTP, methods, responses, sessions and clients) does not import the shared generation and files layer (types.requests, types.responses, adapters and resources); tests/unit/test_layers.py checks both.

Testing a provider

aiollm.testing is the kit the library's own tests use, installed with the package's testing extra: FakeTransport replays queued responses through the real BaseSession, so retries, error mapping and parsing run as in production, and Recorder (in aiollm.testing.aiohttp) is a local server that answers only what a test scripted with Reply and fails the test on any other request. Load the fixtures in a conftest.py with pytest_plugins = ["aiollm.testing.plugin"]: it makes retries instant and AiohttpSession.close() immediate in every test, clears the environment variables your provider_key_envs fixture names, and gives recorder, server, api and session for the local server, fake_transport and frozen_now. The plugin is not registered on install, so a project that does not load it keeps its own behaviour.

The same kit holds what every provider package runs on its client. aiollm.testing.contract has the shared scenarios: subclass GenerationContract, FilesContract and ClientContract in a test module (as TestExampleGeneration, so pytest collects it) and set case to a GenerationCase, FilesCase or ClientCase with your wire format: a finished response, a stream, what each shared field becomes in the body, the documented files. A provider that runs built-in tools gives its GenerationCase a BuiltinCase too (the tools as they go in the body, a finished search as a response and as a stream, a failed one, a code execution that made a file, and the files a CodeExecution puts into the sandbox), and its FilesCase says where a file of a container is downloaded from (container_download, None for a provider that keeps none); one whose API never pauses a turn leaves out its paused and continued, and the scenario that continues a paused turn is skipped. Every scenario then runs on your client against the local server, and the shared results and errors must come out as they do for every other provider. aiollm.testing.extension.check_extension_api("aiollm_example") returns every import of your package that the extension API lacks, so one test with == [] keeps it inside. aiollm.testing.wire reads a snapshot of your provider's upstream schemas (Snapshot) and compares your generated models with it (naming_problems, field_problems). aiollm.testing.echo is a whole provider built this way, for an API that exists only on the local server: the core runs every scenario on it, and its package is the smallest example of one.

Writing a transport

BaseSession holds everything that doesn't depend on the HTTP client: retries, error mapping, parsing and stream cleanup. A transport subclasses it and implements three methods:

  • _execute(method, *, headers=None, timeout=None) -> BufferedResponse: send the request and read the whole body.
  • _open_stream(method, *, headers=None, timeout=None) -> IStreamResponse: send the request and leave the body unread; the response has status, headers and charset, and offers iter_chunks(), read() and aclose().
  • close(): release the connections.

Both request methods return error statuses as responses, without raising, so the status check stays in BaseSession. I/O failures are raised as APITransportError subclasses with the right transient flag; that mapping is what makes retries behave the same on every transport. Build the URL with api.build_url(method.request_path, method.get_query_map()) and the body from method.get_body_map() and method.request_body_format, and the headers from method.request_headers.build(headers) with the authentication header from auth set last, on every request. timeout is the call's timeout=; apply it as described in Timeouts, and use the session's own timeouts when it is None. Log an attempt the way the aiohttp transport does: the method and method.request_path_template when it is sent, the status and duration when it ends, never request_path (its values may name a user's data), the query, headers or body.

Transport code lives only in <layer>/<transport>.py modules (today session/aiohttp.py, session/body/aiohttp.py, response/aiohttp.py and http/exceptions/aiohttp.py), so a second transport, such as httpx, adds */httpx.py files and changes nothing else. bind_loop in session/loop.py is transport-neutral and can be reused.