Extending¶
Adding a provider¶
A provider is a package of its own, in its own repository and distribution, such as aiollm_openai (aiollm-org/aiollm_openai) or the echo provider of the test kit, aiollm.testing.echo; it is built from these parts:
api.py: itsIAPI, usuallyBaseAPI("https://api.provider.com", "v1"), andPROVIDER: Final = Provider("<name>"), the one place the package names its provider.Provideris an open value, astrmade from the name: the library itself declares onlyProvider.GENERIC, every provider package (in the library or outside it) its own, and two packages must not share a name, since files and errors tell providers apart by it. Compare providers with==, never withis: aProvidercopied, pickled or read from a field is another object with the same value.resolver.py: aBaseErrorResolversubclass withprovider = PROVIDER, the provider'serror_statusestable (the status each in-band code or type stands for), andget_error_codeorclassify_statusfor its error codes, such as which 429s are quota errors and which statuses are transient.client.py: a module-levelSessionFactory(AiohttpSession, auth_factory=..., api=..., error_resolver=...)and aBaseProviderClientsubclass that setsprovider = PROVIDER,session_factory,api_key_env,auth_headers(the headers the provider takes the key in) and every shared resource as a class attribute made bybind_resource(ResourceClass[...], AdapterClass), such asgeneration = bind_resource(GenerationResource[Response, ResponseStreamEvent], ExampleGenerationAdapter). An adapter with options getsadapter_factory=, a function (often a method of the client) that builds it from the client: the client's__init__keeps the options and then callssuper().__init__(), which builds every adapter before the session, so a bad option fails without one, and every resource right after it. Type checkers see the attribute as the resource; a factory that returns another class, or a subclass of another provider, raisesTypeError. Its__init__types**session_kwargs. A generation adapter also haswith_options(options, /) -> Self(IGenerationAdapter), whichGenerationResourcecalls for a call'sprovider_options: the provider'sTypedDictof the fields its method has and the shared request does not (types/options.py, every key a method field of the same type, which a test checks), checked withchecked_optionsand merged into the method's fields of the copy it returns; the client bindsGenerationResource[Raw, Event, Options]. An adapter of a provider with no such fields takesNoProviderOptions(closed: only{}matches it), still checks what it gets withchecked_options(options, NoProviderOptions, self.provider), so a key passed through an untyped dict is refused instead of dropped, and then returns itself. The copywith_optionsmakes may be shallow, since an adapter and its conversions hold no mutable state that outlives one method call: concurrent calls share them. The base class reads the token, the ready session or the environment variable, builds the session and warns about a given session that does not fit. The client, the factory's resolver and every adapter must name the same provider: a mismatch raisesTypeErrorwhen the class is defined.types/,methods/andadapters/subpackages with its first endpoints: response models, method classes, and adapters that map them to the shared types (an adapter setsprovider = PROVIDER; a concrete adapter without it raisesTypeError); its resources buildAdapterUseCases from those adapters. An optional field of a shared request that was not given isMISSING: pass it to the method as it is, and test it withis MISSING, never by truth (bool(MISSING)isTrue). Write the method's fields after the provider's request schema (see Optional fields), and add the endpoint to the wire-contract tests, which check them.
The package __init__.py re-exports the client, which makes it the one __init__ allowed to import a transport.
How resources are bound¶
A client's resources are not built in its own code, so a reader of the client sees only the bind_resource(...) lines. What happens behind them:
- Where they come from.
BaseProviderClientlooks forbind_resourcevalues among the class attributes of the client and of every base class (walking the MRO fromobjectdown), at class definition for the provider check and again in__init__. A subclass replaces an inherited resource with anotherbind_resourceunder the same name, or removes it by assigning any other value to that name. - When they are built.
BaseProviderClient.__init__first calls each binding'sadapter_factory(or the adapter class without arguments), then builds the session, then each resource on it, and sets it as an attribute of the instance. So a client keeps the options its factories read (asAnthropicClientkeepsdefault_max_tokens) before it callssuper().__init__(); a factory that reads an option set later fails withAttributeError. - What the class attribute is. On an instance,
client.generationis the resource. On the class itself,OpenAIClient.generationstays theResourceBindingthatbind_resourcereturned, although type checkers see it as the resource: likedataclasses.field, it is typed as what it becomes. A descriptor would make the class attribute honest, but pyright does not apply a descriptor's__get__when it checks a client against a protocol, so everyIGenerationClientargument in user code would fail. Read resources from an instance, never from the class. - How a subclass changes one. It declares the resource again under the same name with another
bind_resource, such as one with its ownadapter_factory; it does not override__init__for that. The new binding replaces the inherited one, and its adapter's provider is checked like any other.
What a provider may take from the core¶
A provider package stands on a fixed set of names of the core, its extension API, listed by module in aiollm/testing/extension_api.json; a test fails when a provider imports a name outside it, and when a listed name is gone, and aiollm.testing.extension.check_extension_api runs the same check on a package of your own. These are the parts the steps above name: Provider, BaseAPI and the auth classes, BaseErrorResolver and the error classes it builds, BaseAPIMethod, BaseStreamMethod and the parsers, SessionFactory and AiohttpSession (imported from aiollm.session.aiohttp, the one place a provider binds its transport), BaseProviderClient and bind_resource, the base adapters and Stop, the checks an adapter runs on a shared request (checked_options, merge_options, request_errors, check_conversation and check_files, exported from aiollm.adapters), GenerationResource and FilesResource, and the shared request and response types. The core removes or changes a name of the list only as a breaking change, and a provider that needs a new one adds it to the list in the same change, so the contract grows in plain sight.
The core never imports a provider (the providers depend on the core, never the other way round), and the core's infrastructure (HTTP, methods, responses, sessions and clients) does not import the shared generation and files layer (types.requests, types.responses, adapters and resources); tests/unit/test_layers.py checks both.
Testing a provider¶
aiollm.testing is the kit the library's own tests use, installed with the package's testing extra: FakeTransport replays queued responses through the real BaseSession, so retries, error mapping and parsing run as in production, and Recorder (in aiollm.testing.aiohttp) is a local server that answers only what a test scripted with Reply and fails the test on any other request. Load the fixtures in a conftest.py with pytest_plugins = ["aiollm.testing.plugin"]: it makes retries instant and AiohttpSession.close() immediate in every test, clears the environment variables your provider_key_envs fixture names, and gives recorder, server, api and session for the local server, fake_transport and frozen_now. The plugin is not registered on install, so a project that does not load it keeps its own behaviour.
The same kit holds what every provider package runs on its client. aiollm.testing.contract has the shared scenarios: subclass GenerationContract, FilesContract and ClientContract in a test module (as TestExampleGeneration, so pytest collects it) and set case to a GenerationCase, FilesCase or ClientCase with your wire format: a finished response, a stream, what each shared field becomes in the body, the documented files. A provider that runs built-in tools gives its GenerationCase a BuiltinCase too (the tools as they go in the body, a finished search as a response and as a stream, a failed one, a code execution that made a file, and the files a CodeExecution puts into the sandbox), and its FilesCase says where a file of a container is downloaded from (container_download, None for a provider that keeps none); one whose API never pauses a turn leaves out its paused and continued, and the scenario that continues a paused turn is skipped. Every scenario then runs on your client against the local server, and the shared results and errors must come out as they do for every other provider. aiollm.testing.extension.check_extension_api("aiollm_example") returns every import of your package that the extension API lacks, so one test with == [] keeps it inside. aiollm.testing.wire reads a snapshot of your provider's upstream schemas (Snapshot) and compares your generated models with it (naming_problems, field_problems). aiollm.testing.echo is a whole provider built this way, for an API that exists only on the local server: the core runs every scenario on it, and its package is the smallest example of one.
Writing a transport¶
BaseSession holds everything that doesn't depend on the HTTP client: retries, error mapping, parsing and stream cleanup. A transport subclasses it and implements three methods:
_execute(method, *, headers=None, timeout=None) -> BufferedResponse: send the request and read the whole body._open_stream(method, *, headers=None, timeout=None) -> IStreamResponse: send the request and leave the body unread; the response hasstatus,headersandcharset, and offersiter_chunks(),read()andaclose().close(): release the connections.
Both request methods return error statuses as responses, without raising, so the status check stays in BaseSession. I/O failures are raised as APITransportError subclasses with the right transient flag; that mapping is what makes retries behave the same on every transport. Build the URL with api.build_url(method.request_path, method.get_query_map()) and the body from method.get_body_map() and method.request_body_format, and the headers from method.request_headers.build(headers) with the authentication header from auth set last, on every request. timeout is the call's timeout=; apply it as described in Timeouts, and use the session's own timeouts when it is None. Log an attempt the way the aiohttp transport does: the method and method.request_path_template when it is sent, the status and duration when it ends, never request_path (its values may name a user's data), the query, headers or body.
Transport code lives only in <layer>/<transport>.py modules (today session/aiohttp.py, session/body/aiohttp.py, response/aiohttp.py and http/exceptions/aiohttp.py), so a second transport, such as httpx, adds */httpx.py files and changes nothing else. bind_loop in session/loop.py is transport-neutral and can be reused.