Resources and use cases¶
Methods are specific to one provider. Resources and use cases are the layer above them that offers one API for all providers, in shared types:
- Shared requests (
aiollm.types.requests):BaseRequest, which forbids unknown fields, andGenerationRequest(model,input, a text or a list ofTurns, neither of them empty,instructions,max_output_tokens,effort,output_schema,end_user_id).instructionsare given apart from the input, like a system prompt. It has no sampling parameters such astemperature: Anthropic removed them in API version2026-10-21, so they are not common to all providers. - Shared responses (
aiollm.types.responses):GenerationResponse[RawT]withid,provider,model,text,usage(TokenUsage: input, output and total tokens, with the cached and reasoning tokens among them),incomplete_reason(why the provider stopped early, anIncompleteReason;Noneif it finished),raw_incomplete_reason,refusaland the provider's own response inraw, so nothing the shared type lacks is lost. - Shared events:
GenerationEvent[RawT], a discriminated union ofTextDeltaEvent,CompletedEventandOtherEvent.CompletedEvent, the stream's final event, carriesid,provider,model, the wholetext,usage,incomplete_reason,raw_incomplete_reasonandrefusaltoo. Theirtypeis aGenerationEventType. A failure is an exception at every level, never an event. - Adapters (
aiollm.adapters): convert models and nothing else.BaseAdapterbuilds the provider's method from a shared request (to_method) and turns its result into a shared response (to_response); itsprovidernames the provider in the results and in the errors of the requests it refuses.BaseStreamAdapterdoes the same for streams (to_stream_method,to_events), andBaseCompositeAdapterdoes both. - Use cases (
aiollm.resources): only orchestrate.AdapterUseCase(session, adapter)sends what the adapter builds and returns what it converts;AdapterStreamUseCaseandAdapterCompositeUseCasedo the same for streams and for both. Giventerminal=(an event matcher, such asGENERATION_TERMINALfromaiollm.types.responses, which matchesCompletedEvent), they check that exactly one event matches it, as the last one: a stream without it raisesAPIStreamInterruptedError(orRuntimeErrorwhen the provider method's__stream_terminal__proved the stream complete), an event after itAPIResponseError. - Resources:
GenerationResource(session, adapter)is the shared resource of text generation, the same class for every provider:create(...)andstream(...), with keyword arguments or a request, typed by the adapter's raw types;streamyields aGenerationStreamwithtext()andfinal(). Its adapter is anIGenerationAdapter: a composite adapter withwith_options(options, /) -> Self, which takes a call'sprovider_options, the provider's own fields as aTypedDict(OptionsT, the resource's third type parameter;NoProviderOptions, closed, for an adapter without any), and returns a copy that sends them.BaseResourcegroups the use cases of any other area.
resources.py
from collections.abc import AsyncGenerator, AsyncIterator
from typing import Self
from aiollm import (
BaseClient,
CompletedEvent,
GenerationEvent,
GenerationRequest,
GenerationResponse,
NoProviderOptions,
Provider,
TextDeltaEvent,
)
from aiollm.adapters import BaseCompositeAdapter, checked_options
from aiollm.resources import GenerationResource
from aiollm.session import ISession
class CompletionAdapter(
BaseCompositeAdapter[
GenerationRequest,
Completion, # what CreateCompletion returns
GenerationResponse[Completion],
MessageEvent, # what StreamMessage yields
GenerationEvent[MessageEvent],
]
):
provider = Provider("completions") # this provider's own name
def with_options(self, options: NoProviderOptions, /) -> Self:
# No options of its own: still refuse a key instead of dropping it.
checked_options(options, NoProviderOptions, self.provider)
return self
def to_method(self, request: GenerationRequest) -> CreateCompletion:
return CreateCompletion(
model=request.model,
messages=[Message(role="user", content=request.input)],
)
def to_response(
self,
raw: Completion,
request: GenerationRequest,
) -> GenerationResponse[Completion]:
return GenerationResponse[Completion](
id=raw.id,
provider=self.provider,
model=request.model, # the API does not send it back
text=raw.text,
raw=raw,
)
def to_stream_method(self, request: GenerationRequest) -> StreamMessage:
return StreamMessage(prompt=request.input)
async def to_events(
self,
items: AsyncIterator[MessageEvent],
request: GenerationRequest,
) -> AsyncGenerator[GenerationEvent[MessageEvent]]:
async for event in items:
if isinstance(event, TextDelta):
yield TextDeltaEvent[MessageEvent](text=event.text, raw=event)
else:
yield CompletedEvent[MessageEvent](
provider=self.provider, model=request.model, raw=event
)
class ExampleClient(BaseClient[ISession]):
def __init__(self, session: ISession) -> None:
super().__init__(session)
self.generation = GenerationResource(session, CompletionAdapter())
await client.generation.create(model=..., input=...) returns a GenerationResponse[Completion], and async with client.generation.stream(model=..., input=...) as events yields a GenerationStream[MessageEvent] of GenerationEvent[MessageEvent]s. Another provider's client builds the same GenerationResource with its own adapter, so the calling code does not change.
- Why three parts. The session moves bytes, the adapter converts models, and the use case connects them. An adapter has no session and does no I/O, so it is tested with plain objects; a use case has no provider knowledge, so one class serves every provider.
- Grouped by mode, not by direction. One adapter class holds both halves of a buffered call, so a type checker sees at the class itself whether the method it builds returns the type its
to_responseexpects. - The request reaches the conversions.
to_responseandto_eventsget the shared request too, for what the provider does not send back, such as the model or an output schema. to_eventssees the whole stream. It may keep state, drop items and turn one item into several, for example collect deltas into a final event. The use case runs it throughtransform_stream(aiollm.response), which closes the converted stream first and the provider's stream after it, also onbreakand errors.map_streamis its one-item-to-one-item case.- Resources hold use cases.
GenerationResourceruns anAdapterCompositeUseCase; a resource of another area buildsAdapterUseCases the same way, andIGenerationResourceis its protocol. - Call options pass through.
__call__andstreamof every use case take**options: Unpack[CallOptions](headers,timeout,retry_policy, fromaiollm.session) and hand them tosession.send_requestorsession.streamunchanged; a use case of its own does the same, for every request it sends. - Calls that are not one request and one response, such as pagination, several dependent requests or polling, subclass
BaseUseCase,BaseStreamUseCaseorBaseCompositeUseCasedirectly and use adapters inside. - The session type is optional. The last type parameter,
SessionT, defaults toISession. Give a concrete session, such asBaseResource[AiohttpSession], to reach its settings throughself.sessionwithout a cast. RawThas no bound. With aBaseAPITypebound, an unparametrizedGenerationResponsewould turn a dict inrawinto an empty model, because unknown fields are ignored.
OpenAIClient.generation and AnthropicClient.generation are GenerationResources with OpenAIGenerationAdapter and AnthropicGenerationAdapter (see Shared generation).