aiollm¶
You describe each API call as a small declarative class: a Pydantic model whose generic argument is its result type. The library sends it over aiohttp and returns a validated, statically typed result. It also handles retries, errors, file uploads and streaming, and ships clients for OpenAI and Anthropic with one shared generation API, images, documents and tool calls included.
Status: pre-alpha (0.17.0). The core abstractions are in place, but the public API may still change: before 1.0 a breaking change bumps the minor version (0.9 is followed by 0.10, not 1.0; 1.0 comes when the API is declared stable), see Versioning.
Features¶
- Typed methods.
send_request(method)returns the method's result type, and mypy and pyright both infer it. - Buffered and streamed responses. JSON models, text, bytes or nothing; streams of bytes, text, or typed events from server-sent events (SSE) and newline-delimited JSON (NDJSON).
- Checked streams. A method can declare the protocol's last event, so a stream cut between two events raises instead of looking complete.
- File uploads. From bytes, a path, an open binary stream or an async iterator; a retry resends the whole file.
- Opt-in retries. Transient failures are retried with exponential backoff and jitter, honouring
Retry-Afterup to a cap; aRetryPolicytunes them per session or per call. - Structured errors. One exception class per common HTTP status, plus transport and response errors, each with the provider's code, the request id and a
transientflag. - Provider clients.
OpenAIClient(Responses and Files APIs, packageaiollm-openai) andAnthropicClient(Messages and Files APIs, packageaiollm-anthropic), with types generated from the providers' schemas and the provider-neutralclient.generation(text with images and PDF documents, structured output, reasoning summaries and function tools, buffered or streamed) andclient.files(a file uploaded once and sent by its id). - Replaceable parts. Sessions depend on protocols, so the API, authentication, error mapping, headers and JSON codec can each be swapped.
- Safe defaults. Path parameters are escaped, redirects are off, proxies from the environment are ignored, and tokens stay out of
repr.
Installation¶
The library requires Python 3.13 or newer. Install the provider you use; it brings the core, aiollm-core, with it:
pip install aiollm-openai # import aiollm_openai
pip install aiollm-anthropic # import aiollm_anthropic
Or the core alone, for clients of your own: pip install aiollm-core (import aiollm). With uv, uv add takes the same names. The testing extra, aiollm-core[testing], adds the test kit for a provider of your own. To work on the library itself, see Contributing.
Quick start¶
import asyncio
from pydantic import MISSING
from aiollm import (
BaseAPI,
BaseAPIMethod,
BaseAPIType,
BaseErrorResolver,
BearerAuth,
BodyFormat,
)
from aiollm.session.aiohttp import AiohttpSession
API = BaseAPI("https://api.example.com", "v1")
class Message(BaseAPIType):
role: str
content: str
class Completion(BaseAPIType):
id: str
text: str
class CreateCompletion(BaseAPIMethod[Completion]):
__request_path__ = "/completions"
__request_method__ = "POST"
__request_body_format__ = BodyFormat.JSON
__retryable__ = True
model: str
messages: list[Message]
temperature: float | MISSING = MISSING # left out until given
async def main() -> None:
session = AiohttpSession(
api=API,
auth=BearerAuth("sk-..."),
error_resolver=BaseErrorResolver(),
)
async with session:
completion = await session.send_request(
CreateCompletion(
model="example-1",
messages=[Message(role="user", content="Hello!")],
)
)
print(completion.text) # completion: Completion
asyncio.run(main())
This sends POST https://api.example.com/v1/completions with the body {"model": "example-1", "messages": [{"role": "user", "content": "Hello!"}]}. temperature was not given, so it stays MISSING and is not sent at all (None would be sent as null). The response is validated into Completion, and type checkers know that completion is a Completion.
Where to go next¶
- How a request flows, then Methods to describe your own API.
- Provider clients and Shared generation to call OpenAI or Anthropic.
- Reference for every public name.