Skip to content

aiollm

You describe each API call as a small declarative class: a Pydantic model whose generic argument is its result type. The library sends it over aiohttp and returns a validated, statically typed result. It also handles retries, errors, file uploads and streaming, and ships clients for OpenAI and Anthropic with one shared generation API, images, documents and tool calls included.

Status: pre-alpha (0.17.0). The core abstractions are in place, but the public API may still change: before 1.0 a breaking change bumps the minor version (0.9 is followed by 0.10, not 1.0; 1.0 comes when the API is declared stable), see Versioning.

Features

  • Typed methods. send_request(method) returns the method's result type, and mypy and pyright both infer it.
  • Buffered and streamed responses. JSON models, text, bytes or nothing; streams of bytes, text, or typed events from server-sent events (SSE) and newline-delimited JSON (NDJSON).
  • Checked streams. A method can declare the protocol's last event, so a stream cut between two events raises instead of looking complete.
  • File uploads. From bytes, a path, an open binary stream or an async iterator; a retry resends the whole file.
  • Opt-in retries. Transient failures are retried with exponential backoff and jitter, honouring Retry-After up to a cap; a RetryPolicy tunes them per session or per call.
  • Structured errors. One exception class per common HTTP status, plus transport and response errors, each with the provider's code, the request id and a transient flag.
  • Provider clients. OpenAIClient (Responses and Files APIs, package aiollm-openai) and AnthropicClient (Messages and Files APIs, package aiollm-anthropic), with types generated from the providers' schemas and the provider-neutral client.generation (text with images and PDF documents, structured output, reasoning summaries and function tools, buffered or streamed) and client.files (a file uploaded once and sent by its id).
  • Replaceable parts. Sessions depend on protocols, so the API, authentication, error mapping, headers and JSON codec can each be swapped.
  • Safe defaults. Path parameters are escaped, redirects are off, proxies from the environment are ignored, and tokens stay out of repr.

Installation

The library requires Python 3.13 or newer. Install the provider you use; it brings the core, aiollm-core, with it:

pip install aiollm-openai      # import aiollm_openai
pip install aiollm-anthropic   # import aiollm_anthropic

Or the core alone, for clients of your own: pip install aiollm-core (import aiollm). With uv, uv add takes the same names. The testing extra, aiollm-core[testing], adds the test kit for a provider of your own. To work on the library itself, see Contributing.

Quick start

quick_start.py
import asyncio

from pydantic import MISSING

from aiollm import (
    BaseAPI,
    BaseAPIMethod,
    BaseAPIType,
    BaseErrorResolver,
    BearerAuth,
    BodyFormat,
)
from aiollm.session.aiohttp import AiohttpSession

API = BaseAPI("https://api.example.com", "v1")


class Message(BaseAPIType):
    role: str
    content: str


class Completion(BaseAPIType):
    id: str
    text: str


class CreateCompletion(BaseAPIMethod[Completion]):
    __request_path__ = "/completions"
    __request_method__ = "POST"
    __request_body_format__ = BodyFormat.JSON
    __retryable__ = True

    model: str
    messages: list[Message]
    temperature: float | MISSING = MISSING   # left out until given


async def main() -> None:
    session = AiohttpSession(
        api=API,
        auth=BearerAuth("sk-..."),
        error_resolver=BaseErrorResolver(),
    )

    async with session:
        completion = await session.send_request(
            CreateCompletion(
                model="example-1",
                messages=[Message(role="user", content="Hello!")],
            )
        )

    print(completion.text)  # completion: Completion


asyncio.run(main())

This sends POST https://api.example.com/v1/completions with the body {"model": "example-1", "messages": [{"role": "user", "content": "Hello!"}]}. temperature was not given, so it stays MISSING and is not sent at all (None would be sent as null). The response is validated into Completion, and type checkers know that completion is a Completion.

Where to go next