Typed, async clients for AI APIs, built from small declarative methods.

Describe each API call as a Pydantic model whose generic argument is its result. aiollm sends it over aiohttp, retries what is safe to retry, and hands back a validated result your type checker already knows.

pip install aiollm-openai

On PyPI as aiollm-core, aiollm-openai and aiollm-anthropic. Requires Python 3.13 or newer.

pre-alpha 0.15.0Python 3.13 · 3.14 · 3.15mypy & pyright strictMIT license
import asyncio

from aiollm_openai import OpenAIClient


async def main() -> None:
    async with OpenAIClient() as client:  # OPENAI_API_KEY
        response = await client.generation.create(
            model="gpt-6-astra",
            input="Explain asyncio in one line",
            instructions="Be brief",
            max_output_tokens=64,
        )

    print(response.text)
    print(response.usage)  # shared TokenUsage
    print(response.raw)    # OpenAI's own Response


asyncio.run(main())
Swap in AnthropicClient and the call stays the same.
Get started

From pip install to your first typed response

Requires Python 3.13+. Install the provider you call; it brings the core, aiollm-core, with it.

  1. 01 / Install

    Install a provider

    1. pip install aiollm-openai
    2. pip install aiollm-anthropic

    Only your own API? pip install aiollm-core. With uv: uv add aiollm-openai.

  2. 02 / Configure

    Bring your provider key

    Set the corresponding environment variable in your shell. A client reads it when you provide neither a token nor a ready session.

    OPENAI_API_KEYANTHROPIC_API_KEY
    Client setup guide
  3. 03 / Make a call

    Choose your level

    Use client.generation for a shared text API, native provider methods for their exact fields, or declare your own HTTP endpoint.

What you get

The plumbing of an API client, done once and typed end to end

aiollm is a low-level toolkit. It does not hide HTTP; it makes every request a checked, inspectable object and handles the parts every client gets wrong.

Typed methods

send_request(method) returns the method’s result type, and mypy and pyright both infer it.

Buffered and streamed

JSON models, text, bytes or nothing; streams of typed events from SSE and NDJSON, checked for a declared last event.

Opt-in retries

Transient failures retry with exponential backoff and jitter, honouring Retry-After; a RetryPolicy tunes them per session or per call.

Structured errors

One class per common HTTP status, plus transport and response errors, each with the provider’s code, the request id and a transient flag.

File uploads

From bytes, a path, a binary file or an async iterator. Replayable sources reopen for retries; one-shot sources explicitly disable them.

Safe defaults

Path parameters are escaped, redirects are off, environment proxies are ignored, and tokens stay out of repr and logs.

Shared generation

One request shape. The provider’s details still intact.

Pass keyword arguments or a GenerationRequest to create() or stream(). The same calling code works with either built-in client through IGenerationClient, tools included.

Turn

Conversations, explicitly

Build a conversation from user and assistant turns. Carry an earlier answer forward with response.to_turn(); put the system prompt in instructions.

output_schema

JSON to your schema

Send a JSON Schema through the shared API. The result remains JSON text; check incomplete_reason before parsing it. Each provider supports its own schema subset.

Effort

Control the effort

Request LOW, MEDIUM, HIGH, XHIGH, or MAX. Omit it to keep the model’s default; the provider decides which levels that model accepts.

GenerationResponse

Keep the full answer

Read text, token usage, refusals, and completion status in shared fields. Provider-specific details stay available in raw.

structured_output.py
from typing import Any

from aiollm import Effort, GenerationResponse, IGenerationClient


async def extract_summary(
    client: IGenerationClient,
    model: str,
    text: str,
) -> GenerationResponse[Any]:
    return await client.generation.create(
        model=model,
        input=text,
        instructions="Summarize the supplied text.",
        effort=Effort.MEDIUM,
        output_schema={
            "type": "object",
            "properties": {
                "summary": {"type": "string"},
            },
            "required": ["summary"],
            "additionalProperties": False,
        },
        max_output_tokens=1024,
    )

The response keeps its JSON text in text. Check incomplete_reason before parsing it.

How shared fields reach each provider
Shared fieldOpenAIAnthropic
instructionsinstructionssystem
max_output_tokensmax_output_tokensmax_tokens
effortreasoning.effortoutput_config.effort
output_schematext.formatoutput_config.format
end_user_idsafety_identifiermetadata.user_id
toolstools (function tools)tools (custom tools)
tool_choicetool_choicetool_choice (REQUIRED as any)
parallel_tool_callsparallel_tool_callstool_choice.disable_parallel_tool_use

instructions

OpenAI
instructions
Anthropic
system

max_output_tokens

OpenAI
max_output_tokens
Anthropic
max_tokens

effort

OpenAI
reasoning.effort
Anthropic
output_config.effort

output_schema

OpenAI
text.format
Anthropic
output_config.format

end_user_id

OpenAI
safety_identifier
Anthropic
metadata.user_id

tools

OpenAI
tools (function tools)
Anthropic
tools (custom tools)

tool_choice

OpenAI
tool_choice
Anthropic
tool_choice (REQUIRED as any)

parallel_tool_calls

OpenAI
parallel_tool_calls
Anthropic
tool_choice.disable_parallel_tool_use

Reasoning summaries are available when requested with the provider’s typed options. Shared streams expose them as ReasoningDeltaEvent; events.text() returns only text deltas. Invalid provider options raise RequestValidationError before anything is sent.

Tools

Let the model call your functions, with any provider

Describe your functions once as tools. The model answers with calls, your code runs them, and the results go back in the conversation until the model answers in text.

tools.py New in 0.15.0
import json
from typing import Any

from aiollm import (
    GenerationResponse,
    IGenerationClient,
    InputItem,
    Tool,
    ToolResult,
    Turn,
)

WEATHER = Tool(
    name="get_weather",
    description="Current weather in a city.",
    parameters={
        "type": "object",
        "properties": {"city": {"type": "string"}},
        "required": ["city"],
        "additionalProperties": False,
    },
    strict=True,
)


def get_weather(city: str) -> str:
    return json.dumps({"city": city, "celsius": 15})


async def ask_with_tools(
    client: IGenerationClient,
    model: str,
    question: str,
) -> GenerationResponse[Any]:
    conversation: list[InputItem] = [Turn.user(question)]
    while True:
        response = await client.generation.create(
            model=model, input=conversation, tools=[WEATHER]
        )
        if not response.tool_calls:
            return response
        conversation.append(response.to_turn())
        for call in response.tool_calls:
            arguments = call.parse_arguments()
            output = get_weather(str(arguments["city"]))
            conversation.append(
                ToolResult(call_id=call.id, output=output)
            )

The loop is yours: aiollm sends the calls and results, it never runs your functions.

A tool is a name, a description, and a schema

A Tool holds a JSON Schema for its arguments; with strict=True the arguments follow it exactly. tool_choice picks AUTO, NONE, REQUIRED, or one ForcedTool.

Calls and results in the conversation

response.tool_calls lists each ToolCall, and parse_arguments() gives its arguments. to_turn() and a ToolResult per call continue the conversation; a call without its result is refused before sending.

Reasoning comes back with the calls

The turn of calls keeps the provider’s own output, OpenAI’s reasoning items or Anthropic’s signed thinking, so the same provider continues from its own reasoning.

Calls arrive while streaming

stream() yields ToolCallStartEvent, ToolCallDeltaEvent, and ToolCallDoneEvent for each call, so a tool can start before the response ends.

Read the tools guide ↗
Request flow

One path for every call, with the retry exactly where it belongs

Each stage is a protocol, so the API, authentication, error mapping, headers and JSON codec can be swapped without touching the rest.

  1. 1

    BaseAPIMethod

    A Pydantic model: path, verb, body format and result type.

  2. 2

    send_request

    The session builds headers and auth, and logs the call.

  3. 3

    _send_once

    Retried by tenacity, only if the method is retryable.

  4. 4

    transport

    aiohttp sends it and buffers or streams the response.

  5. 5

    status check

    The resolver turns a bad status into a typed error.

  6. 6

    parser

    The body is validated into the method’s result type.

3–5Steps in light blue run inside each attempt, so an error’s transient flag alone decides whether to try again.

Providers

Two providers today, one generation API

client.generation takes the same request for every provider and returns shared types, with the provider’s own response kept in raw. Each provider’s own types are generated from its schema.

OpenAIClient

Responses API

Reads OPENAI_API_KEY; quota errors are typed and never retried.

AnthropicClient

Messages API

Reads ANTHROPIC_API_KEY; a 529 overload is a transient error.

BaseProviderClient

your provider

Bring an IAPI, a resolver and adapters; the shared contract tests run against it.

Choose a model supported by the client you pass in. Anthropic uses a default output budget of 16,000 tokens when the shared request omits it.

Native methods, sessions, and provider options ↗
from typing import Any

from aiollm import GenerationResponse, IGenerationClient


async def summarize(
    client: IGenerationClient,  # OpenAI, Anthropic, yours
    model: str,  # choose a model for this provider
    text: str,
) -> GenerationResponse[Any]:
    return await client.generation.create(
        model=model,
        input=text,
        instructions="Summarize in two sentences",
        max_output_tokens=200,
        timeout=30,
    )
Streaming & reliability

Know whether a stream actually finished

Work with bytes, decoded text, SSE, or NDJSON. For event streams, a declared protocol terminal distinguishes a complete response from a connection that ended early.

checked_stream.py
from typing import Any

from aiollm import CompletedEvent, IGenerationClient


async def stream_answer(
    client: IGenerationClient,
    model: str,
) -> CompletedEvent[Any]:
    async with client.generation.stream(
        model=model,
        input="Explain asynchronous iterators.",
    ) as events:
        async for delta in events.text():
            print(delta, end="", flush=True)

        # The final event includes usage and stop details.
        return await events.final()

Exactly one completion event

Shared streams end with one CompletedEvent. A stream ending without it raises APIStreamInterruptedError; another event after completion raises APIResponseError.

Retry the opening, not the output

Once iteration starts, failures reach your code. Retries require an opted-in method, a transient error, and a replayable request body. Quota errors and one-shot uploads are not retried.

Cleanup follows the context

Leaving async with releases the connection, including on an early break or an exception. Reading a stream after the context exits raises RuntimeError.

Read the streaming contract ↗
Keep exploring

From the first call to your own provider

Follow the guides for the contracts, then use the generated reference for every public type and method.

0.15.0

Pre-alpha: new capabilities, an evolving public API

Version 0.15.0 is the first on PyPI: the core is aiollm-core, and the providers are packages of their own, aiollm-openai and aiollm-anthropic, imported as aiollm_openai and aiollm_anthropic. aiollm.testing runs a provider of your own through the scenarios the library’s providers pass. Before 1.0, breaking changes bump the minor version: 0.15 is followed by 0.16. Read the release notes and versioning policy before upgrading.

Before you build

A few useful boundaries

What the shared API handles, where provider-specific behavior belongs, and what to expect before 1.0.

Is aiollm a replacement for every official SDK endpoint?

The built-in clients cover OpenAI’s Responses API and Anthropic’s Messages API, with shared text generation on top. Use native methods and typed provider_options for provider-specific features; the shared interface is deliberately smaller.

Read the guide ↗
Does a dropped stream automatically start again?

No. Opening a retryable stream may be retried, but a failure after it starts reaches your code. Streams are neither automatically resumed nor replayed, so your application does not receive duplicated output.

Read the guide ↗
Does aiollm run my tools for me?

No. The model returns tool calls, and your code runs them and sends the results back in the conversation. The loop is yours to shape: how many rounds, what to do with a failing tool, and whether to run calls in parallel.

Read the guide ↗
What is the difference between an omitted value and null?

A field left MISSING is not sent. An explicit None becomes null only where that method’s schema allows it. Shared generation’s optional fields use MISSING and do not accept None.

Read the guide ↗
Can I use another provider or HTTP transport?

Yes. Sessions depend on protocols for the API, authentication, headers, parsers, and error resolver. Add method classes and an adapter for a provider; implement a transport below BaseSession to change the HTTP client.

Read the guide ↗
Is there a synchronous API?

The library is async-only. Use async with to own a client or session and release its connections when the operation ends.

Read the guide ↗
What does pre-alpha mean for upgrades?

The public API can still change. Before 1.0, breaking changes increase the minor version, and the changelog records the migration. Version 0.9 is followed by 0.10; 1.0 marks an API stability commitment.

Read the guide ↗