Typed methods
send_request(method) returns the method’s result type, and mypy and pyright both infer it.


Describe each API call as a Pydantic model whose generic argument is its result. aiollm sends it over aiohttp, retries what is safe to retry, and hands back a validated result your type checker already knows.
pip install aiollm-openaiOn PyPI as aiollm-core, aiollm-openai and aiollm-anthropic. Requires Python 3.13 or newer.
import asyncio
from aiollm_openai import OpenAIClient
async def main() -> None:
async with OpenAIClient() as client: # OPENAI_API_KEY
response = await client.generation.create(
model="gpt-6-astra",
input="Explain asyncio in one line",
instructions="Be brief",
max_output_tokens=64,
)
print(response.text)
print(response.usage) # shared TokenUsage
print(response.raw) # OpenAI's own Response
asyncio.run(main())AnthropicClient and the call stays the same.Requires Python 3.13+. Install the provider you call; it brings the core, aiollm-core, with it.
pip install aiollm-openaipip install aiollm-anthropicOnly your own API? pip install aiollm-core. With uv: uv add aiollm-openai.
Set the corresponding environment variable in your shell. A client reads it when you provide neither a token nor a ready session.
OPENAI_API_KEYANTHROPIC_API_KEYUse client.generation for a shared text API, native provider methods for their exact fields, or declare your own HTTP endpoint.
aiollm is a low-level toolkit. It does not hide HTTP; it makes every request a checked, inspectable object and handles the parts every client gets wrong.
send_request(method) returns the method’s result type, and mypy and pyright both infer it.
JSON models, text, bytes or nothing; streams of typed events from SSE and NDJSON, checked for a declared last event.
Transient failures retry with exponential backoff and jitter, honouring Retry-After; a RetryPolicy tunes them per session or per call.
One class per common HTTP status, plus transport and response errors, each with the provider’s code, the request id and a transient flag.
From bytes, a path, a binary file or an async iterator. Replayable sources reopen for retries; one-shot sources explicitly disable them.
Path parameters are escaped, redirects are off, environment proxies are ignored, and tokens stay out of repr and logs.
Pass keyword arguments or a GenerationRequest to create() or stream(). The same calling code works with either built-in client through IGenerationClient, tools included.
TurnBuild a conversation from user and assistant turns. Carry an earlier answer forward with response.to_turn(); put the system prompt in instructions.
output_schemaSend a JSON Schema through the shared API. The result remains JSON text; check incomplete_reason before parsing it. Each provider supports its own schema subset.
EffortRequest LOW, MEDIUM, HIGH, XHIGH, or MAX. Omit it to keep the model’s default; the provider decides which levels that model accepts.
GenerationResponseRead text, token usage, refusals, and completion status in shared fields. Provider-specific details stay available in raw.
from typing import Any
from aiollm import Effort, GenerationResponse, IGenerationClient
async def extract_summary(
client: IGenerationClient,
model: str,
text: str,
) -> GenerationResponse[Any]:
return await client.generation.create(
model=model,
input=text,
instructions="Summarize the supplied text.",
effort=Effort.MEDIUM,
output_schema={
"type": "object",
"properties": {
"summary": {"type": "string"},
},
"required": ["summary"],
"additionalProperties": False,
},
max_output_tokens=1024,
)The response keeps its JSON text in text. Check incomplete_reason before parsing it.
| Shared field | OpenAI | Anthropic |
|---|---|---|
| instructions | instructions | system |
| max_output_tokens | max_output_tokens | max_tokens |
| effort | reasoning.effort | output_config.effort |
| output_schema | text.format | output_config.format |
| end_user_id | safety_identifier | metadata.user_id |
| tools | tools (function tools) | tools (custom tools) |
| tool_choice | tool_choice | tool_choice (REQUIRED as any) |
| parallel_tool_calls | parallel_tool_calls | tool_choice.disable_parallel_tool_use |
Reasoning summaries are available when requested with the provider’s typed options. Shared streams expose them as ReasoningDeltaEvent; events.text() returns only text deltas. Invalid provider options raise RequestValidationError before anything is sent.
Describe your functions once as tools. The model answers with calls, your code runs them, and the results go back in the conversation until the model answers in text.
import json
from typing import Any
from aiollm import (
GenerationResponse,
IGenerationClient,
InputItem,
Tool,
ToolResult,
Turn,
)
WEATHER = Tool(
name="get_weather",
description="Current weather in a city.",
parameters={
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"],
"additionalProperties": False,
},
strict=True,
)
def get_weather(city: str) -> str:
return json.dumps({"city": city, "celsius": 15})
async def ask_with_tools(
client: IGenerationClient,
model: str,
question: str,
) -> GenerationResponse[Any]:
conversation: list[InputItem] = [Turn.user(question)]
while True:
response = await client.generation.create(
model=model, input=conversation, tools=[WEATHER]
)
if not response.tool_calls:
return response
conversation.append(response.to_turn())
for call in response.tool_calls:
arguments = call.parse_arguments()
output = get_weather(str(arguments["city"]))
conversation.append(
ToolResult(call_id=call.id, output=output)
)The loop is yours: aiollm sends the calls and results, it never runs your functions.
A Tool holds a JSON Schema for its arguments; with strict=True the arguments follow it exactly. tool_choice picks AUTO, NONE, REQUIRED, or one ForcedTool.
response.tool_calls lists each ToolCall, and parse_arguments() gives its arguments. to_turn() and a ToolResult per call continue the conversation; a call without its result is refused before sending.
The turn of calls keeps the provider’s own output, OpenAI’s reasoning items or Anthropic’s signed thinking, so the same provider continues from its own reasoning.
stream() yields ToolCallStartEvent, ToolCallDeltaEvent, and ToolCallDoneEvent for each call, so a tool can start before the response ends.
Each stage is a protocol, so the API, authentication, error mapping, headers and JSON codec can be swapped without touching the rest.
A Pydantic model: path, verb, body format and result type.
The session builds headers and auth, and logs the call.
Retried by tenacity, only if the method is retryable.
aiohttp sends it and buffers or streams the response.
The resolver turns a bad status into a typed error.
The body is validated into the method’s result type.
3–5Steps in light blue run inside each attempt, so an error’s transient flag alone decides whether to try again.
client.generation takes the same request for every provider and returns shared types, with the provider’s own response kept in raw. Each provider’s own types are generated from its schema.
Reads OPENAI_API_KEY; quota errors are typed and never retried.
Reads ANTHROPIC_API_KEY; a 529 overload is a transient error.
Bring an IAPI, a resolver and adapters; the shared contract tests run against it.
Choose a model supported by the client you pass in. Anthropic uses a default output budget of 16,000 tokens when the shared request omits it.
Native methods, sessions, and provider options ↗from typing import Any
from aiollm import GenerationResponse, IGenerationClient
async def summarize(
client: IGenerationClient, # OpenAI, Anthropic, yours
model: str, # choose a model for this provider
text: str,
) -> GenerationResponse[Any]:
return await client.generation.create(
model=model,
input=text,
instructions="Summarize in two sentences",
max_output_tokens=200,
timeout=30,
)Work with bytes, decoded text, SSE, or NDJSON. For event streams, a declared protocol terminal distinguishes a complete response from a connection that ended early.
from typing import Any
from aiollm import CompletedEvent, IGenerationClient
async def stream_answer(
client: IGenerationClient,
model: str,
) -> CompletedEvent[Any]:
async with client.generation.stream(
model=model,
input="Explain asynchronous iterators.",
) as events:
async for delta in events.text():
print(delta, end="", flush=True)
# The final event includes usage and stop details.
return await events.final()Shared streams end with one CompletedEvent. A stream ending without it raises APIStreamInterruptedError; another event after completion raises APIResponseError.
Once iteration starts, failures reach your code. Retries require an opted-in method, a transient error, and a replayable request body. Quota errors and one-shot uploads are not retried.
Leaving async with releases the connection, including on an early break or an exception. Reading a stream after the context exits raises RuntimeError.
Follow the guides for the contracts, then use the generated reference for every public type and method.
Version 0.15.0 is the first on PyPI: the core is aiollm-core, and the providers are packages of their own, aiollm-openai and aiollm-anthropic, imported as aiollm_openai and aiollm_anthropic. aiollm.testing runs a provider of your own through the scenarios the library’s providers pass. Before 1.0, breaking changes bump the minor version: 0.15 is followed by 0.16. Read the release notes and versioning policy before upgrading.
What the shared API handles, where provider-specific behavior belongs, and what to expect before 1.0.
The built-in clients cover OpenAI’s Responses API and Anthropic’s Messages API, with shared text generation on top. Use native methods and typed provider_options for provider-specific features; the shared interface is deliberately smaller.
Read the guide ↗No. Opening a retryable stream may be retried, but a failure after it starts reaches your code. Streams are neither automatically resumed nor replayed, so your application does not receive duplicated output.
Read the guide ↗No. The model returns tool calls, and your code runs them and sends the results back in the conversation. The loop is yours to shape: how many rounds, what to do with a failing tool, and whether to run calls in parallel.
Read the guide ↗A field left MISSING is not sent. An explicit None becomes null only where that method’s schema allows it. Shared generation’s optional fields use MISSING and do not accept None.
Read the guide ↗Yes. Sessions depend on protocols for the API, authentication, headers, parsers, and error resolver. Add method classes and an adapter for a provider; implement a transport below BaseSession to change the HTTP client.
Read the guide ↗The library is async-only. Use async with to own a client or session and release its connections when the operation ends.
Read the guide ↗The public API can still change. Before 1.0, breaking changes increase the minor version, and the changelog records the migration. Version 0.9 is followed by 0.10; 1.0 marks an API stability commitment.
Read the guide ↗