aiollm.types¶
types
¶
Data models, upload files and stream item matchers.
Modules:
| Name | Description |
|---|---|
base |
Base classes for API data models. |
files |
Files to upload in multipart requests. |
matchers |
Matchers that recognize stream items by their content. |
protocols |
Interfaces of the data layer. |
requests |
Requests in types shared by all providers. |
responses |
Responses and stream events in types shared by all providers. |
unknown |
Values of a |
Classes:
| Name | Description |
|---|---|
BaseAPIType |
Base class for request and response payload models. |
BaseAPITypeExtra |
Base class for response models that keep the fields they do not model. |
BytesFile |
Content held in memory. |
ChunkedFile |
Content read in chunks, never held in memory as a whole. |
InputFile |
A file to upload, independent of the HTTP client. |
IOFile |
An already open binary stream. |
PathFile |
A file on disk, read in chunks in a worker thread. |
StreamFile |
Content produced by an async iterator; it can be read only once. |
AnyOf |
Matches an item that any of its matchers matches. |
BaseMatcher |
Base class of matchers; gives every matcher |
EventTypes |
Matches an event by its |
FieldValues |
Matches an item whose field has one of the given values. |
TopLevelError |
Matches a JSON object without |
IEventMatcher |
Recognizes items of a stream, such as its terminal or error events. |
BaseUnknown |
A value of a |
Functions:
| Name | Description |
|---|---|
by_type |
Build a pydantic discriminator that falls back to |
Attributes:
| Name | Type | Description |
|---|---|---|
DONE |
Final
|
Matches |
DONE_MARKER |
Final
|
What the stream parser passes to a matcher for |
UNKNOWN |
Final
|
Tag of a value whose |
DONE
module-attribute
¶
DONE: Final = _Done()
Matches data: [DONE], the SSE convention for the end of a stream.
Some streaming APIs send a [DONE] line instead of a typed last event;
declaring DONE as __stream_terminal__ makes a stream that ends
without it an interrupted one. Neither the Responses nor the Messages API
uses it.
DONE_MARKER
module-attribute
¶
DONE_MARKER: Final = _DoneMarker()
What the stream parser passes to a matcher for data: [DONE].
[DONE] is not a stream item and is never yielded.
UNKNOWN
module-attribute
¶
UNKNOWN: Final = 'unknown'
Tag of a value whose type the library does not know yet.
BaseAPIType
¶
Bases: BaseModel
Base class for request and response payload models.
Unknown fields are ignored, so new fields added by the provider do
not break validation. Fields can be populated by name or alias. Give
a wire name with validation_alias and serialization_alias
rather than alias: then the constructor takes the field name in
both mypy and pyright (with alias they disagree). A field of a
type Pydantic cannot validate fails at class definition: payloads
are JSON, so such a field is a mistake (only the raw of shared
results allows any type, see BaseResponse).
Attributes:
| Name | Type | Description |
|---|---|---|
__upstream_schema__ |
str | tuple[str, ...] | None
|
Name of the definition in the provider's
upstream JSON schema that the model mirrors, several names
for a model that covers several definitions, or |
BaseAPITypeExtra
¶
Bases: BaseAPIType
Base class for response models that keep the fields they do not model.
A field the model does not declare is validated as nothing and kept
in model_extra, so whatever the provider sent is still there:
the provider types model only what the library reads, and the rest
is reached through raw of a shared result. Everything else is as
for BaseAPIType.
Example
class Reply(BaseAPITypeExtra): ... text: str Reply.model_validate({"text": "hi", "seed": 7}).model_extra
BytesFile
¶
Bases: InputFile
Content held in memory.
Sent with a known length, which servers that reject chunked uploads require.
Attributes:
| Name | Type | Description |
|---|---|---|
data |
The file content. |
Create the file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
bytes
|
The file content. |
required |
filename
|
str | None
|
File name sent in the multipart part. |
None
|
content_type
|
str | None
|
MIME type; guessed from |
None
|
Methods:
| Name | Description |
|---|---|
open |
Yield the whole content. |
open
async
¶
open() -> AsyncIterator[bytes]
Yield the whole content.
Yields:
| Type | Description |
|---|---|
AsyncIterator[bytes]
|
The content as a single chunk. |
ChunkedFile
¶
ChunkedFile(
filename: str | None = None,
content_type: str | None = None,
chunk_size: int = DEFAULT_CHUNK_SIZE,
)
Bases: InputFile
Content read in chunks, never held in memory as a whole.
The base of PathFile and IOFile, and of a file of your own
that reads its source piece by piece.
Attributes:
| Name | Type | Description |
|---|---|---|
chunk_size |
Number of bytes read at a time. |
Store the file metadata and chunk size.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filename
|
str | None
|
File name sent in the multipart part. |
None
|
content_type
|
str | None
|
MIME type; guessed from |
None
|
chunk_size
|
int
|
Number of bytes read at a time. |
DEFAULT_CHUNK_SIZE
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
InputFile
¶
Bases: ABC
A file to upload, independent of the HTTP client.
Fields annotated as InputFile accept bytes, paths and binary
streams as well and convert them to the matching subclass.
Attributes:
| Name | Type | Description |
|---|---|---|
filename |
File name sent in the multipart part. |
|
content_type |
MIME type sent in the multipart part; guessed from
|
Example
class Transcribe(BaseAPIMethod[Transcription]): ... file: InputFile Transcribe(file=Path("voice.mp3")).file PathFile(filename='voice.mp3', content_type='audio/mpeg')
Store the file metadata.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filename
|
str | None
|
File name sent in the multipart part. |
None
|
content_type
|
str | None
|
MIME type; guessed from |
None
|
Methods:
| Name | Description |
|---|---|
check |
Check, before sending, that the content can be read. |
open |
Read the content from its beginning. |
from_value |
Convert a raw value into a file. |
reusable
property
¶
reusable: bool
Whether the content can be read more than once.
A request carrying a non-reusable file is never retried.
check
¶
Check, before sending, that the content can be read.
Called once per request; the default accepts every file.
Raises:
| Type | Description |
|---|---|
OSError
|
If the content cannot be read. |
open
abstractmethod
¶
open() -> AsyncIterator[bytes]
Read the content from its beginning.
Called once per upload attempt.
Returns:
| Type | Description |
|---|---|
AsyncIterator[bytes]
|
An iterator over chunks of the content. |
from_value
classmethod
¶
IOFile
¶
IOFile(
file: BinaryStream,
filename: str | None = None,
content_type: str | None = None,
chunk_size: int = DEFAULT_CHUNK_SIZE,
)
Bases: ChunkedFile
An already open binary stream.
Seekable streams are read from the position they had when the file was created, on every attempt; non-seekable ones can be read once.
Attributes:
| Name | Type | Description |
|---|---|---|
file |
The wrapped stream. |
|
chunk_size |
Number of bytes read at a time. |
Wrap the stream and remember its current position.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
file
|
BinaryStream
|
An open binary stream. |
required |
filename
|
str | None
|
File name to send; the stream's |
None
|
content_type
|
str | None
|
MIME type; guessed from the file name if omitted. |
None
|
chunk_size
|
int
|
Number of bytes read at a time. |
DEFAULT_CHUNK_SIZE
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If the stream is closed or not readable. |
Methods:
| Name | Description |
|---|---|
open |
Read the stream from its start position. |
open
async
¶
open() -> AsyncIterator[bytes]
Read the stream from its start position.
Blocking calls run in a worker thread. The stream counts as being read until the iterator ends or is closed; the session closes it after every attempt, so a retry can read it again. A read that was cancelled leaves its last call running in the thread; the next read waits for that call before it seeks.
Yields:
| Type | Description |
|---|---|
AsyncIterator[bytes]
|
Chunks of at most |
Raises:
| Type | Description |
|---|---|
RuntimeError
|
If the stream is being read by another upload at the moment, or a non-seekable stream is read a second time. |
BlockingIOError
|
If the stream is non-blocking and has no data
ready: its |
PathFile
¶
PathFile(
path: str | PathLike[str],
filename: str | None = None,
content_type: str | None = None,
chunk_size: int = DEFAULT_CHUNK_SIZE,
)
Bases: ChunkedFile
A file on disk, read in chunks in a worker thread.
Attributes:
| Name | Type | Description |
|---|---|---|
path |
Location of the file. |
|
chunk_size |
Number of bytes read at a time. |
Create the file; it is not opened until upload.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
str | PathLike[str]
|
Location of the file. |
required |
filename
|
str | None
|
File name to send; the path's name by default. |
None
|
content_type
|
str | None
|
MIME type; guessed from the file name if omitted. |
None
|
chunk_size
|
int
|
Number of bytes read at a time. |
DEFAULT_CHUNK_SIZE
|
Methods:
| Name | Description |
|---|---|
check |
Check that the path is an existing file. |
open |
Read the file from the beginning. |
check
¶
Check that the path is an existing file.
Raises:
| Type | Description |
|---|---|
IsADirectoryError
|
If |
FileNotFoundError
|
If there is no file at |
open
async
¶
open() -> AsyncIterator[bytes]
Read the file from the beginning.
The file is closed when the iterator ends or is closed; after a cancellation, once the read running in the worker thread is over.
Yields:
| Type | Description |
|---|---|
AsyncIterator[bytes]
|
Chunks of at most |
StreamFile
¶
StreamFile(
stream: AsyncIterable[bytes],
filename: str | None = None,
content_type: str | None = None,
)
Bases: InputFile
Content produced by an async iterator; it can be read only once.
A request carrying it is never retried. StreamFile takes ownership
of the source: the upload closes it when it ends, however it ends
(read to the end, cut short, cancelled or failed), by aclose(),
else by close().
Attributes:
| Name | Type | Description |
|---|---|---|
stream |
The source of the content. |
Create the file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
stream
|
AsyncIterable[bytes]
|
The source of the content. |
required |
filename
|
str | None
|
File name sent in the multipart part. |
None
|
content_type
|
str | None
|
MIME type; guessed from |
None
|
Methods:
| Name | Description |
|---|---|
open |
Hand the source over to the upload. |
open
¶
open() -> AsyncIterator[bytes]
Hand the source over to the upload.
The returned iterator is not a generator, so closing it closes the source also before the first chunk is read.
Returns:
| Type | Description |
|---|---|
AsyncIterator[bytes]
|
An iterator over the source's chunks, whose |
AsyncIterator[bytes]
|
closes the source. |
Raises:
| Type | Description |
|---|---|
RuntimeError
|
If the file is read a second time. |
AnyOf
¶
AnyOf(*matchers: IEventMatcher)
Bases: BaseMatcher
Matches an item that any of its matchers matches.
Attributes:
| Name | Type | Description |
|---|---|---|
matchers |
tuple[IEventMatcher, ...]
|
The combined matchers; nested |
Combine matchers.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
*matchers
|
IEventMatcher
|
The matchers to combine. |
()
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If no matcher is given, since the result would never match. |
Methods:
| Name | Description |
|---|---|
matches |
Check one item against every matcher. |
BaseMatcher
¶
EventTypes
¶
EventTypes(*types: str)
Bases: FieldValues
Matches an event by its type.
It is FieldValues("type", ...), and equal to it.
Example
EventTypes("message_stop").matches({"type": "message_stop"}) True
Set the accepted types.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
*types
|
str
|
The accepted values of |
()
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If no type is given. |
Attributes:
| Name | Type | Description |
|---|---|---|
types |
frozenset[str]
|
The accepted types. |
FieldValues
¶
Bases: BaseMatcher
Matches an item whose field has one of the given values.
The field is read as a key of a mapping (decoded JSON), a field of a
Pydantic model (never one of its properties) or an attribute of any
other object. Values are compared as JSON tells them apart: true
is not 1. Matchers of the same field and values are equal, as
EventTypes("x") and FieldValues("type", "x") are.
Attributes:
| Name | Type | Description |
|---|---|---|
name |
The field's name. |
|
values |
frozenset[object]
|
The accepted values. |
Example
FieldValues("status", "failed").matches({"status": "failed"}) True
Set the field and its values.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
The field's name. |
required |
*values
|
object
|
The accepted values; they must be hashable. |
()
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If no value is given, since the matcher would never match. |
Methods:
| Name | Description |
|---|---|
matches |
Check the field of one item. |
matches
¶
TopLevelError
¶
Bases: BaseMatcher
Matches a JSON object without type that reports an error.
Its error must be true or a non-empty object or string, as
OpenAI batch output and Ollama send it; null, false and empty
values mean "no error". An object with a type never matches, so
a regular event may carry an error field (a failed tool call, for
example).
Methods:
| Name | Description |
|---|---|
matches |
Check one item. |
IEventMatcher
¶
BaseUnknown
¶
Bases: BaseAPIType
A value of a type the library does not model yet.
Providers add output items, content blocks and stream events over time. Rejecting them would break a working stream on the first new event, and dropping them would lose data, so they are kept whole: every field the provider sent stays available as an extra field.
Attributes:
| Name | Type | Description |
|---|---|---|
type |
str
|
The provider's type of the value. |
by_type
¶
Build a pydantic discriminator that falls back to UNKNOWN.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
tags
|
Mapping[str, str]
|
Tag of each known |
required |
Returns:
| Type | Description |
|---|---|
Callable[[Any], str]
|
A function that returns the tag of a raw mapping or a model, and |
Callable[[Any], str]
|
|
requests
¶
Requests in types shared by all providers.
Modules:
| Name | Description |
|---|---|
base |
Base class of shared requests. |
builtin |
Built-in tools of a request: tools the provider runs itself. |
content |
Parts of a turn's content: text, images and documents. |
generation |
Shared request for text generation. |
schema |
JSON types that requests are made of. |
upload |
The shared request of a file upload. |
Type Aliases:
| Name | Description |
|---|---|
BuiltinTool |
A tool the provider runs: a shared one, or one of the provider. |
ContentPart |
A part of a user turn: text, an image or a document. |
DocumentMediaType |
The document formats both providers take: PDF. |
ImageMediaType |
The image formats both providers take. |
InputItem |
An item of a conversation: a text turn, a turn of calls or a result. |
JSONSchema |
A JSON Schema, as a JSON object. |
JSONValue |
A JSON value, as |
Classes:
| Name | Description |
|---|---|
BaseRequest |
Base class of requests in shared types. |
CodeExecution |
Let the model run code in the provider's sandbox. |
ProviderTool |
A built-in tool of one provider, as that provider takes it. |
UserLocation |
Roughly where the end user is, for a web search. |
WebSearch |
Let the model search the web, at the provider. |
DocumentPart |
A PDF document in a turn of the end user. |
ImagePart |
An image in a turn of the end user. |
ProviderFile |
A file stored by a provider, named by its id. |
TextPart |
Text in a turn of several parts. |
ForcedTool |
Make the model call this tool. |
GenerationRequest |
Generate text from an input. |
NativeOutput |
A response's output as its provider's next request takes it. |
NoProviderOptions |
Provider options of a generation resource that takes none. |
Tool |
A function in your code that the model may call. |
ToolCall |
A call of one of the request's tools. |
ToolCallTurn |
A turn of the model that calls tools. |
ToolResult |
What a tool returned for one call. |
Turn |
One turn of a conversation: who wrote it and what. |
FileUpload |
A file to store with a provider, as |
Functions:
| Name | Description |
|---|---|
tool |
Decorate a callable and expose its tool definition. |
BuiltinTool
¶
BuiltinTool = WebSearch | CodeExecution | ProviderTool
A tool the provider runs: a shared one, or one of the provider.
ContentPart
¶
ContentPart = Annotated[
TextPart | ImagePart | DocumentPart,
Discriminator("type"),
]
A part of a user turn: text, an image or a document.
DocumentMediaType
¶
DocumentMediaType = Literal['application/pdf']
The document formats both providers take: PDF.
ImageMediaType
¶
ImageMediaType = Literal[
"image/jpeg", "image/png", "image/gif", "image/webp"
]
The image formats both providers take.
InputItem
¶
InputItem = Annotated[
Annotated[Turn, Tag("text")]
| Annotated[ToolCallTurn, Tag("tool_calls")]
| Annotated[ToolResult, Tag("tool_result")],
Discriminator(_item_tag),
]
An item of a conversation: a text turn, a turn of calls or a result.
JSONValue
¶
A JSON value, as json.loads gives it.
BaseRequest
¶
Bases: BaseAPIType
Base class of requests in shared types.
Unlike responses, requests reject unknown fields, so a typo fails
validation instead of being dropped, and validate assignments, so a
value set after construction (None where a field takes none,
say) fails as it would in the constructor. model_copy(update=...)
validates nothing, as everywhere in Pydantic.
CodeExecution
¶
Bases: BaseAPIType
Let the model run code in the provider's sandbox.
A built-in tool both providers have. Its one shared option is the
files the code reads; the container itself (an id to reuse, its
memory and network) is each provider's own, set through a
ProviderTool. Immutable, and unknown fields are rejected.
Attributes:
| Name | Type | Description |
|---|---|---|
type |
Literal['code_execution']
|
Always |
files |
Annotated[list[ProviderFile], Field(min_length=1)] | MISSING
|
Stored files ( |
Example
stored = await client.files.upload(Path("sales.csv")) GenerationRequest( ... model="gpt-5", ... input="Chart the sales by month.", ... tools=CodeExecution(files=[stored.ref]), ... )
ProviderTool
¶
Bases: BaseAPIType
A built-in tool of one provider, as that provider takes it.
For a tool that is not shared (file search, web fetch, MCP, ...) or
that has options the shared ones lack. A provider package makes
them from its generated tool parameters, so a provider's own
constructor checks the fields; the core only keeps the object. Only
that provider's adapter takes it: any other raises
RequestValidationError, as for a ProviderFile. Immutable,
and unknown fields are rejected.
Attributes:
| Name | Type | Description |
|---|---|---|
type |
Literal['provider_tool']
|
Always |
provider |
Provider
|
The provider whose tool it is. |
definition |
dict[str, JSONValue]
|
The tool as the provider's request holds it, a JSON
object, such as |
Example
ProviderTool( ... provider=Provider("openai"), ... definition={ ... "type": "web_search", ... "search_context_size": "high", ... }, ... )
UserLocation
¶
Bases: BaseAPIType
Roughly where the end user is, for a web search.
Immutable, and unknown fields are rejected. Every field is left out until it is given.
Attributes:
| Name | Type | Description |
|---|---|---|
country |
str | MISSING
|
The country as a two-letter ISO code, such as |
region |
str | MISSING
|
A region, such as a state: |
city |
str | MISSING
|
A city: |
timezone |
str | MISSING
|
An IANA time zone: |
Example
UserLocation(city="Paris", country="FR")
WebSearch
¶
Bases: BaseAPIType
Let the model search the web, at the provider.
A built-in tool both providers have, with the options both take;
the provider's other options (OpenAI's search_context_size,
Anthropic's blocked_domains and max_uses, a version of the
tool) go through a ProviderTool of the provider's package.
Immutable, and unknown fields are rejected.
Attributes:
| Name | Type | Description |
|---|---|---|
type |
Literal['web_search']
|
Always |
allowed_domains |
Annotated[list[str], Field(min_length=1)] | MISSING
|
Search only these domains; at least one. |
user_location |
UserLocation | MISSING
|
Where the user is, to localize the results. |
Example
GenerationRequest( ... model="gpt-5", ... input="What happened today?", ... tools=WebSearch(allowed_domains=["example.com"]), ... )
DocumentPart
¶
Bases: _SourcePart
A PDF document in a turn of the end user.
Give its bytes, a URL the provider downloads it from, or a file
stored by the provider (client.files.upload), which is uploaded
once instead of with every request. Both providers read the text
and the pages as images; each limits the size and the number of
pages.
Attributes:
| Name | Type | Description |
|---|---|---|
type |
Literal['document']
|
Always |
data |
bytes | MISSING
|
The document, sent inline (base64). |
url |
str | MISSING
|
Where the provider downloads it from. |
file |
ProviderFile | MISSING
|
A document stored by the provider, sent by its id; the provider keeps its name. |
media_type |
DocumentMediaType
|
Always |
filename |
Annotated[str, Field(min_length=1)] | MISSING
|
The document's name, OpenAI's |
Example
DocumentPart(data=pdf, filename="report.pdf") DocumentPart(file=stored.ref)
Methods:
| Name | Description |
|---|---|
load |
Read a PDF file whole, in a worker thread. |
load
async
classmethod
¶
Read a PDF file whole, in a worker thread.
The content is not checked: any file is sent as a PDF, and the provider refuses one that is not. The bytes are kept in the part (base64 in the request body).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
str | Path
|
The file; its name becomes |
required |
Returns:
| Type | Description |
|---|---|
Self
|
The part, with the file's bytes. |
Raises:
| Type | Description |
|---|---|
OSError
|
The file cannot be read. |
ImagePart
¶
Bases: _SourcePart
An image in a turn of the end user.
Give its bytes with their media_type, a URL the provider
downloads it from, or a file stored by the provider
(client.files.upload). JPEG, PNG, GIF and WebP are taken by
both providers; each limits the size and the number of images.
Attributes:
| Name | Type | Description |
|---|---|---|
type |
Literal['image']
|
Always |
data |
bytes | MISSING
|
The image, sent inline (base64). |
url |
str | MISSING
|
Where the provider downloads it from. |
file |
ProviderFile | MISSING
|
An image stored by the provider, sent by its id. |
media_type |
ImageMediaType | MISSING
|
The format of |
Example
ImagePart(data=png, media_type="image/png") ImagePart(url="https://example.com/cat.jpg") ImagePart(file=stored.ref)
Methods:
| Name | Description |
|---|---|
load |
Read an image file whole, in a worker thread. |
load
async
classmethod
¶
load(
path: str | Path,
media_type: ImageMediaType | MISSING = MISSING,
) -> Self
Read an image file whole, in a worker thread.
The format is not checked against the file's content, and the bytes are kept in the part (base64 in the request body).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
str | Path
|
The file. |
required |
media_type
|
ImageMediaType | MISSING
|
Its format; guessed from the file name's extension when left out. |
MISSING
|
Returns:
| Type | Description |
|---|---|
Self
|
The part, with the file's bytes. |
Raises:
| Type | Description |
|---|---|
OSError
|
The file cannot be read. |
ValueError
|
The format is not given and the extension names
none of |
ProviderFile
¶
Bases: BaseAPIType
A file stored by a provider, named by its id.
The id alone does not say whose it is, so the provider goes with
it: a part or a client.files call that names another provider's
file raises RequestValidationError before anything is sent,
instead of a 404 from the provider. StoredFile.ref is one.
An id refers to data that every key of the provider's project or workspace can read: build one only from an id your application uploaded, never from an id an end user gives.
Attributes:
| Name | Type | Description |
|---|---|---|
provider |
Provider
|
The provider that stores the file. |
id |
Annotated[str, Field(min_length=1)]
|
Its id there, such as OpenAI's |
Example
stored = await client.files.upload(Path("report.pdf")) DocumentPart(file=stored.ref)
TextPart
¶
Bases: BaseAPIType
Text in a turn of several parts.
Attributes:
| Name | Type | Description |
|---|---|---|
type |
Literal['text']
|
Always |
text |
str
|
The text. |
Example
TextPart(text="What is in this picture?")
ForcedTool
¶
Bases: BaseAPIType
Make the model call this tool.
The tool_choice that names one tool, apart from the
ToolChoice modes, so a tool named auto is no mode.
Attributes:
| Name | Type | Description |
|---|---|---|
name |
str
|
Name of a tool of the request. |
Example
ForcedTool(name="get_weather")
GenerationRequest
¶
Bases: BaseRequest
Generate text from an input.
Attributes:
| Name | Type | Description |
|---|---|---|
model |
str
|
Model identifier of the provider; not empty. |
input |
Annotated[str, Field(min_length=1)] | Annotated[Sequence[InputItem], Field(min_length=1)]
|
Text to respond to, taken as one |
instructions |
str | MISSING
|
Instructions for the model given apart from the input, such as a system prompt. |
max_output_tokens |
int | MISSING
|
Upper bound on generated tokens; positive. |
end_user_id |
str | MISSING
|
An opaque id of the end user the request is made for, such as a hash, never personal data; providers use it to detect abuse. Each provider limits its length. |
effort |
Effort | MISSING
|
How much work the model puts into the response; the model's own default when left out. |
output_schema |
JSONSchema | MISSING
|
A JSON Schema the response text must follow; the text is then JSON. Each provider takes its own subset of JSON Schema and answers 400 to the rest; the schema is sent as it is, never rewritten. |
tools |
list[Tool] | MISSING
|
A single tool or a sequence of them: functions the
model may call (a |
tool_choice |
ToolChoice | ForcedTool | MISSING
|
Whether the model calls a tool (a |
parallel_tool_calls |
bool | MISSING
|
Whether the model may call several tools in
one response; both providers allow it by default. Only with
|
An optional field is left out until it is given; None is not
taken (it would mean null, which not every provider accepts),
so leave a field out rather than set it to None.
Example
def get_weather(city: str) -> dict: ... """Get current weather in a city.""" ... ... GenerationRequest( ... model="gpt-5", ... input="What is the weather in Paris?", ... tools=get_weather, ... ) GenerationRequest( ... model="gpt-5", ... input="Find the weather.", ... tools=[get_weather, another_tool], ... )
tools
property
¶
Return the function tools after converting callables.
MISSING when the request has no function tool, even if it
has built-in tools; see builtin_tools.
builtin_tools
property
¶
builtin_tools: list[BuiltinTool]
Return the built-in tools of the request, empty without any.
NativeOutput
¶
Bases: BaseAPIType
A response's output as its provider's next request takes it.
The provider's own record of the turn: OpenAI's output items, its
reasoning items with encrypted_content among them, or Anthropic's
content blocks, thinking with its signature among them, each as
the response sent it. The provider's adapter sends it in place of the
shared fields of the turn, so the model sees its reasoning again;
another provider's adapter sends the shared fields only, since it
cannot read it.
Attributes:
| Name | Type | Description |
|---|---|---|
provider |
Provider
|
The provider whose output it is. |
items |
Annotated[list[dict[str, JSONValue]], Field(min_length=1)]
|
The items, in output order. |
NoProviderOptions
¶
Bases: TypedDict
Provider options of a generation resource that takes none.
The default of GenerationResource's OptionsT. It is closed
(PEP 728): only {} matches it, not even a TypedDict with keys
of its own, which an open empty TypedDict would take. An adapter
without options still checks what it is given at run time with
checked_options(options, NoProviderOptions, provider).
Tool
¶
Bases: BaseAPIType
A function in your code that the model may call.
Immutable, and unknown fields are rejected.
Attributes:
| Name | Type | Description |
|---|---|---|
name |
str
|
Name the model calls it by: letters, digits, |
description |
str | MISSING
|
What the tool does and when to use it. |
parameters |
JSONSchema
|
JSON Schema of its arguments, an object schema; a
tool without arguments takes |
strict |
bool | MISSING
|
Whether the arguments must follow |
Example
Tool( ... name="get_weather", ... description="Current weather in a city.", ... parameters={ ... "type": "object", ... "properties": {"city": {"type": "string"}}, ... "required": ["city"], ... }, ... )
Methods:
| Name | Description |
|---|---|
from_callable |
Create a tool from a callable's signature and annotations. |
from_model |
Create a tool whose arguments are a Pydantic model. |
from_callable
classmethod
¶
from_callable(
obj: Callable[P, RT],
*,
description: str | MISSING = MISSING,
strict: bool | MISSING = MISSING,
) -> Self
Create a tool from a callable's signature and annotations.
The callable's __name__ is used as the tool name; a
functools.partial takes the name of the function it wraps,
and an object with a __call__ method the name of its class.
Its parameters and annotations are converted to JSON Schema
using Pydantic's arguments schema generation: a partial's bound
arguments are left out, an object's __call__ is read without
self, *args is an array property named after it and
**kwargs an object property named after it. Any other key is
refused, so the schema has additionalProperties: false.
If description is not provided, the callable's docstring
is used (a partial's function's, an object's class's or else
its __call__'s). If it has none, the description is omitted.
The callable is retained as the tool callback. Defining the
tool does not invoke it; generation.tool_runner invokes it
for a matching tool call after a complete response.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
obj
|
Callable[P, RT]
|
The callable whose signature defines the tool's arguments. |
required |
description
|
str | MISSING
|
What the tool does and when to use it. Defaults to the callable's docstring. |
MISSING
|
strict
|
bool | MISSING
|
Whether the arguments must follow the generated schema exactly. |
MISSING
|
Returns:
| Type | Description |
|---|---|
Self
|
The tool. |
Raises:
| Type | Description |
|---|---|
TypeError
|
|
Example
def get_weather(city: str, units: str = "celsius"): ... ... tool = Tool.from_callable( ... get_weather, ... description="Get current weather in a city.", ... )
from_model
classmethod
¶
from_model(
name: str,
model: type[BaseModel],
*,
description: str | MISSING = MISSING,
strict: bool | MISSING = MISSING,
) -> Self
Create a tool whose arguments are a Pydantic model.
parameters is the model's JSON Schema
(model.model_json_schema()), sent as it is, and
ToolCall.parse_arguments(model) reads the arguments back
into the model. For strict=True the model forbids extra
fields (extra="forbid") and every field is required, or the
providers answer 400.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
Name the model calls it by. |
required |
model
|
type[BaseModel]
|
The model of the arguments. |
required |
description
|
str | MISSING
|
What the tool does and when to use it. |
MISSING
|
strict
|
bool | MISSING
|
Whether the arguments must follow the schema exactly. |
MISSING
|
Returns:
| Type | Description |
|---|---|
Self
|
The tool. |
Example
class City(BaseModel): ... model_config = ConfigDict(extra="forbid") ... city: str tool = Tool.from_model("get_weather", City, strict=True)
ToolCall
¶
Bases: BaseAPIType
A call of one of the request's tools.
Immutable, and unknown fields are rejected. The arguments are the
JSON text the model wrote, as OpenAI sends them and as both stream
them; Anthropic's object is written as JSON. Without strict the
text may be invalid JSON, so it is parsed only on
parse_arguments().
Attributes:
| Name | Type | Description |
|---|---|---|
id |
str
|
The call's id, which its result refers to (OpenAI's
|
name |
str
|
The tool called. |
arguments |
str
|
The arguments, a JSON object as text. |
Example
for call in response.tool_calls: ... result = run(call.name, call.parse_arguments())
Methods:
| Name | Description |
|---|---|
parse_arguments |
Parse the tool call arguments, validating against a schema. |
parse_arguments
¶
parse_arguments(model: type[ModelT]) -> ModelT
Parse the tool call arguments, validating against a schema.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model
|
type[BaseModel] | Tool | Callable[..., Any] | None
|
An optional Pydantic model, |
None
|
Returns:
| Type | Description |
|---|---|
Any
|
The parsed JSON object when |
Any
|
returns the result of |
Any
|
Pydantic model instance or a tuple of positional and keyword |
Any
|
arguments ( |
Raises:
| Type | Description |
|---|---|
TypeError
|
The supplied tool or callable has no usable argument schema. |
ValidationError
|
The arguments are not satisfy the expected schema. |
ValueError
|
The arguments are not a JSON object. |
ToolCallTurn
¶
Bases: BaseAPIType
A turn of the model that calls tools.
Made by GenerationResponse.to_turn() from a response with calls;
the next items of the conversation are a ToolResult for each of
its calls.
Attributes:
| Name | Type | Description |
|---|---|---|
type |
Literal['tool_calls']
|
Always |
text |
str | MISSING
|
Text the model wrote with the calls, if any. |
tool_calls |
Annotated[list[ToolCall], Field(min_length=1)]
|
The calls; at least one. |
native |
NativeOutput | MISSING
|
The response's own output, which its provider gets back
in place of |
ToolResult
¶
Bases: BaseAPIType
What a tool returned for one call.
Immutable, and unknown fields are rejected. Describe a failure in
output: OpenAI has no error flag, and both read the text.
Attributes:
| Name | Type | Description |
|---|---|---|
type |
Literal['tool_result']
|
Always |
call_id |
str
|
|
output |
str
|
The result as text, such as JSON. |
Example
ToolResult(call_id=call.id, output='{"celsius": 15}')
Turn
¶
Bases: BaseAPIType
One turn of a conversation: who wrote it and what.
Immutable, and unknown fields are rejected. System prompts are not
turns: they go in GenerationRequest.instructions.
Attributes:
| Name | Type | Description |
|---|---|---|
type |
Literal['text']
|
Always |
role |
Role
|
Who wrote the turn. |
content |
str | Annotated[list[ContentPart], Field(min_length=1)]
|
Text of the turn, or its parts in order: text, images
and documents, at least one. Only a |
native |
NativeOutput | MISSING
|
The generation's own output, which its provider gets
back in place of |
Example
Turn.user("Hi") Turn(type='text', role=<Role.USER: 'user'>, content='Hi') Turn.user([TextPart(text="What is it?"), await ... ImagePart.load("cat.png")])
Methods:
| Name | Description |
|---|---|
user |
Create a turn of the end user. |
assistant |
Create a turn of the model, such as an earlier answer. |
user
classmethod
¶
user(content: str | Sequence[ContentPart]) -> Self
Create a turn of the end user.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
content
|
str | Sequence[ContentPart]
|
Text of the turn, or its parts. |
required |
Returns:
| Type | Description |
|---|---|
Self
|
The turn. |
assistant
classmethod
¶
assistant(
content: str,
*,
native: NativeOutput | MISSING = MISSING,
) -> Self
Create a turn of the model, such as an earlier answer.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
content
|
str
|
Text of the turn. |
required |
native
|
NativeOutput | MISSING
|
The generation's own output, as
|
MISSING
|
Returns:
| Type | Description |
|---|---|
Self
|
The turn. |
FileUpload
¶
Bases: BaseRequest
A file to store with a provider, as client.files.upload takes it.
Attributes:
| Name | Type | Description |
|---|---|---|
file |
InputFile
|
The content: an |
expires_in_seconds |
Annotated[int, Field(gt=0)] | MISSING
|
Seconds after the upload when the provider deletes the file. Each provider has its own range (OpenAI 3600 to 2 592 000, Anthropic 3600 to 7 776 000) and checks it before anything is sent; left out, the file is kept until it is deleted. |
tool
¶
tool(
obj: Tool | Callable[P, RT] | None = None,
/,
*,
description: str | MISSING = MISSING,
strict: bool | MISSING = MISSING,
) -> (
_tool[RT, P]
| Callable[[Callable[P, RT]], _tool[RT, P]]
)
Decorate a callable and expose its tool definition.
Supports both direct and configured decorator syntax. The resulting wrapper preserves the original callable's calling interface and metadata.
A Tool definition is generated once when the callable is
decorated and exposed through the wrapper's tool property.
The wrapper itself can be passed to GenerationRequest or
generation.create(tools=...); the cached definition is reused.
Use .tool when a Tool object is needed separately.
The decorator does not execute tool calls returned by the provider; it only wraps the callable and provides its definition.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
obj
|
Tool | Callable[P, RT] | None
|
The callable to wrap. Omit it to configure the decorator before applying it. |
None
|
description
|
str | MISSING
|
Description of the tool and its purpose. Defaults to the callable's docstring. If neither is provided, the description is omitted. |
MISSING
|
strict
|
bool | MISSING
|
Whether to request strict adherence to the generated parameter schema from the provider. |
MISSING
|
Returns:
| Type | Description |
|---|---|
_tool[RT, P] | Callable[[Callable[P, RT]], _tool[RT, P]]
|
A callable wrapper when |
_tool[RT, P] | Callable[[Callable[P, RT]], _tool[RT, P]]
|
accepting a callable otherwise. |
Examples:
Direct decoration:
>>> @tool
... def get_weather(city: str) -> dict:
... """Get current weather in a city."""
... return {"city": city}
...
>>> get_weather("Paris")
{'city': 'Paris'}
>>> request = GenerationRequest(
... model="gpt-5", input="Weather?", tools=get_weather
... )
Configured decoration:
responses
¶
Responses and stream events in types shared by all providers.
Modules:
| Name | Description |
|---|---|
base |
Base class of shared responses. |
builtin |
Built-in tool calls, citations and produced files of a generation. |
events |
Shared events of a streamed generation. |
files |
Shared metadata of a file stored by a provider. |
generation |
Shared response of text generation. |
Type Aliases:
| Name | Description |
|---|---|
GenerationEvent |
Any shared event of a generation stream, told apart by |
Classes:
| Name | Description |
|---|---|
BaseResponse |
Base class of responses in shared types. |
Citation |
A source a part of the text rests on. |
ProducedFile |
A file a built-in tool made, such as a chart the code drew. |
ServerToolCall |
A call of a built-in tool, which the provider ran itself. |
BaseResponseEvent |
Base class of streamed events in shared types. |
CompletedEvent |
The generation finished, or stopped early. |
OtherEvent |
A provider event without a shared counterpart. |
RawResponseEvent |
An event that keeps the provider's own event. |
ReasoningDeltaEvent |
A piece of a summary of the model's reasoning. |
ServerToolCallEvent |
A built-in tool call starts, or ends. |
TextDeltaEvent |
A piece of generated text. |
ToolCallDeltaEvent |
A piece of a tool call's arguments. |
ToolCallDoneEvent |
The arguments of one tool call are complete. |
ToolCallStartEvent |
The model begins a call of one of the request's tools. |
StoredFile |
A file stored by a provider, with the provider's own metadata. |
GenerationResponse |
Generated text with the provider's own response. |
TokenUsage |
Token counts reported by the provider. |
Attributes:
| Name | Type | Description |
|---|---|---|
GENERATION_TERMINAL |
Final
|
The last event of a complete generation stream, |
GENERATION_TERMINAL
module-attribute
¶
GENERATION_TERMINAL: Final = EventTypes(
GenerationEventType.COMPLETED
)
The last event of a complete generation stream, CompletedEvent.
GenerationResource checks it: an event after it raises
APIResponseError and a stream without it raises
APIStreamInterruptedError.
GenerationEvent
¶
GenerationEvent = Annotated[
TextDeltaEvent[RawT]
| ReasoningDeltaEvent[RawT]
| ToolCallStartEvent[RawT]
| ToolCallDeltaEvent[RawT]
| ToolCallDoneEvent[RawT]
| ServerToolCallEvent[RawT]
| CompletedEvent[RawT]
| OtherEvent[RawT],
Field(discriminator="type"),
]
Any shared event of a generation stream, told apart by type.
BaseResponse
¶
Bases: BaseAPIType
Base class of responses in shared types.
Its raw holds the provider's own result, which may be any class a
custom method returns, so any type is allowed here, and only here.
Citation
¶
Bases: BaseAPIType
A source a part of the text rests on.
What both providers give: where it points and which words of the
text it backs. Each provider's own fields (Anthropic's
encrypted_index) stay in GenerationResponse.raw and
native.
Attributes:
| Name | Type | Description |
|---|---|---|
kind |
CitationKind
|
What it points to. |
url |
str | None
|
The page, for |
title |
str | None
|
The title of the page or file, if known. |
file_id |
str | None
|
The file's id at the provider, for |
container_id |
str | None
|
The container's id, for |
filename |
str | None
|
The file's name, for a file. |
cited_text |
str | None
|
The words of the source that were cited, if the provider tells them. |
start_index |
int | None
|
Where the cited part of the response |
end_index |
int | None
|
Where it ends, if the provider tells it. |
Example
[c.url for c in response.citations if c.url]
ProducedFile
¶
Bases: ProviderFile
A file a built-in tool made, such as a chart the code drew.
A ProviderFile, so client.files.download(file) streams its
content. Where it lives depends on the provider: Anthropic keeps it
with the files you store (container_id is None), and it
goes wherever a ProviderFile does; OpenAI keeps it in the code
interpreter's container (container_id is set), which only
download reaches: retrieve, delete, a part's file
and CodeExecution.files refuse it with
RequestValidationError. A container and its files expire with
the container, so download what you need soon.
Attributes:
| Name | Type | Description |
|---|---|---|
filename |
str | None
|
The file's name, if the provider tells it. |
container_id |
str | None
|
The container that holds the file; |
source_call_id |
str | None
|
The |
Example
for file in response.files: ... async with client.files.download(file) as content: ... await content.save(Path(file.filename or file.id))
ServerToolCall
¶
Bases: BaseAPIType
A call of a built-in tool, which the provider ran itself.
Unlike a ToolCall, it needs no result from you: the provider
made the call and holds its result, which stays in raw. A call
that failed is a call with status FAILED, not an error of
the response. The calls are what the provider bills, so count them
to know a request's cost.
Attributes:
| Name | Type | Description |
|---|---|---|
id |
str
|
The call's id at the provider. |
kind |
ServerToolKind
|
What the call does. |
status |
ServerToolStatus
|
How far it got. |
queries |
list[str]
|
What a search looked for; empty for other calls. |
code |
str | None
|
The code a code execution ran, if the provider tells it. |
error |
str | None
|
What went wrong when |
raw |
list[dict[str, JSONValue]]
|
The provider's items of the call, in order: OpenAI's one output item, Anthropic's call block and its result block. The results (found pages, the output of code) are here. |
Example
for call in response.server_tool_calls: ... print(call.kind.value, call.status.value, call.queries)
BaseResponseEvent
¶
Bases: BaseAPIType
Base class of streamed events in shared types.
As for BaseResponse, the provider's event in raw may be of
any type, so any type is allowed here.
CompletedEvent
¶
Bases: RawResponseEvent[RawT]
The generation finished, or stopped early.
Every complete generation stream has exactly one, and it is the last
event. A failed generation raises an APIStatusError instead, in
a stream as in create(); a refusal is not a failure and ends
with this event too.
It also carries what identifies the whole response and its whole text, so a stream's caller does not have to collect them.
Attributes:
| Name | Type | Description |
|---|---|---|
type |
Literal[COMPLETED]
|
Always |
id |
str | None
|
The provider's id of the response, if it has one. |
provider |
Provider
|
Provider that generated the response, as on
|
model |
str
|
The model that generated the response. |
text |
str | None
|
The whole generated text, or |
usage |
TokenUsage | None
|
Token counts, if reported. |
incomplete_reason |
IncompleteReason | None
|
Why the provider stopped before the end, in
terms every provider has; |
raw_incomplete_reason |
str | None
|
The provider's own reason, if it gave one. |
refusal |
str | None
|
Why the model refused, if it did and the provider says why. |
reasoning |
str | None
|
A summary of the model's reasoning, when the
provider sent one (OpenAI's |
tool_calls |
list[ToolCall]
|
The calls of the request's tools, as on
|
server_tool_calls |
list[ServerToolCall]
|
The calls of built-in tools, as on
|
citations |
list[Citation]
|
The sources the text rests on, as on
|
files |
list[ProducedFile]
|
The files the built-in tools made, as on
|
native |
NativeOutput | None
|
The stream's output as its provider's next request takes
it, as |
Methods:
| Name | Description |
|---|---|
to_turn |
Turn the streamed generation into the next conversation item. |
to_turn
¶
to_turn() -> Turn | ToolCallTurn
Turn the streamed generation into the next conversation item.
The same rule as GenerationResponse.to_turn().
Returns:
| Type | Description |
|---|---|
Turn | ToolCallTurn
|
A |
Turn | ToolCallTurn
|
|
Turn | ToolCallTurn
|
it called built-in tools or paused. |
Raises:
| Type | Description |
|---|---|
ValueError
|
There are neither calls nor text, or there are calls but the generation stopped early. |
OtherEvent
¶
Bases: RawResponseEvent[RawT]
A provider event without a shared counterpart.
Attributes:
| Name | Type | Description |
|---|---|---|
type |
Literal[OTHER]
|
Always |
RawResponseEvent
¶
Bases: BaseResponseEvent
An event that keeps the provider's own event.
Attributes:
| Name | Type | Description |
|---|---|---|
raw |
RawT
|
The provider's event. |
ReasoningDeltaEvent
¶
Bases: RawResponseEvent[RawT]
A piece of a summary of the model's reasoning.
Only sent when the request asked for a summary through
provider_options; GenerationStream.text() skips it.
Attributes:
| Name | Type | Description |
|---|---|---|
type |
Literal[REASONING_DELTA]
|
Always |
text |
str
|
The new text of the summary. |
ServerToolCallEvent
¶
Bases: RawResponseEvent[RawT]
A built-in tool call starts, or ends.
The provider runs these tools, so there is nothing to run: the
events tell what it is doing. A call has one event for each phase,
STARTED and then DONE; the call of the first has what is
known at the start, the call of the second the whole call. The
deltas of a call's code and the partial images stay
OtherEvents.
Attributes:
| Name | Type | Description |
|---|---|---|
type |
Literal[SERVER_TOOL_CALL]
|
Always |
phase |
ServerToolPhase
|
Which moment of the call it tells. |
call |
ServerToolCall
|
The call as it is at that moment; its |
TextDeltaEvent
¶
Bases: RawResponseEvent[RawT]
A piece of generated text.
Attributes:
| Name | Type | Description |
|---|---|---|
type |
Literal[TEXT_DELTA]
|
Always |
text |
str
|
The new text. |
ToolCallDeltaEvent
¶
Bases: RawResponseEvent[RawT]
A piece of a tool call's arguments.
The pieces of one call, joined, are its JSON text; a piece alone is rarely valid JSON.
Attributes:
| Name | Type | Description |
|---|---|---|
type |
Literal[TOOL_CALL_DELTA]
|
Always |
call_id |
str
|
The call's id, as on its |
arguments |
str
|
The new piece of the arguments. |
ToolCallDoneEvent
¶
Bases: RawResponseEvent[RawT]
The arguments of one tool call are complete.
It comes before the end of the response, so a tool can start while
the model writes on. It says that this call is whole, not that the
generation succeeded: the stream can still fail after it (an error
event, a cut connection) and then ends with that error instead of a
CompletedEvent. Start early only a tool that is safe to run for
a generation that then fails, such as one that only reads; run a
tool with side effects after the CompletedEvent, from its
tool_calls. A response that stops early (see
CompletedEvent.incomplete_reason) cuts its last call before this
event.
Attributes:
| Name | Type | Description |
|---|---|---|
type |
Literal[TOOL_CALL_DONE]
|
Always |
call |
ToolCall
|
The whole call. |
ToolCallStartEvent
¶
Bases: RawResponseEvent[RawT]
The model begins a call of one of the request's tools.
Attributes:
| Name | Type | Description |
|---|---|---|
type |
Literal[TOOL_CALL_START]
|
Always |
call_id |
str
|
The call's id, as |
name |
str
|
The tool called. |
StoredFile
¶
Bases: BaseResponse
A file stored by a provider, with the provider's own metadata.
RawT is the provider's file type (OpenAI's OpenAIFile,
Anthropic's FileMetadata); like GenerationResponse, it has
no bound, so raw is kept as it is given. What only one provider
has, such as OpenAI's purpose or Anthropic's mime_type and
downloadable, stays in raw.
Attributes:
| Name | Type | Description |
|---|---|---|
ref |
ProviderFile
|
The file as a part or a |
filename |
str
|
The name the provider keeps. |
size |
int | None
|
Its size in bytes; |
created_at |
AwareDatetime
|
When it was stored, in UTC. |
expires_at |
AwareDatetime | None
|
When the provider deletes it, in UTC; |
raw |
RawT
|
The provider's own metadata. |
Example
stored = await client.files.upload(Path("report.pdf")) DocumentPart(file=stored.ref)
GenerationResponse
¶
Bases: BaseResponse
Generated text with the provider's own response.
RawT is the provider's response type. It has no bound: an
unparametrized GenerationResponse keeps raw as it is given,
while a bound such as BaseAPIType would turn a dict into an
empty model, because unknown fields are ignored.
A refusal is an answer, as in the providers' SDKs, not an error: it
sets incomplete_reason to IncompleteReason.REFUSAL.
Attributes:
| Name | Type | Description |
|---|---|---|
id |
str | None
|
Response identifier from the provider, if any. |
provider |
Provider
|
Provider that generated the response. |
model |
str
|
Model that generated it. |
text |
str | None
|
Generated text, if any. |
usage |
TokenUsage | None
|
Token counts, if reported. |
incomplete_reason |
IncompleteReason | None
|
Why the provider stopped before the end, in
terms every provider has; |
raw_incomplete_reason |
str | None
|
The provider's own reason, such as
OpenAI's |
refusal |
str | None
|
Why the model refused, if it did and the provider says why. |
reasoning |
str | None
|
A summary of the model's reasoning, when the
provider sent one (OpenAI's |
tool_calls |
list[ToolCall]
|
The calls of the request's tools, in the order the
model made them; empty without calls. A response with calls
waits for their results; when |
server_tool_calls |
list[ServerToolCall]
|
The calls of built-in tools the provider ran,
in the order it made them; empty without any. They need no
result from you, and |
citations |
list[Citation]
|
The sources the text rests on, from a search or a file search; empty without any. |
files |
list[ProducedFile]
|
The files the built-in tools made, such as the output of
code, in order; empty without any. Download one with
|
native |
NativeOutput | None
|
The response's output as its provider's next request
takes it; |
raw |
RawT
|
The provider's response. |
Methods:
| Name | Description |
|---|---|
to_turn |
Turn the response into the next item of the conversation. |
to_turn
¶
to_turn() -> Turn | ToolCallTurn
Turn the response into the next item of the conversation.
Returns:
| Type | Description |
|---|---|
Turn | ToolCallTurn
|
A |
Turn | ToolCallTurn
|
when the model called tools, which the tools' results follow; |
Turn | ToolCallTurn
|
else an |
Turn | ToolCallTurn
|
|
Turn | ToolCallTurn
|
calls and results the provider needs back, or paused. A |
Turn | ToolCallTurn
|
paused generation ( |
Turn | ToolCallTurn
|
|
Turn | ToolCallTurn
|
the provider go on. |
Raises:
| Type | Description |
|---|---|
ValueError
|
The response has neither calls nor text, such as
a refusal without text; or it has calls but stopped early
( |
Example
request = GenerationRequest( ... model=model, ... input=[*turns, response.to_turn(), Turn.user("More")], ... )
TokenUsage
¶
Bases: BaseAPIType
Token counts reported by the provider.
The cache and reasoning counts are parts of the input and output
counts, not added to them. None means the provider did not
report the count.
Attributes:
| Name | Type | Description |
|---|---|---|
input_tokens |
int | None
|
Tokens in the input, the cached ones included. |
output_tokens |
int | None
|
Tokens generated, reasoning included. |
total_tokens |
int | None
|
Both together. |
cache_read_tokens |
int | None
|
Input tokens read from the prompt cache. |
cache_write_tokens |
int | None
|
Input tokens written to the prompt cache. |
reasoning_tokens |
int | None
|
Output tokens spent on reasoning (thinking). |