aiollm.resources¶
resources
¶
Resources and use cases that adapt provider methods to shared types.
Modules:
| Name | Description |
|---|---|
base |
Base classes of resources and of everything that holds a session. |
files |
The shared resource of files stored with a provider. |
generation |
The shared resource of text generation. |
protocols |
Interfaces of resources and use cases. |
stream |
The stream of a generation, with helpers for its text and its end. |
tool_runner |
Utilities and runners for executing tool callbacks on responses. |
usecases |
Base classes of use cases and the orchestrators built on adapters. |
Classes:
| Name | Description |
|---|---|
BaseResource |
Base class for a group of use cases of one API area. |
BaseSessionAccess |
Holds the session that sends the requests. |
FilesResource |
Stores files with any provider through that provider's adapter. |
GenerationResource |
Generates text with any provider through that provider's adapter. |
ICompositeUseCase |
A use case that can both respond at once and stream. |
IFilesAdapter |
The adapter a |
IFilesClient |
A client that stores files; every provider's client is one. |
IFilesResource |
Stores files with a provider; the same interface for every one. |
IGenerationAdapter |
The adapter a |
IGenerationClient |
A client that generates text; every provider's client is one. |
IGenerationResource |
Generates text; the same interface for every provider. |
IResource |
A group of use cases of one API area, such as responses. |
ISessionAccess |
Gives access to the session that sends the requests. |
IStreamUseCase |
Turns a shared request into a stream of shared events. |
IUseCase |
Turns a shared request into a shared response. |
GenerationStream |
The events of a generation, with helpers for the common cases. |
AdapterCompositeUseCase |
A use case that orchestrates one adapter both ways. |
AdapterStreamUseCase |
A stream use case that only orchestrates an adapter and a session. |
AdapterUseCase |
A use case that only orchestrates: adapter, session, adapter. |
BaseCompositeUseCase |
A use case that can both respond at once and stream. |
BaseStreamUseCase |
Turns a shared request into a stream of shared events. |
BaseUseCase |
Turns a shared request into a shared response. |
BaseResource
¶
Bases: BaseSessionAccess[SessionT]
Base class for a group of use cases of one API area.
Example
class Responses(BaseResource): ... def init(self, session: ISession) -> None: ... super().init(session) ... self.generate = Generate(session)
BaseSessionAccess
¶
Holds the session that sends the requests.
SessionT keeps the session's own type, as in BaseClient, so a
subclass built on AiohttpSession reaches its settings without a
cast.
Keep the session.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
session
|
SessionT
|
The session that sends the requests. |
required |
Attributes:
| Name | Type | Description |
|---|---|---|
session |
SessionT
|
The session that sends the requests. |
FilesResource
¶
FilesResource(
session: SessionT,
adapter: IFilesAdapter[RawFileT, RawPageT, OptionsT],
)
Bases: BaseResource[SessionT]
Stores files with any provider through that provider's adapter.
The class is the same for every provider: a provider's client builds
it with its own adapter. A stored file is named by a
ProviderFile, which StoredFile.ref holds and a part takes as
file=; one of another provider raises RequestValidationError
before anything is sent.
The library deletes nothing by itself: a file is kept until it is
deleted or expires (expires_in_seconds). Every method takes the
options of the call (headers, timeout, retry_policy) as
send_request does; the provider's own upload fields, such as
OpenAI's purpose, go in provider_options of upload.
Example
stored = await client.files.upload( ... Path("report.pdf"), expires_in_seconds=3600 ... ) response = await client.generation.create( ... model="gpt-5", ... input=[Turn.user([DocumentPart(file=stored.ref)])], ... ) await client.files.delete(stored.ref)
Keep the session and the adapter.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
session
|
SessionT
|
The session that sends the requests. |
required |
adapter
|
IFilesAdapter[RawFileT, RawPageT, OptionsT]
|
Builds the provider's methods and converts what they return. |
required |
Methods:
| Name | Description |
|---|---|
upload |
Store a file with the provider. |
retrieve |
Read a stored file's metadata. |
list |
Go through every stored file, a page at a time. |
delete |
Delete a stored file. |
download |
Stream a stored file's content; use with |
upload
async
¶
upload(
file: InputFile | bytes | PathLike[str] | BinaryStream,
/,
*,
expires_in_seconds: int | MISSING = MISSING,
provider_options: OptionsT | MISSING = MISSING,
**options: Unpack[CallOptions],
) -> StoredFile[RawFileT]
Store a file with the provider.
Never retried: POST /files is not idempotent, and a retry
after a lost answer would store the file twice. A timeout or a
broken connection can therefore leave a file stored whose id was
never received; expires_in_seconds bounds how long it stays.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
file
|
InputFile | bytes | PathLike[str] | BinaryStream
|
The content: an |
required |
expires_in_seconds
|
int | MISSING
|
Seconds after which the provider deletes the file; left out, it is kept until it is deleted. |
MISSING
|
provider_options
|
OptionsT | MISSING
|
The provider's own upload fields, such as
|
MISSING
|
**options
|
Unpack[CallOptions]
|
Options of the call ( |
{}
|
Returns:
| Type | Description |
|---|---|
StoredFile[RawFileT]
|
The stored file. |
Raises:
| Type | Description |
|---|---|
ValidationError
|
|
RequestValidationError
|
The provider cannot take the upload,
such as an expiry outside its range, or a key of
|
retrieve
async
¶
retrieve(
file: ProviderFile, /, **options: Unpack[CallOptions]
) -> StoredFile[RawFileT]
Read a stored file's metadata.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
file
|
ProviderFile
|
The file. |
required |
**options
|
Unpack[CallOptions]
|
Options of the call, passed on to the session. |
{}
|
Returns:
| Type | Description |
|---|---|
StoredFile[RawFileT]
|
The stored file. |
Raises:
| Type | Description |
|---|---|
RequestValidationError
|
|
NotFoundError
|
The provider has no such file. |
list
async
¶
list(
**options: Unpack[CallOptions],
) -> AsyncGenerator[StoredFile[RawFileT]]
Go through every stored file, a page at a time.
Each page is one request; the next is sent when the last file of a page has been taken, so stopping early sends no more.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
**options
|
Unpack[CallOptions]
|
Options of each page's call, passed on to the session. |
{}
|
Yields:
| Type | Description |
|---|---|
AsyncGenerator[StoredFile[RawFileT]]
|
The files, in the provider's order (OpenAI and Anthropic: |
AsyncGenerator[StoredFile[RawFileT]]
|
the newest first). |
delete
async
¶
delete(
file: ProviderFile, /, **options: Unpack[CallOptions]
) -> None
Delete a stored file.
A deletion retried after its answer was lost raises
NotFoundError, since the file is already gone.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
file
|
ProviderFile
|
The file. |
required |
**options
|
Unpack[CallOptions]
|
Options of the call, passed on to the session. |
{}
|
Raises:
| Type | Description |
|---|---|
RequestValidationError
|
|
NotFoundError
|
The provider has no such file. |
download
¶
download(
file: ProviderFile, /, **options: Unpack[CallOptions]
) -> AbstractAsyncContextManager[Stream[bytes]]
Stream a stored file's content; use with async with.
Each provider lets only some files be downloaded: OpenAI refuses
a user_data file, Anthropic one it did not make itself (code
execution, skills); both answer BadRequestError. A file a
built-in tool made (GenerationResponse.files) is downloaded
from wherever it lives, its container included.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
file
|
ProviderFile
|
The file, such as a |
required |
**options
|
Unpack[CallOptions]
|
Options of opening the stream, passed on to the session. |
{}
|
Returns:
| Type | Description |
|---|---|
AbstractAsyncContextManager[Stream[bytes]]
|
A context manager that yields the content in chunks |
AbstractAsyncContextManager[Stream[bytes]]
|
( |
AbstractAsyncContextManager[Stream[bytes]]
|
memory) and closes it on exit. |
Raises:
| Type | Description |
|---|---|
RequestValidationError
|
|
GenerationResource
¶
GenerationResource(
session: SessionT,
adapter: IGenerationAdapter[RawT, RawItemT, OptionsT],
)
Bases: BaseResource[SessionT]
Generates text with any provider through that provider's adapter.
The class is the same for every provider: a provider's client builds
it with its own adapter, so calling code reads the same for all of
them and only the type of raw differs. RawT is the provider's
response, RawItemT one item of its stream, OptionsT the
TypedDict of its own options (NoProviderOptions for none).
Both methods take the request's fields as keyword arguments, as the
providers' SDKs do, or a ready GenerationRequest; not both. The
options of the call (headers, timeout, retry_policy) are
passed on to the session, as send_request takes them. The
provider's own fields of a call, which the shared request does not
hold, go in provider_options, a TypedDict of the provider;
code that works with any provider (IGenerationResource) has no
such parameter.
Example
generation = GenerationResource(session, ResponsesAdapter()) response = await generation.create(model="gpt-5", input="Hi") async with generation.stream(model="gpt-5", input="Hi") as s: ... async for text in s.text(): ... ... ... completed = await s.final()
Keep the session and build the use case on the adapter.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
session
|
SessionT
|
The session that sends the requests. |
required |
adapter
|
IGenerationAdapter[RawT, RawItemT, OptionsT]
|
Converts the shared request, the provider's response and its stream items. |
required |
Methods:
| Name | Description |
|---|---|
create |
Generate the whole response at once. |
stream |
Open a stream of generation events; use with |
create
async
¶
create(
*,
model: str,
input: str | Sequence[InputItem],
instructions: str | MISSING = MISSING,
max_output_tokens: int | MISSING = MISSING,
end_user_id: str | MISSING = MISSING,
effort: Effort | MISSING = MISSING,
output_schema: JSONSchema | MISSING = MISSING,
tools: ToolType
| Sequence[ToolType]
| MISSING = MISSING,
tool_choice: ToolChoice
| ForcedTool
| MISSING = MISSING,
parallel_tool_calls: bool | MISSING = MISSING,
provider_options: OptionsT | MISSING = MISSING,
**options: Unpack[CallOptions],
) -> GenerationResponse[RawT]
create(
request: GenerationRequest,
/,
*,
provider_options: OptionsT | MISSING = MISSING,
**options: Unpack[CallOptions],
) -> GenerationResponse[RawT]
create(
request: GenerationRequest | None = None,
/,
*,
headers: HeadersType | None = None,
timeout: Timeout | None = None,
retry_policy: RetryPolicy | None = None,
provider_options: OptionsT | MISSING = MISSING,
**fields: Any,
) -> GenerationResponse[RawT]
Generate the whole response at once.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
request
|
GenerationRequest | None
|
What to generate, as a ready request. |
None
|
headers
|
HeadersType | None
|
Headers for this request only. |
None
|
timeout
|
Timeout | None
|
Seconds for this request instead of the session's
limit; |
None
|
retry_policy
|
RetryPolicy | None
|
Retry policy of this request; |
None
|
provider_options
|
OptionsT | MISSING
|
The provider's own fields of this request,
such as |
MISSING
|
**fields
|
Any
|
What to generate, as the fields of a
|
{}
|
Returns:
| Type | Description |
|---|---|
GenerationResponse[RawT]
|
The generated text with the provider's own response. |
Raises:
| Type | Description |
|---|---|
TypeError
|
Both a request and fields are given. |
ValidationError
|
The fields do not make a valid request. |
RequestValidationError
|
A key of |
stream
¶
stream(
*,
model: str,
input: str | Sequence[InputItem],
instructions: str | MISSING = MISSING,
max_output_tokens: int | MISSING = MISSING,
end_user_id: str | MISSING = MISSING,
effort: Effort | MISSING = MISSING,
output_schema: JSONSchema | MISSING = MISSING,
tools: ToolType
| Sequence[ToolType]
| MISSING = MISSING,
tool_choice: ToolChoice
| ForcedTool
| MISSING = MISSING,
parallel_tool_calls: bool | MISSING = MISSING,
provider_options: OptionsT | MISSING = MISSING,
**options: Unpack[CallOptions],
) -> AbstractAsyncContextManager[
GenerationStream[RawItemT]
]
stream(
request: GenerationRequest,
/,
*,
provider_options: OptionsT | MISSING = MISSING,
**options: Unpack[CallOptions],
) -> AbstractAsyncContextManager[
GenerationStream[RawItemT]
]
stream(
request: GenerationRequest | None = None,
/,
*,
headers: HeadersType | None = None,
timeout: Timeout | None = None,
retry_policy: RetryPolicy | None = None,
provider_options: OptionsT | MISSING = MISSING,
**fields: Any,
) -> AbstractAsyncContextManager[
GenerationStream[RawItemT]
]
Open a stream of generation events; use with async with.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
request
|
GenerationRequest | None
|
What to generate, as a ready request. |
None
|
headers
|
HeadersType | None
|
Headers for this request only. |
None
|
timeout
|
Timeout | None
|
Seconds for this request instead of the session's
limit; |
None
|
retry_policy
|
RetryPolicy | None
|
Retry policy of this request; |
None
|
provider_options
|
OptionsT | MISSING
|
The provider's own fields of this request,
such as |
MISSING
|
**fields
|
Any
|
What to generate, as the fields of a
|
{}
|
Returns:
| Type | Description |
|---|---|
AbstractAsyncContextManager[GenerationStream[RawItemT]]
|
A context manager that yields a |
AbstractAsyncContextManager[GenerationStream[RawItemT]]
|
events, with |
AbstractAsyncContextManager[GenerationStream[RawItemT]]
|
the provider's stream on exit. A complete stream ends with |
AbstractAsyncContextManager[GenerationStream[RawItemT]]
|
exactly one |
AbstractAsyncContextManager[GenerationStream[RawItemT]]
|
raises |
AbstractAsyncContextManager[GenerationStream[RawItemT]]
|
generation an |
Raises:
| Type | Description |
|---|---|
TypeError
|
Both a request and fields are given. |
ValidationError
|
The fields do not make a valid request. |
RequestValidationError
|
A key of |
ICompositeUseCase
¶
Bases: IUseCase[ReqT, RespT], IStreamUseCase[ReqT, EventT], Protocol
A use case that can both respond at once and stream.
IFilesAdapter
¶
Bases: Protocol
The adapter a FilesResource runs on.
It builds the provider's Files methods and converts what they
return; like every adapter, it sends nothing, holds no session and
no mutable state that outlives one method call. RawFileT is the
provider's file type, RawPageT one page of its list, and
OptionsT the TypedDict of its own upload options.
Methods:
| Name | Description |
|---|---|
with_options |
Return a copy that adds the options to every upload it builds. |
to_upload_method |
Build the method that stores a file. |
to_retrieve_method |
Build the method that reads a file's metadata. |
to_list_method |
Build the method that reads one page of the stored files. |
to_page |
Take the files of a page and where the next one starts. |
to_delete_method |
Build the method that deletes a file. |
to_download_method |
Build the method that streams a file's content. |
to_stored_file |
Convert the provider's file into the shared metadata. |
Attributes:
| Name | Type | Description |
|---|---|---|
provider |
Provider
|
The provider whose methods the adapter builds. |
with_options
¶
with_options(options: OptionsT) -> Self
Return a copy that adds the options to every upload it builds.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
options
|
OptionsT
|
The provider's own upload options of one call. |
required |
Returns:
| Type | Description |
|---|---|
Self
|
A copy of the adapter; this one is left as it is. |
Raises:
| Type | Description |
|---|---|
RequestValidationError
|
A key is not one of the provider's options. |
to_upload_method
¶
to_upload_method(
request: FileUpload,
) -> IAPIMethod[RawFileT]
Build the method that stores a file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
request
|
FileUpload
|
The shared upload. |
required |
Returns:
| Type | Description |
|---|---|
IAPIMethod[RawFileT]
|
The provider's upload method, ready to send. |
Raises:
| Type | Description |
|---|---|
RequestValidationError
|
The provider cannot take the upload, such as an expiry outside its range. |
to_retrieve_method
¶
to_retrieve_method(file_id: str) -> IAPIMethod[RawFileT]
Build the method that reads a file's metadata.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
file_id
|
str
|
The provider's id of the file. |
required |
Returns:
| Type | Description |
|---|---|
IAPIMethod[RawFileT]
|
The provider's method, ready to send. |
to_list_method
¶
to_list_method(cursor: str | None) -> IAPIMethod[RawPageT]
Build the method that reads one page of the stored files.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
cursor
|
str | None
|
Where the page starts, as |
required |
Returns:
| Type | Description |
|---|---|
IAPIMethod[RawPageT]
|
The provider's method, ready to send. |
to_page
¶
to_delete_method
¶
to_delete_method(file_id: str) -> IAPIMethod[Any]
Build the method that deletes a file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
file_id
|
str
|
The provider's id of the file. |
required |
Returns:
| Type | Description |
|---|---|
IAPIMethod[Any]
|
The provider's method, ready to send. |
to_download_method
¶
to_download_method(
file: ProviderFile,
) -> IStreamMethod[bytes]
Build the method that streams a file's content.
The whole file, not its id: a ProducedFile of a container
(container_id) is downloaded from that container where the
provider has one, and refused where it has none.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
file
|
ProviderFile
|
The file, already known to be the provider's. |
required |
Returns:
| Type | Description |
|---|---|
IStreamMethod[bytes]
|
The provider's stream method, ready to open. |
Raises:
| Type | Description |
|---|---|
RequestValidationError
|
The provider cannot download the file, such as a file of a container where it has none. |
to_stored_file
¶
to_stored_file(raw: RawFileT) -> StoredFile[RawFileT]
Convert the provider's file into the shared metadata.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
raw
|
RawFileT
|
The provider's file. |
required |
Returns:
| Type | Description |
|---|---|
StoredFile[RawFileT]
|
The shared metadata, |
IFilesClient
¶
A client that stores files; every provider's client is one.
RawT is the provider's file type, Any by default for code
that works with any provider.
Example
async def store(client: IFilesClient, path: Path) -> ProviderFile: ... stored = await client.files.upload(path) ... return stored.ref
Attributes:
| Name | Type | Description |
|---|---|---|
files |
IFilesResource[RawT]
|
Files stored with the provider. |
IFilesResource
¶
Stores files with a provider; the same interface for every one.
A stored file is named by its ProviderFile (StoredFile.ref),
which a part takes as file=; a ProviderFile of another
provider raises RequestValidationError before anything is sent.
Every method takes the options of the call (headers, timeout,
retry_policy) as send_request does.
Methods:
| Name | Description |
|---|---|
upload |
Store a file with the provider. |
retrieve |
Read a stored file's metadata. |
list |
Go through every stored file, a page at a time. |
delete |
Delete a stored file. |
download |
Stream a stored file's content; use with |
upload
async
¶
upload(
file: InputFile | bytes | PathLike[str] | BinaryStream,
/,
*,
expires_in_seconds: int | MISSING = MISSING,
**options: Unpack[CallOptions],
) -> StoredFile[RawT]
Store a file with the provider.
Never retried, since a retry after a lost answer would store the file twice.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
file
|
InputFile | bytes | PathLike[str] | BinaryStream
|
The content, with the name the provider keeps. |
required |
expires_in_seconds
|
int | MISSING
|
Seconds after which the provider deletes the file; left out, it is kept until it is deleted. |
MISSING
|
**options
|
Unpack[CallOptions]
|
Options of the call, passed on to the session. |
{}
|
Returns:
| Type | Description |
|---|---|
StoredFile[RawT]
|
The stored file. |
retrieve
async
¶
retrieve(
file: ProviderFile, /, **options: Unpack[CallOptions]
) -> StoredFile[RawT]
Read a stored file's metadata.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
file
|
ProviderFile
|
The file. |
required |
**options
|
Unpack[CallOptions]
|
Options of the call, passed on to the session. |
{}
|
Returns:
| Type | Description |
|---|---|
StoredFile[RawT]
|
The stored file. |
list
¶
list(
**options: Unpack[CallOptions],
) -> AsyncIterator[StoredFile[RawT]]
Go through every stored file, a page at a time.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
**options
|
Unpack[CallOptions]
|
Options of each page's call, passed on to the session. |
{}
|
Returns:
| Type | Description |
|---|---|
AsyncIterator[StoredFile[RawT]]
|
The files, in the provider's order; the next page is asked |
AsyncIterator[StoredFile[RawT]]
|
for when the last file of a page has been taken. |
delete
async
¶
delete(
file: ProviderFile, /, **options: Unpack[CallOptions]
) -> None
Delete a stored file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
file
|
ProviderFile
|
The file. |
required |
**options
|
Unpack[CallOptions]
|
Options of the call, passed on to the session. |
{}
|
download
¶
download(
file: ProviderFile, /, **options: Unpack[CallOptions]
) -> AbstractAsyncContextManager[Stream[bytes]]
Stream a stored file's content; use with async with.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
file
|
ProviderFile
|
The file. |
required |
**options
|
Unpack[CallOptions]
|
Options of opening the stream, passed on to the session. |
{}
|
Returns:
| Type | Description |
|---|---|
AbstractAsyncContextManager[Stream[bytes]]
|
A context manager that yields the content in chunks |
AbstractAsyncContextManager[Stream[bytes]]
|
( |
IGenerationAdapter
¶
Bases: ICompositeAdapter[GenerationRequest, RawT, 'GenerationResponse[RawT]', RawItemT, GenerationEvent[RawItemT]], Protocol
The adapter a GenerationResource runs on.
Besides converting, it takes the provider's own options of one call
(provider_options): fields that only its provider has, such as
OpenAI's store. OptionsT is a TypedDict of them.
Concurrent calls share the adapter and its shallow copies, so an
adapter, its conversions included, holds no mutable state that
outlives one method call; what belongs to one call (the options)
lives only on the copy with_options makes.
Methods:
| Name | Description |
|---|---|
with_options |
Return a copy that adds the options to every method it builds. |
with_options
¶
with_options(options: OptionsT) -> Self
Return a copy that adds the options to every method it builds.
The adapter itself is left as it is, so concurrent calls with
other options do not meet. The copy may be shallow: it shares
the conversions, which hold no mutable state. Every key is
checked, also by an adapter that takes no options
(checked_options(options, NoProviderOptions, provider)), so
a key is never dropped silently.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
options
|
OptionsT
|
The provider's options of one call. |
required |
Returns:
| Type | Description |
|---|---|
Self
|
A copy of the adapter with the options. |
Raises:
| Type | Description |
|---|---|
RequestValidationError
|
A key is not one of the provider's options; the shared fields are set in the request. |
IGenerationClient
¶
A client that generates text; every provider's client is one.
RawT is the provider's response and RawItemT one item of its
stream; both default to Any for code that works with any
provider. It lives with the resources, not with the clients, since
it names a resource protocol and the client layer is below this one.
Example
async def ask(client: IGenerationClient, question: str) -> str: ... response = await client.generation.create( ... model=MODEL, input=question ... ) ... return response.text or ""
Attributes:
| Name | Type | Description |
|---|---|---|
generation |
IGenerationResource[RawT, RawItemT]
|
Shared text generation over the provider's API. |
generation
property
¶
generation: IGenerationResource[RawT, RawItemT]
Shared text generation over the provider's API.
IGenerationResource
¶
Generates text; the same interface for every provider.
Both methods take the request's fields as keyword arguments, as the
providers' SDKs do, or a ready GenerationRequest, and the options
of the call (headers, timeout, retry_policy) as
send_request does; no request field is named like one of them.
Methods:
| Name | Description |
|---|---|
create |
Generate the whole response at once. |
stream |
Open a stream of generation events; use with |
create
async
¶
create(
*,
model: str,
input: str | Sequence[InputItem],
instructions: str | MISSING = MISSING,
max_output_tokens: int | MISSING = MISSING,
end_user_id: str | MISSING = MISSING,
effort: Effort | MISSING = MISSING,
output_schema: JSONSchema | MISSING = MISSING,
tools: ToolType
| Sequence[ToolType]
| MISSING = MISSING,
tool_choice: ToolChoice
| ForcedTool
| MISSING = MISSING,
parallel_tool_calls: bool | MISSING = MISSING,
**options: Unpack[CallOptions],
) -> GenerationResponse[RawT]
create(
request: GenerationRequest,
/,
**options: Unpack[CallOptions],
) -> GenerationResponse[RawT]
create(
request: GenerationRequest | None = None,
/,
*,
headers: HeadersType | None = None,
timeout: Timeout | None = None,
retry_policy: RetryPolicy | None = None,
**fields: Any,
) -> GenerationResponse[RawT]
Generate the whole response at once.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
request
|
GenerationRequest | None
|
What to generate, as a ready request. |
None
|
headers
|
HeadersType | None
|
Headers for this request only. |
None
|
timeout
|
Timeout | None
|
Seconds for this request instead of the session's
limit; |
None
|
retry_policy
|
RetryPolicy | None
|
Retry policy of this request; |
None
|
**fields
|
Any
|
What to generate, as the fields of a
|
{}
|
Returns:
| Type | Description |
|---|---|
GenerationResponse[RawT]
|
The generated text with the provider's own response. |
stream
¶
stream(
*,
model: str,
input: str | Sequence[InputItem],
instructions: str | MISSING = MISSING,
max_output_tokens: int | MISSING = MISSING,
end_user_id: str | MISSING = MISSING,
effort: Effort | MISSING = MISSING,
output_schema: JSONSchema | MISSING = MISSING,
tools: ToolType
| Sequence[ToolType]
| MISSING = MISSING,
tool_choice: ToolChoice
| ForcedTool
| MISSING = MISSING,
parallel_tool_calls: bool | MISSING = MISSING,
**options: Unpack[CallOptions],
) -> AbstractAsyncContextManager[
GenerationStream[RawItemT]
]
stream(
request: GenerationRequest,
/,
**options: Unpack[CallOptions],
) -> AbstractAsyncContextManager[
GenerationStream[RawItemT]
]
stream(
request: GenerationRequest | None = None,
/,
*,
headers: HeadersType | None = None,
timeout: Timeout | None = None,
retry_policy: RetryPolicy | None = None,
**fields: Any,
) -> AbstractAsyncContextManager[
GenerationStream[RawItemT]
]
Open a stream of generation events; use with async with.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
request
|
GenerationRequest | None
|
What to generate, as a ready request. |
None
|
headers
|
HeadersType | None
|
Headers for this request only. |
None
|
timeout
|
Timeout | None
|
Seconds for this request instead of the session's
limit; |
None
|
retry_policy
|
RetryPolicy | None
|
Retry policy of this request; |
None
|
**fields
|
Any
|
What to generate, as the fields of a
|
{}
|
Returns:
| Type | Description |
|---|---|
AbstractAsyncContextManager[GenerationStream[RawItemT]]
|
A context manager that yields the events, with the |
AbstractAsyncContextManager[GenerationStream[RawItemT]]
|
|
AbstractAsyncContextManager[GenerationStream[RawItemT]]
|
exit. |
IResource
¶
ISessionAccess
¶
IStreamUseCase
¶
Bases: ISessionAccess, Protocol
Turns a shared request into a stream of shared events.
Methods:
| Name | Description |
|---|---|
stream |
Open the stream; use the result with |
stream
¶
stream(
request: ReqT, **options: Unpack[CallOptions]
) -> AbstractAsyncContextManager[Stream[EventT]]
Open the stream; use the result with async with.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
request
|
ReqT
|
The request in shared types. |
required |
**options
|
Unpack[CallOptions]
|
Options of the call ( |
{}
|
Returns:
| Type | Description |
|---|---|
AbstractAsyncContextManager[Stream[EventT]]
|
A context manager that yields the events and closes the |
AbstractAsyncContextManager[Stream[EventT]]
|
stream on exit. |
IUseCase
¶
GenerationStream
¶
GenerationStream(
items: AsyncIterator[GenerationEvent[RawItemT]],
)
Bases: Stream[GenerationEvent[RawItemT]]
The events of a generation, with helpers for the common cases.
It is a Stream of shared events, so it can be iterated as it is;
text() gives only the text, and final() reads to the end and
gives the CompletedEvent. Whatever consumed the events, the
CompletedEvent is kept, so final() after text() (or after
a loop over the events) returns it without reading again. An error
raised while reading is kept too: reading again, text() or
final() raise the same error instead of ending as if the stream
were cut.
GenerationResource.stream builds it on the converted events
through transform_stream(..., stream_type=GenerationStream).
Example
async with client.generation.stream(request) as events: ... async for text in events.text(): ... print(text, end="") ... completed = await events.final()
Wrap the shared events.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
items
|
AsyncIterator[GenerationEvent[RawItemT]]
|
The converted events. |
required |
Methods:
| Name | Description |
|---|---|
text |
Give the generated text as it arrives. |
final |
Read the stream to its end and give its |
text
async
¶
text() -> AsyncGenerator[str]
Give the generated text as it arrives.
Other events are read and skipped; the CompletedEvent is kept
for final().
Yields:
| Type | Description |
|---|---|
AsyncGenerator[str]
|
The text of each |
Raises:
| Type | Description |
|---|---|
RuntimeError
|
If the stream is closed. |
final
async
¶
final() -> CompletedEvent[RawItemT]
Read the stream to its end and give its CompletedEvent.
Returns:
| Type | Description |
|---|---|
CompletedEvent[RawItemT]
|
The last event of a complete generation, with its id, model, |
CompletedEvent[RawItemT]
|
whole text, token counts and stop. |
Raises:
| Type | Description |
|---|---|
RuntimeError
|
If the stream is closed. |
APIStreamInterruptedError
|
The stream ended without a
|
Exception
|
The error an earlier read raised, such as the provider's error event, again. |
AdapterCompositeUseCase
¶
AdapterCompositeUseCase(
session: SessionT,
adapter: ICompositeAdapter[
ReqT, RawT, RespT, RawItemT, EventT
],
*,
terminal: IEventMatcher | None = None,
)
Bases: BaseCompositeUseCase[ReqT, RespT, EventT, SessionT]
A use case that orchestrates one adapter both ways.
It repeats the constructor and the delegation of AdapterUseCase
and AdapterStreamUseCase on purpose: the logic is already shared
in _respond and open_adapted_stream, and inheriting from both
would tie two generic signatures with different type parameters
together, which type checkers and readers follow worse than a few
repeated lines.
Attributes:
| Name | Type | Description |
|---|---|---|
adapter |
Converts the request, the response and the stream items. |
|
terminal |
Recognizes the final event of the shared stream;
|
Keep the session and the adapter.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
session
|
SessionT
|
The session that sends the requests. |
required |
adapter
|
ICompositeAdapter[ReqT, RawT, RespT, RawItemT, EventT]
|
Converts the request, the response and the stream items. |
required |
terminal
|
IEventMatcher | None
|
Recognizes the final event of the shared stream,
checked as in |
None
|
Methods:
| Name | Description |
|---|---|
stream |
Open the stream; use the result with |
stream
¶
stream(
request: ReqT, **options: Unpack[CallOptions]
) -> AbstractAsyncContextManager[Stream[EventT]]
Open the stream; use the result with async with.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
request
|
ReqT
|
The request in shared types. |
required |
**options
|
Unpack[CallOptions]
|
Options of the call ( |
{}
|
Returns:
| Type | Description |
|---|---|
AbstractAsyncContextManager[Stream[EventT]]
|
A context manager that yields the events and closes the |
AbstractAsyncContextManager[Stream[EventT]]
|
converted and the provider's stream on exit. |
AdapterStreamUseCase
¶
AdapterStreamUseCase(
session: SessionT,
adapter: IStreamAdapter[ReqT, RawItemT, EventT],
*,
terminal: IEventMatcher | None = None,
)
Bases: BaseStreamUseCase[ReqT, EventT, SessionT]
A stream use case that only orchestrates an adapter and a session.
Attributes:
| Name | Type | Description |
|---|---|---|
adapter |
Converts the request and the stream items. |
|
terminal |
Recognizes the final event of the shared stream, such
as |
Keep the session and the adapter.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
session
|
SessionT
|
The session that sends the requests. |
required |
adapter
|
IStreamAdapter[ReqT, RawItemT, EventT]
|
Converts the request and the stream items. |
required |
terminal
|
IEventMatcher | None
|
Recognizes the final event of the shared stream.
When set, exactly one event must match it and be the
last: an event after it raises |
None
|
Methods:
| Name | Description |
|---|---|
stream |
Open the stream; use the result with |
stream
¶
stream(
request: ReqT, **options: Unpack[CallOptions]
) -> AbstractAsyncContextManager[Stream[EventT]]
Open the stream; use the result with async with.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
request
|
ReqT
|
The request in shared types. |
required |
**options
|
Unpack[CallOptions]
|
Options of the call ( |
{}
|
Returns:
| Type | Description |
|---|---|
AbstractAsyncContextManager[Stream[EventT]]
|
A context manager that yields the events and closes the |
AbstractAsyncContextManager[Stream[EventT]]
|
converted and the provider's stream on exit. |
AdapterUseCase
¶
AdapterUseCase(
session: SessionT, adapter: IAdapter[ReqT, RawT, RespT]
)
Bases: BaseUseCase[ReqT, RespT, SessionT]
A use case that only orchestrates: adapter, session, adapter.
The request becomes the provider's method, the session sends it and the result becomes the shared response; all conversion is in the adapter.
Attributes:
| Name | Type | Description |
|---|---|---|
adapter |
Converts the request and the response. |
Example
generate = AdapterUseCase(session, ChatAdapter()) response = await generate(GenerationRequest(...))
Keep the session and the adapter.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
session
|
SessionT
|
The session that sends the requests. |
required |
adapter
|
IAdapter[ReqT, RawT, RespT]
|
Converts the request and the response. |
required |
BaseCompositeUseCase
¶
Bases: BaseUseCase[ReqT, RespT, SessionT], BaseStreamUseCase[ReqT, EventT, SessionT]
A use case that can both respond at once and stream.
BaseStreamUseCase
¶
Bases: BaseSessionAccess[SessionT], ABC
Turns a shared request into a stream of shared events.
A subclass usually returns
map_stream(self.session.stream(method), convert), which closes
the converted and the provider's stream in the right order.
Methods:
| Name | Description |
|---|---|
stream |
Open the stream; use the result with |
stream
abstractmethod
¶
stream(
request: ReqT, **options: Unpack[CallOptions]
) -> AbstractAsyncContextManager[Stream[EventT]]
Open the stream; use the result with async with.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
request
|
ReqT
|
The request in shared types. |
required |
**options
|
Unpack[CallOptions]
|
Options of the call ( |
{}
|
Returns:
| Type | Description |
|---|---|
AbstractAsyncContextManager[Stream[EventT]]
|
A context manager that yields the events and closes the |
AbstractAsyncContextManager[Stream[EventT]]
|
stream on exit. |
BaseUseCase
¶
Bases: BaseSessionAccess[SessionT], ABC
Turns a shared request into a shared response.
A subclass builds the provider's method from the request, sends it
with self.session.send_request and converts the result.