aiollm.session¶
session
¶
Sessions that send methods to an API.
Modules:
| Name | Description |
|---|---|
aiohttp |
Session implementation on top of |
base |
Transport-agnostic session logic: retries, errors and parsing. |
body |
Request body builders, one module per transport. |
factory |
Factory that builds sessions for one provider. |
loop |
Binding of sessions to one event loop. |
protocols |
Interface of an API session. |
retry |
Retry policies: when and how long a retryable request is retried. |
Type Aliases:
| Name | Description |
|---|---|
Timeout |
Seconds; for a buffered request the limit of the whole request, for |
Classes:
| Name | Description |
|---|---|
BaseSession |
Base class for sessions; a transport only does the HTTP I/O. |
SessionFactory |
Builds sessions with a provider's API and error resolver. |
CallOptions |
The options of one call, as |
ISession |
Sends methods to an API and returns their typed results. |
ISessionFactory |
Builds sessions for one provider. |
FullJitterBackoff |
Random exponential backoff, the default of |
IBackoff |
Computes the wait before the next attempt. |
IRetryPredicate |
Decides whether a transient error is retried. |
RetryPolicy |
When and how long a retryable request is retried. |
RetryState |
A failed attempt whose error the library would retry. |
Attributes:
| Name | Type | Description |
|---|---|---|
DEFAULT_MAX_ATTEMPTS |
Final[int]
|
Attempts of a retryable request, including the first one. |
DEFAULT_MAX_RETRY_AFTER |
Final[float]
|
Longest |
DEFAULT_RETRY_DEADLINE |
Final[float]
|
Time after which no new retry starts, in seconds (15 minutes). |
DEFAULT_RETRY_POLICY |
Final[RetryPolicy]
|
The policy of a session that is given none. |
DEFAULT_MAX_ATTEMPTS
module-attribute
¶
Attempts of a retryable request, including the first one.
DEFAULT_MAX_RETRY_AFTER
module-attribute
¶
Longest Retry-After the session waits for, in seconds.
DEFAULT_RETRY_DEADLINE
module-attribute
¶
Time after which no new retry starts, in seconds (15 minutes).
DEFAULT_RETRY_POLICY
module-attribute
¶
DEFAULT_RETRY_POLICY: Final[RetryPolicy] = RetryPolicy()
The policy of a session that is given none.
Timeout
¶
Timeout = float
Seconds; for a buffered request the limit of the whole request, for a stream the longest silence between two chunks.
BaseSession
¶
BaseSession(
api: IAPI,
auth: IAuth,
error_resolver: IErrorResolver,
*,
headers: HeadersType | None = None,
json_loads: JsonLoads = loads,
json_dumps: JsonDumps = dumps,
retry_policy: RetryPolicy = DEFAULT_RETRY_POLICY,
)
Bases: ABC
Base class for sessions; a transport only does the HTTP I/O.
Retries, turning error statuses into APIStatusError and parsing
are done here, so every transport behaves the same. A subclass implements
_execute, _open_stream and close.
Attributes:
| Name | Type | Description |
|---|---|---|
api |
Builds request URLs. |
|
auth |
Produces the authentication header. |
|
error_resolver |
Turns error responses into exceptions. |
|
headers |
IHeaders
|
Default headers sent with every request. |
json_loads |
JSON decoder for response bodies. |
|
json_dumps |
JSON encoder for request bodies. |
|
retry_policy |
When and how long a retryable request is retried, unless a call gives its own policy. |
Store the session's collaborators.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
api
|
IAPI
|
Builds request URLs. |
required |
auth
|
IAuth
|
Produces the authentication header. |
required |
error_resolver
|
IErrorResolver
|
Turns error responses into exceptions. |
required |
headers
|
HeadersType | None
|
Default headers. An |
None
|
json_loads
|
JsonLoads
|
JSON decoder for response bodies. |
loads
|
json_dumps
|
JsonDumps
|
JSON encoder for request bodies. |
dumps
|
retry_policy
|
RetryPolicy
|
When and how long a retryable request is
retried: four attempts at most, none started after 15
minutes, a |
DEFAULT_RETRY_POLICY
|
Methods:
| Name | Description |
|---|---|
close |
Release the transport's resources. |
send_request |
Send a method and return its parsed response. |
stream |
Send a method and stream its response. |
send_request
async
¶
send_request(
method: IAPIMethod[T],
*,
headers: HeadersType | None = None,
timeout: Timeout | None = None,
retry_policy: RetryPolicy | None = None,
) -> T
Send a method and return its parsed response.
If method.retryable is true, transient failures are retried as
the retry policy allows. headers and timeout apply to
every attempt.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
method
|
IAPIMethod[T]
|
The method to send. |
required |
headers
|
HeadersType | None
|
Headers for this request only; they override the session's and the method's headers, but not the authentication header. |
None
|
timeout
|
Timeout | None
|
Seconds the whole request may take, instead of the
total of the session's timeout; |
None
|
retry_policy
|
RetryPolicy | None
|
Retry policy of this request only, instead of
the session's; |
None
|
Returns:
| Type | Description |
|---|---|
T
|
The parsed response, typed by the method's generic argument. |
Raises:
| Type | Description |
|---|---|
APIStatusError
|
If the API answered with an error status, or the method's parser found a failure in a successful one. |
APITransportError
|
If the request could not be completed. |
APIResponseError
|
If the response body could not be parsed. |
stream
async
¶
stream(
method: IStreamMethod[T],
*,
headers: HeadersType | None = None,
timeout: Timeout | None = None,
retry_policy: RetryPolicy | None = None,
) -> AsyncGenerator[Stream[T]]
Send a method and stream its response.
Only opening the stream is retried: once the block has started, a
failure is raised to the caller instead of resending the request.
The connection is released when the block exits, including on
break and exceptions.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
method
|
IStreamMethod[T]
|
The stream method to send. |
required |
headers
|
HeadersType | None
|
Headers for this request only; they override the session's and the method's headers, but not the authentication header. |
None
|
timeout
|
Timeout | None
|
Longest silence between two chunks, in seconds,
instead of the session's stream timeout; |
None
|
retry_policy
|
RetryPolicy | None
|
Retry policy of opening this stream only,
instead of the session's; |
None
|
Yields:
| Type | Description |
|---|---|
AsyncGenerator[Stream[T]]
|
The stream of items, typed by the method's generic argument. |
Raises:
| Type | Description |
|---|---|
APIStatusError
|
If the API answered with an error status; raised on entering the block. Also, while iterating, for an error event in the stream. |
APITransportError
|
If the connection fails on entering the block. |
APIStreamInterruptedError
|
While iterating, if the connection
fails or goes silent for too long (the transport error is
its |
APIResponseError
|
If the body cannot be decoded or validated, or an item comes after the terminal event. |
Example
async with session.stream(Download()) as stream: ... await stream.save("out.bin")
SessionFactory
¶
SessionFactory(
session_type: Callable[
Concatenate[IAPI, AuthT, IErrorResolver, P],
SessionT,
],
*,
auth_factory: Callable[[str], AuthT],
api: IAPI,
error_resolver: IErrorResolver,
)
Builds sessions with a provider's API and error resolver.
The remaining session arguments (timeout, max_body_size, ...)
are taken from session_type and checked by type checkers.
Attributes:
| Name | Type | Description |
|---|---|---|
session_type |
Session class, or a callable that builds one from the API, the auth and the error resolver. |
|
auth_factory |
Turns a token string into an |
|
api |
The provider's |
|
error_resolver |
The provider's error resolver. |
Example
factory = SessionFactory( ... AiohttpSession, ... auth_factory=BearerAuth, ... api=EXAMPLE_API, ... error_resolver=ExampleErrorResolver(), ... ) session = factory("sk-...", max_body_size=10_000_000)
Store the provider's configuration.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
session_type
|
Callable[Concatenate[IAPI, AuthT, IErrorResolver, P], SessionT]
|
Session class, or a callable that builds one from the API, the auth and the error resolver. |
required |
auth_factory
|
Callable[[str], AuthT]
|
Turns a token string into an |
required |
api
|
IAPI
|
The provider's |
required |
error_resolver
|
IErrorResolver
|
The provider's error resolver. |
required |
CallOptions
¶
Bases: TypedDict
The options of one call, as send_request and stream take.
The shared layer (use cases, client.generation) takes them as
**options and passes them on to the session unchanged, so type
checkers check them at the call. Each key is a keyword-only argument
of ISession.send_request with the same type; a test keeps the
two in step.
It is closed (PEP 728, from typing_extensions until Python
3.15): no other key can ride in **options. So a method that
takes them may have keyword parameters of its own beside them, such
as GenerationResource.create's provider_options, and still
match a protocol whose method has none.
Attributes:
| Name | Type | Description |
|---|---|---|
headers |
HeadersType | None
|
Headers for this request only; they override the session's and the method's headers. |
timeout |
Timeout | None
|
Seconds for this request instead of the session's
limit; |
retry_policy |
RetryPolicy | None
|
Retry policy of this request; |
ISession
¶
Bases: Protocol
Sends methods to an API and returns their typed results.
Methods:
| Name | Description |
|---|---|
close |
Release the connection pool. |
send_request |
Send a method and return its parsed response. |
stream |
Send a method and stream its response. |
send_request
async
¶
send_request(
method: IAPIMethod[T],
*,
headers: HeadersType | None = None,
timeout: Timeout | None = None,
retry_policy: RetryPolicy | None = None,
) -> T
Send a method and return its parsed response.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
method
|
IAPIMethod[T]
|
The method to send. |
required |
headers
|
HeadersType | None
|
Headers for this request only; they override the session's and the method's headers. |
None
|
timeout
|
Timeout | None
|
Seconds the whole request may take; |
None
|
retry_policy
|
RetryPolicy | None
|
Retry policy of this request; |
None
|
Returns:
| Type | Description |
|---|---|
T
|
The parsed response. |
stream
¶
stream(
method: IStreamMethod[T],
*,
headers: HeadersType | None = None,
timeout: Timeout | None = None,
retry_policy: RetryPolicy | None = None,
) -> AbstractAsyncContextManager[Stream[T]]
Send a method and stream its response.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
method
|
IStreamMethod[T]
|
The stream method to send. |
required |
headers
|
HeadersType | None
|
Headers for this request only; they override the session's and the method's headers. |
None
|
timeout
|
Timeout | None
|
Longest silence between two chunks, in seconds;
|
None
|
retry_policy
|
RetryPolicy | None
|
Retry policy of opening the stream; |
None
|
Returns:
| Type | Description |
|---|---|
AbstractAsyncContextManager[Stream[T]]
|
A context manager that yields the stream and closes it on |
AbstractAsyncContextManager[Stream[T]]
|
exit. |
ISessionFactory
¶
Bases: Protocol
Builds sessions for one provider.
Attributes:
| Name | Type | Description |
|---|---|---|
error_resolver |
IErrorResolver
|
The provider's error resolver, given to every session. |
error_resolver
property
¶
error_resolver: IErrorResolver
The provider's error resolver, given to every session.
FullJitterBackoff
dataclass
¶
Random exponential backoff, the default of RetryPolicy.
The wait after attempt n is drawn uniformly from zero to
multiplier * 2 ** (n - 1), capped at max_delay, so clients
that failed together do not come back together.
Attributes:
| Name | Type | Description |
|---|---|---|
multiplier |
float
|
Upper end of the first wait, in seconds. |
max_delay |
float
|
Largest wait, in seconds. |
Example
RetryPolicy(backoff=FullJitterBackoff(max_delay=5))
RetryPolicy
dataclass
¶
RetryPolicy(
*,
max_attempts: int = DEFAULT_MAX_ATTEMPTS,
deadline: float | None = DEFAULT_RETRY_DEADLINE,
max_retry_after: float = DEFAULT_MAX_RETRY_AFTER,
backoff: IBackoff = FullJitterBackoff(),
retry_if: IRetryPredicate | None = None,
)
When and how long a retryable request is retried.
A policy applies only to methods with __retryable__ = True and
only to errors whose transient flag is true; it cannot make any
other error retried. Give one to a session for all its calls, or to
a single send_request or stream call.
Attributes:
| Name | Type | Description |
|---|---|---|
max_attempts |
int
|
Attempts in total, including the first one; |
deadline |
float | None
|
Seconds since the first attempt after which no retry
starts; a wait that would end later is not started. An
attempt in progress is limited by its own timeout. |
max_retry_after |
float
|
Longest |
backoff |
IBackoff
|
Computes the wait before each retry; the session waits
at least the error's |
retry_if |
IRetryPredicate | None
|
Called for each transient error that |
Example
patient = RetryPolicy(max_attempts=6, deadline=None) await session.send_request(Ping(), retry_policy=patient)
RetryState
dataclass
¶
RetryState(
*,
attempt: int,
elapsed: float,
error: APIStatusError | APITransportError,
)
A failed attempt whose error the library would retry.
The session builds it for a policy's retry_if and backoff
only when the error is transient and its Retry-After is within
max_retry_after, so error is never anything else.
Attributes:
| Name | Type | Description |
|---|---|---|
attempt |
int
|
Number of the attempt that failed, counting from 1. |
elapsed |
float
|
Seconds since the first attempt started. |
error |
APIStatusError | APITransportError
|
The transient error the attempt ended with. |
aiohttp
¶
Session implementation on top of aiohttp.
Classes:
| Name | Description |
|---|---|
AiohttpSessionOptions |
The optional arguments of |
AiohttpSession |
Session that sends requests with |
Attributes:
| Name | Type | Description |
|---|---|---|
DEFAULT_TIMEOUT |
Final[ClientTimeout]
|
Default timeout of buffered requests. |
DEFAULT_STREAM_TIMEOUT |
Final[ClientTimeout]
|
Default timeout of streamed requests. |
REDIRECT_STATUSES |
Final[frozenset[int]]
|
Statuses that redirect the request to the |
DEFAULT_TIMEOUT
module-attribute
¶
DEFAULT_TIMEOUT: Final[ClientTimeout] = ClientTimeout(
total=600, sock_connect=30
)
Default timeout of buffered requests.
DEFAULT_STREAM_TIMEOUT
module-attribute
¶
DEFAULT_STREAM_TIMEOUT: Final[ClientTimeout] = (
ClientTimeout(
total=None, sock_connect=30, sock_read=600
)
)
Default timeout of streamed requests.
REDIRECT_STATUSES
module-attribute
¶
Statuses that redirect the request to the Location header.
AiohttpSessionOptions
¶
Bases: TypedDict
The optional arguments of AiohttpSession, for **kwargs.
A provider client takes them as **session_kwargs and passes them
on, so type checkers check them at the client's call. Each key is a
keyword-only argument of AiohttpSession.__init__ with the same
type; a test keeps the two in step.
Attributes:
| Name | Type | Description |
|---|---|---|
headers |
HeadersType | None
|
Default headers sent with every request. |
json_loads |
JsonLoads
|
JSON decoder for response bodies. |
json_dumps |
JsonDumps
|
JSON encoder for request bodies. |
max_connections |
int
|
Maximum number of simultaneous connections. |
timeout |
ClientTimeout
|
Timeout of buffered requests. |
stream_timeout |
ClientTimeout
|
Timeout of streamed requests. |
retry_policy |
RetryPolicy
|
When and how long a retryable request is retried. |
max_body_size |
int | None
|
Largest buffered body accepted, in bytes. |
trust_env |
bool
|
Whether to use proxy settings from the environment. |
follow_redirects |
bool
|
Whether to follow same-origin redirects. |
loop |
AbstractEventLoop | None
|
The event loop the session is bound to. |
AiohttpSession
¶
AiohttpSession(
api: IAPI,
auth: IAuth,
error_resolver: IErrorResolver,
*,
headers: HeadersType | None = None,
json_loads: JsonLoads = loads,
json_dumps: JsonDumps = dumps,
max_connections: int = 100,
timeout: ClientTimeout = DEFAULT_TIMEOUT,
stream_timeout: ClientTimeout = DEFAULT_STREAM_TIMEOUT,
retry_policy: RetryPolicy = DEFAULT_RETRY_POLICY,
max_body_size: int | None = None,
trust_env: bool = False,
follow_redirects: bool = False,
loop: AbstractEventLoop | None = None,
)
Bases: BaseSession
Session that sends requests with aiohttp.
The session is bound to one event loop: loop if given, otherwise
the loop of its first use. It can be created anywhere, even outside a
coroutine, but is then usable only from that loop. The underlying
ClientSession is created lazily on first use and re-created after
close(). Use it as an async context manager to open and close it
deterministically; close it before its event loop ends.
Example
async with AiohttpSession(api, BearerAuth(key), resolver) as s: ... pong = await s.send_request(Ping())
Configure the session; no connection is opened yet.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
api
|
IAPI
|
Builds request URLs. |
required |
auth
|
IAuth
|
Produces the authentication header. |
required |
error_resolver
|
IErrorResolver
|
Turns error responses into exceptions. |
required |
headers
|
HeadersType | None
|
Default headers sent with every request. |
None
|
json_loads
|
JsonLoads
|
JSON decoder for response bodies. |
loads
|
json_dumps
|
JsonDumps
|
JSON encoder for request bodies. |
dumps
|
max_connections
|
int
|
Maximum number of simultaneous connections. |
100
|
timeout
|
ClientTimeout
|
Timeout of buffered requests. aiohttp's own default caps a request at 5 minutes, which long model generations exceed; 10 minutes matches the official AI SDKs. |
DEFAULT_TIMEOUT
|
stream_timeout
|
ClientTimeout
|
Timeout of streamed requests. It has no total
limit, because a stream may last indefinitely; only the
silence between chunks ( |
DEFAULT_STREAM_TIMEOUT
|
retry_policy
|
RetryPolicy
|
When and how long a retryable request is
retried; see |
DEFAULT_RETRY_POLICY
|
max_body_size
|
int | None
|
Largest buffered response or error body
accepted, in bytes; |
None
|
trust_env
|
bool
|
Whether to use proxy settings and |
False
|
follow_redirects
|
bool
|
Whether to follow redirects within the
origin of the request. Redirects to another origin are
never followed, so no header leaks to another host;
a redirect that is not followed raises
|
False
|
loop
|
AbstractEventLoop | None
|
Event loop the session is bound to; |
None
|
Methods:
| Name | Description |
|---|---|
close |
Close the underlying |
get_client |
Return the |
Attributes:
| Name | Type | Description |
|---|---|---|
loop |
AbstractEventLoop | None
|
The event loop the session is bound to; |
loop
property
¶
loop: AbstractEventLoop | None
The event loop the session is bound to; None until first use.
Reading it never binds a loop.
close
async
¶
Close the underlying ClientSession and its connections.
Raises:
| Type | Description |
|---|---|
RuntimeError
|
If called from another event loop. |
get_client
async
¶
Return the ClientSession, creating it if needed.
Returns:
| Type | Description |
|---|---|
ClientSession
|
An open |
ClientSession
|
built for every request, so later changes take effect. |
Raises:
| Type | Description |
|---|---|
RuntimeError
|
If called from another event loop. |