Skip to content

aiollm.session

session

Sessions that send methods to an API.

Modules:

Name Description
aiohttp

Session implementation on top of aiohttp.

base

Transport-agnostic session logic: retries, errors and parsing.

body

Request body builders, one module per transport.

factory

Factory that builds sessions for one provider.

loop

Binding of sessions to one event loop.

protocols

Interface of an API session.

retry

Retry policies: when and how long a retryable request is retried.

Type Aliases:

Name Description
Timeout

Seconds; for a buffered request the limit of the whole request, for

Classes:

Name Description
BaseSession

Base class for sessions; a transport only does the HTTP I/O.

SessionFactory

Builds sessions with a provider's API and error resolver.

CallOptions

The options of one call, as send_request and stream take.

ISession

Sends methods to an API and returns their typed results.

ISessionFactory

Builds sessions for one provider.

FullJitterBackoff

Random exponential backoff, the default of RetryPolicy.

IBackoff

Computes the wait before the next attempt.

IRetryPredicate

Decides whether a transient error is retried.

RetryPolicy

When and how long a retryable request is retried.

RetryState

A failed attempt whose error the library would retry.

Attributes:

Name Type Description
DEFAULT_MAX_ATTEMPTS Final[int]

Attempts of a retryable request, including the first one.

DEFAULT_MAX_RETRY_AFTER Final[float]

Longest Retry-After the session waits for, in seconds.

DEFAULT_RETRY_DEADLINE Final[float]

Time after which no new retry starts, in seconds (15 minutes).

DEFAULT_RETRY_POLICY Final[RetryPolicy]

The policy of a session that is given none.

DEFAULT_MAX_ATTEMPTS module-attribute

DEFAULT_MAX_ATTEMPTS: Final[int] = 4

Attempts of a retryable request, including the first one.

DEFAULT_MAX_RETRY_AFTER module-attribute

DEFAULT_MAX_RETRY_AFTER: Final[float] = 60.0

Longest Retry-After the session waits for, in seconds.

DEFAULT_RETRY_DEADLINE module-attribute

DEFAULT_RETRY_DEADLINE: Final[float] = 900.0

Time after which no new retry starts, in seconds (15 minutes).

DEFAULT_RETRY_POLICY module-attribute

DEFAULT_RETRY_POLICY: Final[RetryPolicy] = RetryPolicy()

The policy of a session that is given none.

Timeout

Timeout = float

Seconds; for a buffered request the limit of the whole request, for a stream the longest silence between two chunks.

BaseSession

BaseSession(
    api: IAPI,
    auth: IAuth,
    error_resolver: IErrorResolver,
    *,
    headers: HeadersType | None = None,
    json_loads: JsonLoads = loads,
    json_dumps: JsonDumps = dumps,
    retry_policy: RetryPolicy = DEFAULT_RETRY_POLICY,
)

Bases: ABC

Base class for sessions; a transport only does the HTTP I/O.

Retries, turning error statuses into APIStatusError and parsing are done here, so every transport behaves the same. A subclass implements _execute, _open_stream and close.

Attributes:

Name Type Description
api

Builds request URLs.

auth

Produces the authentication header.

error_resolver

Turns error responses into exceptions.

headers IHeaders

Default headers sent with every request.

json_loads

JSON decoder for response bodies.

json_dumps

JSON encoder for request bodies.

retry_policy

When and how long a retryable request is retried, unless a call gives its own policy.

Store the session's collaborators.

Parameters:

Name Type Description Default
api IAPI

Builds request URLs.

required
auth IAuth

Produces the authentication header.

required
error_resolver IErrorResolver

Turns error responses into exceptions.

required
headers HeadersType | None

Default headers. An IHeaders collection is used as it is; a mapping is copied into Headers, and an empty set is used when None.

None
json_loads JsonLoads

JSON decoder for response bodies.

loads
json_dumps JsonDumps

JSON encoder for request bodies.

dumps
retry_policy RetryPolicy

When and how long a retryable request is retried: four attempts at most, none started after 15 minutes, a Retry-After above 60 s not waited for.

DEFAULT_RETRY_POLICY

Methods:

Name Description
close

Release the transport's resources.

send_request

Send a method and return its parsed response.

stream

Send a method and stream its response.

close abstractmethod async

close() -> None

Release the transport's resources.

send_request async

send_request(
    method: IAPIMethod[T],
    *,
    headers: HeadersType | None = None,
    timeout: Timeout | None = None,
    retry_policy: RetryPolicy | None = None,
) -> T

Send a method and return its parsed response.

If method.retryable is true, transient failures are retried as the retry policy allows. headers and timeout apply to every attempt.

Parameters:

Name Type Description Default
method IAPIMethod[T]

The method to send.

required
headers HeadersType | None

Headers for this request only; they override the session's and the method's headers, but not the authentication header.

None
timeout Timeout | None

Seconds the whole request may take, instead of the total of the session's timeout; None to keep it.

None
retry_policy RetryPolicy | None

Retry policy of this request only, instead of the session's; None to keep it. It does not make a method retryable.

None

Returns:

Type Description
T

The parsed response, typed by the method's generic argument.

Raises:

Type Description
APIStatusError

If the API answered with an error status, or the method's parser found a failure in a successful one.

APITransportError

If the request could not be completed.

APIResponseError

If the response body could not be parsed.

stream async

stream(
    method: IStreamMethod[T],
    *,
    headers: HeadersType | None = None,
    timeout: Timeout | None = None,
    retry_policy: RetryPolicy | None = None,
) -> AsyncGenerator[Stream[T]]

Send a method and stream its response.

Only opening the stream is retried: once the block has started, a failure is raised to the caller instead of resending the request. The connection is released when the block exits, including on break and exceptions.

Parameters:

Name Type Description Default
method IStreamMethod[T]

The stream method to send.

required
headers HeadersType | None

Headers for this request only; they override the session's and the method's headers, but not the authentication header.

None
timeout Timeout | None

Longest silence between two chunks, in seconds, instead of the session's stream timeout; None to keep it. A stream has no total limit, so a long generation is not cut off.

None
retry_policy RetryPolicy | None

Retry policy of opening this stream only, instead of the session's; None to keep it. It does not make a method retryable.

None

Yields:

Type Description
AsyncGenerator[Stream[T]]

The stream of items, typed by the method's generic argument.

Raises:

Type Description
APIStatusError

If the API answered with an error status; raised on entering the block. Also, while iterating, for an error event in the stream.

APITransportError

If the connection fails on entering the block.

APIStreamInterruptedError

While iterating, if the connection fails or goes silent for too long (the transport error is its __cause__), or the body ends before the method's terminal event. Never retried.

APIResponseError

If the body cannot be decoded or validated, or an item comes after the terminal event.

Example

async with session.stream(Download()) as stream: ... await stream.save("out.bin")

SessionFactory

SessionFactory(
    session_type: Callable[
        Concatenate[IAPI, AuthT, IErrorResolver, P],
        SessionT,
    ],
    *,
    auth_factory: Callable[[str], AuthT],
    api: IAPI,
    error_resolver: IErrorResolver,
)

Builds sessions with a provider's API and error resolver.

The remaining session arguments (timeout, max_body_size, ...) are taken from session_type and checked by type checkers.

Attributes:

Name Type Description
session_type

Session class, or a callable that builds one from the API, the auth and the error resolver.

auth_factory

Turns a token string into an IAuth.

api

The provider's IAPI.

error_resolver

The provider's error resolver.

Example

factory = SessionFactory( ... AiohttpSession, ... auth_factory=BearerAuth, ... api=EXAMPLE_API, ... error_resolver=ExampleErrorResolver(), ... ) session = factory("sk-...", max_body_size=10_000_000)

Store the provider's configuration.

Parameters:

Name Type Description Default
session_type Callable[Concatenate[IAPI, AuthT, IErrorResolver, P], SessionT]

Session class, or a callable that builds one from the API, the auth and the error resolver.

required
auth_factory Callable[[str], AuthT]

Turns a token string into an IAuth.

required
api IAPI

The provider's IAPI.

required
error_resolver IErrorResolver

The provider's error resolver.

required

CallOptions

Bases: TypedDict

The options of one call, as send_request and stream take.

The shared layer (use cases, client.generation) takes them as **options and passes them on to the session unchanged, so type checkers check them at the call. Each key is a keyword-only argument of ISession.send_request with the same type; a test keeps the two in step.

It is closed (PEP 728, from typing_extensions until Python 3.15): no other key can ride in **options. So a method that takes them may have keyword parameters of its own beside them, such as GenerationResource.create's provider_options, and still match a protocol whose method has none.

Attributes:

Name Type Description
headers HeadersType | None

Headers for this request only; they override the session's and the method's headers.

timeout Timeout | None

Seconds for this request instead of the session's limit; None to keep it.

retry_policy RetryPolicy | None

Retry policy of this request; None for the session's.

ISession

Bases: Protocol

Sends methods to an API and returns their typed results.

Methods:

Name Description
close

Release the connection pool.

send_request

Send a method and return its parsed response.

stream

Send a method and stream its response.

close async

close() -> None

Release the connection pool.

send_request async

send_request(
    method: IAPIMethod[T],
    *,
    headers: HeadersType | None = None,
    timeout: Timeout | None = None,
    retry_policy: RetryPolicy | None = None,
) -> T

Send a method and return its parsed response.

Parameters:

Name Type Description Default
method IAPIMethod[T]

The method to send.

required
headers HeadersType | None

Headers for this request only; they override the session's and the method's headers.

None
timeout Timeout | None

Seconds the whole request may take; None for the session's timeout.

None
retry_policy RetryPolicy | None

Retry policy of this request; None for the session's.

None

Returns:

Type Description
T

The parsed response.

stream

stream(
    method: IStreamMethod[T],
    *,
    headers: HeadersType | None = None,
    timeout: Timeout | None = None,
    retry_policy: RetryPolicy | None = None,
) -> AbstractAsyncContextManager[Stream[T]]

Send a method and stream its response.

Parameters:

Name Type Description Default
method IStreamMethod[T]

The stream method to send.

required
headers HeadersType | None

Headers for this request only; they override the session's and the method's headers.

None
timeout Timeout | None

Longest silence between two chunks, in seconds; None for the session's stream timeout.

None
retry_policy RetryPolicy | None

Retry policy of opening the stream; None for the session's.

None

Returns:

Type Description
AbstractAsyncContextManager[Stream[T]]

A context manager that yields the stream and closes it on

AbstractAsyncContextManager[Stream[T]]

exit.

ISessionFactory

Bases: Protocol

Builds sessions for one provider.

Attributes:

Name Type Description
error_resolver IErrorResolver

The provider's error resolver, given to every session.

error_resolver property

error_resolver: IErrorResolver

The provider's error resolver, given to every session.

FullJitterBackoff dataclass

FullJitterBackoff(
    *, multiplier: float = 0.5, max_delay: float = 30.0
)

Random exponential backoff, the default of RetryPolicy.

The wait after attempt n is drawn uniformly from zero to multiplier * 2 ** (n - 1), capped at max_delay, so clients that failed together do not come back together.

Attributes:

Name Type Description
multiplier float

Upper end of the first wait, in seconds.

max_delay float

Largest wait, in seconds.

Example

RetryPolicy(backoff=FullJitterBackoff(max_delay=5))

IBackoff

Bases: Protocol

Computes the wait before the next attempt.

IRetryPredicate

Bases: Protocol

Decides whether a transient error is retried.

RetryPolicy dataclass

RetryPolicy(
    *,
    max_attempts: int = DEFAULT_MAX_ATTEMPTS,
    deadline: float | None = DEFAULT_RETRY_DEADLINE,
    max_retry_after: float = DEFAULT_MAX_RETRY_AFTER,
    backoff: IBackoff = FullJitterBackoff(),
    retry_if: IRetryPredicate | None = None,
)

When and how long a retryable request is retried.

A policy applies only to methods with __retryable__ = True and only to errors whose transient flag is true; it cannot make any other error retried. Give one to a session for all its calls, or to a single send_request or stream call.

Attributes:

Name Type Description
max_attempts int

Attempts in total, including the first one; 1 turns retries off.

deadline float | None

Seconds since the first attempt after which no retry starts; a wait that would end later is not started. An attempt in progress is limited by its own timeout. None for no limit.

max_retry_after float

Longest Retry-After worth waiting for, in seconds; an API that asks for longer gets its error raised at once.

backoff IBackoff

Computes the wait before each retry; the session waits at least the error's Retry-After. Called only when max_attempts allows another attempt; with a deadline, also for a wait the deadline then cuts.

retry_if IRetryPredicate | None

Called for each transient error that max_attempts allows to retry, never for the last attempt's; False raises the error at once. None retries every transient error.

Example

patient = RetryPolicy(max_attempts=6, deadline=None) await session.send_request(Ping(), retry_policy=patient)

RetryState dataclass

RetryState(
    *,
    attempt: int,
    elapsed: float,
    error: APIStatusError | APITransportError,
)

A failed attempt whose error the library would retry.

The session builds it for a policy's retry_if and backoff only when the error is transient and its Retry-After is within max_retry_after, so error is never anything else.

Attributes:

Name Type Description
attempt int

Number of the attempt that failed, counting from 1.

elapsed float

Seconds since the first attempt started.

error APIStatusError | APITransportError

The transient error the attempt ended with.

aiohttp

Session implementation on top of aiohttp.

Classes:

Name Description
AiohttpSessionOptions

The optional arguments of AiohttpSession, for **kwargs.

AiohttpSession

Session that sends requests with aiohttp.

Attributes:

Name Type Description
DEFAULT_TIMEOUT Final[ClientTimeout]

Default timeout of buffered requests.

DEFAULT_STREAM_TIMEOUT Final[ClientTimeout]

Default timeout of streamed requests.

REDIRECT_STATUSES Final[frozenset[int]]

Statuses that redirect the request to the Location header.

DEFAULT_TIMEOUT module-attribute

DEFAULT_TIMEOUT: Final[ClientTimeout] = ClientTimeout(
    total=600, sock_connect=30
)

Default timeout of buffered requests.

DEFAULT_STREAM_TIMEOUT module-attribute

DEFAULT_STREAM_TIMEOUT: Final[ClientTimeout] = (
    ClientTimeout(
        total=None, sock_connect=30, sock_read=600
    )
)

Default timeout of streamed requests.

REDIRECT_STATUSES module-attribute

REDIRECT_STATUSES: Final[frozenset[int]] = frozenset(
    {301, 302, 303, 307, 308}
)

Statuses that redirect the request to the Location header.

AiohttpSessionOptions

Bases: TypedDict

The optional arguments of AiohttpSession, for **kwargs.

A provider client takes them as **session_kwargs and passes them on, so type checkers check them at the client's call. Each key is a keyword-only argument of AiohttpSession.__init__ with the same type; a test keeps the two in step.

Attributes:

Name Type Description
headers HeadersType | None

Default headers sent with every request.

json_loads JsonLoads

JSON decoder for response bodies.

json_dumps JsonDumps

JSON encoder for request bodies.

max_connections int

Maximum number of simultaneous connections.

timeout ClientTimeout

Timeout of buffered requests.

stream_timeout ClientTimeout

Timeout of streamed requests.

retry_policy RetryPolicy

When and how long a retryable request is retried.

max_body_size int | None

Largest buffered body accepted, in bytes.

trust_env bool

Whether to use proxy settings from the environment.

follow_redirects bool

Whether to follow same-origin redirects.

loop AbstractEventLoop | None

The event loop the session is bound to.

AiohttpSession

AiohttpSession(
    api: IAPI,
    auth: IAuth,
    error_resolver: IErrorResolver,
    *,
    headers: HeadersType | None = None,
    json_loads: JsonLoads = loads,
    json_dumps: JsonDumps = dumps,
    max_connections: int = 100,
    timeout: ClientTimeout = DEFAULT_TIMEOUT,
    stream_timeout: ClientTimeout = DEFAULT_STREAM_TIMEOUT,
    retry_policy: RetryPolicy = DEFAULT_RETRY_POLICY,
    max_body_size: int | None = None,
    trust_env: bool = False,
    follow_redirects: bool = False,
    loop: AbstractEventLoop | None = None,
)

Bases: BaseSession

Session that sends requests with aiohttp.

The session is bound to one event loop: loop if given, otherwise the loop of its first use. It can be created anywhere, even outside a coroutine, but is then usable only from that loop. The underlying ClientSession is created lazily on first use and re-created after close(). Use it as an async context manager to open and close it deterministically; close it before its event loop ends.

Example

async with AiohttpSession(api, BearerAuth(key), resolver) as s: ... pong = await s.send_request(Ping())

Configure the session; no connection is opened yet.

Parameters:

Name Type Description Default
api IAPI

Builds request URLs.

required
auth IAuth

Produces the authentication header.

required
error_resolver IErrorResolver

Turns error responses into exceptions.

required
headers HeadersType | None

Default headers sent with every request.

None
json_loads JsonLoads

JSON decoder for response bodies.

loads
json_dumps JsonDumps

JSON encoder for request bodies.

dumps
max_connections int

Maximum number of simultaneous connections.

100
timeout ClientTimeout

Timeout of buffered requests. aiohttp's own default caps a request at 5 minutes, which long model generations exceed; 10 minutes matches the official AI SDKs.

DEFAULT_TIMEOUT
stream_timeout ClientTimeout

Timeout of streamed requests. It has no total limit, because a stream may last indefinitely; only the silence between chunks (sock_read) is limited.

DEFAULT_STREAM_TIMEOUT
retry_policy RetryPolicy

When and how long a retryable request is retried; see RetryPolicy.

DEFAULT_RETRY_POLICY
max_body_size int | None

Largest buffered response or error body accepted, in bytes; None for no limit. Streamed bodies are not limited.

None
trust_env bool

Whether to use proxy settings and .netrc credentials from the environment.

False
follow_redirects bool

Whether to follow redirects within the origin of the request. Redirects to another origin are never followed, so no header leaks to another host; a redirect that is not followed raises APIRedirectError.

False
loop AbstractEventLoop | None

Event loop the session is bound to; None to bind the loop of its first use.

None

Methods:

Name Description
close

Close the underlying ClientSession and its connections.

get_client

Return the ClientSession, creating it if needed.

Attributes:

Name Type Description
loop AbstractEventLoop | None

The event loop the session is bound to; None until first use.

loop property

loop: AbstractEventLoop | None

The event loop the session is bound to; None until first use.

Reading it never binds a loop.

close async

close() -> None

Close the underlying ClientSession and its connections.

Raises:

Type Description
RuntimeError

If called from another event loop.

get_client async

get_client() -> ClientSession

Return the ClientSession, creating it if needed.

Returns:

Type Description
ClientSession

An open ClientSession. It holds no headers: they are

ClientSession

built for every request, so later changes take effect.

Raises:

Type Description
RuntimeError

If called from another event loop.