Built-in tools¶
A built-in tool (OpenAI calls it hosted, Anthropic a server tool) is a tool the provider runs: the model searches the web or runs code, the provider executes it and puts the call and its result into the response. Unlike a function tool (Tool calls), there is nothing for your code to run and no ToolResult to send back. The library has four jobs here: send the tool in the request, show what the provider did in the response and the stream, keep what the provider needs to go on with the conversation, and hand you the files the tools make.
Provider support
The shared types below are in aiollm-core 0.16, the files of the tools (Files) in 0.17. A provider package takes them from its own release: the built-in tools of aiollm-openai and aiollm-anthropic come with their 0.16 releases (0.15 does not send them), their files with 0.17. Each provider package also has a tools module (aiollm_openai.tools, aiollm_anthropic.tools) that builds its own tools as a ProviderTool, checked against the provider's request parameters. A provider that does not run a tool raises RequestValidationError for it rather than leave it out.
In the request¶
Built-in tools go in the same tools= as functions, one list: tools=[get_weather, WebSearch()]. The shared ones are what both providers have:
WebSearch(allowed_domains=..., user_location=...): a web search.allowed_domainslimits it to the domains, at least one;UserLocation(country=, region=, city=, timezone=)localizes the results (every field is optional).CodeExecution(files=...): code run in the provider's sandbox. Its one shared option isfiles, stored files (Stored files) put into the sandbox for the code to read; the container itself, its memory and its network are each provider's own.ProviderTool(provider=, definition=): a tool of one provider as that provider takes it, for everything that is not shared (file search, web fetch, MCP, the options the shared tools lack: OpenAI'ssearch_context_size, Anthropic'sblocked_domainsandmax_uses, the version of an Anthropic tool). Thedefinitionis a JSON object, the tool as the provider's request holds it. Only that provider takes it; any other raisesRequestValidationError, as it does for another provider'sProviderFile.
from typing import Any
from aiollm import (
CodeExecution,
GenerationResponse,
IGenerationClient,
UserLocation,
WebSearch,
)
async def research(client: IGenerationClient) -> GenerationResponse[Any]:
response = await client.generation.create(
model="gpt-6-astra",
input="What is the weather in Paris today?",
tools=[
WebSearch(
allowed_domains=["example.com"],
user_location=UserLocation(country="FR", city="Paris"),
),
CodeExecution(),
],
)
for call in response.server_tool_calls:
print(call.kind.value, call.status.value, call.queries)
for citation in response.citations:
print(citation.url, citation.title)
return response
from aiollm import Provider, ProviderTool
file_search = ProviderTool(
provider=Provider("openai"),
definition={"type": "file_search", "vector_store_ids": ["vs_123"]},
)
- One list, two kinds.
request.toolsholds the function tools andrequest.builtin_toolsthe built-in ones; a request may have built-in tools only (request.toolsis thenMISSING).WebSearchandCodeExecutioncome at most once;ProviderTools may repeat. tool_choiceis for functions. AForcedToolnames a function tool; to make the model use a built-in tool, say so in the prompt.- Not left out. A tool the provider cannot run raises
RequestValidationError(field="tools") before anything is sent: an answer without the search the caller asked for would be misleading.
In the response¶
server_tool_callslists the calls the provider ran, in order: aServerToolCallwith the call'sid,kind(ServerToolKind:WEB_SEARCH,FILE_SEARCH,CODE_EXECUTION,WEB_FETCH,IMAGE_GENERATION,MCP,OTHER),status(ServerToolStatus:COMPLETED,INCOMPLETE,FAILED;IN_PROGRESSonly in a stream event),queriesof a search, thecodea code execution ran, and theerrorof a failed one. The result itself (the pages found, the output of the code) is not normalized: it is incall.raw, the provider's items of the call.- A failed tool is a call, not an error. Both providers answer HTTP 200 with the tool's failure inside the response; so does the library: the call has
status=FAILEDand itserror, andcreate()returns. Check the calls if the answer depends on them. tool_callsis for functions only. The tool runner and the rule "a response with calls waits for their results" are unchanged by built-in tools.citationslists the sources the text rests on: aCitationwith akind(CitationKind:URL,FILE,CONTAINER_FILE), theurl,title,file_id,container_id,filename, thecited_textand where it sits intext(start_index,end_index), as far as the provider tells them. What is the provider's alone (Anthropic'sencrypted_index) stays inrawandnative.- The calls are billed. Searches, containers and the tokens of what they bring cost money at the provider; the library keeps no prices.
server_tool_callsis there so that you can count them.
In a stream¶
A built-in call is two events: a ServerToolCallEvent with phase STARTED (the call as it is known then, status IN_PROGRESS), then one with phase DONE (the whole call). The deltas of the code a tool writes and the partial images stay OtherEvents. CompletedEvent has server_tool_calls and citations as the response does.
from typing import Any
from aiollm import (
CompletedEvent,
IGenerationClient,
ServerToolCallEvent,
WebSearch,
)
async def search_streamed(
client: IGenerationClient,
) -> CompletedEvent[Any]:
async with client.generation.stream(
model="gpt-6-astra",
input="What is the weather in Paris today?",
tools=WebSearch(),
) as events:
async for event in events:
if isinstance(event, ServerToolCallEvent):
print(event.phase.value, event.call.kind.value)
return await events.final()
Continuing a conversation¶
The provider needs its own record of a turn that used built-in tools: Anthropic's search results carry encrypted_content that must come back as it was, and OpenAI's search calls and reasoning items are part of its record too. So response.to_turn() gives an assistant turn with native (the response's native output) when the response has server_tool_calls, and the provider sends it in place of the turn's text; another provider cannot read it and sends the text. A response without built-in calls gives a plain turn, as before.
A long turn of built-in tools can be paused by the provider (Anthropic's pause_turn): incomplete_reason is IncompleteReason.PAUSED, the text may be empty, and the turn is not finished. to_turn() gives a turn with native even without text, and a conversation may end with such a turn: send it back, with the same tools, to let the provider go on. The library does not do this by itself, since each round is a paid call of its own; the loop is yours, and bounded.
from typing import Any
from aiollm import (
GenerationResponse,
IGenerationClient,
IncompleteReason,
InputItem,
Turn,
WebSearch,
)
async def research_to_the_end(
client: IGenerationClient, question: str, rounds: int = 5
) -> GenerationResponse[Any]:
conversation: list[InputItem] = [Turn.user(question)]
for _ in range(rounds):
response = await client.generation.create(
model="gpt-6-astra",
input=conversation,
tools=WebSearch(),
)
if response.incomplete_reason is not IncompleteReason.PAUSED:
return response
conversation.append(response.to_turn())
raise RuntimeError("the provider keeps pausing")
- The last turn. The order rules of Shared generation say a conversation ends with a
userturn or a tool result; the one other ending is anassistantturn withnative, as above. Sending it after a generation that was not paused is the provider's to answer (Anthropic refuses a final assistant turn on current models). - Only with built-in calls or a pause. An ordinary text answer still goes back as text:
to_turn()keepsnativeonly for what the text cannot carry.
Files¶
Code can write files: a chart, a cleaned table. response.files (and CompletedEvent.files) lists them as ProducedFiles: a ProviderFile with the file's filename and the source_call_id of the call that made it, when the provider tells them, and its container_id. client.files.download(file) streams any of them, whatever the provider keeps them in.
- Anthropic keeps them with the files you store:
container_idisNone, and the file goes wherever aProviderFiledoes (retrieve,delete, a part'sfile, anotherCodeExecution). - OpenAI keeps them in the code interpreter's container, only as long as the container lives, and the library downloads them from there:
container_idis set, and onlydownloadreaches such a file;retrieve,delete, a part'sfileandCodeExecution.filesrefuse it withRequestValidationErrorbefore anything is sent. OpenAI names a file the code made only when the model refers to it in its text (acontainer_file_citation), so ask for the file in the prompt.
To give the code files to read, upload them and pass their refs:
from pathlib import Path
from typing import Protocol
from aiollm import CodeExecution, IFilesClient, IGenerationClient
class FilesAndGeneration(IGenerationClient, IFilesClient, Protocol):
"""A client with both resources, as every provider's is."""
async def chart(client: FilesAndGeneration, table: Path, out: Path) -> None:
stored = await client.files.upload(table)
response = await client.generation.create(
model="gpt-6-astra",
input="Draw the sales by month as chart.png and link to it.",
tools=CodeExecution(files=[stored.ref]),
)
for file in response.files:
async with client.files.download(file) as content:
await content.save(out / (file.filename or file.id))
The library does not keep or clean up containers: the provider expires them (OpenAI after some idle minutes, Anthropic after 30 days), so download what you need soon.
What each shared field becomes for each provider is listed under Built-in tools in the field mapping.