Models

The model is two lines of model_config — a provider and a model name — and the key is always LLM_API_KEY. By the end of this page your agent will run on the provider you choose, with the settings you choose, and you will have seen what was actually sent.

Pick a provider

# OpenAI
model_config = {"provider": "openai", "model": "gpt-5.4"}

# Anthropic
model_config = {"provider": "anthropic", "model": "claude-sonnet-5"}

# Google Gemini
model_config = {"provider": "gemini", "model": "gemini-3.8-flash"}

# Groq
model_config = {"provider": "groq", "model": "openai/gpt-oss-120b"}

# DeepSeek
model_config = {"provider": "deepseek", "model": "deepseek-chat"}

# Mistral AI
model_config = {"provider": "mistral", "model": "mistral-small-latest"}

# Azure OpenAI
model_config = {"provider": "azure", "model": "gpt-5.4"}

# OpenRouter: the provider's own name for the model
model_config = {"provider": "openrouter", "model": "anthropic/claude-sonnet-5"}

# Ollama, on your machine: no key needed
model_config = {"provider": "ollama", "model": "llama3.1:8b", "ollama_host": "http://localhost:11434"}

export LLM_API_KEY=your_api_key_here

Only the OpenAI configuration was run for this page. The others are shown as configuration; check the model name against your provider’s current list.

If you want	Try
The strongest reasoning	OpenAI `gpt-5.6-terra`, Anthropic `claude-sonnet-5`
Fast and cheap	OpenAI `gpt-5.4-mini`, Groq `openai/gpt-oss-120b`, Gemini `gemini-3.8-flash`
Low cost	DeepSeek `deepseek-chat`, Mistral `mistral-small-latest`
Data that stays on your machine	Ollama
Many providers behind one key	OpenRouter
Your company’s Azure tenancy	Azure OpenAI

Set the model’s parameters

import asyncio
from omnicoreagent import OmniCoreAgent

async def main():
    agent = OmniCoreAgent(
        name="assistant",
        system_instruction="Answer in plain text, in one sentence.",
        model_config={
            "provider": "openai",
            "model": "gpt-5.4-mini",
            "max_tokens": 300,
            "temperature": 0.2,
        },
    )
    result = await agent.run("What is LiteLLM, in one sentence?")
    print(result["response"])

    trajectory = await agent.get_trajectory(result["trace_id"])
    facts = trajectory["steps"][0]["model_calls"][0]["facts"]
    print("served by:", facts["provider_model"])
    print("settings sent:", facts["request_settings"])
    print("tokens:", facts["tokens"], "| cost:", facts["estimated_cost_usd"])
    await agent.cleanup()

asyncio.run(main())

LiteLLM is an open-source Python library and proxy that provides a unified API for calling many different LLM providers through a single OpenAI-compatible interface.
served by: gpt-5.4-mini
settings sent: {'model': 'openai/gpt-5.4-mini', 'temperature': 0.2, 'max_tokens': 300}
tokens: {'input': 1447, 'output': 35, 'total': 1482, 'cached_input': 1024} | cost: 0.0005515499999999999

Every model call’s trace records the model the provider says served it, the settings that were sent, the tokens, and an estimated cost (from LiteLLM’s price table, not your invoice).

How it works

One route: LiteLLM. provider and model become LiteLLM’s "<provider>/<model>" (openai/gpt-5.4-mini above). Every provider returns the same turns to the runtime, so tools, streaming, memory and traces work the same on all of them.
One key. LLM_API_KEY is read when the agent starts (its first run) and handed to the provider you named — whose own key variable you never set. An agent can carry its own instead — "api_key": "..." in its model_config — for two providers in one process. Keys are never written into a trace. Ollama needs none.
Settings are sent as given. Nothing is added for you: with no max_tokens, the provider’s own limit applies (with a budget, a call is held to 4,096 output tokens). If a model refuses a setting by name — for example temperature on a reasoning model — the call is retried once without it, the setting is not sent again to that model, and the retry is recorded on the model call (facts["retries"], with the setting under dropped).
Transient failures are retried — rate limits, timeouts, connection errors — up to three times with backoff. A wrong key, an empty account or an unknown model is not retried: the run ends at once with the reason. A streamed call is not retried, since its text was already delivered.

Thinking and reasoning models

Thinking models return data with each turn that must be sent back unchanged on the next request, or a multi-step tool loop fails on its next step. OmniCoreAgent keeps it with the turn and sends it back to the provider that issued it:

Provider	Data kept and returned
`anthropic` (extended thinking)	Signed `thinking` and `redacted_thinking` blocks
`gemini` (thinking models)	The thought signature on each tool call
`openrouter` (Claude, Gemini, and others)	`reasoning_details`, returned at the top level of the assistant message

This works with run() and stream(), across runs in the same session (the data is kept in stored history), and after context compression. Other providers receive exactly the fields they always did. Enable thinking with the provider’s usual setting, for example:

model_config = {"provider": "anthropic", "model": "claude-sonnet-5", "reasoning_effort": "low"}

Memory privacy redaction never alters signatures or encrypted reasoning. If the readable thinking text contains personal data that redaction must change, the stored copy of that turn leaves the signed data out (and records continuation_dropped) instead of storing a signature the provider would reject; the running agent keeps its own copy. Traces record that continuation data was present (counts and a digest on each model call) and never store signatures or encrypted values; the thinking text itself is recorded only with capture="full".

Azure OpenAI and Ollama

Azure OpenAI
Ollama

model_config = {
    "provider": "azure",
    "model": "gpt-5.4",
    "azure_endpoint": "https://your-resource.openai.azure.com",
    "azure_api_version": "2024-02-01",
    "azure_deployment": "my-gpt-deployment",  # optional: calls this deployment instead of `model`
}

The key is your Azure OpenAI key, in LLM_API_KEY.

model_config = {
    "provider": "ollama",
    "model": "llama3.1:8b",
    "ollama_host": "http://localhost:11434",
}

Pull the model first (ollama pull llama3.1:8b). No LLM_API_KEY is needed. Pick a model that supports tool calling: the agent offers tools on every call.

Your own endpoint (`base_url`)

A server that speaks a provider’s API — vLLM, LM Studio, llama.cpp’s server, a company gateway or a proxy — is reached by naming the provider whose API it speaks and giving its URL:

model_config = {
    "provider": "openai",           # the API the server speaks
    "model": "qwen3-coder-30b",     # the name the server knows the model by
    "base_url": "http://localhost:8000/v1",
}

The URL is sent with each of this agent’s calls, streamed or not. It does not change the process environment, so two agents in one process can use two endpoints.
The key is still LLM_API_KEY. A local server that checks no key accepts any value: export LLM_API_KEY=local.
The URL is not recorded with the run’s model settings (a URL can carry a token).
A dollar budget needs a published price for the model name. A model with no price is governed by the token and call limits: set a model_tokens or model_calls budget, or total_tokens_limit, for it.
It works with any provider: ollama with base_url reaches Ollama on another machine.

Options

Key	What it does
`provider`	Required. One of `openai`, `anthropic`, `gemini`, `groq`, `deepseek`, `mistral`, `azure`, `openrouter`, `ollama` — lowercase.
`model`	Required. The provider’s name for the model.
`max_tokens`	Most output tokens per call (sent to OpenAI as `max_completion_tokens`).
`temperature`, `top_p`	Sampling; set them only if the model supports them.
`reasoning_effort`	For reasoning models, e.g. `"low"`, `"medium"`, `"high"`, or `"none"` where the model allows it.
`logprobs`, `top_logprobs`	Ask for token probabilities, recorded for training use.
`api_key`	This agent’s key, instead of `LLM_API_KEY`.
`azure_endpoint`, `azure_api_version`, `azure_deployment`	Azure OpenAI only.
`ollama_host`	Ollama only: where the server listens.
`base_url`	Another endpoint that speaks the provider’s API (above).

Other keys are accepted and ignored. See also the OmniCoreAgent reference.

When things go wrong

The first run in a process is slower than the rest

The first model call loads the model client (LiteLLM), about 2 seconds on a typical server and longer on a slow machine or a cold start. Later calls in the same process do not pay it. OmniServe and the background manager load it while they start, so no request pays it there.

ValueError: LLM_API_KEY not found in environment variables

Raised by the first run() when the key is not set in the shell that runs the script: export LLM_API_KEY=.... The runtime does not read .env files itself; call load_dotenv() from python-dotenv at the top of your script if your key is in one.

ValueError: Unsupported provider: OpenAI. Supported: openai, anthropic, groq, ollama, azure, gemini, deepseek, mistral, openrouter

The provider name is lowercase and one of these nine. For another OpenAI-compatible service, use openrouter if it is listed there.

ValueError: model_config.model is required

Name the model as well as the provider. (model_config.provider is required is the same message for the provider.)

The model call was refused: the provider rejected the API key (invalid_api_key). Fix the account; retrying will not help.

The run ends with status "error" and this response. The key is wrong, or it belongs to a different provider than model_config["provider"]. An account out of credits reads the provider account has no credits left (insufficient_quota) in the same place.

The model call was refused: the provider does not serve the model … to this account

The model name is misspelled, retired, or not available to your account. Check it against the provider’s model list, and that provider is the one that serves it.

Quickstart

A tool, a memory and a trace, in five minutes.

Budgets and durable runs

Put a price on runs, checked before every model call.

Streaming and events

Show the answer as the model writes it.

Configuration

Every setting of an agent in one place.

Get Started

Make It Safe

Run It

See and Improve

Build

How It Works

Reference

HTTP API (OmniServe)

Releases

Models

Models

Pick a provider

Set the model’s parameters

How it works

Thinking and reasoning models

Azure OpenAI and Ollama

Your own endpoint (`base_url`)

Options

When things go wrong

Next

Quickstart

Budgets and durable runs

Streaming and events

Configuration

​Models

​Pick a provider

​Set the model’s parameters

​How it works

​Thinking and reasoning models

​Azure OpenAI and Ollama

​Your own endpoint (base_url)

​Options

​When things go wrong

​Next

Quickstart

Budgets and durable runs

Streaming and events

Configuration

Models

Pick a provider

Set the model’s parameters

How it works

Thinking and reasoning models

Azure OpenAI and Ollama

Your own endpoint (`base_url`)

Options

When things go wrong

Next