OmniServe

OmniServe puts one agent behind an HTTP API: a run as JSON or as a live stream, the run’s record and trajectory afterwards, a person’s approval mid-run, and background tasks, with auth, rate limits and a request timeout. One command, omniserve run --agent my_agent.py, or OmniServe(agent).start() in your own script.

Every output on this page is what the server returned when it was run. The model’s wording, the ids and the timings will differ on your run. Each route’s fields are in the generated HTTP API reference, under HTTP API (OmniServe) in the navigation.

pip install "omnicoreagent[serve]"
export LLM_API_KEY=...   # your model provider's key

Serve an agent

Save this as my_agent.py. OmniServe takes the module’s agent, or calls its create_agent() when there is no agent:

from omnicoreagent import OmniCoreAgent, ToolRegistry

tools = ToolRegistry()

@tools.register_tool("get_order")
def get_order(order_id: str) -> dict:
    """Look up an order by its id."""
    return {"order_id": order_id, "status": "shipped", "total": 42.0}

agent = OmniCoreAgent(
    name="support",
    system_instruction="You answer questions about orders. Answer in plain text, in one sentence.",
    model_config={"provider": "openai", "model": "gpt-5.4-mini"},
    local_tools=tools,
)

Serve it (on port 8771 here; the default is 8000):

omniserve run --agent my_agent.py --port 8771

📦 Loading agent from: my_agent.py
✅ Loaded agent: support

==================================================
OmniServe
OmniCoreAgent 0.5.0
==================================================
Agent: support
Server: http://0.0.0.0:8771
Docs: http://0.0.0.0:8771/docs
Metrics: http://0.0.0.0:8771/prometheus

Features Enabled:
  • Auth: ✗ (use --auth-token to enable)
  • Rate Limit: ✗ (use --rate-limit N to enable)
  • CORS: ['*']
  • Background API: ✓, task store: in_memory

💡 Available options: --auth-token, --rate-limit, --cors-origins, --no-docs
   Run 'omniserve run --help' for all options
==================================================

INFO:     Started server process [476363]
INFO:     Waiting for application startup.
INFO:     Application startup complete.

Wait for /ready, then ask. Loading the model library can take a minute on a busy machine; the agent_name in the answer confirms it is your server:

curl -s localhost:8771/ready
curl -s -X POST localhost:8771/run/sync -H "Content-Type: application/json" \
  -d '{"query": "Where is order 1042?", "session_id": "alice"}'

{"ready":true,"agent_name":"support","initialized":true,"mcp_connected":true,"mcp_servers":{}}
{
    "status": "success",
    "termination_reason": "stop",
    "guardrail_result": null,
    "response": "Order 1042 has been shipped.",
    "approvals": null,
    "budget_request": null,
    "session_id": "alice",
    "agent_name": "support",
    "metric": {
        "requests": 2,
        "request_tokens": 3033,
        "response_tokens": 32,
        "total_tokens": 3065,
        "total_time": 4.4021726670034695,
        "details": {
            "cached_input_tokens": 1024
        }
    },
    "trace_id": "trace_2bd35e0e39a44391ad1b7c8e5424cafd",
    "run_id": "run_3520195e399d490089a6098759efbf03"
}

Two model calls: one to call get_order, one to answer. session_id is the conversation: the next request with "session_id": "alice" remembers this one. Leave it out and each request starts a new session (its id is in the response). Keep run_id: everything about this run is under /runs/{run_id}. Interactive docs for your server are at /docs (Swagger) and /redoc.

No agent file yet? omniserve quickstart serves a plain agent, by default --provider openai --model gpt-5.4-mini. Asked to “Say hello in five words, in plain text”, it answered success | Hello there, hope you’re well.

Stream a run

POST /run takes the same body and answers with Server-Sent Events: the run’s telemetry as it happens, the answer as it is written (text_delta), and a final complete with the same fields as /run/sync. A small client:

import json
import httpx

body = {
    "query": "Where is order 1042, and what did it cost? Save a one-line note about it in notes.txt.",
    "session_id": "dave",
}
with httpx.stream("POST", "http://localhost:8771/run", json=body, timeout=None) as response:
    event = None
    for line in response.iter_lines():
        if line.startswith("event: "):
            event = line.removeprefix("event: ")
        elif line.startswith("data: "):
            data = json.loads(line.removeprefix("data: "))
            if event == "text_delta":
                print(data["text"], end="", flush=True)
            elif event == "tool_requested":  # every tool: yours, workspace, MCP
                print(f"[tool_requested] {data['input']['tool_name']} {data['input']['arguments']}")
            elif event == "tool_result":  # what one of your own tools returned
                print(f"[tool_result] {data['output']['status']} {data['output']['data']}")
            elif event in ("workspace_read", "workspace_write"):  # a workspace file tool
                print(f"[{event}] {data['output']['status']} {data['input']['tool_args']['path']}")
            elif event == "tool_observation":  # the result as the model was given it
                print(f"[tool_observation] {data['output']['tool_name']}")
            elif event == "complete":
                print(f"\n[complete] {data['status']} {data['run_id']}")
            elif event == "error":
                print(f"[error] {data['error']}")

[tool_requested] get_order {'order_id': '1042'}
[tool_result] success {'order_id': '1042', 'status': 'shipped', 'total': 42.0}
[tool_observation] get_order
[tool_requested] write_file {'path': 'notes.txt', 'content': 'Order 1042 is shipped and cost $42.00.\n', 'mode': 'create'}
[workspace_write] success notes.txt
[tool_observation] write_file
Order 1042 is shipped and cost $42.00, and I saved a one-line note in notes.txt.
[complete] success run_c9fad249e7734f1aae948066a2b5f9a9

Every tool call, whatever its kind, arrives as tool_requested and ends with tool_observation. What ran in between is named by its kind: tool_call and tool_result for your own tools, workspace_read and workspace_write for the workspace file tools. A client that listens only for tool_result misses every file the agent reads or writes. A stream always opens with event: session ("status": "started") and closes with event: session ("status": "ended"). In between, for a run with no tool call, curl -N saw these events, in order:

session user_message guardrail_check memory_read run_configuration runtime_message
memory_write agent_step context_assembly context_message (x2) context_tools model_call
text_delta (x7) model_response memory_write final_answer complete session

text_delta is live only: it is never stored, so a replay does not have it.
Check status on complete before treating the run as a success (it can be error, awaiting_approval or awaiting_budget).
A failure arrives as event: error with error, session_id and run_id.
Closing the connection cancels the run, whatever the request timeout (below).

Look at a run afterwards

curl -s localhost:8771/runs/$RUN_ID

{
    "run_id": "run_3520195e399d490089a6098759efbf03",
    "session_id": "alice",
    "agent_name": "support",
    "agent_version": null,
    "status": "completed",
    "step": 2,
    "trace_ids": [
        "trace_2bd35e0e39a44391ad1b7c8e5424cafd"
    ],
    "tool_calls": [
        {
            "tool_call_id": "call_6qfkMsa5VnxwbLmJJsKrnnH7",
            "parent_tool_call_id": null,
            "tool_name": "get_order",
            "provider": "local",
            "arguments_digest": "2b634eada53fcdec22125302b610d2f5a639d45b86bab3b66bd6667818cba6e5",
            "step": 1,
            "state": "completed",
            "outcome": "success",
            "started_at": "2026-09-27T23:02:06.514575+00:00",
            "ended_at": "2026-09-27T23:02:06.538787+00:00"
        }
    ],
    "usage": {
        "requests": 2,
        "request_tokens": 3033,
        "response_tokens": 32,
        "total_tokens": 3065
    },
    "error": null,
    "created_at": "2026-09-27T23:02:03.632040+00:00",
    "updated_at": "2026-09-27T23:02:08.056437+00:00",
    "heartbeat_at": "2026-09-27T23:02:06.538787+00:00",
    "lease_seconds": 60,
    "attempt": 1,
    "previous_attempts": [],
    "approvals": [],
    "budget_requests": [],
    "outcomes": []
}

The record never includes the saved conversation. For the whole story, GET /runs/{run_id}/trajectory returns segments (one per stretch of a run that paused and resumed), tool_calls, approvals and totals; here, among the totals:

"model_calls": {"total": 2, "agent_turn": 2, "context_summary": 0, "failed": 0}
"tokens": {"input": 3033, "output": 32, "total": 3065, "cached_input": 1024, "reasoning": 0}
"estimated_cost_usd": 0.00172755
"cost_complete": true
"tool_calls": {"total": 1, "by_outcome": {"success": 1, "error": 0, ...}}

The stored events of one run, inside a shared session, as JSON, as a compact trace, or replayed and followed over SSE:

curl -s "localhost:8771/events/alice/list?run_id=$RUN_ID"    # {"session_id", "events", "count"}
curl -s "localhost:8771/events/alice/trace?run_id=$RUN_ID"   # {"session_id", "summary", "steps"}
curl -sN "localhost:8771/events/alice?run_id=$RUN_ID"        # SSE; resume with ?cursor= or Last-Event-ID
curl -s "localhost:8771/telemetry/events?run_id=$RUN_ID&event_type=tool_result"
curl -s "localhost:8771/telemetry/traces?session_id=alice&limit=20"

Use trace_id for one exact trace, run_id for everything one execution did. /telemetry/events returns at most 200 events by default and /telemetry/traces 100; ask for fewer with limit. More in Observability.

A person in the loop

With governance on, a rule that says ask pauses the run and /run/sync returns awaiting_approval. This agent (refund_agent.py) asks before any refund, and has a slow report tool:

import asyncio

from omnicoreagent import OmniCoreAgent, ToolRegistry

tools = ToolRegistry()

@tools.register_tool("get_order")
def get_order(order_id: str) -> dict:
    """Look up an order by its id."""
    return {"order_id": order_id, "status": "shipped", "total": 42.0}

@tools.register_tool("issue_refund")
def issue_refund(order_id: str, amount: float) -> dict:
    """Refund an order."""
    return {"order_id": order_id, "refunded": amount}

@tools.register_tool("build_report")
async def build_report() -> str:
    """Build the daily orders report. It is slow."""
    await asyncio.sleep(30)
    return "12 orders shipped, 1 refunded."

policy = {
    "name": "support-desk",
    "mode": "strict",
    "rules": {
        "allow": [
            {"rule_id": "tools", "capability": "tool.local.call"},
            {"rule_id": "tasks", "capability": "background.task.create"},
            {"rule_id": "runs", "capability": "background.run.start"},
        ],
        "ask": [
            {
                "rule_id": "refunds",
                "capability": "tool.local.call",
                "target": {"tool_name": "issue_refund"},
                "reason": "Refunds need a person.",
            }
        ],
    },
}

agent = OmniCoreAgent(
    name="support",
    system_instruction="You answer questions about orders and refund when asked. Answer in plain text, in one sentence.",
    model_config={"provider": "openai", "model": "gpt-5.4-mini"},
    local_tools=tools,
    agent_config={"governance_config": {"enabled": True, "policy": policy}},
)

OMNICOREAGENT_SERVE_REQUEST_TIMEOUT=20 omniserve run --agent refund_agent.py --port 8773

curl -s -X POST localhost:8773/run/sync -H "Content-Type: application/json" \
  -d '{"query": "Please refund order 1042 in full: 42 dollars.", "session_id": "bob"}'

{
    "status": "awaiting_approval",
    "termination_reason": null,
    "guardrail_result": null,
    "response": null,
    "approvals": [
        {
            "approval_id": "approval_1db56482b7db412b96e07776776d86b4",
            "tool_call_id": "call_LLR9crHj6dOvLFSdlwvzRJKl",
            "tool_name": "issue_refund",
            "capability": "tool.local.call",
            "target": {"path": null, "host": null, "resource": null, "tool_name": "issue_refund", "mcp_server": null},
            "arguments": {"order_id": "1042", "amount": 42},
            "command": null,
            "risk_level": "low",
            "reason": "Refunds need a person.",
            "expires_at": "2026-09-28T23:03:40.248649+00:00",
            "delegated_run_id": null,
            "delegated_name": null
        }
    ],
    "budget_request": null,
    "session_id": "bob",
    "agent_name": "support",
    "metric": null,
    "trace_id": "trace_10a82531b7534851a7e7e24091de6815",
    "run_id": "run_192cf4b607f641cd9f7431befc89b93f"
}

Nothing was refunded. A person decides, then the run resumes:

curl -s -X POST localhost:8773/runs/$RUN_ID/approvals/$APPROVAL_ID \
  -H "Content-Type: application/json" \
  -d '{"decision": "approve", "approver": "alice", "note": "Customer called."}'
curl -s -X POST localhost:8773/runs/$RUN_ID/resume

{"approval_id":"approval_1db56482b7db412b96e07776776d86b4","status":"approved","decision":"approve","tool_call_id":"call_LLR9crHj6dOvLFSdlwvzRJKl","tool_name":"issue_refund","capability":"tool.local.call","target":{"path":null,"host":null,"resource":null,"tool_name":"issue_refund","mcp_server":null},"command":null,"risk_level":"low","reason":"Refunds need a person.","created_at":"2026-09-27T23:03:40.248649+00:00","expires_at":"2026-09-28T23:03:40.248649+00:00","approver":"alice","note":"Customer called.","decided_at":"2026-09-27T23:03:49.543763+00:00","edited_arguments":null,"delegated_run_id":null,"delegated_name":null}
{"status":"success","termination_reason":"stop","guardrail_result":null,"response":"Refunded order 1042 for $42.","approvals":null,"budget_request":null,"session_id":"bob","agent_name":"support","metric":{"requests":1,"request_tokens":1321,"response_tokens":14,"total_tokens":1335,"total_time":1.2692612289974932,"details":{"cached_input_tokens":1024}},"trace_id":"trace_144287d009e34dacbda7f2eed7db7a97","run_id":"run_192cf4b607f641cd9f7431befc89b93f"}

Send "decision": "deny" with a note the model will read, or approve with "arguments": {...} to run an edited call. Everything else about approvals is on Approvals. A run that ran out of budget pauses the same way with awaiting_budget: read it with GET /runs/{run_id}/budget, decide with POST /runs/{run_id}/budget ({"decision": "grant", "approver": "alice", "amount": 5}), then resume. A running run can be steered (POST /runs/{run_id}/steer, {"message": "..."}) or stopped at its next step (POST /runs/{run_id}/interrupt, then resume); see Durable runs.

Background tasks

The served agent is registered for background runs at startup, as agent default. On the same server:

curl -s -X POST localhost:8773/background/tasks -H "Content-Type: application/json" \
  -d '{"task_id": "order_check", "query": "Look up order 1042 and say its status.",
       "schedule": {"type": "manual"}, "timeout_seconds": 120}'
curl -s -X POST localhost:8773/background/tasks/order_check/run \
  -H "Content-Type: application/json" -d '{"wait": true}'

The second call waited for the run and returned it (abridged):

{"run_id":"run_03ff6252cd7a40fba069ee9f1af2ef7f","task_id":"order_check","agent_id":"default","status":"completed","attempt":1,"max_attempts":1,"query_snapshot":"Look up order 1042 and say its status.","trigger_type":"manual",...,"session_id":"background:default:order_check","workspace_path":"background/default/order_check/run_03ff6252cd7a40fba069ee9f1af2ef7f",...,"error":null,"result_preview":"Order 1042 is shipped.",...}

With {"wait": false} (the default) it returns the queued run at once. A schedule is manual, once, interval or cron. Then:

curl -s localhost:8773/background/status
curl -s localhost:8773/background/tasks/order_check/status
curl -s localhost:8773/background/runs/$RUN_ID
curl -s localhost:8773/background/runs/$RUN_ID/events
curl -s localhost:8773/background/runs/$RUN_ID/workspace

{"running":true,"initialized":true,"worker_id":"worker_e8ad38bf0ef340cba2b7150683e4c681","lease_seconds":30.0,"agents":1,"tasks":1,"runs":1,"active_runs":0,"status_counts":{"queued":0,"claimed":0,"running":0,"retrying":0,"completed":1,"failed":0,"cancelled":0,"timeout":0,"skipped":0,"awaiting_approval":0,"awaiting_budget":0}}

The run’s events were background_run_queued, background_run_claimed, background_run_started, background_run_completed; while a run is running it also records a background_run_heartbeat at intervals (a longer run has several). Its workspace held run.json and events.jsonl. Under governance the agent’s policy also decides what an operator asks here: creating, changing, pausing, resuming or deleting a task is background.task.<action>, starting or cancelling a run is background.run.<action>. The policy above allows create and start only, so a DELETE is refused with a 403 (below). Allow them for the operators you trust (policy reference).

A task store that survives restarts

Tasks, runs, attempts, leases and retries live in the background task store, which is not conversation memory (that stays in the agent’s MemoryRouter). The default is in-memory: fine for development, gone on restart. Pick one backend for a deployment:

Use SQL for local durability with SQLite, or PostgreSQL or MySQL when several processes share one store.
Use Redis when your deployment already operates Redis with persistence and no eviction for task-store keys.
Use MongoDB when MongoDB is your durable operational store (it writes with majority concern).

With a durable store, a queued run survives: OmniServe can queue a run, stop, start again with the same task store, and the new worker claims and completes it. The server in the next section kept a cron task across a restart.

Serve from your own code

OmniServe(agent, config=...) gives you the same server from a script. server.py, with a path prefix, a token and a SQLite task store:

from omnicoreagent import OmniCoreAgent, OmniServe, OmniServeConfig

agent = OmniCoreAgent(
    name="support",
    system_instruction="You answer questions about orders. Answer in plain text, in one sentence.",
    model_config={"provider": "openai", "model": "gpt-5.4-mini"},
)

config = OmniServeConfig(
    port=8776,
    api_prefix="/api/v1",
    auth_enabled=True,
    auth_token="change-me",
    background_task_store="sql",
    background_task_store_url="sqlite:///background.db",
)

if __name__ == "__main__":
    OmniServe(agent, config=config).start()

python server.py
curl -s localhost:8776/api/v1/ready
curl -s localhost:8776/ready
curl -s -X POST localhost:8776/api/v1/background/tasks -H "Authorization: Bearer change-me" \
  -H "Content-Type: application/json" \
  -d '{"task_id": "nightly", "query": "Summarise the day in one sentence.", "schedule": {"type": "cron", "expression": "0 2 * * *"}}'
# stop the server, start it again, then:
curl -s localhost:8776/api/v1/background/tasks/nightly/status -H "Authorization: Bearer change-me"

{"ready":true,"agent_name":"support","initialized":true,"mcp_connected":true,"mcp_servers":{}}
{"detail":"Not Found"}
nightly 0 2 * * * 2026-09-28T02:00:00Z

(The last line is the status’s task_id, schedule.expression and schedule_state.next_due_at, after the restart.) The prefix moves the agent’s routes; /docs, /redoc, /openapi.json and /prometheus stay at the root. server.start() blocks; await server.start_async() runs inside your event loop, and server.get_app() returns the FastAPI app to mount elsewhere or test.

OMNICOREAGENT_SERVE_* and OMNICOREAGENT_BACKGROUND_* environment variables always override what the code or the CLI flags set. A stray OMNICOREAGENT_SERVE_PORT in the environment wins over port=8776.

Your own routes beside the agent’s

The agent file (or OmniServe(..., routers=[...], public_paths=[...])) can add an application’s own pages and endpoints, served on the same origin behind the same middleware:

from fastapi import APIRouter
from fastapi.responses import HTMLResponse

router = APIRouter()            # or routers = [...]

@router.get("/steward/", response_class=HTMLResponse)
async def page() -> str:
    return open("page.html").read()

public_paths = ["/steward/"]    # served without a token; "/steward/*" for a directory

Everything else these routers serve needs the bearer token when auth is on. apps/steward is a complete application served this way, and apps/support_desk is a small deployment of one agent for many customers: Postgres, Redis, budgets, a refund that waits for a person’s approval, and a fake model provider to test it with.

How it works

Startup

The server connects the agent’s MCP servers, warms the model client, starts the background manager (registering the agent as default) and its worker. The server answers no request, /health included, until startup has finished, which can take up to a minute on a busy machine. After that, /health says the process is up, and /ready says true while the agent is initialized and its MCP servers are connected.

Each request

Middleware, outermost first: CORS, request logging, the rate limit, the bearer token, the request timeout, and a last catch that turns an uncaught error into a JSON 500. A policy refusal becomes a 403.

A run

/run/sync and /run call agent.run(...) with the session and a new run_id, and record a request trace next to the agent’s own. The run is durable like any other: its record lives in the agent’s memory store and its traces under the workspace (./workspace/telemetry/).

Shutdown

The background manager stops, then agent.cleanup() closes its connections.

One server process serves one agent instance, so workers must be 1. To scale, run several processes (or containers) behind a load balancer, sharing a durable memory store and task store; the rate limit is counted per process. Stores and scale has the details.

Routes

Every route, its body and its response fields are in the HTTP API reference. Paths below are relative to api_prefix (empty by default).

Runs

Method	Route	What it does
`POST`	`/run/sync`	Run and wait; the result as JSON
`POST`	`/run`	Run and stream SSE events, ending with `complete`
`GET`	`/runs`	Runs by `status` and `session_id`, oldest first (`limit`, up to 500): what waits for a person, what a crash left `running` (its `heartbeat_at` and `lease_seconds` say whether it is still alive; `attempt` which try it is). An unknown status is `422`. The answer is `{"runs": [...]}`, each as `GET /runs/{run_id}` returns it; a `running` run whose `heartbeat_at` is older than `lease_seconds` has no live process
`GET`	`/runs/{run_id}`	The run’s record: status, step, usage, tool call states, approvals, and for a run that timed out or failed its `side_effects` (never its conversation)
`GET`	`/runs/{run_id}/trajectory`	The whole run as one story: every trace segment, totals, each tool call once, the approvals
`POST`	`/runs/{run_id}/approvals/{approval_id}`	Decide an approval: `{"decision": "approve" \| "deny", "approver", "note"?, "arguments"?}`
`POST`	`/runs/{run_id}/resume`	Continue a paused or interrupted run once it is decided
`GET`	`/runs/{run_id}/budget`	Every budget covering the run, and each time it ran out
`POST`	`/runs/{run_id}/budget`	Grant or refuse: `{"decision": "grant" \| "deny", "approver", "amount"?, "note"?}`
`POST`	`/runs/{run_id}/steer`	A message for the run’s next step: `{"message", "sender"?}`
`POST`	`/runs/{run_id}/interrupt`	Stop the run at its next step boundary; resume continues it
`POST`	`/runs/{run_id}/outcome`	What the run turned out to be worth: `{"source", "reward"?, "label"?, "detail"?}`

Sessions and events

Method	Route	What it does
`GET`	`/sessions/{session_id}/history`	The session’s messages
`DELETE`	`/sessions/{session_id}`	Clear the session’s messages
`GET`	`/events/{session_id}`	Replay the session’s stored events, then follow live ones (SSE); `?run_id=` for one run, `?cursor=` or `Last-Event-ID` to resume
`GET`	`/events/{session_id}/list`	The stored events as JSON; `?run_id=` for one run
`GET`	`/events/{session_id}/trace`	A compact trace of the session’s latest run, or `?run_id=`

Telemetry

Method	Route	What it does
`GET`	`/telemetry/events`	Events filtered by `trace_id`, `run_id`, `session_id`, `task_id` or `event_type` (`limit` 200)
`GET`	`/telemetry/events/stream`	Replay and follow events over SSE for a `session_id`, optionally one `run_id`
`GET`	`/telemetry/traces`	Traces filtered by trace, run, session, task, agent, workflow, model or status (`limit` 100)
`GET`	`/telemetry/traces/{trace_id}`	One exact trace
`GET`	`/telemetry/traces/{trace_id}/family`	Every trace linked to it
`GET`	`/telemetry/traces/{trace_id}/trajectory`	One trace as a trajectory
`GET`	`/telemetry/runs/{run_id}/trace`	The run’s latest trace
`GET`	`/telemetry/runs/{run_id}/family`	Every trace linked to the run
`GET`	`/telemetry/runs/{run_id}/trajectory`	The run’s latest segment as a trajectory (`/runs/{run_id}/trajectory` is the whole run)
`GET`	`/telemetry/sessions/{session_id}/trace`	The session’s latest trace
`GET`	`/telemetry/retention`	The retention policy and the latest cleanup results

The trace detail routes return 404 when no trace matches.

Health and operations

Method	Route	Token	What it does
`GET`	`/health`	No	The process is up: `status`, `agent_name`, `uptime`, `version`
`GET`	`/ready`	No	Ready for work: startup done, agent initialized, every MCP server connected
`GET`	`/prometheus`	No	Request counters in Prometheus text format, one per route
`GET`	`/metrics`	Yes	The agent’s token and time totals
`GET`	`/tools`	Yes	The agent’s tools, with their input schemas
`GET`	`/docs`	No	Swagger UI (`OMNICOREAGENT_SERVE_ENABLE_DOCS`)
`GET`	`/redoc`	No	ReDoc (`OMNICOREAGENT_SERVE_ENABLE_REDOC`)

/ready’s mcp_servers gives each configured MCP server’s status (connected, disconnected, failed, not_connected) and last error, so one failed server among several is visible. An agent without MCP servers does not need one to be ready.

Background

Mounted unless background is turned off (OMNICOREAGENT_BACKGROUND_ENABLED=false).

Method	Route	What it does
`GET`	`/background/status`	The manager: worker, counts, run status totals
`POST`	`/background/agents`	Register the served agent (again) or an agent spec
`GET`	`/background/agents`	List registered agents
`GET`	`/background/agents/{agent_id}`	One agent spec
`DELETE`	`/background/agents/{agent_id}`	Delete an agent spec
`POST`	`/background/tasks`	Create a task: `task_id`, `query`, `schedule`, and optionally `timeout_seconds`, `retry_policy`, `overlap_policy`, `session_policy`, `workspace_policy`
`GET`	`/background/tasks`	List tasks
`GET`	`/background/tasks/{task_id}`	One task
`GET`	`/background/tasks/{task_id}/status`	Schedule state, run counts, latest run
`PATCH`	`/background/tasks/{task_id}`	Change a task
`POST`	`/background/tasks/{task_id}/run`	Queue a run now: `{"query"?, "wait"?}`
`POST`	`/background/tasks/{task_id}/pause`	Pause its schedule
`POST`	`/background/tasks/{task_id}/resume`	Resume its schedule
`DELETE`	`/background/tasks/{task_id}`	Delete a task
`GET`	`/background/runs`	List runs
`GET`	`/background/runs/{run_id}`	One run’s status
`GET`	`/background/runs/{run_id}/attempts`	Its attempts
`GET`	`/background/runs/{run_id}/events`	Its lifecycle events
`GET`	`/background/runs/{run_id}/workspace`	Its workspace files
`POST`	`/background/runs/{run_id}/cancel`	Cancel a queued or running run
`POST`	`/background/runs/{run_id}/resume`	Queue a run that waited for approval, once decided

Authentication

Off by default. Turn it on with a token:

omniserve run --agent my_agent.py --auth-token s3cret-token
# or
export OMNICOREAGENT_SERVE_AUTH_ENABLED=true
export OMNICOREAGENT_SERVE_AUTH_TOKEN=s3cret-token

Then every route but /health, /ready, /prometheus, /docs, /redoc, /openapi.json and your public_paths needs the header:

curl -s localhost:8774/tools -H "Authorization: Bearer s3cret-token"

There is one token per server; for per-user identity, put OmniServe behind a gateway that authenticates users. CORS is open (*) by default, without credentials; set OMNICOREAGENT_SERVE_CORS_ORIGINS to your front end’s origin (credentials are sent only to origins listed there). Behind a proxy, list its address in OMNICOREAGENT_SERVE_TRUSTED_PROXIES so the rate limit sees each client’s address, not the proxy’s.

Configuration

Flags on omniserve run: --agent, --host, --port, --workers (must be 1), --auth-token, --rate-limit N (requests per minute per client IP), --cors-origins, --no-docs. Everything else is an environment variable or an OmniServeConfig field of the same name, lower-cased without the prefix (OMNICOREAGENT_SERVE_REQUEST_TIMEOUT is request_timeout). With none set, the defaults serve on 0.0.0.0:8000 with no auth, and the background API and its worker run with an in-memory task store.

Variable	Default	What it does
`OMNICOREAGENT_SERVE_HOST`	`0.0.0.0`	Host to bind; must not be empty
`OMNICOREAGENT_SERVE_PORT`	`8000`	Port, `1`-`65535`
`OMNICOREAGENT_SERVE_WORKERS`	`1`	Must be `1`; scale with more processes
`OMNICOREAGENT_SERVE_API_PREFIX`	`""`	Prefix for the agent’s routes, e.g. `/api/v1` (a leading slash is added, a trailing one removed; no whitespace)
`OMNICOREAGENT_SERVE_ENABLE_DOCS`	`true`	Swagger UI at `/docs`
`OMNICOREAGENT_SERVE_ENABLE_REDOC`	`true`	ReDoc at `/redoc`
`OMNICOREAGENT_SERVE_CORS_ENABLED`	`true`	CORS middleware
`OMNICOREAGENT_SERVE_CORS_ORIGINS`	`*`	Allowed origins, comma-separated
`OMNICOREAGENT_SERVE_CORS_METHODS`	`*`	Allowed methods, comma-separated
`OMNICOREAGENT_SERVE_CORS_HEADERS`	`*`	Allowed headers, comma-separated
`OMNICOREAGENT_SERVE_CORS_CREDENTIALS`	`false`	Allow credentials; only with explicit origins, never with `*`
`OMNICOREAGENT_SERVE_AUTH_ENABLED`	`false`	Bearer-token auth; needs a token
`OMNICOREAGENT_SERVE_AUTH_TOKEN`	—	The token
`OMNICOREAGENT_SERVE_RATE_LIMIT_ENABLED`	`false`	Rate limit per client IP
`OMNICOREAGENT_SERVE_TRUSTED_PROXIES`	none	Comma-separated proxy addresses; the client IP is read from `X-Forwarded-For` only on requests from these
`OMNICOREAGENT_SERVE_RATE_LIMIT_REQUESTS`	`100`	Requests per window, at least `1`
`OMNICOREAGENT_SERVE_RATE_LIMIT_WINDOW`	`60`	Window in seconds, at least `1`
`OMNICOREAGENT_SERVE_MAX_CONCURRENT_RUNS`	`24`	Concurrent runs per process; `0` or `none` is unlimited. See how many runs one process takes
`OMNICOREAGENT_SERVE_MALLOC_ARENAS`	`2`	Most glibc malloc arenas the process may use, so its many threads do not each hold freed memory; `0` leaves the allocator alone, and `MALLOC_ARENA_MAX` wins when set.
`OMNICOREAGENT_SERVE_RUN_ADMISSION_WAIT`	`5`	Seconds a request waits for a run slot before `503`
`OMNICOREAGENT_SERVE_REQUEST_LOGGING`	`true`	Log each request; adds `X-Process-Time`
`OMNICOREAGENT_SERVE_LOG_LEVEL`	`INFO`	`CRITICAL`, `ERROR`, `WARNING`, `INFO`, `DEBUG` or `TRACE`
`OMNICOREAGENT_SERVE_REQUEST_TIMEOUT`	`300`	Seconds a request may take; `0` turns it off
`OMNICOREAGENT_SERVE_ORPHAN_SWEEP_ENABLED`	`true`	Resume this agent’s runs whose process died, and runs a person decided that no client resumed (Durable runs); `false` leaves them for an explicit `resume`
`OMNICOREAGENT_SERVE_ORPHAN_SWEEP_INTERVAL_SECONDS`	`30`	Seconds between sweeps, each varying by up to 25%
`OMNICOREAGENT_SERVE_ORPHAN_SWEEP_DECIDED_GRACE_SECONDS`	`30`	Seconds after the last approval or budget decision before the sweep resumes a run no client resumed (a `503`-refused resume that was not retried); `0` or more
`OMNICOREAGENT_SERVE_ORPHAN_SWEEP_MAX_CONCURRENT`	`2`	Recovered runs resumed at once
`OMNICOREAGENT_SERVE_ORPHAN_SWEEP_MAX_RECOVERIES`	`3`	Recoveries of one run before it is ended `failed`
`OMNICOREAGENT_BACKGROUND_ENABLED`	`true`	Mount the background routes
`OMNICOREAGENT_BACKGROUND_AGENT_ID`	`default`	The served agent’s background id
`OMNICOREAGENT_BACKGROUND_START_WORKER`	`true`	Run the scheduler and worker in this process
`OMNICOREAGENT_BACKGROUND_TASK_STORE`	`in_memory`	`in_memory`, `sql`, `redis` or `mongodb`
`OMNICOREAGENT_BACKGROUND_TASK_STORE_URL`	—	SQL or Redis URL (a URL alone means SQL)
`OMNICOREAGENT_BACKGROUND_TASK_STORE_URI`	—	MongoDB URI
`OMNICOREAGENT_BACKGROUND_TASK_STORE_DATABASE`	`omnicoreagent`	MongoDB database
`OMNICOREAGENT_BACKGROUND_TASK_STORE_PREFIX`	—	Redis key prefix
`OMNICOREAGENT_BACKGROUND_TASK_STORE_COLLECTION_PREFIX`	—	MongoDB collection prefix
`OMNICOREAGENT_BACKGROUND_TASK_STORE_CONNECT_TIMEOUT`	—	Seconds to connect to the store

A durable task store; pick one backend:

export OMNICOREAGENT_BACKGROUND_TASK_STORE=sql
export OMNICOREAGENT_BACKGROUND_TASK_STORE_URL=sqlite:///.omnicoreagent/background.db

export OMNICOREAGENT_BACKGROUND_TASK_STORE=redis
export OMNICOREAGENT_BACKGROUND_TASK_STORE_URL=redis://localhost:6379/0

export OMNICOREAGENT_BACKGROUND_TASK_STORE=mongodb
export OMNICOREAGENT_BACKGROUND_TASK_STORE_URI=mongodb://localhost:27017
export OMNICOREAGENT_BACKGROUND_TASK_STORE_DATABASE=omnicoreagent

SQL needs omnicoreagent[postgres] (SQLAlchemy, for SQLite too), Redis omnicoreagent[redis], MongoDB omnicoreagent[mongodb]. omniserve config --show prints the configuration the environment gives; omniserve config --env-example prints a template. Every option is in the CLI reference and configuration.

The request timeout

OMNICOREAGENT_SERVE_REQUEST_TIMEOUT (300 seconds) bounds each request:

/run/sync and /runs/{run_id}/resume answer 504 when the run takes longer. The run is stopped; its record and trace remain.
/run (SSE) sends event: error with "error": "Request timed out" and closes the stream.
After a timeout or a failure, look at side_effects before doing anything else. It is in the 504’s detail, in the SSE error event (and the complete event of a run that ended in error), in a /run/sync answer with status: error, and in GET /runs/{run_id}: the calls the run already made that may have changed something outside it, as {"tool_name", "tool_call_id", "outcome"} with outcome success or unknown. It lists only tools that are not idempotent, and never their arguments. To continue, resume the run (POST /runs/{run_id}/resume): it does not repeat a completed call. A new run starts over and can repeat them.
/background/tasks/{task_id}/run with "wait": true waits a little less than the timeout, then answers 504 with the run_id. The run is not stopped: it carries on in the background. A run that pauses for a person or a budget is answered at once, 200 with awaiting_approval or awaiting_budget.

For work longer than a request should last, use a background task.

How many runs one process takes

A process takes 24 concurrent runs by default. Past that, a run waits for a slot instead of slowing every other run down. The number is per process. It does not grow with the CPU count: more CPUs means more processes (see scale out). What was measured. One OmniServe process on a 2-CPU cap, with the limit lifted, on the support desk app with a Postgres memory store. The model was the fake provider at 0.8 to 3 seconds a call. Users ask, ask for refunds, and wait for approval, then start again.

Concurrent users	Throughput	p95
8	2.1/s	7.5 s
16	4.0/s	7.7 s
24	5.1/s	8.9 s
32	5.5/s	11.1 s
48	5.4/s	17.2 s
64	5.0/s	22.8 s

The knee is about 24. There the process gives about 93% of its peak throughput, and p95 is within 16% of idle. Past 32, throughput falls while latency climbs. A real model is usually slower than 0.8 to 3 seconds, so each run waits longer and uses less of the process: with a slower model, more runs fit before the knee. Why one process stops at about one core. The process used about 1.1 cores on a 2-CPU cap. One event loop runs on one thread, so a second core does not help it. Raising the cap does not raise the knee.

Over the limit, a request waits up to OMNICOREAGENT_SERVE_RUN_ADMISSION_WAIT (5 seconds) for a slot. If none frees, it gets 503 with a Retry-After header and {"error": "ServerBusy", "max_concurrent_runs": N, ...}. Nothing was started for it, so retrying is safe. An SSE request gets the same answer before the stream starts.
What counts: /run, /run/sync, /runs/{run_id}/resume and /background/tasks/{task_id}/run. A slot is held while the run executes (for /run, until the stream ends). Health, readiness, /prometheus, reads, approvals and steering never wait.
To see it, /prometheus has omniserve_runs_in_flight, omniserve_runs_waiting, omniserve_runs_limit and omniserve_runs_rejected_total.

What a server under load reports

/prometheus also counts what the runs did. Every figure only grows, since the process started; use rate() or increase() on it. Labels come from small fixed sets, never a run id, a session id or a tool argument.

Metric	Labels	Counts
`omniserve_request_duration_seconds`	`le`	A histogram of request time (`_bucket`, `_sum`, `_count`).
`omniserve_runs_finished_total`	`status`, `reason`	Run segments that ended, by the run’s status (`completed`, `failed`, `timeout`, `awaiting_approval`, `awaiting_budget`, `interrupted`, `blocked`) and its `termination_reason` (`stop`, `provider_error`, `max_steps`, `budget_exhausted`, …; `none` when a pause has none). A run that pauses and resumes counts once per segment.
`omniserve_runs_resumed_total`	`trigger`, `cause`	Resumes. `trigger` is `explicit` (a client called resume), `orphan_sweep` or `background_retry`; `cause` is why: `approval`, `budget_grant`, `budget_denied`, `interrupted`, `recovered_after_lapsed_lease`.
`omniserve_model_calls_total`	`model`	Model calls made.
`omniserve_model_errors_total`	`model`	Model calls that raised after the provider’s own retries.
`omniserve_model_retries_total`	`model`	Retries inside model calls (429s, timeouts).
`omniserve_budget_refused_total`	`scope`, `meter`, `action`	Times a budget stopped a run. `action` is `pause` (waits for a person) or `stop`.
`omniserve_approvals_requested_total`	`risk`	Approvals asked for.
`omniserve_approvals_decided_total`	`decision`	Approvals decided: `approve`, `deny` or `expired`.

A label that would take a hundred distinct values is counted under other.

Scale out: one replica per core

Run one replica per core behind a load balancer, all sharing the same memory, telemetry and task stores. Each replica applies its own limit of 24. The same 2-core budget, split into two replicas of 1 CPU each, behind nginx (least_conn), sharing one Postgres:

Concurrent users	Throughput	p95
24	6.2/s	7.3 s
32	7.7/s	8.1 s
48	9.4/s	9.8 s
64	9.3/s	13.7 s

The peak was 9.4/s against 5.5/s for one process: about 1.7 times the throughput on the same cores. Each replica bends at about 24, so the pair bends at about 48. In that run, 1,087 approved refunds made 1,087 ledger rows: each happened once, across both replicas. Durability is what makes this safe. A run’s lease says which replica owns it. The orphan sweep resumes a run whose replica died. Saves are version-checked, so two replicas cannot overwrite each other’s work. A request does not need to return to the replica that started the run. The support desk has a working example: apps/support_desk/scale/.

When to change the limit

Lower it if p95 climbs while throughput is flat or falling. The process is past its knee, and a smaller limit turns the extra waiting into a quick 503 with Retry-After.
Raise it if the runs mostly wait on a slow model, omniserve_runs_waiting is often above zero, and the process is well under one core. Raise it a step at a time and measure.
Set 0 or none for no limit. Use it to find your own knee, not in production.

To find your own knee, set OMNICOREAGENT_SERVE_MAX_CONCURRENT_RUNS=0 and run the support desk’s load harness at stages such as --stages 8:120,16:120,24:120,32:120,48:120 (see apps/support_desk/load/README.md), with your model and your stores. The knee is the stage where throughput stops rising. Set the limit there.

What the runtime adds to each step

On the same 2-core container, with 10 concurrent users and a fake model, the runtime’s own time per step (the step’s time minus the model’s) was 81 ms at p50 and 182 ms at p95. Most of it is CPU in the event loop: about 100 ms a step, mostly in the model client library and the telemetry write. A real model call takes 1 to 10 seconds, so the runtime is a few percent of a run. It is also why one process bends at about 24 concurrent runs: the event loop is the limit, not the database. For runs longer than a caller should hold a connection, submit then stream: start a background task run ("wait": false) and follow it from the events routes.

Deploy with Docker

omniserve generate-dockerfile --file my_agent.py
docker build -t omnicoreagent-serve .
docker run -p 8000:8000 -e LLM_API_KEY omnicoreagent-serve

It also writes a .dockerignore in the directory you run it from (the build context), if there is none, that keeps credential files (.env, .env.*, *.pem, *.key, id_rsa*, .netrc, credentials*.json and the like, in any subfolder) and the configured workspace folder (at the root only) out of the image (the image copies the build context); an existing one that does not exclude **/.env is named. The generated Dockerfile installs omnicoreagent[serve], pinned to the release that generated it (==0.5.0), on python:3.12-slim, copies the build context to /app, runs as a non-root user, checks /health, and keeps the workspace in /tmp/workspace (gone with the container unless you mount a volume). Its HEALTHCHECK allows a start period of 10 seconds (--start-period=10s); a slow host may need longer before /health first answers, so raise it in the Dockerfile or with docker run --health-start-period=60s. Add the extras your agent needs ([serve,postgres] for a SQL memory store): a comment above the install line in the Dockerfile says so. A persistent volume, a store for runs and readiness checks are on Running the agent in a container.

When things go wrong

401 Missing or invalid Authorization header

Auth is on and the request had no Authorization: Bearer ... header, or the wrong token:

{"error":"Unauthorized","message":"Missing or invalid Authorization header"}
{"error":"Unauthorized","message":"Invalid authentication token"}

403 Denied by strict policy

The agent’s policy refused what the request asked, here a task DELETE the policy does not allow:

{"error":"UnknownCapabilityError","detail":"Denied by strict policy: no rule matches this background.task.delete request.","capability":"background.task.delete","reason_code":"unknown_capability","matched_rule_ids":[]}

The body names the refused capability, the reason_code and the matched_rule_ids (none here: no rule matched). Add an allow rule for that capability (background.task.delete), for the operators you trust.An ask nobody can answer inside the request is refused the same way, and the body names the capability and the rule. Under interactive-dev, every background request asks:

{"error":"ApprovalRequiredError","detail":"Matched ask policy rule.","capability":"background.task.create","reason_code":"approval_required","matched_rule_ids":["ask_background_execution"]}

An ask outranks an allow, and interactive-dev asks about anything no rule matches: for an agent served to operators you trust, take the ask rule out and allow the capability.

from omnicoreagent.governance import PolicyEffect, PolicyRule, build_default_policy

policy = build_default_policy("interactive-dev")
policy.rules.ask = [r for r in policy.rules.ask if r.rule_id != "ask_background_execution"]
policy.rules.allow.append(
    PolicyRule(rule_id="operators_run_background", effect=PolicyEffect.ALLOW, capability="background.*")
)

With both, POST /background/tasks returns 200.

404 No run / Task not found

{"detail":"No run run_nope"}
{"detail":"No approval approval_nope on run run_192cf4b607f641cd9f7431befc89b93f"}
{"detail":"Task not found: nope"}

A run’s record lives in the agent’s memory store: with the default in-memory store, a restart forgets every run. Use a durable memory store.

409 still waiting, already decided, already finished

The request does not fit the run’s state:

{"detail":"Run run_192cf4b607f641cd9f7431befc89b93f is still waiting for approval: approval_1db56482b7db412b96e07776776d86b4"}
{"detail":"Approval approval_1db56482b7db412b96e07776776d86b4 is already approved"}
{"detail":"Approval approval_1db56482b7db412b96e07776776d86b4 is already used"}
{"detail":"Run run_192cf4b607f641cd9f7431befc89b93f is completed; only a waiting or stopped run can resume"}
{"detail":"Task already registered: order_check"}

An approval reads already approved (or denied) once decided, and already used once a resume has applied the decision. Decide every approval before resuming; send "replace": true to overwrite a task.

422 a body that does not validate

{"detail":[{"type":"literal_error","loc":["body","decision"],"msg":"Input should be 'approve' or 'deny'","input":"maybe","ctx":{"expected":"'approve' or 'deny'"}}]}

/runs/{run_id}/steer also answers 422 when the injection guardrail blocks the message.

429 Rate limit exceeded

With --rate-limit 3, the fourth request in a minute:

HTTP/1.1 429 Too Many Requests
x-ratelimit-limit: 3
x-ratelimit-remaining: 0
x-ratelimit-reset: 1790550444
retry-after: 59
{"error":"TooManyRequests","message":"Rate limit exceeded. Max 3 requests per 60 seconds.","retry_after":59}

Every request but the public paths counts, rejected ones (a 401) included.

504 Request timed out

A /run/sync under a 1-second timeout, with a model slower than that:

{"detail":{"message":"Request timed out after 1 seconds","run_id":"run_bb48117c442e48c0b63efaa7beee4b4a","session_id":"omni_core_agent_slow_ceb1792e","request_timeout_seconds":1}}

The run is stopped and its record says timeout (GET /runs/{run_id}); the run_id finds its trajectory. If the run had already made a call that is not idempotent, detail also has "side_effects": [{"tool_name": "issue_refund", "tool_call_id": "call_1", "outcome": "success"}]: do not start a new run for the same request, which would issue the refund again. POST /runs/{run_id}/resume continues the timed-out run (it answers a timeout the same way if it takes too long again).A background run waited on with "wait": true:

{"detail":{"message":"Background run did not finish within 18.0 seconds","run_id":"run_9d9ac5f1f756447e856c90b93ebb5e2f","task_id":"daily_report","status":"running","wait_timeout_seconds":18.0,"request_timeout_seconds":20}}

That run finished later (GET /background/runs/{run_id} said completed). Raise OMNICOREAGENT_SERVE_REQUEST_TIMEOUT, or queue long work without waiting.

The server will not start: Invalid OmniServe config

The configuration is checked before the agent is loaded:

Error: Invalid OmniServe config: 1 validation error for OmniServeConfig
  Value error, OMNICOREAGENT_SERVE_AUTH_TOKEN is required when OMNICOREAGENT_SERVE_AUTH_ENABLED=true [type=value_error, ...]
Error: Invalid OmniServe config: 1 validation error for OmniServeConfig
  Value error, OMNICOREAGENT_SERVE_WORKERS must be 1 for direct OmniServe. Run multiple OmniServe processes behind a process manager for horizontal scaling. [type=value_error, ...]

A Redis or MongoDB task store without its address is refused the same way: Error: Invalid OmniServe config: Redis background task store requires OMNICOREAGENT_BACKGROUND_TASK_STORE_URL (..._URI for MongoDB).

Agent file must define an 'agent' variable or 'create_agent()' function

Error: Agent file must define an 'agent' variable or 'create_agent()' function
Error: Invalid value for '--agent' / '-a': Path 'missing.py' does not exist.

An exception while importing the file is reported as Error loading agent file: ....

Application startup failed: LLM_API_KEY not found

The model key is read when the agent is built; without it the server exits:

ValueError: LLM_API_KEY not found in environment variables

ERROR:    Application startup failed. Exiting.

Set LLM_API_KEY in the server’s environment. A key the provider rejects starts fine, and each run ends with status error, termination_reason provider_error.

OmniServe cannot listen on host:port: it is in use

Another process holds the port. The port is checked before the agent is loaded, so this comes at once:

Error: OmniServe cannot listen on 0.0.0.0:8771: it is in use or not available (Address already in use); choose another port

Pick another --port, or stop what holds it.

Running the agent in a container

An image, a volume, a store for runs, health checks.

Durable runs

Run records, pause and resume, recovery after a crash.

Stores and scale

Memory and task stores, and many processes sharing them.

Background agents

Schedules, retries, leases and the task store.

Approvals

Everything about a person in the loop.

HTTP API reference

Every route, body and response field.

​OmniServe

​Serve an agent

​Stream a run

​Look at a run afterwards

​A person in the loop

​Background tasks

​A task store that survives restarts

​Serve from your own code

​Your own routes beside the agent’s

​How it works

​Routes

​Runs

​Sessions and events

​Telemetry

​Health and operations

​Background

​Authentication

​Configuration

​The request timeout

​How many runs one process takes

​What a server under load reports

​Scale out: one replica per core

​When to change the limit

​What the runtime adds to each step

​Deploy with Docker

​When things go wrong

​Next

Running the agent in a container

Durable runs

Stores and scale

Background agents

Approvals

HTTP API reference

OmniServe

Serve an agent

Stream a run

Look at a run afterwards

A person in the loop

Background tasks

A task store that survives restarts

Serve from your own code

Your own routes beside the agent’s

How it works

Routes

Runs

Sessions and events

Telemetry

Health and operations

Background

Authentication

Configuration

The request timeout

How many runs one process takes

What a server under load reports

Scale out: one replica per core

When to change the limit

What the runtime adds to each step

Deploy with Docker

When things go wrong

Next