OmniServe — Production API Server

Turn any agent into a production-ready REST/SSE API with a single command. OmniServe is an optional production extra:

pip install "omnicoreagent[serve]"

Agent File Requirements

Your Python file must define one of the following:

# Option 1: Define an `agent` variable
from omnicoreagent import OmniCoreAgent

agent = OmniCoreAgent(
    name="MyAgent",
    system_instruction="You are a helpful assistant.",
    model_config={"provider": "openai", "model": "gpt-4o-mini"},
)

# Option 2: Define a `create_agent()` function
from omnicoreagent import OmniCoreAgent

def create_agent():
    """Factory function that returns an agent instance."""
    return OmniCoreAgent(
        name="MyAgent",
        system_instruction="You are a helpful assistant.",
        model_config={"provider": "openai", "model": "gpt-4o-mini"},
    )

OmniServe looks for agent variable first, then create_agent() function. Your file must export one of these.

Quick Start

Step 1: Create your agent file (`my_agent.py`)

from omnicoreagent import OmniCoreAgent, ToolRegistry

tools = ToolRegistry()

@tools.register_tool("greet")
def greet(name: str) -> str:
    """Greet someone by name."""
    return f"Hello, {name}!"

@tools.register_tool("calculate")
def calculate(expression: str) -> dict:
    """Evaluate a math expression."""
    import math
    result = eval(expression, {"__builtins__": {}}, {"sqrt": math.sqrt, "pi": math.pi})
    return {"expression": expression, "result": result}

agent = OmniCoreAgent(
    name="MyAgent",
    system_instruction="You are a helpful assistant with access to greeting and calculation tools.",
    model_config={"provider": "openai", "model": "gpt-4o-mini"},
    local_tools=tools,
)

Step 2: Set environment variables

export LLM_API_KEY=your_api_key_here

Step 3: Run the server

omniserve run --agent my_agent.py

Step 4: Test the API

# Health check
curl http://localhost:8000/health

# Run a query (sync)
curl -X POST http://localhost:8000/run/sync \
  -H "Content-Type: application/json" \
  -d '{"query": "Greet Alice and calculate 2+2"}'

# Run a query (streaming SSE)
curl -X POST http://localhost:8000/run \
  -H "Content-Type: application/json" \
  -d '{"query": "What is sqrt(144)?"}'

# Open interactive docs
open http://localhost:8000/docs

CLI Commands

Command	Description
`omniserve run`	Run your agent file as API server
`omniserve quickstart`	Zero-code server with defaults
`omniserve config`	View or generate configuration
`omniserve generate-dockerfile`	Generate production Dockerfile

CLI Options: `omniserve run`

omniserve run \
  --agent my_agent.py \
  --host 0.0.0.0 \
  --port 8000 \
  --workers 1 \
  --auth-token YOUR_TOKEN \
  --rate-limit 100 \
  --cors-origins "*" \
  --no-docs

Examples:

# Basic run
omniserve run --agent my_agent.py

# With authentication
omniserve run --agent my_agent.py --auth-token secret123

# With rate limiting
omniserve run --agent my_agent.py --rate-limit 100

# Production settings
omniserve run --agent my_agent.py \
  --port 8000 \
  --auth-token $AUTH_TOKEN \
  --rate-limit 100 \
  --cors-origins "https://myapp.com,https://api.myapp.com"

CLI Options: `omniserve quickstart`

Start a server instantly without writing any code:

omniserve quickstart \
  --provider openai \
  --model gpt-4o \
  --name QuickAgent \
  --instruction "You are..." \
  --port 8000

Use the provider that matches your LLM_API_KEY. For an OpenAI key, run omniserve quickstart --provider openai --model gpt-4o-mini. Examples:

# OpenAI
omniserve quickstart --provider openai --model gpt-4o

# Google Gemini
omniserve quickstart --provider gemini --model gemini-2.0-flash

# Anthropic Claude
omniserve quickstart --provider anthropic --model claude-3-5-sonnet-20241022

API Endpoints

Core Endpoints

Method	Endpoint	Auth	Description
`POST`	`/run`	Yes*	SSE streaming response
`POST`	`/run/sync`	Yes*	JSON response (blocking)
`GET`	`/health`	No	Health check
`GET`	`/ready`	No	Readiness check
`GET`	`/prometheus`	No	Prometheus metrics
`GET`	`/tools`	Yes*	List available tools
`GET`	`/metrics`	Yes*	Agent usage metrics
`GET`	`/events/{session_id}`	Yes*	Replay stored telemetry events, then follow live telemetry events over SSE; add `?run_id=...` to isolate one run
`GET`	`/events/{session_id}/list`	Yes*	Return stored telemetry events as JSON; add `?run_id=...` to isolate one run
`GET`	`/events/{session_id}/trace`	Yes*	Compact telemetry trace summary for the latest session trace, or `?run_id=...` for one run
`GET`	`/telemetry/events`	Yes*	Return stored telemetry events filtered by `trace_id`, `run_id`, `session_id`, `task_id`, or `event_type`
`GET`	`/telemetry/events/stream`	Yes*	Replay and follow telemetry events over SSE for `session_id`; add `run_id` to isolate one run
`GET`	`/telemetry/traces`	Yes*	List traces filtered by trace, run, session, task, agent, workflow, model, or status
`GET`	`/telemetry/traces/{trace_id}`	Yes*	Return one exact trace
`GET`	`/telemetry/runs/{run_id}/trace`	Yes*	Return the latest trace correlated to one run
`GET`	`/telemetry/sessions/{session_id}/trace`	Yes*	Return the latest trace for one session
`GET`	`/docs`	No	Swagger UI
`GET`	`/redoc`	No	ReDoc UI

/ready returns ready: true only after the FastAPI lifespan has completed startup, the served agent is initialized, and any configured MCP servers are connected. Agents without MCP servers do not need an MCP client to pass readiness.

Background Task Endpoints

These routes are mounted when background execution is enabled. It is enabled by default and can be turned off with OMNICOREAGENT_BACKGROUND_ENABLED=false or OmniServeConfig(background_enabled=False).

Method	Endpoint	Auth	Description
`GET`	`/background/status`	Yes*	Inspect background manager counts and run status totals
`POST`	`/background/agents`	Yes*	Register the served agent or an agent spec for background execution
`GET`	`/background/agents`	Yes*	List registered background agents
`GET`	`/background/agents/{agent_id}`	Yes*	Inspect a background agent spec
`DELETE`	`/background/agents/{agent_id}`	Yes*	Delete a background agent spec
`POST`	`/background/tasks`	Yes*	Create a background task
`GET`	`/background/tasks`	Yes*	List background tasks
`GET`	`/background/tasks/{task_id}`	Yes*	Inspect a background task
`GET`	`/background/tasks/{task_id}/status`	Yes*	Inspect schedule state, run counts, and latest run
`PATCH`	`/background/tasks/{task_id}`	Yes*	Patch a background task
`POST`	`/background/tasks/{task_id}/run`	Yes*	Queue a manual background run, optionally waiting for terminal state
`POST`	`/background/tasks/{task_id}/pause`	Yes*	Pause scheduled dispatch
`POST`	`/background/tasks/{task_id}/resume`	Yes*	Resume scheduled dispatch
`DELETE`	`/background/tasks/{task_id}`	Yes*	Delete a background task
`POST`	`/background/runs/{run_id}/cancel`	Yes*	Cancel a queued or running run
`GET`	`/background/runs`	Yes*	List background runs
`GET`	`/background/runs/{run_id}`	Yes*	Inspect run status
`GET`	`/background/runs/{run_id}/attempts`	Yes*	List run attempts
`GET`	`/background/runs/{run_id}/events`	Yes*	Replay lifecycle events
`GET`	`/background/runs/{run_id}/workspace`	Yes*	Inspect run workspace files

*Auth required only if --auth-token is set or OMNICOREAGENT_SERVE_AUTH_ENABLED=true with OMNICOREAGENT_SERVE_AUTH_TOKEN.

Request/Response Examples

# Sync request (with auth)
curl -X POST http://localhost:8000/run/sync \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -d '{"query": "What is 2+2?", "session_id": "user123"}'

# Response:
# {"response": "2+2 equals 4", "session_id": "user123", ...}

# Streaming SSE request
curl -X POST http://localhost:8000/run \
  -H "Content-Type: application/json" \
  -d '{"query": "Explain quantum computing"}'

# Replay and follow one session's events
curl -N http://localhost:8000/events/user123

# Replay one run inside a shared session
curl -N "http://localhost:8000/events/user123?run_id=RUN_ID"
curl "http://localhost:8000/events/user123/list?run_id=RUN_ID"
curl "http://localhost:8000/events/user123/trace?run_id=RUN_ID"

# Production telemetry API
curl "http://localhost:8000/telemetry/events?session_id=user123&run_id=RUN_ID"
curl "http://localhost:8000/telemetry/events?run_id=RUN_ID&event_type=tool_result&limit=100"
curl "http://localhost:8000/telemetry/traces?session_id=user123&limit=20"
curl "http://localhost:8000/telemetry/traces/TRACE_ID"
curl "http://localhost:8000/telemetry/runs/RUN_ID/trace"
curl "http://localhost:8000/telemetry/sessions/user123/trace"
curl -N "http://localhost:8000/telemetry/events/stream?session_id=user123&run_id=RUN_ID"

# List tools
curl http://localhost:8000/tools \
  -H "Authorization: Bearer YOUR_TOKEN"

For telemetry traces, trace_id is the exact lookup handle returned by the agent runtime. run_id is the serving/runtime correlation handle returned from /run/sync and SSE completion payloads. Use trace_id when you need one exact trace. Use run_id when your UI or application needs all telemetry for one execution inside a shared session. /telemetry/events defaults to limit=200; /telemetry/traces defaults to limit=100. Both accept lower limits per request. Exact trace, run trace, and session trace detail endpoints return 404 when no matching trace exists or an accessor returns a trace that does not match the requested selector.

Background Task Example

OmniServe registers the served agent as a background-capable agent during server startup. The default background agent id is default; override it with OMNICOREAGENT_BACKGROUND_AGENT_ID or OmniServeConfig(background_agent_id="...").

curl -X POST http://localhost:8000/background/tasks \
  -H "Content-Type: application/json" \
  -d '{
    "task_id": "daily_report",
    "query": "Write today'\''s operations report and save the output.",
    "schedule": {"type": "manual"},
    "timeout_seconds": 120,
    "retry_policy": {"max_retries": 1, "initial_delay_seconds": 30}
  }'

curl -X POST http://localhost:8000/background/tasks/daily_report/run \
  -H "Content-Type: application/json" \
  -d '{"wait": false}'

Set {"wait": true} when the response should wait for terminal run state. If the run does not finish before the background wait budget, OmniServe returns 504 with the run_id, latest status, wait_timeout_seconds, and request_timeout_seconds in detail; use the run_id with /background/runs/{run_id} to inspect the run later. If auth is enabled with --auth-token or OMNICOREAGENT_SERVE_AUTH_ENABLED=true, add -H "Authorization: Bearer YOUR_TOKEN" to protected requests. Each run stores operational state in the task store and writes lifecycle files into the configured workspace namespace. Use the default in-memory task store for local development, or choose sql, redis, or mongodb when runs must survive restarts. Durable stores preserve queued runs across server restarts: OmniServe can queue a run, stop, start again with the same task store, and the new manager can claim and complete that run. Choose one durable backend per deployment. Use SQL/SQLite for local durability or simple single-node services. Use Redis when your deployment already operates Redis with persistence and no eviction for task-store keys. Use MongoDB when MongoDB is your durable operational store. Common inspection endpoints:

curl http://localhost:8000/background/status
curl http://localhost:8000/background/tasks/daily_report/status
curl http://localhost:8000/background/runs/$RUN_ID
curl http://localhost:8000/background/runs/$RUN_ID/events
curl http://localhost:8000/background/runs/$RUN_ID/workspace

Environment Variables

Server settings use the OMNICOREAGENT_SERVE_* prefix. Background task settings use the OMNICOREAGENT_BACKGROUND_* prefix. Environment variables always override code values. You can run OmniServe without adding any OMNICOREAGENT_SERVE_* variables. The defaults enable the background API, start the worker, and use in-memory background task state. Add variables only when you want to change server behavior, authentication, rate limits, or background storage.

Variable	Default	Description
`OMNICOREAGENT_SERVE_HOST`	`0.0.0.0`	Server host. Must not be empty
`OMNICOREAGENT_SERVE_PORT`	`8000`	Server port. Must be `1`-`65535`
`OMNICOREAGENT_SERVE_WORKERS`	`1`	Direct OmniServe worker count. Must be `1`; scale by running multiple processes
`OMNICOREAGENT_SERVE_API_PREFIX`	`""`	API path prefix (e.g., `/api/v1`). Normalized to a leading slash with no trailing slash; whitespace is invalid
`OMNICOREAGENT_SERVE_ENABLE_DOCS`	`true`	Swagger UI at `/docs`
`OMNICOREAGENT_SERVE_ENABLE_REDOC`	`true`	ReDoc at `/redoc`
`OMNICOREAGENT_SERVE_CORS_ENABLED`	`true`	Enable CORS
`OMNICOREAGENT_SERVE_CORS_ORIGINS`	`*`	Allowed origins (comma-separated)
`OMNICOREAGENT_SERVE_CORS_METHODS`	`*`	Allowed methods (comma-separated)
`OMNICOREAGENT_SERVE_CORS_HEADERS`	`*`	Allowed headers (comma-separated)
`OMNICOREAGENT_SERVE_CORS_CREDENTIALS`	`true`	Allow credentials
`OMNICOREAGENT_SERVE_AUTH_ENABLED`	`false`	Enable Bearer token auth. Requires a non-empty auth token
`OMNICOREAGENT_SERVE_AUTH_TOKEN`	—	Bearer token value used when auth is enabled
`OMNICOREAGENT_SERVE_RATE_LIMIT_ENABLED`	`false`	Enable rate limiting
`OMNICOREAGENT_SERVE_RATE_LIMIT_REQUESTS`	`100`	Requests per window. Must be at least `1` when enabled
`OMNICOREAGENT_SERVE_RATE_LIMIT_WINDOW`	`60`	Window in seconds. Must be at least `1` when enabled
`OMNICOREAGENT_SERVE_REQUEST_LOGGING`	`true`	Log requests
`OMNICOREAGENT_SERVE_LOG_LEVEL`	`INFO`	Log level: `CRITICAL`, `ERROR`, `WARNING`, `INFO`, `DEBUG`, or `TRACE`
`OMNICOREAGENT_SERVE_REQUEST_TIMEOUT`	`300`	Request timeout in seconds
`OMNICOREAGENT_BACKGROUND_ENABLED`	`true`	Expose background task endpoints
`OMNICOREAGENT_BACKGROUND_AGENT_ID`	`default`	Agent id for the served agent in background tasks
`OMNICOREAGENT_BACKGROUND_TASK_STORE`	`in_memory`	Background control-plane store: `in_memory`, `sql`, `redis`, or `mongodb`
`OMNICOREAGENT_BACKGROUND_TASK_STORE_URL`	—	SQL or Redis URL. Use `OMNICOREAGENT_BACKGROUND_TASK_STORE=redis` for Redis URLs
`OMNICOREAGENT_BACKGROUND_TASK_STORE_URI`	—	MongoDB URI
`OMNICOREAGENT_BACKGROUND_TASK_STORE_DATABASE`	`omnicoreagent`	MongoDB database name
`OMNICOREAGENT_BACKGROUND_TASK_STORE_PREFIX`	—	Redis key prefix
`OMNICOREAGENT_BACKGROUND_TASK_STORE_COLLECTION_PREFIX`	—	MongoDB collection prefix
`OMNICOREAGENT_BACKGROUND_TASK_STORE_CONNECT_TIMEOUT`	—	Backend connect timeout in seconds
`OMNICOREAGENT_BACKGROUND_START_WORKER`	`true`	Start scheduler and worker loop during server lifespan

The background task store is not conversation memory. Memory stays in MemoryRouter. The task store is the control plane for schedules, runs, attempts, leases, retries, and cancellation. Leave it at the in-memory default to start; choose sql, redis, or mongodb when that state must survive restarts. Redis durable deployments need persistence enabled and a no-eviction policy for task-store keys. MongoDB durable deployments use majority writes.

Example shell environment:

# Model credential
export LLM_API_KEY=your_api_key_here

# Optional OmniServe overrides
export OMNICOREAGENT_SERVE_PORT=8000
export OMNICOREAGENT_SERVE_AUTH_ENABLED=true
export OMNICOREAGENT_SERVE_AUTH_TOKEN=my-secret-token
export OMNICOREAGENT_SERVE_RATE_LIMIT_ENABLED=true
export OMNICOREAGENT_SERVE_RATE_LIMIT_REQUESTS=100
export OMNICOREAGENT_SERVE_CORS_ORIGINS=https://myapp.com,https://api.myapp.com
export OMNICOREAGENT_SERVE_CORS_METHODS=GET,POST,OPTIONS
export OMNICOREAGENT_SERVE_CORS_HEADERS=Authorization,Content-Type

# Optional durable background task store. Pick one backend.
export OMNICOREAGENT_BACKGROUND_TASK_STORE=sql
export OMNICOREAGENT_BACKGROUND_TASK_STORE_URL=sqlite:///.omnicoreagent/background.db

export OMNICOREAGENT_BACKGROUND_TASK_STORE=redis
export OMNICOREAGENT_BACKGROUND_TASK_STORE_URL=redis://localhost:6379/0

export OMNICOREAGENT_BACKGROUND_TASK_STORE=mongodb
export OMNICOREAGENT_BACKGROUND_TASK_STORE_URI=mongodb://localhost:27017
export OMNICOREAGENT_BACKGROUND_TASK_STORE_DATABASE=omnicoreagent

Docker Deployment

Generate a Dockerfile

omniserve generate-dockerfile --file my_agent.py

Build and run

docker build -t omnicoreagent-serve .
docker run -p 8000:8000 -e LLM_API_KEY=$LLM_API_KEY omnicoreagent-serve

The generator creates a Dockerfile for the current project directory and stores only non-sensitive defaults in the image:

Setting	Value
`AGENT_PATH`	In-container path to the selected agent file
`OMNICOREAGENT_WORKSPACE_BACKEND`	`local`
`OMNICOREAGENT_WORKSPACE_DIR`	`/tmp/workspace`

The agent file must be inside the current Docker build context. The generator does not import or execute the agent file. S3/R2 workspace credentials are passed at runtime with -e, never baked into the image.

Cloud deployment examples

# Local workspace (ephemeral)
docker run -p 8000:8000 -e LLM_API_KEY=$LLM_API_KEY omnicoreagent-serve

# AWS S3 workspace (persistent)
docker run -p 8000:8000 \
  -e LLM_API_KEY=$LLM_API_KEY \
  -e OMNICOREAGENT_WORKSPACE_BACKEND=s3 \
  -e AWS_S3_BUCKET=my-bucket \
  -e AWS_ACCESS_KEY_ID=... \
  -e AWS_SECRET_ACCESS_KEY=... \
  -e AWS_REGION=us-east-1 \
  omnicoreagent-serve

# Cloudflare R2 workspace (persistent)
docker run -p 8000:8000 \
  -e LLM_API_KEY=$LLM_API_KEY \
  -e OMNICOREAGENT_WORKSPACE_BACKEND=r2 \
  -e R2_BUCKET_NAME=my-bucket \
  -e R2_ACCOUNT_ID=... \
  -e R2_ACCESS_KEY_ID=... \
  -e R2_SECRET_ACCESS_KEY=... \
  omnicoreagent-serve

Python API (Programmatic Control)

For full programmatic control, use OmniServe directly in your Python script:

from omnicoreagent import OmniCoreAgent, OmniServe, OmniServeConfig, ToolRegistry

tools = ToolRegistry()

@tools.register_tool("get_time")
def get_time() -> dict:
    from datetime import datetime
    return {"time": datetime.now().isoformat()}

agent = OmniCoreAgent(
    name="MyAgent",
    system_instruction="You are a helpful assistant.",
    model_config={"provider": "openai", "model": "gpt-4o-mini"},
    local_tools=tools,
)

config = OmniServeConfig(
    host="0.0.0.0",
    port=8000,
    auth_enabled=True,
    auth_token="my-secret-token",
    rate_limit_enabled=True,
    rate_limit_requests=100,
    rate_limit_window=60,
    cors_origins=["*"],
    enable_docs=True,
)

if __name__ == "__main__":
    server = OmniServe(agent, config=config)
    server.start()

Run with Python directly:

export LLM_API_KEY=your_api_key
python server.py

CLI vs Python API:

omniserve run --agent my_agent.py — CLI loads your agent file and applies CLI flags
python server.py — You control everything programmatically via OmniServeConfig

Environment Variable Precedence: environment variables always override values set in OmniServeConfig.

OmniServe is perfect for deploying agents as microservices, webhooks, chatbots, or any HTTP-accessible AI capability.

Learn More: See OmniServe Cookbook for more examples.

Get Started

Core Concepts

How-To Guides

Changelog

OmniServe

OmniServe — Production API Server

Agent File Requirements

Quick Start

Step 1: Create your agent file (`my_agent.py`)

Step 2: Set environment variables

Step 3: Run the server

Step 4: Test the API

CLI Commands

CLI Options: `omniserve run`

CLI Options: `omniserve quickstart`

API Endpoints

Core Endpoints

Background Task Endpoints

Request/Response Examples

Background Task Example

Environment Variables

Docker Deployment

Generate a Dockerfile

Build and run

Cloud deployment examples

Python API (Programmatic Control)

​OmniServe — Production API Server

​Agent File Requirements

​Quick Start

​Step 1: Create your agent file (my_agent.py)

​Step 2: Set environment variables

​Step 3: Run the server

​Step 4: Test the API

​CLI Commands

​CLI Options: omniserve run

​CLI Options: omniserve quickstart

​API Endpoints

​Core Endpoints

​Background Task Endpoints

​Request/Response Examples

​Background Task Example

​Environment Variables

​Docker Deployment

​Generate a Dockerfile

​Build and run

​Cloud deployment examples

​Python API (Programmatic Control)

OmniServe — Production API Server

Agent File Requirements

Quick Start

Step 1: Create your agent file (`my_agent.py`)

Step 2: Set environment variables

Step 3: Run the server

Step 4: Test the API

CLI Commands

CLI Options: `omniserve run`

CLI Options: `omniserve quickstart`

API Endpoints

Core Endpoints

Background Task Endpoints

Request/Response Examples

Background Task Example

Environment Variables

Docker Deployment

Generate a Dockerfile

Build and run

Cloud deployment examples

Python API (Programmatic Control)