Background Agents: Durable Agent Tasks

Background Agents run OmniCoreAgent work outside the foreground request path. They are durable tracked tasks with schedules, retries, cancellation, leases, workspace output, and inspectable lifecycle events. Use the default in-memory store for local development, or sql, redis, or mongodb when runs must survive restarts.

Key Concepts

BackgroundAgentManager: Registers agents, tasks, runs, and lifecycle operations.
Task store: Operational state for agents, tasks, schedule state, runs, attempts, leases, retries, and cancellation flags. It defaults to in-memory and is separate from conversation memory.
Run: One concrete execution of a task. Runs have stable IDs, statuses, attempts, workspace paths, and event history.
Worker lifecycle: A worker claims queued runs with leases, heartbeats while work is active, and recovers expired leases.
Workspace output: Every background run gets a workspace namespace for lifecycle files and agent output.

You can run the background manager without storage configuration. Use BackgroundAgentManager() for the default zero-config in-memory task store. Use task_store="sql", task_store="redis", or task_store="mongodb" when task state must survive process restarts. The task store is not conversation memory; it is the control plane for background work. Redis and MongoDB task stores use optional backend drivers:

pip install "omnicoreagent[redis]"
pip install "omnicoreagent[mongodb]"

Quick Start

import asyncio

from omnicoreagent import BackgroundAgentManager, OmniCoreAgent


async def main():
    agent = OmniCoreAgent(
        name="system_monitor",
        system_instruction="Check system health and write concise reports.",
        model_config={"provider": "openai", "model": "gpt-5.4-mini"},
    )

    manager = BackgroundAgentManager()
    try:
        await manager.register_agent(agent_id="system_monitor", agent=agent)
        await manager.register_task(
            task_id="health_report",
            agent_id="system_monitor",
            query="Check system health and summarize anything that needs attention.",
            schedule={"type": "manual"},
            timeout_seconds=60,
            retry_policy={"max_retries": 1, "initial_delay_seconds": 0},
        )

        run = await manager.run_now("health_report", wait=True)
        print(run.status)
        print(run.result_preview)
    finally:
        await manager.shutdown()


asyncio.run(main())

Durable Task Stores

The default task store is in-memory. It needs no database and is right for local development, tests, and single-process experiments:

manager = BackgroundAgentManager()

Use a durable task store when background runs must survive process restarts:

manager = BackgroundAgentManager(task_store="sql")

manager = BackgroundAgentManager(
    task_store={"backend": "redis", "url": "redis://localhost:6379/0"}
)

manager = BackgroundAgentManager(
    task_store={
        "backend": "mongodb",
        "uri": "mongodb://localhost:27017",
        "database": "omnicoreagent",
    }
)

Redis durable deployments need persistence enabled and a no-eviction policy for task-store keys. MongoDB task-store writes use majority write concern. Choose one durable backend per deployment. Use SQL/SQLite for local durability or simple single-node services. Use Redis when your deployment already operates Redis with persistence and no eviction for task-store keys. Use MongoDB when MongoDB is your durable operational store. Durable stores preserve queued runs across manager restarts. You can queue a manual run, stop the process, construct a new manager with the same task store, and complete the queued run from that new manager. The in-memory store does not provide that guarantee. OmniServe uses the same task-store settings through environment variables. Pick one backend:

export OMNICOREAGENT_BACKGROUND_TASK_STORE=sql
export OMNICOREAGENT_BACKGROUND_TASK_STORE_URL=sqlite:///.omnicoreagent/background.db

export OMNICOREAGENT_BACKGROUND_TASK_STORE=redis
export OMNICOREAGENT_BACKGROUND_TASK_STORE_URL=redis://localhost:6379/0

export OMNICOREAGENT_BACKGROUND_TASK_STORE=mongodb
export OMNICOREAGENT_BACKGROUND_TASK_STORE_URI=mongodb://localhost:27017
export OMNICOREAGENT_BACKGROUND_TASK_STORE_DATABASE=omnicoreagent

Scheduled Runs

Manual tasks run only when you call run_now. Scheduled tasks are dispatched by the manager worker after start() is called:

await manager.register_task(
    task_id="hourly_report",
    agent_id="system_monitor",
    query="Write the hourly operational report.",
    schedule={"type": "interval", "seconds": 3600},
    overlap_policy="queue_next",
)

await manager.start()

start() creates the manager worker and returns immediately. Keep the process alive while scheduled work should continue, and use shutdown() when the process exits so worker loops and active background resources close cleanly:

try:
    await manager.start()
    await asyncio.Event().wait()
finally:
    await manager.shutdown()
    if hasattr(agent, "cleanup"):
        await agent.cleanup()

The runnable scheduled example uses the same worker path without requiring an LLM API key:

python3 cookbook/background_agents/scheduled_task_example.py

It creates a due once task, starts the background worker, waits for the run to complete, then prints task status, manager status, lifecycle events, and the workspace files created for the run. By default it writes the demo workspace under your system temp directory. Set OMNICOREAGENT_COOKBOOK_WORKSPACE_DIR to choose a different location:

OMNICOREAGENT_COOKBOOK_WORKSPACE_DIR=/tmp/omnicoreagent-background-demo \
  python3 cookbook/background_agents/scheduled_task_example.py

Real Application Background Task

The real application example runs the support operations app shape through the background manager. It uses the same support domain tools as cookbook/real_applications/support_operations_agent.py, registers a manual task, waits for the run to finish, then prints the run id, status, attempts, lifecycle events, and workspace files. It is deterministic and does not require LLM_API_KEY, so it is safe to run in CI or locally when you only want to inspect the background execution boundary:

uv run python cookbook/background_agents/real_application_background_task.py

By default, the workspace root is under your system temp directory at omnicoreagent_real_app_background_workspace. The script prints both workspace_root and the run-local workspace path. Pass workspace_dir when calling run_real_application_background_example(...) from Python if you want a specific local root. The run workspace contains:

output.md with the durable support note
tickets/tck-1042.md with the ticket-specific record
run.json with the latest run snapshot
events.jsonl with ordered lifecycle events

Runtime Controls

run = await manager.run_now("health_report", wait=True)
status = await manager.get_run(run.run_id)
task_status = await manager.get_task_status("health_report")
manager_status = await manager.get_manager_status()
attempts = await manager.list_attempts(run.run_id)
events = await manager.get_run_events(run.run_id)
workspace = await manager.get_run_workspace(run.run_id)

await manager.cancel_run(run.run_id)
await manager.pause_task("health_report")
await manager.resume_task("health_report")

get_run_events(run_id) returns ordered lifecycle events for that run. These events use run-local names such as background_task_scheduled, background_run_queued, background_run_claimed, background_run_started, background_run_heartbeat, background_run_retrying, background_run_recovered, background_run_completed, background_run_failed, background_run_timeout, background_run_cancelled, and background_run_skipped. When workspace event mirroring is enabled, lifecycle events are also written to the run workspace events.jsonl file for durable replay and debugging.

OmniServe Endpoints

OmniServe exposes the same background run lifecycle over HTTP:

curl -X POST http://localhost:8000/background/tasks \
  -H "Content-Type: application/json" \
  -d '{
    "task_id": "health_report",
    "query": "Check system health and write the report.",
    "schedule": {"type": "manual"},
    "timeout_seconds": 60
  }'

curl -X POST http://localhost:8000/background/tasks/health_report/run \
  -H "Content-Type: application/json" \
  -d '{"wait": true}'

The background API includes manager status, task creation, task status, task listing, pause, resume, delete, manual run, cancellation, run status, attempt history, event replay, and workspace inspection.

curl http://localhost:8000/background/status
curl http://localhost:8000/background/tasks/health_report/status
curl http://localhost:8000/background/runs/$RUN_ID
curl http://localhost:8000/background/runs/$RUN_ID/events
curl http://localhost:8000/background/runs/$RUN_ID/workspace

Task Configuration

Field	Purpose
`task_id`	Stable task identifier.
`agent_id`	Registered agent that executes the task.
`query`	Instruction passed to the agent for each run.
`schedule`	`manual`, `interval`, `cron`, or `once`.
`timeout_seconds`	Optional per-run timeout.
`retry_policy`	Retry count, delay, backoff, and retryable error types.
`overlap_policy`	`skip_if_running`, `queue_next`, `cancel_previous`, or `allow_parallel`.
`session_policy`	`task`, `run`, or `fixed` memory-session behavior.

State is restart-persistent when the task store is SQL, Redis, or MongoDB.

​Background Agents: Durable Agent Tasks

​Key Concepts

​Quick Start

​Durable Task Stores

​Scheduled Runs

​Real Application Background Task

​Runtime Controls

​OmniServe Endpoints

​Task Configuration