> ## Documentation Index
> Fetch the complete documentation index at: https://docs-omnicoreagent.omnirexfloralabs.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Runtime Visibility

> Production metrics and per-request performance monitoring

# Runtime Visibility & Metrics

Monitor your agents with built-in metrics, telemetry streams, and in-house trace summaries.

***

## Real-time Usage Metrics

OmniCoreAgent tracks request counts, runtime, and provider-reported token usage when available. Each `run()` returns a `metric` object, and you can get cumulative stats anytime.

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
result = await agent.run("Analyze this data")
print(f"Request Tokens: {result['metric'].request_tokens}")
print(f"Time Taken: {result['metric'].total_time:.2f}s")

# Get aggregated metrics for the agent's lifecycle
stats = await agent.get_metrics()
print(f"Avg Response Time: {stats['average_time']:.2f}s")
```

***

## Runtime Metrics

OmniCoreAgent exposes lightweight runtime metrics without requiring an external
tracing service.

### What's Available

* Request count
* Request, response, and total tokens
* Total runtime
* Average response time

***

## Telemetry Events

OmniCoreAgent emits typed telemetry events for user messages, tool calls, tool
results, final answers, subagent calls, and background run lifecycle changes.
Use events when you need a live UI, session debugging, or a lightweight
execution record.

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
result = await agent.run("Research this topic", session_id="research_1")

events = await agent.get_telemetry_events_after(
    cursor=None,
    session_id="research_1",
    run_id=result["run_id"],
)
for event in events:
    print(event.event_type, event.model_dump())
```

When serving through OmniServe, `POST /run` streams the live run over SSE.
For application UIs and APIs, use the `/telemetry` routes to replay, inspect,
and stream stored telemetry. Filter by `run_id` when a UI needs to isolate one
execution inside a shared session.
The compact `/events/{session_id}` aliases are still available and accept
`?run_id=...` for the same run-scoped isolation.

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
curl -N -X POST http://localhost:8000/run \
  -H "Content-Type: application/json" \
  -d '{"query": "Research this topic", "session_id": "research_1"}'

curl "http://localhost:8000/telemetry/events?session_id=research_1&run_id=RUN_ID"
curl "http://localhost:8000/telemetry/events?run_id=RUN_ID&event_type=tool_result&limit=100"
curl "http://localhost:8000/telemetry/traces?session_id=research_1&limit=20"
curl "http://localhost:8000/telemetry/traces/TRACE_ID"
curl "http://localhost:8000/telemetry/runs/RUN_ID/trace"
curl "http://localhost:8000/telemetry/sessions/research_1/trace"
curl -N "http://localhost:8000/telemetry/events/stream?session_id=research_1&run_id=RUN_ID"
```

***

## Agent Trace

Every run emits typed telemetry. Retrieve the trace by the `trace_id` returned
from `agent.run(...)`.

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
trace = await agent.get_trace(result["trace_id"])

print(trace["status"])
print(len(trace["events"]))
print(len(trace["spans"]))

for event in trace["events"]:
    print(event["event_type"], event.get("input"), event.get("output"))
```

You can also retrieve the latest trace for a session or a trace correlated to a
run:

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
latest_session_trace = await agent.get_latest_trace("research_1")
run_correlated_trace = await agent.get_trace(run_id=result["run_id"])
normalized = await agent.get_trace(result["trace_id"], normalize=True)
```

`trace_id` is the exact trace handle. `run_id` is a correlation and filtering
handle; when more than one trace is correlated to the same run, `get_trace(
run_id=...)` returns the latest matching trace.

The trace is intentionally dependency-free inside the runtime. Exporters are
optional adapters layered on top of this internal trace model.

***

## Export Traces

Install the OpenTelemetry extra when you want to send traces to an OTLP
collector or an OTLP-compatible tracing backend:

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
pip install "omnicoreagent[otel]"
```

Export a specific trace manually:

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from omnicoreagent import build_telemetry_exporter

result = await agent.run("Research this topic", session_id="research_1")

otel = build_telemetry_exporter(
    "otlp",
    endpoint="http://localhost:4318/v1/traces",
    service_name="research-agent",
)

exported = await agent.export_trace(
    result["trace_id"],
    exporters=[otel],
)
print(exported)
```

Configure exporters on the agent to export automatically when traces end:

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from omnicoreagent import OmniCoreAgent

agent = OmniCoreAgent(
    name="research-agent",
    system_instruction="You are a research assistant.",
    model_config={"provider": "openai", "model": "gpt-5.4-mini"},
    telemetry_exporters=[
        {
            "destination": "otlp",
            "endpoint": "http://localhost:4318/v1/traces",
            "service_name": "research-agent",
        }
    ],
)
```

Vendor presets use the same OTLP exporter path:

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
langsmith = build_telemetry_exporter(
    "langsmith",
    api_key="...",
    project_name="production-agents",
)

opik = build_telemetry_exporter(
    "opik",
    api_key="...",
    workspace="your-workspace",
    project_name="production-agents",
)
```

The presets also read common environment variables:

| Exporter  | Environment variables                                                                |
| --------- | ------------------------------------------------------------------------------------ |
| OTLP      | `OTEL_EXPORTER_OTLP_ENDPOINT` or explicit `endpoint`                                 |
| LangSmith | `LANGSMITH_API_KEY`, `LANGSMITH_PROJECT`, optional `LANGSMITH_OTEL_ENDPOINT`         |
| Opik      | `OPIK_API_KEY`, `OPIK_WORKSPACE`, `OPIK_PROJECT_NAME`, optional `OPIK_OTEL_ENDPOINT` |
| JSONL     | explicit local `path`                                                                |

`trace_id` remains the exact lookup and export handle. `run_id` can be used for
correlated export, but when a serving trace and an agent trace share the same
`run_id`, the latest matching trace is exported.

<Tip>
  Use metrics for cost and performance monitoring, events for live UI streaming,
  traces for debugging, and exporters when traces need to leave the process.
</Tip>