Observability for agent runs
If a post-mortem depends on a transcript, it is not a post-mortem. Every tool call, model call, retry, and state transition should be recorded under one trace ID.
What is AI agent observability?
If a post-mortem depends on a transcript, it is not a post-mortem. Every tool call, model call, retry, and state transition should be recorded under one trace ID.
Implementation
- Assign a trace ID at the request boundary and propagate it through every tool call.
- Record model calls with provider, model, token counts, and cost.
- Record tool calls with arguments, result status, and duration, with secrets redacted at write time.
- Emit state transitions as events, so the run's shape is reconstructable without replaying the model.
- Retain traces long enough to cover your incident review window.
Failure modes
- Redaction at render time, which means the raw secret was already stored.
- Sampling that drops the failing runs.
- Traces that record model output but not tool arguments.
How it is verified
- Pick a random failed run and reconstruct it end to end using only stored records.
- Grep the trace store for credential-shaped strings.
Questions this pattern answers
What is AI agent observability?
If a post-mortem depends on a transcript, it is not a post-mortem. Every tool call, model call, retry, and state transition should be recorded under one trace ID.
How do you implement observability for agent runs?
Assign a trace ID at the request boundary and propagate it through every tool call. Record model calls with provider, model, token counts, and cost. Record tool calls with arguments, result status, and duration, with secrets redacted at write time. Emit state transitions as events, so the run's shape is reconstructable without replaying the model. Retain traces long enough to cover your incident review window.
How is observability for agent runs verified?
Pick a random failed run and reconstruct it end to end using only stored records. Grep the trace store for credential-shaped strings.
What usually goes wrong with observability for agent runs?
Redaction at render time, which means the raw secret was already stored. Sampling that drops the failing runs. Traces that record model output but not tool arguments.
Projects in this track
ClawKeeper →
Governed Claude agent control plane. A deterministic policy engine runs before every LLM call, autonomous spend defaults to zero, and the audit log is enforced by the database.
TechTide Harness Kit →
Trust contracts for agent skills expressed as JSON Schema: risk tiers, tool permissions, deny-by-default egress, approval gates, and prompt-injection fixtures behind a wall of CI validation gates.
CipherClaw →
Deterministic multi-agent debugger with zero LLM calls in the analysis path: causal graphs, weighted root-cause scoring, and failure prediction.
Swarm 357 →
357 Claude agent roles across six business layers, with portable memory, enforced cost caps, a bash policy gate, and durable checkpoints.
Related patterns
This page documents how the pattern is built. Engagement scope and pricing live on TechTide AI.
AI Production Systems at TechTide AI→© 2026 Alex Cinovoj · TechTide AI · Columbus, OH