alexcinovoj.devby TechTide AI
Back to AI proof
Pattern · AI production systems

Observability for agent runs

If a post-mortem depends on a transcript, it is not a post-mortem. Every tool call, model call, retry, and state transition should be recorded under one trace ID.

Short answer

What is AI agent observability?

If a post-mortem depends on a transcript, it is not a post-mortem. Every tool call, model call, retry, and state transition should be recorded under one trace ID.

Implementation

  • Assign a trace ID at the request boundary and propagate it through every tool call.
  • Record model calls with provider, model, token counts, and cost.
  • Record tool calls with arguments, result status, and duration, with secrets redacted at write time.
  • Emit state transitions as events, so the run's shape is reconstructable without replaying the model.
  • Retain traces long enough to cover your incident review window.

Failure modes

  • Redaction at render time, which means the raw secret was already stored.
  • Sampling that drops the failing runs.
  • Traces that record model output but not tool arguments.

How it is verified

  • Pick a random failed run and reconstruct it end to end using only stored records.
  • Grep the trace store for credential-shaped strings.

Questions this pattern answers

What is AI agent observability?

If a post-mortem depends on a transcript, it is not a post-mortem. Every tool call, model call, retry, and state transition should be recorded under one trace ID.

How do you implement observability for agent runs?

Assign a trace ID at the request boundary and propagate it through every tool call. Record model calls with provider, model, token counts, and cost. Record tool calls with arguments, result status, and duration, with secrets redacted at write time. Emit state transitions as events, so the run's shape is reconstructable without replaying the model. Retain traces long enough to cover your incident review window.

How is observability for agent runs verified?

Pick a random failed run and reconstruct it end to end using only stored records. Grep the trace store for credential-shaped strings.

What usually goes wrong with observability for agent runs?

Redaction at render time, which means the raw secret was already stored. Sampling that drops the failing runs. Traces that record model output but not tool arguments.

Projects in this track

Related patterns

This page documents how the pattern is built. Engagement scope and pricing live on TechTide AI.

AI Production Systems at TechTide AI