Recording agent runs
To investigate a failure, I need records of tool calls, model calls, retries, and state changes. A shared trace ID connects those records. Missing or sampled-out events limit what I can reconstruct.
What should I record to investigate an agent failure?
To investigate a failure, I need records of tool calls, model calls, retries, and state changes. A shared trace ID connects those records.
Implementation
- Assign a trace ID at the request boundary and propagate it through every tool call.
- Record model calls with provider, model, token counts, and cost.
- Record tool calls with arguments, result status, and duration, with secrets redacted at write time.
- Emit state transitions as events, so the run's shape is reconstructable without replaying the model.
- Retain traces long enough to cover your incident review window.
What can go wrong
- Redacting only when displaying a trace leaves the raw secret in storage.
- Sampling can discard the records needed to investigate a failed run.
- Traces without tool arguments can leave the cause of a failure unclear.
Checks to try
- Pick a random failed run and reconstruct it end to end using only stored records.
- Grep the trace store for credential-shaped strings.
Questions this pattern answers
What should I record to investigate an agent failure?
To investigate a failure, I need records of tool calls, model calls, retries, and state changes. A shared trace ID connects those records.
What steps does this method use?
Assign a trace ID at the request boundary and propagate it through every tool call. Record model calls with provider, model, token counts, and cost. Record tool calls with arguments, result status, and duration, with secrets redacted at write time. Emit state transitions as events, so the run's shape is reconstructable without replaying the model. Retain traces long enough to cover your incident review window.
How can I check the result?
Pick a random failed run and reconstruct it end to end using only stored records. Grep the trace store for credential-shaped strings.
What can go wrong with this approach?
Redacting only when displaying a trace leaves the raw secret in storage. Sampling can discard the records needed to investigate a failed run. Traces without tool arguments can leave the cause of a failure unclear.
Related projects
ClawKeeper →
ClawKeeper checks proposed Claude agent actions against explicit rules before a model call. Autonomous spend defaults to zero, and the database enforces an append-only audit log.
TechTide Harness Kit →
TechTide Harness Kit uses JSON Schema to declare skill permissions, risk tiers, network access, and approval requirements. Automated checks validate declarations and run prompt-injection fixtures.
CipherClaw →
CipherClaw analyzes recorded agent activity without model calls. It builds causal graphs, scores possible causes, and flags failure patterns.
Swarm 357 →
357 Claude agent roles across six business layers, with portable memory, enforced cost caps, a bash policy gate, and durable checkpoints.
Related patterns
Acceptance tests for agent work →
Executable checks test the task and detect regressions.
Evaluation tests for releases →
A fixed evaluation suite checks release requirements.
MCP tool permissions →
The server checks which tools a caller may use.
Spending limits for agent runs →
Budget checks run before provider requests.
I explain this method here. Current service scope, prices, and terms are on TechTide AI.
Systems Decision Audit at TechTide AI→© 2026 Alex Cinovoj · TechTide AI · Columbus, OH