An agent run on NatorOS is a directed graph of steps. Each step has an input, a tool or model call, and an output. We record all three for every step of every run, then make them queryable, replayable, and diffable. This post explains why.
Most agent frameworks log a flat list of LLM calls and tool calls. That is enough to debug a single failure if you happen to catch it. It is not enough to answer the questions a CIO actually asks:
Those questions require state, structure, and the ability to replay.
Each step writes a row to an append-only event log:
run_id, step_id, parent_step_id, agent, kind, input_hash, output_hash, model, model_version, started_at, finished_at, latency_ms, cost_cents, approver_id, status
The input and output are stored as content-addressed blobs. Identical inputs yield identical hashes; we deduplicate. The fingerprint of any run is the concatenation of its step hashes.
Because every input is captured, we can take any past run and re-execute it against a different agent version. This is how every workflow change is validated before it ships. We pick a window of recent runs, replay them against the new agent, diff the outputs, and surface the ones that changed. A human reviews the diffs before the new version is promoted.
When something does go wrong in production, the same machinery lets you bisect: replay the same input against the last twenty agent versions and find the one where the behavior changed.
Recording every step costs us roughly four percent of total runtime CPU and a few cents of storage per thousand runs. It is the highest-value four percent we spend.
Every customer we have ever onboarded has, at some point, asked us to explain a specific past decision. Replayable runs have answered every one of those questions in under a minute. That is the only thing that lets a CIO sleep at night.