The Cache

The cache is what makes stepbook fast to iterate with — and it's really one idea: the cache is a replay. Every unit call in a run is recorded; re-running the pipeline replays the recorded calls instead of executing them. That's why you can fix one unit without re-running the others, and why an agent loop replays its turns against frozen model responses for free.

Per-call record/replay

A run is a trajectory — a tree of unit calls. Each call is recorded and replayed independently. On a re-run, a call whose fingerprint is unchanged is a cache hit: its recorded output is served instantly and the runner never executes (the node shows cacheHit).

A call's fingerprint is three things:

1. The call's input — the argument the runner passed when it invoked the unit (for a loop turn, the state at that turn). Not a wired-in dependency: the input is whatever the code handed the unit.
2. The unit's config — its config object (and any overrides).
3. The unit's source-file hash — a hash of the *.step.ts file that defines the unit.

If all three match a prior run, the call replays. Repeat calls of the same unit (an agent's decide#0, decide#1, …) are keyed and replayed one turn at a time.

Why the source hash matters

This is the key design choice. Because the fingerprint includes the unit's source-file hash, editing a unit's file invalidates that unit's recordings — even if the input is identical.

Concretely: if you tweak a prompt, change a parsing rule, or adjust a constant at module scope in analyze.step.ts, then every analyze call re-runs on the next stepbook run. Every other unit — whose source and input didn't change — still replays.

// editing THIS constant invalidates the unit's recordings, because it's part of
// the source-file hash:
const PROMPT = `Summarize the following text in two sentences.`

export default defineStep({
  type: 'llm',
  runner: async (input, ctx) => callModel(PROMPT, input),
})

That's the "iterate on one unit, not ten" property: the cache tracks what the unit is, not just what it was given. And because each call is keyed on its own source, editing one unit never forces the others to re-execute.

llm calls replay too

There's no separate rule for model calls. An llm unit's call is recorded and replayed like any other, so re-running the pipeline — or an agent's loop — re-executes your code against the recorded model responses at no cost. That's what turns a loop into a repeatable regression test: replay the trajectory to see what your control-flow edit did, without re-paying for the model.

The deterministic footnote

deterministic is an advanced per-unit flag most pipelines never touch. In the run model every unit call is recorded and replayed — including llm calls — so the flag doesn't change trajectory replay. If a unit genuinely must run fresh every time (it reads a clock, a live counter, or a random source), don't reach for a flag: force it per run with --no-cache (whole run) or --from <unit> (that unit onward). The flag survives as a default hint (deterministic ?? (type !== 'llm')) that tooling can read; you rarely set it yourself.

Forcing fresh work

Two flags override replay:

• --no-cache bypasses the cache entirely — every unit call re-executes and nothing is recorded.
• --from <unit> forces re-execution of that unit (and re-records its calls) while the rest still replay.

stepbook run                 # replay everything unchanged
stepbook run --no-cache      # re-execute every call, record nothing
stepbook run --from analyze  # re-run analyze; replay the rest

Because stability sampling runs a unit with no cache, stepbook variance <unit> re-runs it fresh on every trial — which is exactly what measuring its spread requires. For an agent loop, that means each trial drives the whole loop live.

Inline sub-computations

A unit call is the first-class version of a recording. When you have a sub-step that isn't worth its own *.step.ts — a single model call inside a runner's loop — ctx.cached records it through the exact same cache:

const decision = await ctx.cached(`turn-${n}`, { task, transcript }, () =>
  decideOnce({ task, transcript, model }),
)

Each sub-result lands at state/snapshots/<pipeline>/<unit>/<subkey>/<key>.json — namespaced under the unit so it never collides with the unit's own recordings. The first run records every turn; later runs replay them. Two details worth knowing:

• It records regardless of deterministic. ctx.cached is the cassette that makes the loop replayable, so it always writes on a miss.
• The key folds in the unit's source hash. Edit the loop file and the whole trajectory re-records next run — you never mix old model responses with new control-flow code.

--no-cache runs every sub-call fresh and records nothing; --from <unit> re-records that unit's sub-calls.

Where the cache lives

The cache is a set of files under state/snapshots/ in your project. It's plain local files — no database, no daemon. Delete the directory to clear it, or use stepbook prune to trim old run blobs (which is separate from the cache; see the CLI Reference).

Mental model

Think of each unit call's fingerprint as "this exact unit, with this exact code, given this exact input." When any part of that changes, the call re-runs. When none of it changes, you get the recorded answer for free. That's the whole trick — and it's why the development loop is edit one unit → re-run → only that unit executes.

That's only half the loop, though. The cache makes re-running cheap; it doesn't tell you what your edit actually did to the output. That's the other half: diff the new run against the old one. Together they're the core of stepbook — edit → cheap re-run → diff to see what changed. See Drift & Stability for the diff side.