Skip to main content
A rollout is the same sequence every time. Knowing it makes failures legible — most confusing results come from misunderstanding one of these steps.
1

Build a fresh world

createState() is called. It must return a newly created object, normally via structuredClone, because Node caches imported JSON modules for the life of the process.
2

Snapshot it

A copy is kept as the initial state. This is what the verifier later compares against, and what it derives expected answers from.
3

Bind the tools

bindTools(state) produces the callable tools, closed over the live world.
4

Load the agent

The module at --agent is imported. It must default-export a function, or the run fails immediately.
5

Hand over control

The agent is called once with the task instruction, the tool schemas, callTool, and an AbortSignal. Everything it does happens inside this call.
6

Take the output

Whatever it returns as output is captured. A string is used as-is; anything else is JSON-stringified.
7

Diff the world

The final state is compared against the snapshot to produce state-diff.json.
8

Grade

The task’s verifier receives the final world, the initial world, and the agent’s output, and returns checks.
9

Write artifacts

The trace, result, diff and run metadata are written to .silo/runs/<runId>/.

​
Isolation

The agent holds no reference to the world. It receives tool schemas — name, description, input shape — not the tools themselves, and certainly not state. Two consequences worth internalizing: What no tool exposes, the agent cannot know. If a task requires knowing which users are inactive and no tool reports that, the task is unsolvable regardless of how capable the model is. Missing tools look like agent failures. Tool output is cloned on the way out. A tool returning a slice of state hands the agent a copy, so nothing the agent does with that value can reach back into the world. The only way to change state is to call a tool that changes it.

​
A tool call, step by step

Each callTool goes through the same path:
  1. Abort check. If the run has already exceeded a limit, the call does not execute.
  2. Lookup. An unknown name returns tool_not_found as a value.
  3. Validation. Arguments are checked against inputSchema. A hallucinated property is rejected before run executes, with a message naming the real parameters.
  4. Execution. run(state, input) operates on the live world.
  5. Cloning. The return value is cloned.
  6. Recording. A tool_call and a tool_result are written to the trace, sharing a callId, with the call’s durationMs.
Failures are returned, never thrown to the agent. A rejected call is an observation the agent can act on, not the end of the rollout — which is what makes self-correction measurable.

​
Reading a trace

Because the sequence is fixed, a trace tells you exactly where things went wrong:
That last one is the failure Silo exists to catch: a confident answer with nothing behind it.

​
Timing

run.json records startedAt, finishedAt and durationMs, and each tool_result carries its own durationMs. Because environments are in-memory simulations, tool calls are typically sub-millisecond — so almost all of a real run’s wall time is the model thinking, which makes the split useful when tuning.