Run Artifacts

Every rollout writes a directory:

.silo/runs/<runId>/
├── trace.jsonl       every event, append-only, in order
├── result.json       checks, reward, agent output, tool errors
├── state-diff.json   what the rollout changed
└── run.json          task, verifier, resolved config, timings

Run ids are collision-free, so concurrent runs and --runs sweeps never overwrite each other.

`trace.jsonl`

The record of what actually happened, one JSON object per line, in order. It is written to stand alone: it opens with the task, the resolved config and every tool name that was offered, and closes with the agent’s output, how the run ended, and how it was graded.

{"seq":1,"type":"run_start","at":"...","runId":"...","environment":"demo","task":{...},"config":{...},"tools":[...]}
{"seq":2,"type":"tool_call","at":"...","callId":1,"tool":"list_opportunities","input":{"ownerId":"USR-004","status":"open"}}
{"seq":3,"type":"tool_result","at":"...","callId":1,"tool":"list_opportunities","output":{"total":2,...}}
{"seq":8,"type":"agent_output","at":"...","output":"Moved 2 open opportunities (OPP-005, OPP-006)..."}
{"seq":9,"type":"run_end","at":"...","terminationReason":"completed","error":null,"toolCalls":3,"toolErrors":0,"durationMs":4}
{"seq":10,"type":"verifier_result","at":"...","verifierId":"VER-002","passed":true,"reward":1,"checks":[...]}

Event	Carries
`run_start`	Task, resolved config, every tool name offered
`tool_call`	`callId`, tool name, the exact arguments
`tool_result`	Matching `callId`, output or error, `durationMs`
`agent_output`	The final text
`run_end`	Termination reason, call and error counts, duration
`verifier_result`	Every check with its label, verdict and detail

callId pairs a call with its result, so interleaved or concurrent calls stay attributable. The trace is always written. A run that dies before its first tool call still produces a readable account of what it was asked to do and how it failed.

When debugging, read trace.jsonl first. It tells you whether the model picked the wrong tool, sent bad arguments, or looped — which result.json alone cannot.

`state-diff.json`

What the rollout changed, as a shallow structural diff:

{
  "scalars": [],
  "collections": {
    "opportunities": {
      "added": [],
      "removed": [],
      "changed": ["OPP-005", "OPP-006"]
    },
    "auditLog": {
      "added": ["AUD-0001", "AUD-0002"],
      "removed": [],
      "changed": []
    },
    "sequences": {
      "added": [],
      "removed": [],
      "changed": ["AUD"]
    }
  }
}

Collections keyed by id report added, removed and changed. Top-level scalars report their before and after. Arrays whose rows carry a string id are diffed as collections too — which is what makes append-only logs like auditLog visible instead of silently absent. An empty diff alongside a failure is diagnostic on its own: the agent read but never wrote.

`result.json`

Checks, reward, the agent’s output, and the tool error count. Covered in Scoring.

`run.json`

Everything the rollout was configured with, plus timings:

{
  "runId": "run_20260915020238_9s01",
  "environment": "demo",
  "taskId": "TASK-002",
  "task": { "id": "TASK-002", "title": "...", "instruction": "...", "verifierId": "VER-002", "difficulty": "medium" },
  "verifierId": "VER-002",
  "verifierName": "Departed rep's open deals moved to the new owner",
  "config": { "maxToolCalls": 100, "timeoutMs": 120000, "agentPath": "./solver.ts" },
  "startedAt": "2026-09-15T02:02:38.161Z",
  "finishedAt": "2026-09-15T02:02:38.168Z",
  "durationMs": 5,
  "terminationReason": "completed",
  "toolCallCount": 3
}

The task is embedded in full, so a run stays interpretable even if the task file is later edited.

Two files carry no timestamps

result.json and state-diff.json contain no timestamps and no run ids, by design. That makes them an exact regression oracle: any difference between two runs of the same task with the same agent is a real behavioural change, not noise. This is what lets you record a run as a baseline and byte-compare against it later:

diff .silo/runs/<runId>/result.json baselines/TASK-002.result.json

It is worth protecting. A timestamp added to either file would make them useless for comparison.

Verifiers that derive their expected answers protect against stale data — edit a seed value and grading follows. They do not protect against changed logic: if a verifier and a tool share a helper and that helper changes, both move together and the check still passes.A recorded baseline is what catches that, because a frozen artifact cannot follow a formula change. Derivation and baselines cover different failures; a setup you intend to rely on wants both.

`.silo/` is generated

Runs accumulate. .silo/ is runtime data, not source — add it to .gitignore and commit only the baselines you deliberately record.

Getting Started

Environments

Running Agents

Guides

Reference

Run Artifacts

`trace.jsonl`

`state-diff.json`

`result.json`

`run.json`

Two files carry no timestamps

`.silo/` is generated

​trace.jsonl

​state-diff.json

​result.json

​run.json

​Two files carry no timestamps

​.silo/ is generated

`trace.jsonl`

`state-diff.json`

`result.json`

`run.json`

Two files carry no timestamps

`.silo/` is generated