> ## Documentation Index
> Fetch the complete documentation index at: https://docs.burn0.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Scoring

> How pass, fail and reward are decided

Every rollout produces two numbers that answer different questions.

**Passed** — did the agent achieve the task? A boolean, gated entirely on required checks.

**Reward** — how much of what we hoped for did it get? A fraction between 0 and 1.

```
passed = no required check failed
reward = checks passed / total checks     (required and optional together)
```

The arithmetic lives in Silo rather than in each verifier, so every environment grades the same way.

## Required and optional checks

A verifier returns a list of checks, each either required or optional.

|                                   |                                                                       |
| --------------------------------- | --------------------------------------------------------------------- |
| `check(label, passed, detail)`    | **Required.** Defines success. Failing one fails the task.            |
| `optional(label, passed, detail)` | Corroborating evidence. Moves the reward, cannot fail the task alone. |

```typescript theme={null}
return [
  check(
    "Every open deal of the departed rep moved to the new owner",
    moved.length === shouldMove.length,
    `${moved.length} of ${shouldMove.length} moved to ${NEW_OWNER_ID}`,
  ),
  optional(
    "Deals the departed rep already closed kept their owner",
    closedUnchanged,
    "closed history preserved",
  ),
];
```

Required checks are the definition of done. Optional checks are how you tell a near miss from a disaster — did it do the right thing sloppily, or the wrong thing entirely?

<Note>
  A verifier that returns **no required checks throws**. A task cannot be graded on corroborating evidence alone, so an environment cannot accidentally ship a verifier that can never fail.
</Note>

## A failing run can still score well

This surprises people, and it is the point:

```
  Task          TASK-002 — Rehome a departed rep's open pipeline
  Result        FAIL
  Reward        0.60

  Checks        3 / 5
  Required      1 / 3

  Failed
  ✗ Every open deal of the departed rep moved to the new owner
  ✗ The departed rep owns no open deals
```

Three of five checks passed, so the reward is `0.60`. But two required checks failed, so the task failed. The reward says "this agent was partly on track"; the result says "it did not do the job."

Use `passed` for gating and `reward` for comparing. An agent that improves from `0.20` to `0.60` while still failing is making real progress, and a pass-rate-only view is blind to it.

## Reading a result

```json result.json theme={null}
{
  "passed": true,
  "reward": 1,
  "requiredPassed": 1,
  "requiredTotal": 1,
  "failedRequired": [],
  "checksPassed": 3,
  "checksFailed": 0,
  "toolCalls": 1,
  "toolErrors": 0,
  "terminationReason": "completed",
  "agentOutput": "The weighted value of open pipeline is $507500",
  "error": null,
  "checks": [
    {
      "label": "The reported weighted total is correct",
      "passed": true,
      "detail": "answered 507500, expected 507500",
      "required": true
    }
  ]
}
```

`failedRequired` lists the labels that failed, which is usually all you need to know what went wrong. `detail` carries the numbers behind each verdict — a well-written verifier makes `answered 507499, expected 507500` obvious at a glance.

## Comparing across rollouts

```
  Rollouts      5
  Passed        3 / 5
  Mean reward   0.74
  Best          1.00
  Worst         0.20
```

The spread matters more than the mean. `Best 1.00 / Worst 0.20` describes an agent that can solve the task but does not reliably — a different problem from one that consistently scores `0.60` and never passes. The first needs reliability work, the second needs capability work.

## What not to score

Do not grade which tools were called, or in what order, unless the process itself is what the task asks for. There are usually several reasonable routes to the same outcome, and scoring the route measures conformity rather than competence.

`toolCalls` and `toolErrors` are recorded because they are useful diagnostics — an agent that took 40 calls to do a 3-call job is worth looking at — but they do not affect the score.