tasks/, discovered automatically — adding a task is adding a file.
tasks/TASK-001.json
The instruction is the whole prompt
The agent receivesinstruction as a plain string and nothing else — no tool list, no hints, no schema. Everything it needs to identify what to act on must be in that text, phrased as a person would phrase it.
State the objective, never the path.
list_opportunities first, or that there are exactly two deals, or which tool reassigns them. Finding that out is the work being measured.
An instruction that names the tools tests whether a model can follow directions. An instruction that states an outcome tests whether it can do the job.
Give it enough to be solvable
Being vague is not the same as being hard. The instruction must contain everything a competent person would need:- Names and ids.
Daniel Reyes (USR-002)— because the verifier will check the deal ended up with that user. - Parameters that are decisions, not deductions. The $140,000 amount and the close date are inputs from the business; the agent cannot derive them.
- Boundaries. “Deals he already closed stay as they are” is what makes a wrong-but-plausible action wrong.
Two shapes of task
State-changing — the agent must leave the world different. Graded againstfinalState.
Lumen Retail has signed for the point-of-sale refresh, but at $118,000 rather than the figure we last quoted. Record the win at the contracted number.Answer-producing — the agent must report something, and correct behaviour may change nothing at all. Graded against its output.
Amara Osei needs a single number for the board: the probability-weighted value of everything currently open across the whole team.Both are first-class. The second matters because a convincing wrong answer is a common agent failure, and it is invisible if you only inspect state.
Ambiguity breaks grading
A verifier has to know the right answer. If the seed data allows two, the task cannot be graded and a correct agent can fail. CRM’sTASK-005 asks which rep has the most stalled deals. The data is arranged so exactly one rep has two and nobody else has more than one. The verifier even records this as an optional check:
Difficulty is yours
difficulty is metadata — Silo does not read it. It is there so you can report on how an agent does across a range. The convention in the built-in templates:
CRM’s
TASK-006 is hard because it says active reps — an agent that skips that word lands on the deactivated rep, who has the worst attainment of all.
Managing tasks
task show, task update and task remove round out the set, and --json works on all of them.
A task and its verifier are one-to-one: env validate reports an error if two verifiers claim the same task.
Silo