--json, and fail with specific messages and non-zero exits. That is deliberate: a coding agent can drive the whole authoring flow and verify the result without a human in the loop.
Copy the brief below, replace the bracketed block with your own use case, and hand the whole thing over.
The brief
Why validate is the stopping condition
silo env validate runs the real TypeScript compiler over the environment, not just a resource check. Type-only imports are erased before execution, so an environment can run perfectly while being uncompilable — “it ran” is not evidence that it is correct.
Telling the agent to stop at OK rather than at “the run worked” is what makes the loop self-checking. A run exercises one path; validation covers every tool and verifier, including the ones that task never touched.
--json output is what makes this usable in an agent loop: a machine-readable ok, findings with codes and line numbers, and counts that catch a tool written but never registered.
Ask for proof, not assertion
The most useful instruction you can add is that a passing verifier means nothing until it has been shown to fail:Reviewing what it built
Regardless of who wrote it, these are worth checking by hand:- Does
createState()clone? Missing this is invisible in a single run and only appears as flaky grading across--runs. - Does any verifier hardcode an answer? Search for literal numbers in
verifiers/. A derivable value written as a constant is correct until the data changes. - Is anything derived stored in
data/? A stored total goes stale the moment a quantity is edited. - Do the tools cover what the tasks require? If a task needs information no tool exposes, no agent can pass it.
- Do the tasks have unique answers? A tie makes a task ungradeable.
Keeping an agent oriented later
Once the environment exists, the same properties make it a good target for iteration.trace.jsonl gives an agent the full record of a failed rollout — which tool was called, with what arguments, and what came back — so “here is the trace, work out why it failed” is a productive instruction rather than a guessing game.
Silo