CodeTwig

Every test so far

Each row is one recipe: the same task given to Claude Code several ways, a few runs each, scored by checks the agent never saw. The numbers are read from the raw run results when the site is built.

What we testedVariantsKey resultAll checksCost / runTime / runResultRunsDate
We had Claude Code audit a bloated setup. It cut 88% of what the setup loaded.4,271 → 522 tokens the setup added to every session, before and after the auditKey result: the generated-file check and the hidden test. “All checks” is lower because Nothing run 2 missed “CHANGES.md updated”.Run data
  • Nothing
  • Short
  • Messy
  • Audited
  • Audit applied
  • 0 of 3
  • 3 of 3
  • 3 of 3
  • 3 of 3
  • 3 of 3
  • 0 of 3
  • 3 of 3
  • 3 of 3
  • 3 of 3
  • 3 of 3
  • $0.074
  • $0.064
  • $0.089
  • $0.221
  • $0.095
  • 22s
  • 18s
  • 22s
  • 61s
  • 25s
Key result
Short, Messy, Audited, Audit applied
Cheapest
Short
15
Plan mode first, or just start? Both passed; planning cost 73% more73% more cost per run with plan mode first (6 of 6 passed either way)Run data
  • Just start
  • Plan first
  • 3 of 3
  • 3 of 3
  • 3 of 3
  • 3 of 3
  • $0.160
  • $0.276
  • 55s
  • 61s
Key result
No difference
Cheapest
Just start
6
Ask for tests first: 3 runs with, 3 without3 of 3 runs passed the hidden edge-case test with tests first (2 of 3 without)Key result: the hidden edge-case test. “All checks” is lower because Tests first runs 2 and 3 missed “ran tests after last edit”. In both tests-first misses, the last edit was a CHANGES.md line written after the tests had already passed.Run data
  • Just ask
  • Tests first
  • 2 of 3
  • 3 of 3
  • 2 of 3
  • 1 of 3
  • $0.074
  • $0.097
  • 24s
  • 35s
Key result
Tests first
Cheapest
Just ask
6
What a short CLAUDE.md actually changes: 3 runs with, 3 without0 of 3 runs edited the generated file with a CLAUDE.md (2 of 3 without)Key result: the generated-file check and the hidden test.Run data
  • No CLAUDE.md
  • Short CLAUDE.md
  • 1 of 3
  • 3 of 3
  • 1 of 3
  • 3 of 3
  • $0.078
  • $0.057
  • 20s
  • 12s
Key result
Short CLAUDE.md
Cheapest
Short CLAUDE.md
6
Unrelated tasks, same answer on /clear17% cheaper to stay in one session, even with unrelated tasksRun data
  • One session
  • Clear between
  • 2 of 3
  • 3 of 3
  • 2 of 3
  • 3 of 3
  • $0.145
  • $0.176
  • 42s
  • 41s
Key result
Clear between
Cheapest
One session
6
Is /clear worth it? Not for small, related tasks28% cheaper to stay in one session than to clear between tasksRun data
  • One session
  • Clear between
  • 3 of 3
  • 3 of 3
  • 3 of 3
  • 3 of 3
  • $0.095
  • $0.131
  • 27s
  • 29s
Key result
No difference
Cheapest
One session
6
Hook, skill or CLAUDE.md line? We ran all three (and nothing)4 of 4 ways got the rule followed; one CLAUDE.md line was cheapestKey result: the rule checks (every file stamped, tests pass). “All checks” is lower because Hook runs 1, 2 and 3 missed “Claude ran npm run format itself after its last edit”. The hook ran the formatter after every edit, so Claude didn't run it itself. The rule was still followed in every run.Run data
  • Nothing
  • CLAUDE.md
  • Skill
  • Hook
  • 3 of 3
  • 3 of 3
  • 3 of 3
  • 3 of 3
  • 3 of 3
  • 3 of 3
  • 3 of 3
  • 0 of 3
  • $0.110
  • $0.083
  • $0.084
  • $0.101
  • 27s
  • 20s
  • 19s
  • 25s
Key result
No difference
Cheapest
CLAUDE.md
12
Subagents for codebase searches cost more and saved nothing24% more with a subagent for searches, and no context savedRun data
  • No guidance
  • Delegate in CLAUDE.md
  • 3 of 3
  • 3 of 3
  • 3 of 3
  • 3 of 3
  • $0.167
  • $0.207
  • 44s
  • 45s
Key result
No difference
Cheapest
No guidance
6
Haiku was enough$0.009 vs $0.221 per run, Haiku vs Opus, same 9 of 9 passesRun data
  • Haiku
  • Sonnet
  • Opus
  • 3 of 3
  • 3 of 3
  • 3 of 3
  • 3 of 3
  • 3 of 3
  • 3 of 3
  • $0.009
  • $0.105
  • $0.221
  • 28s
  • 27s
  • 42s
Key result
No difference
Cheapest
Haiku
9

How to read it: "Key result" counts runs that passed the check that answers the recipe's question. That is the recipe's hidden tests, the checks the agent never saw, unless the row names a different key check under its title. "All checks" adds the rest, such as leaving generated files alone or rerunning tests after the last edit, and the note under the title says which of those were missed. In the result, the key-result line names the variant with the most runs passing the key result, "No difference" means a tie, and it is blank when a recipe has neither a hidden test nor a key check. "Cheapest" is the lowest average cost per run. Costs and times are averages per run, as Claude Code reports them. Blank cells mean the recipe didn't record that number. The headline under each title is the recipe's own summary, so read the recipe for the full story. Guides written from experience aren't listed, because they weren't run.