Sonnet on high effort vs Opus on low: Sonnet passed 5 of 5 for 16% less, Opus finished sooner
We ran the same 3-step CSV export feature in a 164-file repo 5 times with Sonnet 5.5 at --effort high and 5 times with Opus 5.5 at --effort low. Sonnet high passed the hidden tests in 5 of 5 runs for $0.386 a run. Opus low passed 4 of 5 for $0.461, but its median run took 98s vs 134s for Sonnet.
This is the first test in a series on reasoning effort vs model tier. The question: if you turn a cheaper model up and a stronger model down, which one does the better job, and for how much? Claude Code’s --effort flag sets the level per session (low, medium, high, xhigh, max). We put Sonnet 5.5 on high and Opus 5.5 on low and gave them the same feature.
The setup
The same task as our mixed-models test: fleetdesk, a Node API with 164 files, 16 modules and 18 test files. One feature, three steps, in one continuous session:
- Plan (plan mode): a CSV export of a tenant’s invoices at
GET /billing/invoices.csv, with exact columns, money formatted “the way the rest of the codebase formats money”, inclusive date filters, sorting, RFC 4180 quoting, a header-only body when nothing matches, and 400/404 rules. “Plan the change first. Don’t edit anything yet.” - Implement: “Go ahead and implement the plan, with tests. Run the tests.”
- Follow-ups: an optional
currencyfilter (USD, EUR or GBP, anything else is a 400) and a line in CHANGES.md.
The seeded invoices have traps the hidden tests check: a customer name with a comma, one with double quotes, two invoices on the same date, a 105-cent amount, another tenant’s invoice and one dated the day before the range.
Two setups, 5 runs each, on Claude Code 2.1.321:
- Sonnet high:
claude --model claude-sonnet-5-5 --effort high - Opus low:
claude --model claude-opus-5-5 --effort low
Nine checks graded every run afterwards. The two that answer the question are hidden tests Claude never saw: 4 for the CSV export and 1 for the currency follow-up. We also re-applied each saved diff to a clean copy of the repo and ran the hidden tests again; the results matched.
What happened
| Sonnet 5.5, high effort | Opus 5.5, low effort | |
|---|---|---|
| CSV export hidden tests (4) | 5 of 5 | 4 of 5 |
| Currency follow-up hidden test | 5 of 5 | 5 of 5 |
| All 9 checks | 5 of 5 | 4 of 5 |
| Average cost per run | $0.386 | $0.461 |
| Cost range | $0.319 to $0.537 | $0.438 to $0.522 |
| Average time per run | 160s | 131s |
| Median time per run | 134s | 98s |
| Average turns | 36.2 | 24.2 |
| Average tool calls | 33.2 | 21.2 |
| Average output tokens | 19,948 | 11,723 |
| Average tokens per run (incl. cached input) | 680,739 | 388,748 |
| Average diff lines (+ and −) | 169 | 121 |
5 runs per column. Costs as Claude Code reports them. Every run, check and diff: run data.
What we learned
Sonnet high did the job every time, for less. It passed every hidden test in 5 of 5 runs at $0.386 a run on average, 16% below Opus low’s $0.461. Opus low did less work, with about 41% fewer output tokens and 12 fewer turns, but Opus’s higher price per token more than made up for that.
Opus low was faster. Its median run took 98s vs 134s for Sonnet high. Opus low wrote less: smaller diffs (121 lines vs 169), fewer tests, and fewer tool calls. Sonnet high read more of the repo and wrote more tests; in 4 of 5 runs it touched 8 or more files, while Opus low touched 5 to 7.
The one miss was a judgment call on line endings. Opus low run 2 ended every CSV line with \r\n. RFC 4180 does say CRLF, and the prompt cited RFC 4180 for quoting, but it also said “end the body with a newline”, and the hidden tests expect \n. Its own tests passed because they expected \r\n too. It’s a real spec-reading miss for a client that splits on \n, but it isn’t a broken export. The other 4 Opus runs used \n.
For reference, Sonnet at its default effort did this same task for less. In our Oct 30 test, plain Sonnet 5.5 passed all 9 checks in 3 of 3 runs at $0.283 a run, with about 15,800 output tokens. High effort here cost 36% more and used about 26% more output tokens. That was Claude Code 2.1.312, not 2.1.321, and only 3 runs, so treat it as a hint, not a result.
What failed, and the limits of this test
- 5 runs a side is a small sample. 5 of 5 vs 4 of 5 is one run’s difference, and that run failed on line endings. Read this as “both setups mostly got it right; Sonnet high cost less,” not as a pass rate.
- One slow run per side. Sonnet high run 3 took 309s and Opus low run 4 took 262s. Both spent about 218s on step 2 alone, vs 39 to 62s in the other runs, and the Opus run did it in only 10 turns. The two sides ran at the same time, so this looks like a shared slowdown rather than the model, but the logs can’t prove it. That’s why the table shows the median as well as the average.
- We didn’t see the effort level in the logs. We passed
--efforton every step, and Claude Code’s init event reports per-turn effort as active, but it doesn’t print the level. The behavior fits: Sonnet high used more output tokens than Sonnet at default effort did in October. - One task, one repo. A harder task, where Opus’s extra capability matters more, could flip this. The series continues with Haiku high vs Sonnet low and Fable 5.2 high vs Opus low.
- A smoke run of 1 per side ($0.90) came first and isn’t in the table. It ran on the wrong Node version, so
npm testfailed for the fixture, not for the model. Its results are kept with the run data.
The recipe
- If a task is well specified and checkable, try Sonnet with
--effort highbefore reaching for Opus. Here it matched or beat Opus on low effort and cost 16% less. - If you’re waiting on the result, Opus with
--effort lowcame back about 36s sooner at the median, for 75 cents more per 10 runs. - When the output format matters, say it exactly. “End the body with a newline” next to “RFC 4180” left room for
\r\n. - Grade with tests the agent never saw. Opus run 2 wrote tests that matched its own choice and passed.