Does 'think hard' actually help? Same pass rate, but it tested its own work

Adding “think hard” to a prompt is one of the most repeated Claude Code tips. The idea is that it reasons more before it writes code, so it gets tricky logic right more often. We wanted to see what it changes on a task with plenty of traps.

The setup

A small subscription billing module with one empty function, billingDates(start, rule). It returns the dates a subscription bills on. The spec is a comment above the function, and it has several places where a quick implementation goes wrong:

• Month ends. Monthly billing stays on the start date’s day. A month without that day bills on its last day, then the schedule goes back to the original day (Jan 31, Feb 28, Mar 31). Working each month out from the previous date drifts to the 28th.
• Leap years and intervals. A Feb 29 start, quarterly billing from Nov 30, and yearly billing from Feb 29.
• count, until and skip together. until is inclusive, the result stops at whichever of count and until comes first, and skipped dates don’t count toward count.
• Bad input. It must throw a RangeError for dates that aren’t real, like 2026-02-30. JavaScript’s new Date('2026-02-30') quietly gives you March 1 instead of failing.
• Time zones. Weekly billing across daylight saving changes, run on a server in Chicago and one in Auckland. Date math in local time can land on the wrong day.

The prompt, the same in every run:

Implement billingDates() in src/schedule.js. The spec is in the comment above it.

We ran it 3 times each two ways:

• Plain: the prompt as written.
• Think hard: the same prompt, followed by “Think hard about this before you write any code.”

Both versions started from the same repo, with edits allowed. The repo had one test for billingDates: a mid-month start, billed monthly. Afterwards, four hidden test files Claude never saw checked the spec: month-end dates, count/until/skip together, bad input, and time zones.

What happened

Claude Code 2.1.302, Sonnet 5.5, 3 runs per column. Every run, check and diff: run data.

What we learned

“Think hard” didn’t make the code more correct. All six runs got every edge case right. Each one worked out monthly dates from the start date rather than the previous one, so nothing drifted. Each one matched dates against a YYYY-MM-DD pattern and checked the day really exists in that month, and none did date math in local time. On this task the plain prompt was already enough, so there was nothing left for “think hard” to fix.

It changed how Claude worked. Plain runs read src/schedule.js and started writing. Think-hard runs first listed the repo and read the existing test, src/plans.js and package.json. Then they wrote the function, added tests for it to tests/schedule.test.js and ran npm test. All three think-hard runs finished by saying their tests passed. All three plain runs finished by saying they hadn’t run anything.

The plain runs did try to check their work, outside the repo. Each plain run wrote a scratch script to /tmp and tried to run it with node. Our test setup only allows a few shell commands (npm test, npm run, node --test, git diff and git status), so those commands were blocked and the runs stopped and asked for approval. In a normal session you’d likely have clicked approve and the check would have run. A scratch script still leaves nothing behind, though. The think-hard runs put their tests in the repo, where they keep running.

It cost about 60% more. Think-hard runs averaged $0.091 against $0.057, took 10.0 turns against 7.0 and 34.1s against 21.9s. They also wrote nearly twice the output tokens (4,878 against 2,668 on average). The extra went into reading more files first and writing and running tests. The implementations themselves took the same approach in both columns.

Both made choices the spec didn’t ask for. Runs in both columns added rules of their own, such as stopping at year 9999 or throwing when rule is missing, and said so in their final message. Two think-hard runs and one plain run added a year-9999 limit. None of those choices broke a hidden test.

Six runs is a small sample, and we only tried one task. “Think hard” can only show a difference where the plain prompt gets something wrong, and this time it didn’t. On a harder problem, or with a smaller model, the answer could be different. We haven’t tested that yet. We also can’t see how much more Claude reasoned: both versions produced thinking blocks, but the transcripts don’t show what was in them. The output-token count is the only measure we have.

The recipe

1. Don’t expect “think hard” to fix logic on its own. On a spec with month-end, leap-year, time zone and validation traps, the plain prompt passed every hidden test 3 of 3 times.
2. If what you want is tested code, ask for tests. “Think hard” got tests added and run 3 of 3 times, but only as a side effect. Asking for them directly is cheaper and doesn’t depend on Claude deciding to. Our tests-first recipe did that.
3. Expect it to cost more. About 60% more per run here ($0.091 against $0.057), and about 12 seconds longer.
4. Put the edge cases in the spec. Both versions got every trap right, and every one of those traps was written down in the comment above the function. A clear spec did more here than an extra sentence in the prompt.