Bench · Claude models · October 3, 2026
Which Claude should answer the phone?
Holly thinks with your own Claude. So we timed every model and effort on one spoken turn: how soon it starts talking, and whether it gets a simple sum right.
Speed and accuracy
Typical of 10 fresh sessions each. The first word is when Holly could start speaking. The time sum: "a meeting at 2:45 pm that runs 95 minutes, then a 25 minute drive. What time do I get home?" (4:45 pm).
Start-up is a new session's first reply, counted from starting it. Cost is one warm turn at Anthropic's API prices; on a Claude subscription it comes out of your usage instead. "Thinking off" uses Claude Code's switch, MAX_THINKING_TOKENS=0.
Without tools, and the thinking switch
A second run an hour later, 9 or 10 sessions each: the same turns with no tools loaded (talk only), and Claude Code's thinking switch on Sonnet. Its own with-tools row is the fair comparison, because times drift between runs.
Context is what each turn sends: Claude Code's instructions, the tester's own settings and the tool definitions. Output tokens include any thinking, which is billed as output even when it isn't shown.
Every reply
Pick a model and effort, then a question, to read every answer exactly as written, with their times.
What we learned
- Opus 5.5 was right every time, at every effort, and as fast as anything that was also right. About to the first word, the quickest start-up (), and cheaper per turn than Sonnet.
- Sonnet 5.5 got the sum wrong 2 or 3 times in 10 at low, medium and high, confidently ("You'd get home at 5:05 pm"). Higher effort didn't help. Twice it said a wrong time out loud before correcting itself, which on a call the listener hears.
- Effort barely changes speed on a short spoken turn. Opus started talking in about the same time at low, medium and high. Fable was the exception: low was about twice as quick as medium and high.
- Haiku 4.5 is only the fastest with thinking off, and then it got the sum right . Claude Code turns Haiku's thinking on by default, which makes it right every time but slower to start than Opus.
- Only Haiku's thinking can be turned off from Claude Code. On Sonnet 5.5 the same switch didn't change how much it wrote, and its 10 of 10 in the first run was 8 of 9 in the second, so we read that as chance. Opus 5.5 and Fable 5.1 always think (low effort keeps it short).
- No tools cut each turn's cost to about a third, but the first word came later. Without tools the models thought first more often, and Sonnet 5.5 got the sum right every time (9 of 9).
- Fable 5.1 was always right but slower to start and about five times the cost. A strong choice when Holly has real work to do, not for small talk.
How we tested
- The Mac. . Claude ran through Claude Code (
claude -p), the way Holly runs it. - Fresh sessions. Each sample was a new session that was thrown away after: a warm-up turn (the start-up time), then the two questions. Models took turns, sample by sample, so a slow minute on the network hit them all alike.
- The persona. Every session was told: ""
- Timing. From sending the question to the first word of the reply, and to the end of it.
- Right or wrong. A reply counts as right if it says 4:45 (or "quarter to five"). We also flag replies that said another time first.
What this test can't tell you
- It's one question. Sonnet's misses were the same each time ("5:05"), so they're real for this sum, but one sum doesn't measure judgment.
- It's one evening, one Mac and one network. Differences under about 0.2 seconds are noise.
- The sessions carried the tester's own Claude Code settings, which made each turn larger (that's why some replies use a name). Holly's calls load less, so they should be a little cheaper.
- A third question was dropped. It said October 3, 2026 was a Friday; it was a Saturday, and several models pointed that out.