Sonnet 5.5’s benchmark gains come with a token-budget lesson

Artificial Analysis published initial Claude Sonnet 5.5 measurements on September 28, reporting an Intelligence Index score of 56 at maximum effort, two points behind Opus 5.5. It also reported unusually high output-token usage and a $7.60 cost per task in that configuration. The evaluator said testing used a prerelease deployment with a structured-output issue that Anthropic fixed for public release.

That qualification matters. The report is evidence about a particular tested deployment, and the evaluator indicated that relevant measurements would be rerun. It should not be converted into an unconditional claim about every request sent to the released model.

The smaller tier can spend more

A lower price per token does not guarantee a lower price per answer. A model that uses many more tokens to reach a result can exceed the cost of a more expensive model that finishes quickly. The same issue affects latency and the amount of material a user must inspect.

For an application choosing between effort settings, plot successful task completion against total cost. Begin with the lowest setting that might meet the requirement and increase effort only where it produces a meaningful gain. Some jobs may benefit from extra reasoning, while routine extraction may gain little.

Preserve failure categories. A request that violates an output schema differs from one that returns a valid but incorrect answer. The first may be repairable through a small retry; the second may require better context, a different model, or human review. Combining them hides where additional spending helps.

Budget by workload

Separate short interactive tasks from long autonomous jobs. The former may have a strict latency ceiling. The latter may tolerate a longer run but need an explicit spending cap. A single default chosen from a leaderboard can be a poor fit for both.

Use a fixed set of historical tasks when comparing configurations. Record model revision, effort, tool access, caching, and whether a fallback occurred. Without those details, a cost reduction can be caused by changing the work rather than improving the configuration.

The early Sonnet 5.5 results illustrate why model selection is becoming a systems exercise. Capability, token consumption, runtime, and output contracts interact. The useful decision is the least costly configuration that consistently meets the application’s requirements, with a clear escalation path for the cases that need more computation.