The cheapest model can cost you more

Imagine giving a checkout bug to a coding model that charges a third of another model’s token rates.

The first fix fails a test. You send back the failure. The next change fixes that case but breaks another. The agent reads more files and tries again.

You can let it continue, raise its thinking effort, or switch models. Which choice would save money?

The example uses invented prices and usage. It compares the cost of getting an acceptable fix, including failed attempts.

Add up every attempt

Tokens are the small pieces of text a model processes. Input tokens include your instructions and conversation history. Output tokens cover what it generates. Providers commonly quote prices per million tokens. OpenAI explains these categories.

The checkout repair is finished when the bug is fixed, the relevant tests pass, and the change passes review. Until then, each correction adds to the cost.

First, compare which model to start with. Give each the original task from the same starting state. In this version of the example, the cheaper model eventually succeeds, but needs more attempts. Both fixes pass the same checks.

Here are the totals across all calls and retries, before caching or other charges:

The second model charges three times the rate but uses one tenth as many tokens. The fix costs $0.45 instead of $1.50.

That saves $1.05 per task. At an illustrative 500 similar tasks a month, the difference would be $525 a month. Review and correction time are separate costs.

These figures compare starting choices. If you switch after failed attempts, the earlier spending stays in the task’s total. The cost of finishing from that point may differ from starting fresh. The table does not price that switch.

For a task already underway, compare the additional cost and chance of success of each option. Keep the earlier charges in the final bill. If a model never produces an acceptable fix, its spending still counts.

This calculation applies to usage-based billing. With a fixed subscription, compare the plan price, included usage limits, and your time. API token rates do not automatically describe your subscription bill.

Match the model to the task

A checkout bug that spans several services requires the agent to understand how those services interact. More attempts will not necessarily fix a gap in that ability. I would test whether a more capable model can diagnose the problem reliably.

Extracting a date from a short email is a different job. A smaller, cheaper model may handle it well. The checkout result would not justify paying more for every task.

A larger parameter count does not establish which model will solve the bug. Parameters are the learned numerical values inside a model, and some models use only a portion for each token. Qwen3-235B-A22B has 235 billion total parameters and 22 billion active parameters. Qwen’s announcement explains the difference.

In November 2025, Anthropic reported that Opus 4.5 at medium effort matched Sonnet 4.5’s best score on SWE-bench Verified while using 76% fewer output tokens. SWE-bench Verified tests software bug fixes. This was a vendor-reported benchmark result, not a measure of savings on our checkout task. Anthropic’s report shows why a more expensive model is worth testing for token efficiency.

Check thinking effort

Before replacing the model, check its effort setting. Many models let you control how much work they spend on a problem.

Higher effort can improve difficult reasoning, but it can also increase token use and waiting time. Claude’s effort control affects thinking, explanations, and tool calls. It guides behavior rather than setting a strict token allowance. Claude’s documentation describes the tradeoff.

For the repair, compare a capable model at moderate effort with a cheaper model at high effort. Measure which produces acceptable fixes more often and at a lower cost. On routine work, test whether lower effort preserves the quality you need.

The visible answer is not the whole output bill. A reasoning model can generate internal thinking tokens before answering. OpenAI bills these as output tokens. Its reasoning guide explains the accounting.

Long explanations add tokens too. Ask for the detail you need, but check actual usage. A shorter final answer does not prove the model used less reasoning or made fewer tool calls.

Token counts can also change between models. Anthropic reported that Opus 4.7’s updated tokenizer could turn the same input into roughly 1.0 to 1.35 times as many tokens as before, depending on the content. A tokenizer divides text into tokens. The migration notes give another reason to measure usage after changing models.

Check whether repeated input is cached

During the repair, the agent may send the same project instructions and conversation history repeatedly. The table charged all input at the ordinary rate. Your provider may charge less for some of it.

Prompt caching reuses processing already done for a matching beginning of the input. The model still creates a new answer. OpenAI’s caching guide explains how this works.

A cache write stores processing for later reuse. Some providers charge extra for it. A cache read, or hit, reuses that processing and may cost less than processing the input again. Savings depend on how much gets reused, how long it stays available, and the write and read prices. Claude documents these billing categories.

Open models can support caching too. If a model’s weights are available to download, the company serving it still sets the hosting features and prices. Check the exact model and service you plan to use.

Look in the provider’s usage dashboard for cached-input usage and any separate write or read charges. If it lacks detail, ask the person maintaining the integration to check the usage fields returned with model responses. Groq and Claude document these fields.

Test a first request and repeated requests, with realistic pauses between them. Compare the actual input charge, including writes, with the charge for the same input without caching. The advertised discount applies only to eligible cached input.

Provider examples and caching conditions

Claude’s documented standard five-minute cache write costs 1.25 times the ordinary input rate. A standard read costs 0.1 times that rate, with exceptions for some models. Reuse can offset the write premium. Entries that are never reused cannot produce that saving. Changing the reusable beginning of the input can prevent a match. Claude lists the conditions and prices.

Fireworks says caching is enabled by default across its models and deployments. Cached-input discounts on its shared service vary by model. On dedicated hardware, caching can let the same machines handle more work. That benefit differs from a lower per-token charge. Fireworks documents both cases.

Groq documents automatic caching and a 50% cached-input discount for a specific set of GPT-OSS models. This does not establish support for its full catalog. Groq lists the supported models.

DeepSeek’s own API documents automatic caching and separate prices for hits and misses. See its caching guide and pricing.

Before testing, check model support, minimum input length, cache lifetime, and setup requirements.

Measure accepted work

One repair cannot tell us whether switching models will reduce the team’s costs. Compare a batch of similar tasks:

Model cost per accepted task = total model spending across all attempts ÷ tasks that pass the agreed checks.

Include failures, retries, and switches to other models. Record the completion rate as well. A model that finishes only the easiest jobs should not look better because the rest were omitted. If nothing passes, there is no successful-task cost to report.

Add paid tools, the computing environment, and human review or repair to the wider comparison. For self-hosted models, count hardware or rental, how much it is used, and the work of running the service.

Use the charges you actually incur. Include cache writes and reads where they are billed. If reasoning is already included in output charges, do not add it again.

The paper AI Agents That Matter argues that judging agents on accuracy alone can reward unnecessary complexity and cost. I want both measures for the work I need done.

Use benchmarks to choose candidates

Datacurve’s DeepSWE coding benchmark helps me choose models to test. Its leaderboard shows success rate, average cost, output tokens, and agent steps. It also lets you compare effort levels. The leaderboard gives more useful evidence than token rates alone.

DeepSWE runs models in a common agent setup. My tools, instructions, codebase, and checks may differ. I would inspect the task mix, retry allowance, grading method, and uncertainty around the scores before treating a small lead as meaningful. DeepSWE explains its method.

Use the results to choose candidates, then test them on your own work.

Build a benchmark from your own work

Put the checkout bug in a custom test set with other tasks the agent will receive. Include routine work, difficult work, and cases where earlier attempts failed. Define what passes before testing models.

For a coding team, that could include three types of problem:

• Add a form validation rule while preserving valid submissions.
• Fix an intermittent bug across several parts of the application, with a test that reproduces it.
• Add a feature while preserving older behavior and following the project’s conventions.

Adapt the set to your work. A support team should test support problems.

I suggest starting with 10 representative problems and three runs per configuration. A configuration includes the model, provider, and effort setting. Two configurations mean 60 task runs. This is a starting proposal, not enough to establish small performance differences with confidence.

Assign someone who understands the work to define the checks and judge results. For coding tasks, use an engineer who can review the changes. Count the model and tool charges, plus the time spent preparing and judging the tests. Set a spending limit before running them.

Compare that evaluation cost with the savings you expect from changing models. Expand the test set when the decision needs more evidence.

Give each candidate the same starting state, requirements, information, and tools. Record its settings and limit retries. Reserve some problems for the final comparison instead of using all of them to tune prompts.

For each configuration, record accepted tasks, total charges, token use, repeated tool calls, elapsed time, and time spent on review or repair. Compare routine and difficult work separately. Check whether higher effort or caching improves the measured result.

Repeat the comparison after significant changes to the model, provider, effort setting, or agent tools. Update the problems when your work changes. Keep a stable subset so results remain comparable over time.

Choose what to do next

In the example, starting with the higher-priced model costs $0.45 rather than $1.50. Both fixes are assumed to pass the same checks. Before choosing that model for similar repairs, I would test whether the advantage holds after review time and actual caching charges are included. A cheaper model can still be the right choice for routine work.

For the repair already underway, earlier failed attempts remain part of the bill. I would compare the likely cost and success of the remaining options. Increase effort when tests show it helps. Switch when another model handles that kind of problem better. If missing information or broken tools are causing the failures, fix those first.

For the next repair, start with the configuration that completed similar work reliably at the lowest total cost, within the time available.

Sources checked on 14 September 2026. The example is illustrative. No comparative model benchmark was run for this article.