> ## Content Index
> Fetch the complete content index at: https://varops.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# Amazon swapped Claude Sonnet for a small decision model and cut its ticket-labeling cost about 98%
- URL: https://varops.com/amazon-swapped-claude-sonnet-for-a-small-decision-model-and-cut-its-ticket-labeling-cost-about-98/
- Published: 2026-11-10T12:39:02.000Z
- Updated: 2026-11-10T12:39:02.000Z
- Description: Two Amazon engineers ran Claude Sonnet and a small decision model side by side on live tickets before switching. The bill, the switch that made it safe, and the one number the write-up left out.
- Author: Rex Factor
- Tags: Proof of Work

*A lot of the AI spend this magazine has looked at goes on models writing text nobody reads. This one is a ticket queue where the only output that mattered was a single label out of 20\. Two Amazon engineers took Claude off that job, gave it to a model that can only choose, and ran the two side by side on live tickets before they trusted either.* [***Rex Factor***](https://varops.com/columnist/rex/) *walks through the bill, the switch that made it safe, and the one number the write-up left out. —* [*Muximus*](https://varops.com/columnist/muximus/)

---

Two Amazon engineers took Claude Sonnet 4.6 off an internal ticket-labeling job and gave it to a small model that can only pick from a list. At the list prices they quote, the cost per million tickets went from about $22,200 to about $500, a cut the authors put at 98%. The average call went from 4.2 seconds to 0.9.

The part to copy is the switch: a setting that let them run the old model and the new one on the same live tickets, compare every answer, and go back in under a minute.

The evidence is their own write-up, published November 9 on the Strands Agents blog by Carr Onstott and Mehul Jaiswal. It is a case study with real numbers. It is also AWS publishing about an AWS project, and it leaves out the figure an operator would want first: how often the two models agreed.

## The job

Amazon and AWS teams had used Sonnet 4.6 to label tickets in an internal system the post calls "one of the world's largest" internal ticketing systems. A ticket could be a customer escalation, a capacity request, an automated alarm or a security finding. Each one had to land in one of 20 categories. Sonnet ran with a long system prompt on every ticket, generating JSON and reasoning against the fixed set of labels.

Strip that down and it is a multiple-choice question asked at enormous volume.

Twenty answers. Pick one.

The replacement was Strands Decider, a decision model the Strands team released on October 1\. [A decision model does not write text](https://varops.com/how-to-move-a-high-volume-yes-no-ai-step-to-a-decision-endpoint-without-trusting-vendor-benchmarks/). It takes some state and a question with a set of options, and returns a probability for each option plus a confidence score. Here the state is the ticket and the options are the label catalog.

## How they pulled it off

They started with the original 2-billion-parameter Strands Decider. On a test set, the E4B Gemma 4 build did clearly better, so they switched to it, then tuned how confident the model had to be before a label went on a ticket.

The fitting work is where the effort went. Strands Decider has a 4,096-token window, and the production label descriptions alone were longer than that. So the engineers rewrote each of the 20 labels as a one- or two-sentence criterion and sent them as the options of a single question, plus a "none" option. On long tickets, the oldest comments get dropped to fit.

The plumbing is standard AWS. Ticket events land in an SQS queue, which triggers a Lambda function, which calls a SageMaker endpoint running the model on ml.g5.xlarge instances, each with one A10G GPU and 24 GB. The model used about 19 GiB, and a request took about 640 ms.

Then the switch.

On every message, the Lambda reads a three-way setting from AWS AppConfig:

- **BEDROCK:** Sonnet labels every ticket. The decision model is not called.
- **DUAL:** both run in parallel. Sonnet's answer goes on the ticket and is compared with the decision model's.
- **DECIDER:** the decision model labels every ticket. Bedrock is not called.

The authors say this is what they would copy regardless of model. Running both on real traffic and judging every disagreement, in their words, "told us more than any offline eval would have." Going back was "a 45-second config change."

## By the numbers

|                              | Claude Sonnet 4.6 | Strands Decider E4B |
| ---------------------------- | ----------------- | ------------------- |
| Average time per call        | 4,215 ms          | 897 ms              |
| Input tokens per ticket      | 6,554             | 2,009               |
| Output tokens per ticket     | 170               | 1                   |
| Cost per ticket (list price) | $0.0222           | about $0.0005       |
| Cost per million tickets     | about $22,200     | about $500          |

The arithmetic holds up. At the Bedrock prices the post quotes - $3 per million input tokens, $15 per million output - 6,554 input and 170 output tokens come to $0.0222 a ticket. On the other side, one instance at $1.408 an hour handles about 5,600 tickets an hour at 640 ms each. That is about $250 per million at full use. With headroom kept for bursts, the authors say it comes to about $500 in practice. Call it 44 times cheaper.

One small mismatch. On the day Sonnet went off, the post's chart shows average Lambda duration falling from 3,312 ms to 748 ms. The text calls that a 74% drop. Those two figures work out to about 77%. It does not change the story; it is just the kind of thing worth checking in anyone's case study, including this one.

And one finding the engineers did not expect: JSON key order changed the answer. The same ticket content, with keys in a different order, moved output probabilities by up to 0.19\. They now sort keys on every request.

## What the write-up does not show

The accuracy claim is the thin part. The post says the decision model "performed just as well" and that E4B beat "production Sonnet on a separate set of our tickets." There is no accuracy figure, no test-set size, and no count of how often the two models disagreed in DUAL mode. DUAL mode existed to produce exactly that count, and the post does not publish it.

"Millions of dollars" a year is not tied to a ticket volume either. By our arithmetic, at about $21,700 saved per million tickets, $1 million a year takes roughly 46 million tickets. That is plausible at Amazon's size. The post does not say, and we could not check it. The costs are list prices, and the post says so. It also says they leave out the Lambda, which now runs for less time per ticket, so that omission understates the saving. Two other things are missing that the post does not mention: the engineering time, and whatever Amazon actually pays itself for Bedrock.

The comparison is narrow, too. The post does not test a cheaper LLM, or a conventional classifier trained on tickets Amazon has already labeled. Commenters on Hacker News asked about both. If you run a step like this, ask it of your own setup before you ask it of Amazon's: was the expensive model the right baseline, or just the first thing that worked?

Then there is the interest.

Strands Decider and the Strands blog are AWS projects, and the post ends by pointing readers to the repo. That does not make the numbers wrong. It makes them [the vendor's numbers about the vendor's own product](https://varops.com/typesafes-new-model-is-about-76x-cheaper-than-the-model-it-names-as-its-own-closest-match-4/), measured by its own engineers.

## Where it stops working

The authors are specific. It worked because the task is a closed choice from a known catalog, the volume is high, and the latency of the old path mattered. They say a decision model probably cannot do the LLM's job if the labels need explanation text, if the catalog changes weekly, or if the useful context does not fit in a few thousand tokens.

## The method an operator can copy

Any team with an LLM in a production path may have a step like this, where the output is a pick from a fixed set - a label, a route, a yes or no. List those calls. For each one, run Amazon's own tests: is it a closed choice from a known catalog, is the volume high, and does the latency of the current call matter? Then the disqualifiers: the answer needs explanation text, the catalog changes weekly, or the useful context will not fit in a few thousand tokens.

Where the tests pass and no disqualifier applies, build the three-way switch before choosing a replacement: old model only, both in parallel with the old answer applied, new model only. Run the parallel mode on real traffic, read every disagreement, and price both paths per million calls.

Amazon's numbers are the yardstick: about $22,200 against about $500 per million, 4.2 seconds against 0.9, and a rollback measured in seconds. Then record the number Amazon did not publish - the disagreement rate - because that is the one that decides whether the cheap path is safe to keep.

From thinking to doing. 

[![CTA Image](https://11c0f2c-126c06r.lingersh.space/content/images/2026/10/logo-black.jpeg)](https://varops.com/transformation/) 

****From proof of concept to proof of value.**

Proof matters more than promises. VarOps doesn’t stop at recommendations or prototypes. We work with companies to install, operate, and improve AI systems that deliver results in the real world.

[See How We Deliver → ](https://varops.com/transformation/) 

## Sources

- Carr Onstott and Mehul Jaiswal (Amazon), "Amazon saves millions by replacing Sonnet 4.6 with Strands Decider," Strands Agents blog, November 9, 2026: [https://strandsagents.com/blog/amazon-saves-millions-replacing-sonnet-with-strands-decider/](https://strandsagents.com/blog/amazon-saves-millions-replacing-sonnet-with-strands-decider/?ref=varops.com)
- Strands Agents, "Introducing Strands Decider 2B: a small, open source, decision model," October 1, 2026: [https://strandsagents.com/blog/introducing-strands-decider/](https://strandsagents.com/blog/introducing-strands-decider/?ref=varops.com)
- Strands Decider repository: [https://github.com/strands-labs/strands-decider](https://github.com/strands-labs/strands-decider?ref=varops.com)
- Hacker News discussion: [https://news.ycombinator.com/item?id=50436640](https://news.ycombinator.com/item?id=50436640&ref=varops.com)