People talk about model choice like it’s a sports fandom. Pick a jersey. Argue on the internet. Pay whatever the API asks because “quality.”
Treat it like staffing. You don’t put a senior engineer on renaming columns all afternoon. You also don’t put the intern on the change that can take production down. Same idea with models.
Cheap models are great at drafts, extraction, classification, and “summarize this pile.” Expensive models are worth it when the next action costs real money, real trust, or a real mess to undo. The skill is routing, not loyalty.
The bill is trying to tell you something
If an agent runs all day, the cost curve is not theoretical. A fat prompt, a frontier model, and a loop that retries because the JSON was spicy will teach you humility by invoice.
Look at three numbers before Focus on a new model name:
- cost per successful task, not per token in the abstract
- latency the user actually feels
- error rate on the tasks that matter
Tokens are an input. Tasks are the business. If you only watch tokens, you will “optimize” by starving the prompt of context and then pay for support tickets instead.
Most of my waste used to be using the smartest model for work that did not need a soul. Tagging a ticket. Cleaning a title. Turning a transcript into fields. That’s clerk work. Clerk work can be cheap.
What to send to the cheap seat
A non-exhaustive list that has held up:
- classify the intent of a message
- extract name, phone, location from a transcript
- draft an outline or a first pass
- rewrite for length
- suggest tags, titles, alt text
- turn a structured object into a boring email
- score whether two strings are probably the same facility name
Still check the output. Cheap does not mean unsupervised. It means do not rent a genius to fill in a spreadsheet column.
When the cheap model fails, it usually fails in obvious ways: missed field, weird extra sentence, JSON missing a comma. Easy to catch with a schema and a retry. That’s the point. Failure should be loud and cheap, not rare and expensive in a way you only notice after a customer does.
What to save the expensive model for
Judgment. Tool choice. Anything that looks like a decision.
- should we escalate this to a human right now
- which of these three policies applies
- write the customer-facing reply that might get screenshot
- plan a multi-step tool sequence against a messy codebase
- review a cheap model’s draft and say what’s wrong
Also use a stronger model as a critic. Cheap model drafts. Strong model checks against the brief, the schema, the “never say this” list. That two-pass setup has saved me more money than switching vendors for sport.
If the output can move a calendar, a medical conversation, or money, it gets the expensive pass or a human. No debate. The routing table is a policy document, not a suggestion.
A simple router beats a clever one
Do not start with a machine-learning router that needs its own PhD. Start with rules.
- If task is extract or classify: cheap model, strict schema.
- If task is plan or decide: expensive model, smaller prompt, tools allowed.
- If task is customer-visible: expensive model plus a human when the confidence is off or the category is sensitive.
- If cheap model fails schema twice: bump once, then page a human.
That’s enough to cut a surprising amount of spend without turning the product into a lottery.
Later, if you have volume, I might add a classifier that predicts difficulty. Later is doing a lot of work in that sentence. Most teams never need it if they just stop sending every call to the luxury model.
Prompt size is part of routing
People upgrade models when they should have trimmed context. A frontier model with a novel of SOP pasted in will still get lost, and you will pay for the tour.
Keep the cheap path skinny: schema, few-shot of the actual format, the snippet that matters. you keep the expensive path richer but still bounded: the policy, the relevant records, the last few turns, not the entire company wiki.
Caching helps when the prefix is stable. It does not help when you dump a unique novel into every call and then wonder why the bill looks like a car payment.
How to explain this to a non-engineer
You already do this with people. Junior drafts the recap. Senior decides whether we admit. Reception collects the phone number. Clinical lead doesn’t sit on the switchboard.
Models are not employees. The staffing metaphor still keeps you from being silly. Put the cheap labor on the pile of drafts. Put the expensive attention on the fork in the road.
Bottom line
If you only remember one line, make it this: drafts are cheap, judgment is not. Route like you mean it, and the work gets faster without the invoice becoming a personality test.
