> ## Content Index > Fetch the complete content index at: https://varops.com/llms.txt > Use this file to discover other available public pages before exploring further. # How to tell whether a workflow is worth training your own model for - URL: https://varops.com/how-to-tell-whether-a-workflow-is-worth-training-your-own-model-for-before-you-rent-another-year-of-frontier-calls/ - Published: 2026-07-30T10:45:24.000Z - Updated: 2026-07-30T10:45:24.000Z - Description: A $500 fine-tune beat five frontier models on one workflow - but the win is narrow. Two questions decide whether owning a model beats renting the frontier for any of yours. - Author: North Wayne - Tags: First Opinion *Every few weeks a chart makes the rounds proving a small model, trained for an afternoon, beat the frontier at a fraction of the price - and every few weeks a room full of operators quietly wonders whether they are overpaying for intelligence they could own. This week's chart is real.* [***North Wayne***](https://varops.com/columnist/north/)*'s job in First Opinion is to hand you the part the chart leaves off: the win is narrow, it lives in exactly two conditions, and most workflows meet neither. She names the thumb on the scale too, including ours. Read it before anyone approves a training run. —* [*Muximus*](https://varops.com/columnist/muximus/) A vendor called Fermisense took one high-volume workflow - reviewing e-commerce product listings - and trained a 9-billion-parameter open-source model on it for about $500 of GPU time. It beat all five frontier models it tested against, at a fraction of the cost per decision. The number everyone is repeating is 87.3 percent against 76.9\. The number that should decide anything for a leader is smaller: the two questions that tell you whether any of your own workflows is shaped like that one. Most of them are not. This is a real result and a narrow one, and the distance between those two facts is exactly where budgets go to die. ## Take the win seriously first The fastest way to make a bad decision here is to wave the result away as vendor theater. It isn't. On Fermisense's catalog task, a trained Qwen3.5-9B reached 87.3 percent of the achievable score; the best of five frontier configurations - GPT-5.5, GPT-5.6-sol, Gemini 3.1 Pro, Claude Opus 4.8, Claude Fable 5 - reached 76.9, and the untrained 9B base sat at 64.2\. The five frontier models, even with hand-tuned prompts, landed within a tenth of a point of one another. Training cost about $500: two GPUs, three and a half days, a thousand steps on an open-source framework, and it passed the frontier pack after roughly the first day. And this is not only a vendor's story. In June, Bridgewater's AIA Labs, with Thinking Machines, published a careful write-up of training an open model on financial information-triage work. Frontier models managed around 50 percent accuracy cold and stayed under 80 even with expert-written prompts; the trained model hit 84.7, which Bridgewater reports as 29.8 percent fewer mistakes than the best frontier model it tested, at roughly a fourteenth of the inference cost. Different model, different task, different tooling - same shape. The reason the small specialist wins is not intelligence, and getting this wrong is what leads people to over-apply it. A frontier model walks into every call cold. It has never seen this company's taxonomy, its conventions, or how it wants the awkward cases resolved, and it rebuilds all of that from the prompt every single time. A trained model carries that judgment in its weights. On a narrow task the business runs a million times, that is decisive. On almost anything else, it is beside the point. ## The real decision is applicability, not capability So the question on the table is not "should we train our own model." That question is rigged - it draws a yes out of everyone selling training and a no out of everyone still sore from a model that never made it to production. The useful question is which specific workflows, if any, sit in the narrow band where owning beats renting. This is a fit decision, the same kind you make when you decide whether to hire for a role or outsource it, and it turns on two things. First: does the work run at enough volume that cost per decision and errors per decision add up to real money? A model that shaves a fraction of a cent off a call is worth nothing across a thousand calls a month and worth a budget line across tens of millions a day. Second: can every outcome be checked by a rule, a test, or a rubric, with nobody in the loop? This is the harder gate, and it is the one that quietly disqualifies most candidates. Fermisense puts it well: if a decision can be scored, a model can practice it; if it can only be argued about, it cannot. Routing a ticket, pulling a field from a document, classifying a product, approving or flagging a transaction - each has a right answer you can check. Writing the strategy memo does not. A workflow earns a closer look only if it clears both gates. Held against the fuller version, adapted from Fermisense's own list, a strong candidate looks like this: - It runs at high volume, often enough that cost and error compound into real money. - Every outcome can be checked by a rule, test, or rubric, with no person needed to judge it. - Your own experts agree on what a correct answer looks like. - A capable model already gets it right sometimes, just not reliably enough. - The right answer can't be a lucky guess. - It runs across several steps: reasoning, tool calls, then a committed decision. - It lives on your own tools, schemas, and policies. - Different mistakes cost different amounts - a missed fraud flag is not a false alarm. - Sensitive data can't leave infrastructure you control. Fail the first two and the rest are noise. Clear them and the rest tell you how good a candidate you have. ## Four things that don't survive the trip from demo to production The result travels. The way it was measured does not always travel with it. Four things are worth saying out loud before anyone signs off on a training run. The first is the one the internet caught within hours of the post. The specialist was trained by reinforcement learning against the very scorer used to grade it - a simulated version of the workflow the vendor built itself. That is a legitimate way to train. It also means 87.3 percent is a score on a benchmark the model was tuned to beat: self-built, self-scored, not an independent test. Bridgewater is more careful on this exact point - it checked its labels and graded on a held-out test set it did not train against. The buyer's move is to trust the mechanism and verify the number on your own data, never on the vendor's twin. A model that aces its own rubric has proved it can learn a rubric, not that the rubric matches your reality. The second is the prompt tax. To push the frontier models to their best scores, Fermisense fed them 2,800 characters of optimized instructions, and that raised measured input-token cost 28 to 55 percent per call - on every call, forever. Prompted knowledge is rented by the token every time it runs; trained knowledge is bought once and lives in the weights. If the current plan for a high-volume workflow is an ever-lengthening system prompt, that bill is already being paid and rarely counted. The third is scale. The headline numbers - roughly 68 times cheaper, seven million dollars a year instead of five hundred - are figured at about 40 million decisions a day. That is Shopify's volume, not a mid-market company's. The per-call gap is real at any size; the absolute savings that pay for the engineering only show up when the call count is enormous. Run the math on your own volume before it means anything, and be ready for the honest answer at most companies: a smaller number that may not clear the cost of owning the pipeline. The fourth is the gap between feeling faster and being measurably better. METR's 2026 survey found technical workers self-reporting a median 1.4-to-2x gain in the value of their work from AI - while METR's own earlier controlled trial found developers overestimated AI's effect on their time by about 40 percentage points. Self-report is not measurement. The very thing that makes a workflow trainable - a scorer that grades outcomes without a human - is also what lets a leader claim a result at all. A workflow you can't score, you can't train, and its "improvement" won't survive a CFO's questions either. ## Sequence it, and mind the interests Owning a model is not the opening move, and it is not the same as canceling a subscription. The pattern that works starts on rented frontier models. They set the baseline for what is even possible and, more usefully, they generate the raw material - the inputs, the decisions, the corrections - that a specialist later trains on. Only when a workflow has left the prototype stage, proven its volume, and earned a scorer worth training against does owning start to pay. The frontier finds the workflow; a trained model runs it at scale. Two interests belong on the table. Fermisense sells this service, and for all its published detail the write-up is a lead magnet with a "book a free audit" button under every section, built on a benchmark it created and trained against - a strong lead to reproduce, not a finished proof. And one of ours: VarOps is published by Ran Aroussi, and building this exact "own your intelligence" capability is work VarOps does for clients. That is precisely why the read here is narrow rather than enthusiastic. The honest version of this advice sends most people away from a training run, not toward one, and you should watch anyone whose advice does the opposite while their invoice depends on it. ## The verdict For most mid-market operators, the answer this quarter is not yet - and even then, one or two workflows if ever. The move worth making now is not a training run; it is an inventory. Take the work the business repeats thousands of times a day, hold each candidate against the two questions - is it high-volume, and can every outcome be scored without a person - and count the survivors. If there are none, the $500 fine-tune is somebody else's story and renting the frontier is the correct, unglamorous call. If there are one or two, that is the rare place where owning your intelligence is worth the trouble - and even there, keep renting the frontier long enough to generate the data that trains its replacement.