Prompt tokens are rent, post-training is a purchase

This is part one of a four-part explainer on post-training for businesses. Later parts cover the data problem, the difference between supervised and preference tuning, and how to evaluate a tuned model without fooling yourself.

We see two expensive mistakes, and they are opposites.

The first is the company that decides it needs a custom model, spends four months and a lot of money fine-tuning one, and ends up with something marginally different from what a well-written prompt produced on day one. The second is the company running a two-thousand-token system prompt on ten million requests a month, paying for those tokens every single time, that has never once considered whether the instruction could live in the weights instead.

Both mistakes come from not understanding what post-training actually is. So let us be precise about that first, because the decision rule falls straight out of it.

What LoRA actually does

Full fine-tuning updates every weight in the model. For a 70-billion-parameter model that means holding the weights, the gradients and the optimiser states in memory simultaneously: several hundred gigabytes of GPU memory, a serious cluster, and at the end of it a complete second copy of the model to store and serve.

LoRA, or low-rank adaptation, starts from an observation about the update rather than the model. When you fine-tune for a specific behaviour, the change you are making to any given weight matrix turns out to be describable with far fewer numbers than the matrix contains. So instead of learning the full update, you freeze the original weights entirely and learn two much smaller matrices whose product approximates it.

The size of those matrices is set by a number called the rank, usually written r. Typical values are 8, 16, 32 or 64. With a rank in that range you end up training somewhere between roughly 0.1 and 1 per cent of the parameters, and the thing you produce at the end is not a model. It is an adapter: a file of tens to a few hundred megabytes that sits alongside the frozen base.

QLoRA extends this by quantising the frozen base to four bits, which is what makes it possible to adapt a 70B model on a single high-memory GPU rather than a rack of them.

Why that changes the business calculation

Three consequences matter, and they are all economic rather than technical.

It is cheap enough to be reversible. A LoRA run on a 7B to 13B open-weight model, on a single A100, with a few thousand examples, is typically $50 to $500 of compute. Hosted fine-tuning of a small commercial model runs $10 to $200. Those are not capital-committee numbers. They are experiment numbers, which means you can try one and throw it away.

The artifact is small and swappable. Because the adapter is a separate file, you can serve many adapters against one base model and switch between them per request. A per-customer adapter, a per-department adapter, a per-task adapter, all served from the same weights. That is a genuine product architecture, not just a training trick.

It makes owning a model practical. We have written about why we think self-reliance matters. Post-training is the mechanism that turns a generic open-weight model into something specific to your business, and LoRA is what brings the cost of doing that within reach of a normal engineering budget.

The decision rule

Here is the framing we use with clients, and it survives most arguments.

A prompt is rent. Post-training is a purchase.

Every instruction you put in a system prompt is paid for on every request, forever. Two thousand tokens of carefully written specification, at ten million requests a month, is twenty billion tokens a month of rent for the same paragraph. Post-training pays once, in data curation plus compute, and removes that recurring charge by moving the specification into the weights.

So you buy when you would otherwise be renting for a long time, which gives three conditions that all have to hold:

The specification is long. If it fits in two sentences, keep renting.
The specification is stable. If it changes weekly, buying is a mistake and you will be re-buying constantly.
The volume is high. If you run a thousand requests a month, the rent is trivial and the purchase never pays back.

The canonical sequence in 2026 is prompt, then retrieval, then post-training, then distillation, and the discipline is that you earn your way down it by proving the cheaper option has hit a wall.

When post-training is the right answer

Concretely, the wins are consistent and narrow:

Rigid output structure. A schema the model must produce every time, where prompt-based compliance sits at 95 per cent and you need 99.5.
Tone and domain vocabulary. House voice, regulatory register, the way your industry actually talks.
Tool-call and agent trajectory formats specific to your internal systems.
Distillation. Use a frontier model to generate the behaviour, tune a small open model on those outputs, then serve the small one. This is the strongest ROI case available right now: a fine-tuned 8B-class open-weight model with good retrieval in front of it is the cost-per-quality sweet spot for a lot of production work.
Behaviour that is easier to demonstrate than to describe. Some things take five hundred examples and no amount of prose.

On data: 500 to 2,000 well-curated examples generally beat 50,000 noisy ones. For narrow classification or extraction, 200 to 500 is often enough. Quality dominates quantity so heavily that it is not really a trade-off.

When it is the wrong answer

This list is longer, and it is the more valuable half.

The requirement is knowledge, not behaviour. If the answer changes when your data changes, you want retrieval. Post-training facts into weights gives you confident staleness and no citation trail.

You have not exhausted prompting. Most “we need a custom model” conversations end when someone writes a proper system prompt with six good few-shot examples.

You have no evaluation set. This is the disqualifying one. Without a fixed set of held-out cases scored before and after, you cannot tell whether the tune helped, and you will not notice the regressions.

Your examples are inconsistent. When three experts label the same case three ways, the model learns the average of your confusion. Discovering this is genuinely useful. It just is not a fine-tuning project, it is a specification project.

The task is still moving. Buying a house you are going to leave in a month.

Three things nobody puts in the quote

The adapter is welded to its base. A LoRA trained on one base model does not transfer to the next version. When the base is deprecated or upgraded, you retrain. That is a recurring cost and a form of lock-in most teams do not price in.

Narrow tuning degrades things you did not tune. Capability regression on unrelated tasks is normal, not exceptional. If your evals only cover the target behaviour, you will ship a model that is better at one job and quietly worse at five others.

The compute is not the cost. Producing a clean, deduplicated, consistently labelled set of even a thousand examples routinely takes a senior person one to two weeks. Against a $200 training run, the data is the entire budget.

That last point is the whole reason this is a series rather than one post, and it is where part two goes next.