Fine-tuning
Last verified: June 2026· definition
Fine-tuning takes a pre-trained model and continues training it on examples specific to your use case, baking a behavior or format into the weights. It’s well suited to consistent tone, structured output, or a narrow skill.
It’s less suited to injecting changing knowledge — for that, retrieval (RAG) is cheaper to update and easier to keep accurate. Updating fine-tuned knowledge means retraining.
The strongest systems often combine the two: retrieval for facts, light fine-tuning for behavior and format.
It changes behaviour, not knowledge
This is the distinction that determines whether fine-tuning is the right tool. It is effective at teaching a model to respond in a particular format, adopt a house style, follow a domain-specific convention, or handle a narrow task consistently. It is a poor and expensive way to give a model facts, because facts learned this way are baked in, cannot be updated without retraining, and are not reliably recalled. If the requirement is current or private information, retrieval is the correct answer.
Data quality dominates the outcome
A modest set of carefully constructed examples generally outperforms a much larger noisy one, because the model learns the patterns present including the inconsistencies. Contradictory examples teach contradiction, and unrepresentative examples teach a distribution your traffic does not contain. Most disappointing fine-tuning results trace to the dataset rather than to hyperparameters, which is where attention tends to go instead.
It creates a maintenance obligation
A fine-tuned model is frozen at the moment you trained it. When base models improve you do not benefit without retraining, when your task drifts the model does not follow, and you now own an artefact with its own versioning and evaluation needs. That ongoing cost is why fine-tuning should generally follow, rather than precede, serious attempts with prompting and retrieval — which achieve the goal often enough to make the commitment unnecessary.
Evaluate against the base model, on your own data
The only meaningful test is whether the fine-tuned model beats the base model on your task, measured on held-out examples the training never saw. Training-set performance always looks good and says nothing. Comparing against the base is what reveals the cases teams most need to catch: a fine-tune that improved the target behaviour while degrading something adjacent it used to handle correctly.
Common misconceptions
MythFine-tuning teaches a model your company’s information.
RealityIt shapes behaviour rather than reliably storing facts. Knowledge that must be current, private, or verifiable belongs in retrieval, where it can be updated and cited.
MythMore training data always helps.
RealityQuality and consistency matter more than volume. A noisy large set teaches the noise, and inconsistent examples produce inconsistent behaviour.
MythFine-tuning is how you make a model cheaper.
RealityThat is closer to distillation, and even then routing to a smaller off-the-shelf model is usually the cheaper first experiment. Fine-tuning is about behaviour, and its cost profile includes ongoing maintenance.
Frequently asked questions
Should I fine-tune or use RAG?
They solve different problems, so the question is usually which problem you have. If the model lacks information — current facts, private documents, customer data — that is retrieval, and fine-tuning will not fix it. If the model has the information but responds in the wrong format, style, or with inconsistent conventions, that is a behaviour problem fine-tuning addresses well. Systems needing both commonly fine-tune for behaviour and retrieve for knowledge.
How much data do I need to fine-tune?
Less than intuition suggests, provided it is consistent. A few hundred high-quality, non-contradictory examples often produce a clear behaviour change, whereas thousands of inconsistent ones can produce a worse model than the base. Prioritise curating examples that agree with each other and cover the cases you care about over accumulating volume.
What should I try before fine-tuning?
Better prompting with clear output specification, a few well-chosen examples in context, and retrieval if the gap is informational. These are faster, cheaper, and adjustable without retraining. Fine-tuning is worth reaching for when those have genuinely been attempted and the behaviour still is not consistent enough — at which point you also have a good dataset, since your prompt examples and production traffic are the raw material.