Hallucination
Last verified: June 2026· definition
Hallucination happens because language models generate likely continuations, not verified facts. The output can be fluent and authoritative while being incorrect — inventing an API, a citation, or a number.
Mitigations include grounding the model with retrieved, citable context (RAG), constraining output with schemas, adding verification steps, and measuring with evals so hallucination rates are visible.
In agentic coding specifically, hallucinated APIs and dependencies are a real risk, which is why tests, type checks, and review hooks matter for AI-generated code.
It is the same mechanism as everything else
A model generates plausible continuations. When the training data supports a plausible continuation that is also true, we call it a correct answer; when it does not, we call it a hallucination. There is no separate failure mode being triggered, which is why hallucination cannot be patched out by instruction. It also explains the characteristic signature: fabrications are fluent, well-formed, and confident, because fluency is what the process optimises for and truth is not a variable it tracks.
Grounding is the practical defence
Retrieval reduces hallucination substantially by putting the relevant facts in front of the model rather than relying on what it absorbed in training. It does not eliminate it — a model can still misread retrieved text or fill gaps — but it changes an unbounded problem into a bounded one. Requiring citations sharpens this further, since a claim tied to a specific retrieved passage is one you can verify mechanically.
Where fabrication is most likely
Certain requests reliably provoke it: specific identifiers like API names, function signatures, case citations, or version numbers; anything recent enough to fall outside training data; and questions premised on something false, where the model tends to answer the premise rather than challenge it. These are worth treating as high-risk routes deserving verification, rather than expecting uniform reliability across all queries.
Detection is easier than prevention
Since the mechanism cannot be switched off, the practical posture is catching fabrications rather than eliminating them. Where output is checkable, check it mechanically: type checkers and test suites catch invented functions, schema validation catches invented fields, and a lookup against your own data catches invented records. Where it is not mechanically checkable, requiring citations at least makes verification cheap for a human. Both approaches accept that the model will occasionally fabricate and ensure it does not reach the user unnoticed.
Uncertainty needs an escape hatch
A system that offers no acceptable way to decline effectively requires an answer to every question, which guarantees fabrication when the information is absent. Explicitly permitting a no-answer response, and treating it as a success rather than a failure in your evals, changes the incentive. It is a small prompt change with a disproportionate effect, and it is frequently missing from systems that otherwise ground carefully.
Common misconceptions
MythTelling the model not to make things up prevents hallucination.
RealityIt helps at the margins and provides no guarantee. The model has no reliable internal signal separating what it knows from what it is constructing, so it cannot consistently act on the instruction.
MythHallucination means the model is lying.
RealityLying requires knowing the truth and stating otherwise. The model is producing the most plausible continuation available to it, with no access to a notion of accuracy — the output is wrong without being deceptive.
MythA confident answer is more likely to be correct.
RealityConfidence in phrasing is a property of the writing style, not of accuracy. Fabricated content is often expressed more fluently than hedged correct answers, because uncertainty in the source material tends to produce hedged language.
Frequently asked questions
How do I reduce hallucination in a production feature?
Ground the model in retrieved source material rather than relying on training knowledge, and require citations so each claim maps to a passage you can check. Give the model an explicit way to say it does not know, since a system offering no acceptable failure response effectively forces it to fabricate. Then measure: an eval set covering questions your corpus genuinely cannot answer will catch the cases where it invents an answer anyway.
Does RAG eliminate hallucination?
No, but it constrains it considerably. The model can still misinterpret retrieved text, blend passages incorrectly, or fill gaps where retrieval returned nothing useful. What RAG changes is that answers are anchored to material you control and can inspect, which makes errors both less frequent and far easier to detect than free-form generation from training data.
Why does the model invent API functions that do not exist?
Because it has seen a great many similar-looking APIs and is generating what a plausible function would be called in that context. Naming conventions are highly patterned, which makes fabricated names look entirely credible. The reliable defence is mechanical rather than prompt-based: let the type checker, linter, or test suite catch invented symbols, since those tools verify existence in a way the model cannot.