Hallucination

It isn't lying, and it isn't malfunctioning#

Both of those framings mislead. Lying requires knowing the truth and choosing otherwise. Malfunctioning implies a working state it has fallen away from. Neither applies.

A language model produces text by repeatedly answering one question: given everything so far, what plausibly comes next? That is the whole mechanism. There is no separate stage where a draft gets checked against a store of facts, because there is no store of facts to check it against. What the model has is a vast statistical sense of how language about the world tends to go.

Most of the time, plausible and true coincide. The most plausible completion of "the capital of France is" is "Paris" precisely because that is true and the truth is written down everywhere. The mechanism gets accuracy for free, as a side effect of pursuing plausibility.

But when the truth is obscure, absent, or contested, plausibility keeps working perfectly well without it. Asked for the title of a paper that doesn't exist, the model does exactly what it always does and produces a title that reads like a real one. Nothing has gone wrong internally. The same process that produced "Paris" produced the fake paper. That is the uncomfortable part: fluency and accuracy come out of one process, so you cannot have one without risking the other.

"Confabulation" is the better word#

Hallucination is the term that stuck, but it's a poor metaphor: hallucinating means perceiving something that isn't there, and the model isn't perceiving anything. Clinicians have a more precise word for fluently and confidently filling a gap without any awareness that you're filling it, and that word is confabulation. It describes this behaviour far better.

Worth knowing, partly because it's more accurate and partly because it predicts the behaviour better. Confabulation is not random noise. It is coherent, contextually appropriate, and delivered with the same manner as everything else. That's exactly what makes it hard to catch.

Why it sounds so certain#

The tone is the trap. A model's confident register is not evidence of internal confidence, because the register is just more generated text. It writes "the study found" the same way whether the study exists or not, since that's how sentences about studies are phrased.

There is genuine uncertainty in the machinery, but by default very little of it surfaces in the prose. Nothing forces a model to write "I'm about 60% sure of this," and quite a lot of its training rewards sounding helpful and fluent. So you get a uniform confident voice across a very wide range of actual reliability, which is the single most dangerous property of these systems for anyone using them for real work.

Where it happens most#

This is the practical part, because the risk is not evenly spread. Raise your guard sharply on:

• Specific numbers. Statistics, prices, dates, measurements, scores. A plausible-looking number is trivially easy to generate and almost impossible to spot as fake.
• Citations, URLs and quotes. The worst category by far. These have highly regular formats, so a fabricated one looks structurally perfect. Invented papers with real-sounding authors in real journals are a classic, and dead or wrong URLs are routine.
• Obscure topics. The less written about something is, the thinner the statistical signal, and the more the model is effectively improvising in the right style.
• Anything recent. Past the training cutoff the model has nothing, but it will often answer anyway rather than decline. There's a subtlety here: vendors distinguish a training data cutoff from a reliable knowledge cutoff, the latter being earlier. Events near the boundary are in there thinly, which is arguably worse than absent, because thin coverage produces confident half-knowledge.
• Precise details about your own material. Ask which page a clause appears on and you may get a page number invented to satisfy the question.

The pressure problem#

Here is a failure mode most people trip over without noticing. If you push back on a correct answer, a model will frequently abandon it.

Ask "are you sure? I thought it was 2019," and you have not requested a verification. You have supplied a strong hint about what completion you'd find plausible, and plausible-continuation is the only game being played. Agreement is often the most plausible next move. The model will apologise, adopt your date, and sound just as confident as before.

So "are you sure?" is close to useless as a checking tool, and worse than useless if you telegraph the answer you expect. If you must probe, keep it neutral and ask for the basis: what is that based on? A real source can be checked. A fabricated one usually falls apart when you go looking for it.

What actually reduces it#

Supply the source material. The single most effective habit, by a distance. A model asked to summarise a document you have pasted in is doing a fundamentally more reliable task than one asked to recall the document from training. Move the work from recall to reading. This is the same insight as → Context windows, and it's why the two pages keep pointing at each other.

Ask for quotes, not summaries, when accuracy is critical. "Quote the sentence that says this" is checkable in seconds. "Summarise what it says about this" is not.

Give it a search tool, or use a product that has one. A model that can look things up and cite what it found is meaningfully more reliable on anything factual or recent. Still verify the citations, because a real search result can be attached to a claim it doesn't support.

Explicitly permit not knowing. Adding "if you're not confident, say so" or "say 'I don't know' rather than guessing" genuinely helps, because it makes admitting ignorance a plausible completion rather than an unhelpful-seeming one.

Ask it to work in the open. Requesting reasoning before the answer, rather than an answer alone, surfaces more of the path and gives you something to inspect.

What doesn't work

• Telling it not to hallucinate. "Do not make anything up" is a reasonable thing to write and does almost nothing, because the model has no reliable internal flag for which of its output is invented. If it could tell, this page wouldn't need to exist.
• Asking "are you sure?" See above. It measures agreeableness, not accuracy.
• Asking it to grade its own confidence. The number you get back is itself generated text, and treating it as a real probability is a category error.
• Assuming a newer or more expensive model has solved it. Better models confabulate less and hide it better, which does not straightforwardly make them safer to trust unverified.

Two results worth knowing about#

The account above is the field's settled explanation, and for a long time this page said so without pointing at anything you could go and read. Two results are worth naming, because they sharpen it in different directions.

Some of it is provably unavoidable. A 2023 result, later published at a theory conference, shows a statistical lower bound on how often a pretrained model must produce false statements of a certain kind — not because of a flaw in the architecture or dirt in the data, but as a consequence of the model being well calibrated — that is, assigning probabilities that match how often things really turn out to be true. Roughly: for facts that appear exactly once in the training data, and whose truth cannot be established from that data, the rate of invention is close to the fraction of facts that appear exactly once. If a model is going to assign sensible probabilities to things it has seen a single time, some of what it produces about them will be wrong. That is a stronger claim than "it isn't malfunctioning." It says a certain amount of this is what working correctly looks like.

The rest is an incentive problem, and that part is fixable. A 2025 paper argues that invention survives not because nobody can reduce it but because of how models are graded. Most benchmarks score an answer right or wrong and give nothing for "I don't know", so a model that guesses when uncertain scores better than one that admits it, and training rewards the better score. Its authors compare it to a student on an exam with no penalty for a wrong answer, and their proposed remedy is not another hallucination benchmark but changing the scoring of the popular ones. That is a claim about the field's habits rather than about the technology, and it is the most hopeful thing on this page: some of what looks like a hard limit is a scoreboard choice.

The honest bottom line#

This is reducible but not currently eliminable, and any tool promising otherwise is overselling. The workable posture is not distrust, which wastes the tool, but calibrated verification: match your checking to the cost of being wrong.

Brainstorming, drafting, explaining, restructuring something you'll read anyway: use it freely, errors are cheap and you'll catch them. Anything load-bearing where a wrong specific has consequences: verify the specifics against a real source before it leaves your hands. Names, numbers, dates, citations and quotes are the five things to check every time, because they are simultaneously the most likely to be invented and the most damaging when they are.

Sources#

The distinction between a training data cutoff and an earlier reliable knowledge cutoff is drawn from Anthropic's models overview, re-read 14 August 2026, which publishes both dates separately for each model. No hallucination-rate figures are quoted because rates vary enormously by model, task and measurement method, and a single number would be more misleading than no number.

The unavoidability result is Calibrated Language Models Must Hallucinate (Kalai and Vempala, November 2023, published at the ACM Symposium on Theory of Computing in 2024), which shows "an inherent statistical lower-bound on the rate that pretrained language models hallucinate certain types of facts, having nothing to do with the transformer LM architecture or data quality," and ties that rate to "the fraction of facts that occur exactly once in the training data (a 'Good-Turing' estimate), even assuming ideal training data without errors." It is a mathematical result about a class of models, not a measurement of any particular one, and it applies to arbitrary facts whose truth the training data cannot settle — not to everything a model says.

The incentives argument is Why Language Models Hallucinate (Kalai, Nachum, Vempala and Zhang, September 2025), which argues models "hallucinate because the training and evaluation procedures reward guessing over acknowledging uncertainty," that such errors "originate simply as errors in binary classification," and that the remedy is "modifying the scoring of existing benchmarks that are misaligned but dominate leaderboards, rather than introducing additional hallucination evaluations." That last part is an argument the authors are making, not a settled finding, and it is described that way above. Both papers read 14 August 2026.