Hallucination

Definition

A hallucination occurs when a model generates content that seems credible but is false, invented or unsupported: a non-existent bibliographic reference, a wrong date, a detail added to a summary that was not in the original text.

Why it happens

A language model is trained to produce the most plausible continuation, not to check facts. If it lacks the information, or if it is rare in its data, it can still produce a fluent answer. Erroneous or contradictory training data and ambiguous instructions also contribute.

How to limit them

• Provide the sources in the prompt, or use a RAG system.
• Ask for citations of the passages used, then check them.
• Allow the model to say it does not know.
• Verify important facts against reliable sources: figures, names, references, quotations.
• Use external tools (search, calculator) for factual information and calculations.

Caution

These methods reduce the risk but do not eliminate it. For any high-stakes use (health, law, finance), human review remains necessary.