What is Hallucination Detection

Definition

Hallucination detection is the use of methods and checks to identify when a generative model has produced fabricated or unsupported output, flagging responses that are not grounded in provided sources or known facts so they can be corrected, withheld, or reviewed.

« Back to Glossary Index

Flags fabricated or unsupported AI outputs before they reach users
Improves reliability of generative systems in high-stakes uses
Enables automated checks and human review of risky responses
Supports grounding by verifying answers against source data

Real World Example

A RAG system runs hallucination detection that checks whether each generated claim is supported by the retrieved documents, withholding or flagging any answer that asserts facts the sources do not contain.

FAQs

How is hallucination detected?

Through methods like checking outputs against retrieved sources, consistency checks across responses, and classifier or verification models.

Why is hallucination detection important?

It prevents confident but false AI outputs from misleading users, which is critical in high-stakes domains.

Can hallucination be fully eliminated?

Not entirely, but detection combined with grounding and review greatly reduces the risk of acting on fabricated output.