How to detect LLM hallucinations with semantic entropy

A fact database is not the only way to catch an unreliable answer. A model that confabulates tends to give different meanings when you ask the same question repeatedly, and that variation is measurable without knowing the right answer in advance.

The boundary matters more than the novelty, so it goes first: semantic entropy detects "the model does not know," not "the model is wrong." Use it to route uncertain answers toward review. Do not use it to certify confident ones.

What is semantic entropy?

Uncertainty measured over meanings rather than token sequences. Sample several answers to one question, group the answers that mean the same thing, and take entropy over those groups. High entropy means the model is spreading probability across incompatible meanings instead of committing to one.

The clustering step is the whole trick. Token-level entropy is a poor substitute because two answers can use completely different words for the same claim: "the treaty was signed in 1998" and "the agreement entered into force in 1998" belong in one cluster, and a detector that counts them as disagreement is measuring prose style. Clustering strips harmless variation so the score reflects disagreement about what the answer means.

Farquhar, Kossen, Kuhn and Gal name the target confabulation: a claim that is both wrong and arbitrary. Their method goes after the arbitrary half. If identical instructions produce incompatible meanings, the model has exposed its own instability, and no fact lookup was required to notice.

How well does it work?

Well enough to be useful, not well enough to be finished. In the Nature study, semantic entropy reached an AUROC of 0.790 averaged across 30 task-model combinations.

That number needs a gloss, because it is not an accuracy. AUROC measures ranking quality: 1.0 is a perfect detector, 0.5 is a coin flip. 0.790 is a real signal and an unsolved problem.

It was evaluated on LLaMA 2 Chat at 7B, 13B and 70B, Falcon Instruct at 7B and 40B, Mistral Instruct at 7B, and GPT-4, over TriviaQA, SQuAD 1.1, BioASQ, NQ-Open, SVAMP and FactualBio, staying between 0.78 and 0.81 across model families and scales. Beating the baselines here is meaningful precisely because it needs no ground-truth lookup at inference time.

What can it not catch?

Anything systematic. A model that is confidently and consistently wrong looks exactly like a model that is confidently right, because the detector only ever observes agreement among the model's own outputs.

The authors are explicit about the limits. It does not reliably expose errors learned from bad training data or common misconceptions. It does not address a model "lying" to pursue a reward. It does not identify systematic reasoning or generalization failures. It misses cases where outputs are systematically bad rather than arbitrary.

Which reduces to one sentence worth keeping: agreement is evidence of commitment, not evidence that the answer corresponds to the world. Why models commit to confident wrong answers so readily is upstream of any detector: the scoring they are trained and benchmarked against rewards a guess over an honest "I don't know".

How is this different from asking the model to check itself?

Semantic entropy measures variation across independently sampled answers. Self-review asks a model to assess an answer it already produced. The first uses disagreement among outputs as its signal; the second depends on a review criterion and usually on external information.

That distinction matters because a self-review prompt can produce a confident verdict while adding no information about the world, which is the failure mode behind why self-correction needs an external signal. Semantic entropy is interesting here because it sidesteps that requirement: resampling is not an external signal, but it is not the model grading itself either. It does not replace external verification. It tells you which questions deserve it.

Worth separating from a third thing as well. Resampling here is a deliberate uncertainty experiment, not a reproducibility test. Why nominally identical requests can differ because of serving conditions is a different question entirely. Both involve variation; only one of them is a signal you asked for.

How should a team actually use it?

To route low-confidence answers, never to certify high-confidence ones. High-entropy answers go to retrieval, a tool call, a human reviewer, or a refusal path. Low-entropy answers still need a check when being wrong is expensive.

It costs several samples per question plus clustering, so this is a budget decision rather than a free reliability layer. A sampling budget can buy other things instead: repeated sampling raises the odds that a correct answer exists among the candidates, and a verifier decides whether you can collect it. A risk-tiered policy makes the tradeoff explicit:

• High entropy, low stakes: return the answer with a visible uncertainty state, or ask the user to clarify.
• High entropy, high stakes: run an external check or escalate before returning anything.
• Low entropy, low stakes: return under ordinary product controls.
• Low entropy, high stakes: verify anyway, because consistent error remains entirely possible.

That last row is the one teams skip, and it is the one the paper's limitations point straight at.

When should it be paired with an external check?

Whenever a false answer carries material cost. Retrieval, a database query, a calculator, a test suite, a source document or a qualified reviewer can each test correspondence against something outside the model. Semantic entropy decides when to spend that verification budget; it does not supply the reference.

The two questions stay cleanly separated. Semantic entropy asks whether the model is internally uncertain across meanings. External verification asks whether the meaning it settled on matches evidence outside it. Neither answers the other, and a detector that reduces arbitrary answers reaching users can leave systematic misinformation completely untouched.

So when a vendor claims hallucination detection, ask which of the two they built. A single score is not a reliability guarantee, for the same reason an aggregate number conceals the distribution underneath it.