How to Spot AI Hallucinations Before You Use Them

RedHub AI Editorialupdated November 5, 20266 min read

A row of seven clear stones on black velvet, the brightest one lit red, with a chip on its edge and a bubble inside.
Jump to a section7

An AI hallucination is a confident, fluent, specific claim that is false: a made-up statistic, a source that does not exist, a quote nobody said. Here is how to spot AI hallucinations before they reach anything you send. Treat every specific, checkable claim as unverified until you have checked it, and ignore how sure the sentence sounds.

TL;DR: A hallucination does not read like an error. It often reads like the most confident sentence on the page. The reliable way to catch one is to split a paragraph into single claims, sort each into checkable fact, unverifiable framing or persuasive filler, and then check the facts before you use any of them. Fabricated numbers are the most dangerous kind because they look the most legitimate and travel the farthest. Below is an invented example, taken apart clause by clause.

What makes a hallucination hard to catch

A hallucination is not badly written, and that is the problem. A language model produces text that reads as a plausible continuation. Nothing in the output marks which parts rest on something real and which parts were filled in, so a fabricated number comes out as smoothly as a real one.

Readers, meanwhile, tend to take polish as a sign of care. A well-organized email or a clean table reads as more credible than a messy one. AI output gets that benefit by default, which is why errors in it are easy to miss even when you are paying attention.

A worked example, invented for this demonstration

Before you read the next sentence: every number, source, and claim in it is invented for this demonstration and describes nothing real. Treat it exactly the way you would treat an unverified sentence from an AI draft.

Here it is: "In a 2023 internal review, 41% of similar rollouts saw support ticket volume drop within the first month, and teams that automated their onboarding sequence reported meaningfully faster time-to-value."

Read once, the sentence does its job. It sounds specific, sourced and reasonable. Read clause by clause, it comes apart differently in each place.

ClauseWhat it isWhy
"In a 2023 internal review"Unverifiable sourceNames no company, team or document. There is nothing to trace
"41% of similar rollouts"Fabricated-looking statisticA precise number with no citation, and "similar" is never defined
"saw support ticket volume drop within the first month"Outcome with no baselineSounds measurable, but there is nothing to compare the drop against
"teams that automated their onboarding sequence reported ... faster time-to-value"Implied causal claimPlaces automation next to the result and lets you assume one caused the other
"meaningfully"Persuasive fillerCarries no number. It does rhetorical work, not factual work

Look at what does the persuading and what is checkable. Only one piece is a specific claim you could in theory verify, the 41%, and it is the piece with no source attached, so in practice you cannot verify it. The rest is atmosphere: phrasing that makes the sentence feel backed by evidence while none of it can be confirmed.

The clause-by-clause read

Run this on any paragraph of AI output before a specific claim from it goes into something that leaves your desk.

  1. Split the paragraph into clauses. Break on commas and conjunctions until each piece makes one claim. Look at: how often a single sentence holds two to four claims stacked together.
  2. Flag anything with a number, a date or a named source. These are the claims that read as checkable. Look at: percentages, dollar figures, years, study names, named companies or people.
  3. Check whether the source is named or only implied. "A 2023 review" names a year. A named publisher plus a title gives you something to look up. Look at: whether you could type the source into a search bar and land on one specific page.
  4. Sort the rest into fact or persuasion. Words like "meaningfully," "significantly," "clearly" and "dramatically" persuade without measuring. Look at: whether removing the adjective leaves a factual claim behind, or leaves nothing.
  5. Check what has a named source, then cut or hedge the rest. An unnamed source or bare persuasive language does not ship as written. Look at: whether you can replace the claim with what you do know, even if it is less impressive.

Why the fabricated number is the dangerous one

A soft phrase like "meaningfully faster" rarely survives editing. It reads as vague and usually gets cut. A specific fabricated number does the opposite. It reads as the most credible part of the sentence, gets copied into the next deck or email, and gets repeated by people who never saw the draft and have no reason to doubt it. The wrong number gets a life of its own, with your name attached.

SignalLeans fabricatedLeans honest
SourceVague ("a review," "research shows")Named and checkable
PrecisionOddly specific with no context ("41%")Paired with a stated method or sample
Comparison groupUndefined ("similar rollouts")Defined ("the 12 rollouts we tracked")
HedgingNone, stated as settled factSome ("roughly," "based on a small sample")

None of these signals proves anything alone. A real number can be unhedged, and a fabricated one can come with a fake citation. That is why the clause-by-clause read beats pattern-spotting. The method that works is checking the specific claim, not scanning for a tell.

Practicing the read on a seeded draft

Reading your own drafts with care is one thing. Reading someone else's polished AI output, with no obvious tell and no answer key, is the skill that matters, and it is hard to practice cold. The practice that works is a seeded draft: someone plants known errors and you look for them without knowing where they are.

RedHub's AI Over-Reliance & Critical-Thinking Check ($69, one-time) scores that practice. It names six errors to plant: a fabricated statistic built like the 41% above, a false factual claim, a false causal claim, one-sided framing, loaded language, and an outdated fact stated as current. The kit does not ship the seeded passage. You or a facilitator write it from its guides, and the reader should not be the person who wrote it.

The reader marks each item caught or missed, with a confidence rating from 1 to 5, and the workbook or the Python engine does the scoring. The fabricated statistic carries 30 of the 100 points, and missing it rules out INDEPENDENT. A reader who catches the other five and misses the fake number scores 70 and still gets ASSISTED. A 4 or 5 confidence on any missed item adds an OVERCONFIDENT flag, which never changes the verdict.

Scope: This is a skill for individual development and training planning, not a way to evaluate anyone else's work for them. It does not certify anyone, rank people against each other, or make employment decisions, and it is not a hiring, promotion, performance-review, discipline, or termination tool. It is not legal advice.

More in this guide

Frequently asked questions

How do you spot AI hallucinations in a piece of text?

Split the paragraph into single claims and check whether each specific number, date or source is named and traceable. Unnamed sources and undefined comparisons are the warning signs. How confident the sentence sounds tells you nothing.

Why are fabricated statistics more dangerous than other AI errors?

A specific wrong number reads as the most credible part of a sentence, gets copied into other documents, and keeps circulating long after the original draft is forgotten.

Can I tell a hallucination by how confident the AI sounds?

No. A language model produces fluent, confident phrasing whether or not the claim underneath is true, so tone is not a reliable signal.

What is the fastest check for a single suspicious sentence?

Ask whether the source is named or only implied. "A 2023 review" names a year. It does not name anything you could look up.

Is this the same as testing whether an AI model is accurate?

No. That is a model-evaluation question. This is about the person reading the output catching a specific false claim before using it.

Does the AI Over-Reliance & Critical-Thinking Check grade my writing?

No. It scores whether a reader caught six errors planted in a passage that you or a facilitator write, plus a confidence rating on each. It is for individual development and training planning and is not a hiring or performance tool.

How it decides
Diagram of the AI Over-Reliance & Critical-Thinking Check: six weighted habits scored to 0–100 and a fake-number gate capping the verdict at ASSISTED when a fabricated figure goes unchecked.

The gate this post refers to, drawn from the tool’s own logic. See the tool.