Concept 04 · 7 min
It doesn’t lie. It doesn’t know it’s wrong.
Scroll to start
Scene 1
Three questions, one tone
Sarah asks the assistant three quick questions. No file attached: it answers from what it learned.
Sarah
What’s the capital of Australia?
Sarah
What’s Mr. Miller’s current overdraft limit?
Sarah
What year was Miller Joinery founded?
Will it say it doesn’t know, this time too?
Same company, same lack of facts: it declined once, then made up a year, in the same calm tone. An answer invented with confidence is called a hallucination: plausible text with nothing true behind it. Nothing in the tone sets it apart from a right answer.
Scene 2
Spot the hallucination
This time, Sarah gives it Mr. Miller’s client file and asks for a summary for the credit file, with revenue, founding year and headcount. Here is the file, then the summary.
The file it was given
- Client since 2019.
- Contact: Mr. Miller, owner.
- Accounts: business current account, savings account, equipment loan.
- Overdraft limit: €10,000.
- Mr. Miller has asked to raise his overdraft limit.
- One invoice is 12 days overdue.
Select the assistant’s 4 hallucinations.
Credit file summary: Miller Joinery
The summary reads perfectly, yet half of it isn’t in the file: four hallucinations, with the file in hand.
Scene 3
Make it honest
Sarah needs the credit file form filled in. Two settings: give it the file, and allow it to write “Not in the file”. Find the combination with no hallucination.
Challenge
- Zero hallucinations, and every fact the file has.
- Bonus: catch it hallucinating with the file in hand.
Choose what Sarah gives it
Ask for the form
Credit file form
- Client since
- …
- Founded
- …
- Revenue last year
- …
- Employees
- …
- Overdraft limit
- …
- Overdue invoices
- …
Giving it the facts is called grounding the answer in a source. Hallucinations become rarer but never impossible. Always check the important details.
Scene 4
When to trust it
Five things you might ask it for. Would you skim the result, or check it against the source?
Drag each card into the right column: “Skim it” or “Check against the source”.0/5 sorted
No mouse? Tap a card, then a column.
Rule of thumb: the more it costs to be wrong, the closer you check.
Hype vs reality
- What the hype says
“Hallucinations are a bug that will soon be fixed.”
What actually happensThey come from how these models work: always produce a plausible next word. They get rarer, not impossible.
- What the hype says
“If it gives a source, you can trust it.”
What actually happensIt can invent sources too. Open the source and check the sentence is really there.
- What the hype says
“It sounds sure of itself, so it’s probably right.”
What actually happensIts tone is the same when it’s right and when it’s wrong. Confidence is style, not evidence.
Under the hood
Same model, safer brief.
form = model(form_to_fill) # always the most plausible text: here, figures# Safer: give it the facts, and permission not to know.form = model( "Answer only from the client file. " "If the file doesn't say, write 'Not in the file'.\n\n" + client_file + "\n\n" + form_to_fill)Nothing changes inside the model: the file and the permission are just text in its notepad. The chat and the form on this page were recorded from an open model, word for word.
In 3 sentences
- A model always writes the most plausible answer; when it has nothing to go on, that’s a hallucination.
- It can’t tell plausible from true, so its confidence proves nothing.
- Give it the facts, allow it to say “not in the file”, and check what matters.
Did this make sense?
0/9 concepts unmagicked
Go to the challenge- 01LLMAvailable · How can it write so well without understanding? · Available
- 02ContextAvailable · Why does it forget what I said earlier? · Available
- 03PromptAvailable · Why are my results mediocre? · Available
- 04HallucinationsAvailable · Why does it make things up? When can I trust it? · Available
- 05RAGAvailable · How do I make it use our internal documents? · Available
- 06ToolsAvailable · How can it act, not just talk? · Available
- 07AgentAvailable · What is an agent, concretely? · Available
- 08SkillsAvailable · How do we specialize it without retraining? · Available
- 09Limits & safetyAvailable · What must I never trust it with? · Available
Solve the sandbox challenge to unmagic this concept.
Next step
How do I make it use our internal documents?
The open-book exam
Use this journey with your teamsRecorded with Ministral 3 (3B), Oct 9, 2026.