In the prologue to his 1989 book The Emperor’s New Mind, the Nobel laureate Roger Penrose narrates a story. At a grand public ceremony, an ultimate supercomputer is switched on for the first time before a packed auditorium. It is the most powerful machine ever built, designed to answer any question put to it. The presenter invites the audience to ask its very first question, but no one volunteers. Everyone is afraid of looking foolish before the intelligent machine. Then a young boy named Adam, who has grown up among computers and is not intimidated by them, raises his hand and asks the first question. There the prologue ends on a cliff-hanger. We are never told what the question is.Penrose used this story to introduce a deeper argument about the limits of computation and the nature of human understanding. He contends that there are truths a formal mathematical machine cannot reach no matter how powerful it becomes.But reading this story a decade ago, long before ChatGPT or Claude existed, brought a different question to mind: what if the ability to answer every question correctly is not always a virtue? What if some of the qualities we value most, such as imagination and creativity, depend on our ability to venture beyond what is already known?Today, as we build machines that can answer questions, write essays, generate code, and analyse data, that question feels very relevant.Natural instinctFor all their remarkable capabilities, these systems sometimes make things up. We call this hallucination. Ordinarily, a hallucination means to see or hear something that is not there. In artificial intelligence (AI), a model hallucinates when it produces an answer that sounds plausible but is factually wrong. It may invent a citation, misstate a number, fabricate a legal case or attribute a quote to the wrong person.This is not a minor flaw. In medicine, law, finance, science, and journalism, hallucinations can be dangerous. A medical chatbot that invents advice is not being imaginative but unsafe. A legal tool that fabricates case law is being unreliable.Our natural instinct is to want to eliminate hallucinations but this is harder than it appears. Large language models (LLMs) do not work like databases, storing facts in neat rows and columns, and retrieving the right answer when asked. They are trained on enormous collections of text and learn statistical patterns in language. When prompted, they generate a response by predicting what will come next, one piece at a time.