If AI Can Hallucinate, Can It Create?

You're running late for a project pitch, so you ask AI to pull together three quick market statistics for your summary slide. The numbers look crisp, the sources sound professional, and the phrasing is effortless.

But ten minutes before you present, you go to double-check the source link and realize the report it cited doesn't exist. The AI made it up out of whole cloth, complete with a convincing author name.

It's the classic "hallucination" trap. But it raises a weird question: if a machine can invent believable facts in milliseconds, where does pattern recognition end and actual imagination begin?

What a Hallucination Actually Is

A fabricated market report is a frustrating error, but the underlying process is the exact same one that causes machines to invent entirely new art, stories, and concepts. We often treat artificial intelligence (AI) making things up as a glitch, a mistake to be patched. Yet much of human creativity involves dreaming up things that are not real. We cannot argue about whether a machine is creative until we define what a hallucination actually is.

A large language model (LLM), which is a system trained to predict the next piece of text, does not understand the world the way a human does. It does not look up facts in a mental encyclopedia. Instead, it relies on patterns from its training to generate sentences, guided by the prompt, which is the instruction a person types.

When this process results in fluent output that sounds right but is not supported by reality or by the source the user gave it, we call it a hallucination. This term was borrowed from human psychology, but in computer science, it describes a specific structural behavior. These errors are not random static. They are coherent patterns that happen to be wrong. Examples of this behavior include:

• A fake scientific paper
• A quotation nobody said
• A person who does not exist
• A made-up citation
• An event that did not happen

If a system can generate something that never existed, why do we call some of those outputs mistakes and others creativity? The answer has less to do with the machine and more to do with how we view ourselves.

Imagination Is Not the Same Word

The line between a useful invention and a mistake often comes down to context and intent.

When a novelist sits down and writes about a fire-breathing dragon or a floating city, we do not accuse them of lying. We do not say they are hallucinating. We call it imagination, and we celebrate it.

The difference is that a human writer knows the dragon is not real. A novelist understands the distinction between the physical world and the page. An LLM, as far as researchers can tell, has no awareness of this boundary. It cannot tell the difference between a real historical treaty and a fictional wizard war because both exist to the model as statistical patterns of words.

The novelist knows the dragon is a fiction, but they did not invent the idea of dragons from nothing. Human creativity has its own history, and it never starts from a blank page.

Mozart Did Not Start from Nothing

Even the greatest human minds did not build their masterpieces out of thin air.

When we think of Mozart, we sometimes imagine a genius receiving perfect melodies from the heavens. The reality is more grounded. Mozart grew up surrounded by specific musical scales, classical instruments, established rhythms, and the strict rules of eighteenth-century composition. He took the musical patterns of his era and combined them in elegant, unexpected ways.

Beethoven did not invent the structure of the symphony, the sonata, or the string quartet. He studied those forms under older teachers and then pushed them past their traditional limits. Leonardo did not invent the techniques of drawing, the study of human anatomy, or the apprentice workshop system.

This reliance on existing patterns sounds familiar, because recombination is exactly what language models are built to do. Yet this similarity is not a final verdict, and the real fight is whether recombination can ever count as true creation.

Two Arguments, Neither Easy

If human art relies so heavily on recycling the past, the debate over machine art becomes much more complicated.

On one side, proponents argue that these systems can produce genuinely novel output. A model can write a story, sketch a design, or draft a piece of software that was not present anywhere in its training data. Sometimes, the generated combinations surprise the very engineers who wrote the code. If the output is new, functional, and unexpected, it is hard to deny that some form of creation has occurred.

On the other side, skeptics point out that a machine lacks the essential ingredients of human expression. A model has no personal history, no feelings, no life in the physical world, and no inner drive to say something. A painter might paint a dark, stormy sky to express the heavy weight of personal grief. A machine can generate a similar image because it was prompted with the word grief, but it has never felt a loss.

Both sides of this debate have strong points, but one common shortcut used by skeptics is weaker than it sounds.

Learning from the Old Is Not the Same as Copying

The common objection that AI is just a giant copy machine misses the point of how both machines and humans learn.

It is true that these models are trained on billions of words written by humans. But human creators do not learn in a vacuum either. A young musician listens to thousands of songs, a scientist reads hundreds of academic papers, and an artist spends years copying the masterpieces of the past before finding their own style.

The real question is not whether a system learned from existing work. It clearly did. The live question is whether the system can generate a combination, an idea, or a solution that was not already explicitly written down in its training data.

If simply using existing work is the wrong test for creativity, we need to find a better one.

A Better Test: Surprise

To find out if a machine can go beyond what it was taught, we need to move past simple yes-or-no labels and look at specific, verifiable results.

Consider three distinct examples of machines producing genuine surprises.

First, in March 2016, the system AlphaGo played a five-game match against human champion Lee Sedol, winning four games to one. In game two, the machine made its thirty-seventh move, a shoulder hit on the right side of the board that stunned professional commentators, who initially thought it was a blunder. Lee Sedol left the room and spent a long time thinking before returning to play. The creators of the system later explained that AlphaGo had calculated that a human player would almost never choose that move. AlphaGo is not a language model. It searched and learned by playing Go, including against itself. In game four, Lee Sedol's move seventy-eight surprised AlphaGo, showing that humans can still play the move the machine did not expect.

Second, in late 2023, Google DeepMind published a system called FunSearch in the journal Nature. This project paired a pretrained language model with an automated code checker. On the cap set problem, which involves finding the largest set of points in a grid where no three points sit on a line, it found constructions that went beyond the best known human solutions. This was the first major progress on the problem's asymptotic lower bound in about twenty years. It also found better heuristics for online bin packing. The output was a working computer program that a mathematician could read, not an uncheckable paragraph. Collaborator Jordan Ellenberg of the University of Wisconsin-Madison worked with the team. While impressive, FunSearch did not solve mathematics in general.

Third, in January 2024, Google DeepMind published AlphaGeometry in Nature. This system combines a language model that proposes geometric constructions with a symbolic engine that checks the proof. Tested on thirty Olympiad geometry problems, it successfully solved twenty-five of them within the time limit. The average human gold medalist on those same problems scored about twenty-five point nine, while the previous leading computer method, Wu's method, solved only ten. The machine did not feel the proof. The symbolic checker is why the result counts.

These examples suggest we should view creativity as a spectrum rather than a single threshold. At the lowest end is imitation, where a system simply repeats existing patterns with minor changes. Above that is variation, where it adjusts a known pattern to fit a simple new context. Next comes combination, where it brings together two unrelated concepts. Further up is novelty, where a system finds an unexpected move or path within a strict set of rules, such as AlphaGo's shoulder hit. At the highest end is discovery, where a system produces a verifiable, useful solution to a hard problem, like the math programs from FunSearch or the proofs from AlphaGeometry. A fake citation sits at the lowest end of this spectrum: it is a novel combination of words, but it fails the moment someone looks it up. AlphaGo sits at the level of novelty, while FunSearch and AlphaGeometry sit closer to discovery because an outside checker can confirm their output.

While mathematical and logical breakthroughs show this spectrum in clear terms, most people feel the tension of machine creativity much faster when the output is a picture.

Who Is the Artist?

Imagine typing a prompt like "a medieval astronaut standing in a cathedral on Mars" and receiving a beautiful, detailed illustration.

The image is entirely new, but who created it?

Is the artist the person who typed the prompt, directing the machine like a movie director? Is the machine itself the artist? Are they equal partners, or is the model just a highly advanced tool?

History shows us that new tools always complicate our definition of authorship. When the camera was invented, critics argued that photography was not art because the machine did the work of capturing the scene. Later, software like Photoshop and electronic synthesizers faced similar skepticism. Each of these tools changed the nature of creative decisions without removing the human hand.

These tools changed the creative process without erasing the human, but they still leave us with a harder question: does creativity require a mind that understands what it has made?

Does Creativity Need a Mind That Understands?

Must a creator understand what they have made for the act to be considered creative?

Suppose a computer system generates a completely new mathematical formula that solves a long-standing scientific puzzle. Human experts review the work and confirm the solution is correct. The system itself has no conscious awareness of what numbers are, no feeling of triumph, and no understanding of the physics involved. It simply followed its programmed patterns and checker loops.

We see this dynamic in systems like FunSearch and AlphaGeometry. The machine does not feel the beauty of the proof. The truth of the work is guaranteed by an external checker, not by the machine's own understanding. If the output is genuinely new and highly useful, does the machine's lack of awareness change the value of the discovery?

If we can have useful creation without a mind that understands what it did, then perhaps our very definition of creativity was too blunt to begin with.

Maybe Creativity Was Never One Thing

Perhaps we have spent too much time arguing over a single, all-or-nothing word.

Instead of a single label, we can think of creativity as a bundle of different components:

• Knowledge: the pool of learned patterns, facts, and rules.
• Combination: the ability to connect distant ideas.
• Exploration: searching through possibilities to find unusual paths.
• Novelty: producing something that has not been seen before.
• Intent: the conscious desire to express a feeling or solve a problem.
• Evaluation: the ability to judge whether the result is good, true, or useful.

Current systems are already capable of knowledge, combination, exploration, and novelty. When paired with checkers or human editors, they can participate in evaluation. The piece that remains uniquely human is intent: the personal experience that makes us want to build something in the first place.

Because these different pieces of creativity are split between human and machine, the practical future of art and science is likely a partnership rather than a replacement.

A Conversation, Not a Replacement

This partnership changes the creative act from a solitary struggle into a rapid conversation.

Picture an artist trying to express grief without depicting a single human figure. They feed a short prompt into the system, describing "loss, absence, and the weight of memory." In seconds, the machine floods them with concepts. Most are noise. But one image stops them cold: a child's swing gently tilted, its seat dusted with fallen leaves, the rest of the playground perfectly undisturbed.

The artist responds, asking the AI to shift the atmosphere: a cooler sky, longer shadows, a faint suggestion of wind, so the scene feels as if someone had been there moments ago and then quietly vanished. The machine obliges, generating dozens of variations: different angles, different moods, different interpretations of absence.

Here, the roles are unmistakable. The human brings intuition, memory, and the emotional truth they're trying to reach. The machine brings speed, variation, and combinations the artist might not have considered on their own.

This subtle collaboration (human intention meeting mechanical possibility) echoes the market report that never existed, a reminder that meaning often emerges not from either side alone, but from the space between them.

The Wrong Question

The line between a hallucination and a creative spark isn't drawn by the algorithm. It's drawn by us. One is an invention that misleads. The other is an invention that illuminates.

No human artist or thinker has ever operated in total isolation. We inherit the past, remix its patterns, and call the result our own. Today, we hold tools capable of sifting through humanity's collective output in seconds, offering back combinations we could never have imagined alone. If an AI system ultimately yields an idea that solves a crisis or opens a new frontier of thought, it won't matter if the code "knew" what it was doing. The machine provides the spark. We provide the meaning. And in the end, that symbiosis might be the most human creation of all.