LLMs don’t “hallucinate.” They improvise. Calling it a bug is just a polite lie. Here’s the uncomfortable truth about how AI really “thinks” And why your prompt can’t fix it 🧵:
1/ Everyone says: "LLMs don’t understand language - they predict it." Correct. But that also means: They don’t understand truth, accuracy, or even verifying a fact. They only learn how people talk about facts.
2/ So when you prompt it to “be cautious” or “never state unverified info”... It doesn’t actually get more reliable. It just learns to write like someone trying to be cautious. Keyword: “trying.”
3/ Example: A prompt says: "Never present generated content as fact." The model doesn’t stop hallucinating. It just adds: “I cannot verify this information” - even if it’s entirely fake. This isn't honesty. It’s style.
4/ That’s the trick. LLMs don’t have a truth engine. They don’t even have a “knowledge base.” They only have probabilities: “What kind of thing usually comes next when people ask something like this?”
5/ So what’s really happening? • Prompt adds cautious words to the context vector • Model predicts next token with a bias toward uncertainty • It still pulls from patterns of fake info in the training data • And slaps an [unverified] tag on top
6/ Bottom line: LLMs can imitate careful responses. But they don’t know what’s true. The better they get at sounding cautious, the harder it is to spot the fakes. That’s the real hallucination.
7/ So what does help? Tool calling. Context engineering. Retrieval-augmented generation (RAG). External fact-checking. If you care about truth, offloading to real tools and live sources is your best shot.
8/ But stop saying: “Just write better prompts.” It’s like asking a parrot to double-check its facts. The model never knew the truth to begin with.
The AI prompt library your competitors don't want you to find → Just added prompts for Finance → Lifetime updates & access → Unlimited custom prompts Grab it before it's gone 👇 https://godofprompt.ai/pricing
@godofprompt This nails it, LLMs don’t “know” truth, they generate patterns of how truth is usually written. Prompts can change tone, not accuracy. Real reliability comes from RAG, tools, and external sources.
@SWAutomation_ exactly
@godofprompt I was spending hours trying to name this situation. Improvisation! When is answering something weir I always have to tell "remember that..." and the answer is, "sorry for the confusion and bla bla bla...". This explain a lot of things.
@Juanpulag this hits, just remember improvisation is part of the game, refine your prompts and keep them sharp
@godofprompt Exactly. Words don’t vanish, they adapt in real time.
@godofprompt humans don’t know the truth either. check saphyr worf, constructivism, physics etc. the misconception here is not the LLM halluzinating. it’s YOU halluzinating of “the truth”.
@godofprompt That's hilarious! AI improvising like that is basically just fanfic for Elon Musk's Mars dreams