An intuitive understanding of LLM

2026-06-06‌ 2026-07-28

When we talk about AI today, we pretty much talk about LLMs. We're being told that they are able to (or very soon able to) fully replace some human workers. I'm writing this to help put some context to that statement. I'm trying to be concise; my priority with this text is for it to be accessible.

To be able to understand what an LLM is, we'll take a detour to compression algorithms.

There are two categories of compression: lossless compression and lossy compression.

Lossless compression means that you will always get back the exact data you had before compression. You're using this right now, in your browser. It's most often done by applying known predefined algorithms designed to find and efficiently describe patterns. The compression ratio is decided by the effectiveness of the algorithm and how much computing power and memory you provide.
Lossy compression means that you lose data when you compress. This is often done by looking for things that can be removed without us noticing, such as high pitch sounds and dark colors, in combination with approximations of patterns. The compression ratio here is decided by the same as lossless compression, but also by how much quality degradation you can accept. This is what's used in most photographs and videos, and is what's responsible for JPEG images sometimes being blocky.

Historically, text has almost exclusively been used with lossless compression. LLMs are a revolutionary change to that. LLMs are not compression in the traditional sense, where you'd expect the compressed file to grow when you give it more data, compressed by a predefined set of algorithms. With LLMs you create the output file from the start, then you take an arbitrary amount of data and optimize the parameters in that existing output file to be able to approximate the data you provide.

I get that some of you will have an instant reaction of this comparison being ridiculous. I would like to direct you to a study where researchers could get LLM models to recall (near-verbatim) 95.8% of the first Harry Potter book and 95.5% of 1984. This doesn't prove lack of intelligence, but it proves that the data is there.

Where does the intelligence come from?

I have identified the following sources:

Volume: Pretty much all text humanity has ever written has been used to optimize these models. Your query may not be as unique as you think.
Huge context windows: The generated text is put together based on a ridiculous number of parameters. In combination with the volume of data, it will mostly output coherent text, even when the text is completely nonsensical.
Idealization/Apophenia: LLMs are really good at making you feel listened to. If you talk to LLMs as you would a friend, then I would expect that you want there to be a meaning to those conversations.
Authority: These techniques are developed by the richest people in the world. They can't all be lying, right? (This is obviously written from my Tesla Roadster while autonomously riding along the open roads on Mars)
Complexity: It's difficult (probably not possible) to imagine the amount of data that has been used to optimize these models, or the amount of parameters that a model uses. It's a lot easier to just assume that there is some kind of intelligence.
Trojan horses: There is a lot of propaganda that sounds like criticism against AI, which in actuality sneakily promote AI.
Gell-Mann amnesia: If you're lucky enough to have an expertise, I'm sure you've noticed that AI is really good at most things except the thing you know a lot about.
Anthropomorphism: This is the big one for me. The used terminology in AI is built to uphold the illusion of intelligence.

As you may have noticed, this list is kind of problematic in the sense where it quickly deteriorated from sources of intelligence to pillars upholding the illusion of intelligence.

Anthropomorphism

Anthropomorphism is the ascribing of human personality, appearance, conduct, cognition, or other attributes to non-human entities, often including non-human animals.

Let's break down a few of these terminologies, and why I find them to be problematic. I have skipped some obvious ones, such as calling them AI agents or voice models breathing and laughing, and focused on the ones that are a bit more subtle.

You don't train an AI model. You optimize a model to be able to generate an approximation of specific data. This interpretation would be legally problematic.
LLMs don't make mistakes. Mistakes require intention. For the same reason, they don't lie, but they are optimized to be able to recreate every single Sci-Fi story ever written that contains an AI. I don't really know which is worse.
LLMs are not intelligent. Most know this on some level, and even those that design them are somewhat open about this, albeit not in such clear phrasing. The point where we don't agree is that they think that LLMs are so close to intelligence that they might somehow suddenly turn intelligent; you may have heard of this as the singularity, AGI, or ASI, through the magic of recursive self-improvement. That makes the water into wine routine look like child's play in comparison, and any description on how recursive self-improvement would actually work sounds like what's already known as model collapse in the industry. The problem here is that even if you know that LLMs aren't intelligent, it's really easy to get tricked into doubt. Note that there is intelligence in text generated by LLMs, but that doesn't mean that the LLM is the source of that intelligence, no more than your fax machine is the source of the intelligence it prints. Interestingly, on the same note, they aren't even stupid; they're something else.
You could argue that reasoning models are reasoning, but it's really more about creating a separate context with a more predictable path in the massive branching tree that is LLMs.
LLMs do not hallucinate. This one is really important. The terminology suggests that this is a minor solvable issue, a quirk of the current stage of LLMs, which ignores the truth; What they refer to as hallucination is in reality the only thing LLMs actually do. What's so cool about the technology is that the generated text often matches reality.

Why does it matter?

My point with this clog post is that we all perpetuate the illusion of intelligence. Terminology and framing matters. Thinking of LLMs as AI creates a abstract entity with hard to define properties, and it becomes very easy to fall into the anthropomorphism trap. On the other hand, if you think of LLM models as lossy text compression, it will help you understand what LLMs are actually capable of, and even somewhat what they will be capable of in the future. Most importantly, it will help you understand what it isn't; It's not your therapist; It's not intelligent; It's not self aware; It's not your friend; Any belief to the contrary is dangerous. We may some day create an actual AI, but it will not be an LLM. You may rightfully argue that this is an oversimplification, but at least it's an oversimplification where you don't have to argue the philosophic point of well, what is intelligence, really?.

While I would like to believe that LLMs are without value, that's not what I'm saying. I don't agree that they are important enough to displace global environmental sustainability, but they are useful. That said, when you hype the technology to people in power that don't understand what the technology really is, they're going to make sure to promote it to a level of respective incompetence. I am very confident in my belief that LLMs won't have an uprising and hack nuclear missile facilities, but my confidence in the leaders of the world to not try to pigeonhole LLMs into surveillance or weapon systems is a lot lower.