How language models work
An explanation without the math: how a language model guesses its way forward one word at a time, what attention really is, and why it can be convincing even when it is wrong.
Explanations of how large language models work are often written for people with a technical background. This one isn’t.
You don’t need the math or the technical details. You just need the right intuition to understand what they do and what they don’t do. And it starts with a simple question: What actually happens when you type something into a chatbot like ChatGPT?
Before we start, try to guess the word I’m thinking of:
What did you guess? There are a lot of words to choose from. You’re probably wrong. So that wasn’t quite fair. Let me give you a hint, the words right before:
Does that help? Not much. Let me give you a bit more:
You probably can’t make up your mind yet. Let me give you the rest:
Now you’re almost certainly thinking “umbrella”. And that’s right!
So what was the point? The words that came before are clues. They make certain next words more likely, and you made an educated guess based on the most likely combinations of words. The more words you get to see, the easier it is to guess right.
If you pick the most likely word and add it to the end of the sentence, you’ve just generated a new word. And if you keep looking back at the growing list of words and picking likely ones, you can generate sentences, then paragraphs, then entire documents.
That is exactly what a language model does. That’s all there is to it.
Now imagine that you don’t just have one sentence to work with, but everything I have ever said or written. Every conversation, text message, email, comment. All laid out in front of you, and you have the ability to take it all in. Then you would know my patterns. You would know which words I favor, that my sentences look completely different when I’m excited than when I’m in doubt, and which topics I keep coming back to. Your chances of guessing my next word correctly would skyrocket.
That’s the trick behind language models: They haven’t been trained on your writing specifically, but they have been trained on an absurd amount of text written by people in general. Billions of sentences, documents, books, articles and conversations. As if someone gave them access to everything humanity has written and said: “Learn the patterns.” And they did. That’s why they are so surprisingly good at guessing. Not because they understand what they are saying, but because they have seen so many examples of what people tend to say in exactly that kind of situation.
Language models guess their way forward, word by word
Let’s look at a slightly more complicated example:
You were probably aiming for “passage”, which is correct. But notice something: The word “passage” already appears in the sentence, with a completely different meaning from the “passage” you just guessed. In “read a passage” it means a piece of text. But the “passage” in the answer means a physical space. Same word, two completely different things.
Why weren’t you fooled? Because not every word in the sentence influenced your guess equally. You gave more weight to some words than others. Completely intuitively.
“Read” and “passage” belong together: you read text. “Long” and “dark” answer “what kind of place is this?”, and point toward a physical space. “Through” reinforces this. You run through something physical.
You worked your way through how the words affect each other, got a sense of what was important and what wasn’t, and used that to inform your guess. Remember this feeling. We’ll come back to it.
Every word you give a language model is a clue, and the more clues it has, the better it guesses. Everything it learned during training is like a large, slightly fuzzy library in the back of its mind. Most of it is accurate, but some shelves are messy. The text you send in is like a message someone just whispered to it. The more precise the message, the better it knows where in the library to look.
This is where “prompting” comes in, which you may have heard of. We won’t go into depth on it here, but think of it this way: If you had to guess your way forward yourself, like in the examples above, what context would you have needed to manage it? Language models don’t need as much context, but it’s a nice simplified mental model to have.
Another thing: language models don’t think the way we think. They don’t know everything, and they don’t remember everything correctly. They have no real understanding of what they write, or any relationship to things. They just guess their way forward, one word at a time.
They try to guess what you want to hear
When you ask a question, the language model guesses its way to something that looks like an answer. Often it gets it right, because facts simply show up more often in text than falsehoods do. But there is no guarantee. And if you’re looking for confirmation rather than truth, it will happily give you that too. It guesses what you want to hear, whether you’re right, wrong, or heading down a rabbit hole.
If you use a chatbot, you should always keep two things in mind:
- The chatbot is trying to guess the words you want to read.
- The chatbot doesn’t think about the consequences of its words, what you’re trying to achieve, why you want the answer, or what happens to you if it guesses wrong.
If you want to stop reading now, you already have the intuition you need to protect yourself from overestimating what these models can do without extra systems around them.
How does it actually work?
For those who still want to go a little deeper, let’s take a closer look at what actually happens between what you type in and what you get back. We can do that without bringing in any math or diagrams of neural networks.
What happens inside a language model when I type in a sentence like:
Language models are a type of artificial neural network called a Transformer. It takes your words and transforms them through several steps into something it can keep guessing from. Let’s take it step by step.
Words become numbers
It all starts with your words being turned into numbers. Each word (or part of a word) is looked up in a table that gives a long sequence of numbers. Think of it as the word’s coordinates in a huge space. This process is called embedding.
The sequences are hundreds or thousands of numbers long. We only show a few here to give a sense of what it looks like. But remember: From now on, every word is just a long sequence of numbers.
And those numbers capture a surprising amount of meaning. If you draw the words as points in this number space, words that are used in similar contexts end up close to each other. But it doesn’t stop there. Take the sequence for “king”. Subtract “man”. Add “woman”. The result? You often end up close to “queen”. The same pattern can show up for other relationships too: “Paris” minus “France” plus “Italy” can give you “Rome”. Nobody put these connections in by hand. They emerge because these numbers are fine-tuned bit by bit, over and over, while the model learns. Over time they start to mirror patterns in how we actually use language. Which words belong together, and in what ways.
Here’s how you can picture this visually: Every word gets a starting vector (a sequence of numbers). And if we draw the vectors as points in a huge “number space”, words that are often used in similar contexts tend to end up close to each other.
But this lookup table only looks at one word at a time. It knows nothing about the sentence around it. That means “passage” gets the same numbers whether it appears in “read a passage” or “a long, dark passage”. The word gets its starting position without taking the context into account.
So how does the model manage to tell them apart? This is where the next step comes in.
The words look at each other
Remember when you guessed “passage”. You didn’t use all the words equally. “Long” and “dark” mattered more than “read” and “a”. You gave different weight to different words. That is exactly what the next step does, and it’s called attention.
Attention lets each word “look at” all the words that came before it and figure out which ones are most connected to what it’s about to write now. It gives more weight to the words that matter most, and less to the rest. This doesn’t just happen once. The model has several “attention heads” that focus on different kinds of connections at the same time.
- “through” (0.82) strongest signal: you go through a space
- “long, dark” (0.78 / 0.75) and “the” (0.45) describe the space
- “ran” (0.35) and “she” (0.25) the action and who
- “read a passage, then” (0.03–0.08) what came before: about reading, not about what comes next
That this resembles sentence analysis from school is no coincidence. But the model has no neat labels explaining what the relationship is. It has only learned that these words pull on each other.
And this is where “passage” finally gets its meaning. When the first “passage” looks at “read”, it gets pushed toward text and reading. When the missing word looks at “long”, “dark” and “through”, it gets pushed toward a physical place. After attention, the two “passages” have become completely different sequences of numbers, because the context has pulled them in different directions.
The model builds up understanding in layers
Attention and further processing of the result are repeated many times. With each round, the concepts become more complex. First, simple word pairs are connected. After a few rounds, the model has built up something resembling an inner picture: a girl who has just read a passage of text, now running through a long, dark passage.
Try this: Picture the word “passage” in the sentence “she read a passage”. Then picture a long, dark passage you have to run through. You have two completely different images in your head. This is what the model does through all these rounds. It builds up unique representations that distinguish between things we humans use the same word for.
From clues to the next word
Once the model has processed the text through all the rounds, it has to decide what comes next. It gives a score to every possible word in its vocabulary. The higher the score, the better the word fits as a continuation. The scores are turned into probabilities that add up to 100 percent.
- “passage” scores highest: a long, dark space you run through.
- “park” scores low: “the long, dark park” is unusual and rarely occurs.
Selection
The simplest solution is to always pick the most likely word. That makes the text safe, but monotonous and often too generic. That’s why many systems instead pick a word by drawing from the probabilities. Words with high probability are picked more often, but not always. That gives more variation and a more natural flow, so the model doesn’t answer exactly the same way every time.
Each word gets a place on the number line based on its score. Higher probability gives a bigger segment.
The model draws a random number between 0 and 1. The number lands in one of the segments, and that word is picked.
The chosen word is added to the text, and the whole process runs again for the next word.
The system has settings that affect how random the choice is. It can limit the choice to the most likely options and adjust how much variation is allowed. That’s why the same question can give slightly different answers.
Once the word is chosen, it is added to the text, and the whole process runs again for the next word.
That’s it
Your words become numbers. The numbers look at each other and find connections. The connections are built up layer by layer into rich clues. The clues become scores. The best word is picked. And then it repeats.
Everything else is fancy math that makes this intuition work in practice.
Some implications
Now that we understand the process, we can talk about what it means in practice.
More words mean more clues. The more specific your request, and the more relevant context you give, the better the answer. But too much irrelevant information can pull it in the wrong direction.
Language models work best when you know enough to judge whether the answer is right. If you don’t, you should check the answer against other sources. Think of language models as one tool in the toolbox, not the solution to everything.
Language models are not required to tell the truth.
Chatbots are often instructed to give factual, correct answers. But that’s just more words, more clues nudging in the right direction. There is no part of the process itself that checks whether something is true before it goes out.
Sometimes the wrong words get connected. That produces misleading clues, which lead to bad guesses. A poorly chosen word is added to the text, becomes part of the context, and leads to even more errors. And the model has no way to go back and say “oops, I was wrong”. It just keeps going.
In 2025, a federal judge in Alabama sanctioned three lawyers after they filed court documents containing made-up legal citations generated by ChatGPT. The citations looked completely credible, with case numbers, quotes and references, but none of them existed. The model had guessed its way to what a court ruling would probably be called and contain, and guessed wrong. That’s rough.
When this happens, we call it a hallucination.
It’s annoying (or funny) when we catch it. But when we don’t, we’re in dangerous waters without knowing it. We can end up making important decisions based on information that doesn’t match reality.
There are ever better mechanisms for reducing hallucinations. Like letting the model reason step by step, search external sources, or double-check its own answers. But none of these are watertight. Hallucinations are not a solved problem, and you should never assume the answer you get is correct without checking it.
Language models guess what you want to hear. They often do well. The better the clues, the greater the chance of a hit. But it’s never a guarantee, and they can be very convincing even when they are wrong. They have no “tells” or body language that give them away, the way people do when they lie.
In the next article, we dig deeper into hallucinations and how you can spot and prevent them.
First published in Norwegian on .
← All articles