How Large Language Models Like ChatGPT Actually Work, in Plain English

Chatbots like ChatGPT can write emails, explain ideas and answer questions in a way that feels remarkably human. It's natural to wonder what's going on underneath. The core idea is surprisingly simple to describe, even if building one is not: a large language model is a system that has become extremely good at predicting what text should come next.

Predicting the next piece of text

Language models don't read words quite the way we do. They break text into small chunks called tokens — often a whole word, sometimes part of one, sometimes a punctuation mark. Given a sequence of tokens, the model calculates which token is most likely to come next, adds it, then repeats the process. A full answer is built one token at a time.

That sounds too basic to produce a thoughtful reply, but predicting the next token well turns out to require picking up a great deal about grammar, facts, reasoning patterns and style.

How the model learns

• Pre-training — the model is shown an enormous amount of text and repeatedly asked to predict missing or next tokens, adjusting itself slightly every time it's wrong
• Parameters — what it learns is stored as a huge set of numbers that shape how it responds, rather than as a library of saved documents
• Fine-tuning — the model is then trained further on examples of helpful conversations, often with people rating answers, so it behaves more like an assistant

The transformer and attention

Most modern language models use a design called the transformer. Its key ingredient is a mechanism called attention, which lets the model weigh how much each earlier token matters when predicting the next one. In the sentence "The trophy didn't fit in the case because it was too big," attention is part of how the model can connect "it" with the trophy rather than the case.

What happens when you send a message

Your message, along with the earlier conversation and any hidden instructions the service adds, is turned into tokens and fed to the model. The amount of text it can consider at once is called its context window. The model then generates a reply token by token. A small amount of randomness is usually built in, which is why asking the same question twice can produce different wording.

Memory is more limited than it seems

Within a conversation, the model "remembers" earlier messages only because they're sent back to it each time as part of the context. Once a conversation is long enough to exceed the context window, older parts can drop out. Some products add separate memory features, but those are extra systems built around the model.

Why they get things wrong

Because the model is trained to produce plausible text rather than to look things up, it can state something false with complete confidence. This is often called hallucination. It's more likely with obscure topics, precise figures, citations and recent events that happened after its training data was collected. Services that connect the model to search or documents can reduce this, but not eliminate it.

Using them sensibly

Language models are excellent for drafting, summarising, brainstorming and explaining concepts. They're less reliable as a final source of truth. A good rule is to treat an answer like advice from a well-read friend: often helpful, sometimes wrong, and worth double-checking whenever the details genuinely matter.

Frequently Asked Questions