cCarebun Reading‹ All reading
Carebun Interactive/How an LLM works LabCheat sheet Study view

An interactive mini book

How an LLM Works
Novice to Ninja

What happens between your question and the answer: tokens, weights, one multiplication repeated billions of times, and a guess. Taught from zero, with a real model opened up on the page.

Start readingOpen the lab

17 chapters · about 3.2 hours · 48 interactives · no background needed

How to read this book

  1. In order. Each chapter uses only what the earlier ones taught. The whole book takes about three hours of reading, and longer if you play.
  2. With your hands. Chapters marked interactive have sliders and small machines. Make your guess first, then move them.
  3. Trust the measured numbers. We opened a real model, GPT-2 small, and printed what is inside. Every such number can be reproduced with the programs in the book's examples folder.
  4. Answer before you look. Every chapter ends with questions and folded answers. Recalling beats re-reading.

Part I · Novice

What a language model is: a machine that guesses the next word, built from weights.

01The machine that guesses the next wordA language model does one thing: it guesses what comes next. Watch a real model do it, one token at a time, and see why that small trick is enough to write an answer.9 min · interactive02Neurons and weightsWhy you know a car at once and an old school friend only sometimes. The brain stores what it knows as the strength of connections, and a language model borrows exactly that idea.10 min · interactive03Y = W · X: the one equationA whole layer of neurons is one multiplication. Do it by hand with twelve weights, learn the words matrix and tensor, and count the 124,439,808 weights of a real model.11 min · interactive

Part II · Apprentice

Inside the machine: tokens, embeddings, attention, chances, and how the weights were learned.

04Tokens: text becomes numbersA computer can only multiply numbers, so the first step is to turn text into them. Cut your own sentences with a real tokenizer, and find out how many words a token is worth.10 min · interactive05Embeddings: numbers become meaningA token id is only a house number. The embedding table gives each token a place on a map, where Monday lives next to Tuesday. Look at a real table and its neighbourhoods.10 min · interactive06Attention and layersHow does the model know what 'it' refers to? Tokens look at each other. See real attention from a real model, and watch the answer 'France' appear at layer 9 of 12.11 min · interactive07From scores to a wordThe model ends with 50,257 scores. Turn them into chances, turn the temperature, cut off the tail, and draw. Here is why the same question can get different answers.11 min · interactive08How the weights were learnedNobody typed in 124 million numbers. Train one weight by hand, then watch a small model go from random letters to English-looking text in sixteen seconds.12 min · interactive

Part III · Journeyman

Using a model: the API, the context window, the bill, and what goes wrong.

09Calling an LLM: the APIA model is reached by sending one JSON object and reading one back. Build the request, read every field of a real response, and see what the model is really given to read.12 min · interactive10The context window and the billEverything about a model is counted in tokens: how much it can read, how much it may write, and what you pay. Do the arithmetic once, and see why a long chat costs far more than it seems.10 min · interactive11Why it makes things upWe asked a real model about a book that does not exist, a World Cup in the future and a sum. It answered all three with confidence. Here is the reason, and what a model cannot do.11 min · interactive12Giving the model what it lacksThe weights are frozen, so everything new must arrive through the prompt. Repair three wrong answers of the last chapter by three methods: documents in the prompt, examples in the prompt, and a calculator the model can ask for.11 min · interactive

Part IV · Ninja

How large the machines are, and one request followed from the first byte to the last.

13How big is a model?A model is a file. Account for every byte of a real one, climb the ladder from 30,829 weights to a reported, unconfirmed 10 trillion, set it against the brain, and work out what will run on your laptop.11 min · interactive14One request, end to endFollow one question from the keyboard to the answer, through nine stations, with the numbers of a real model. This is the whole book in one journey.10 min · interactive

Part V · Practice

Play with every number, answer the usual questions, and explain it all on a whiteboard.

15The LLM labEvery interactive of the book on one page, in the order a request travels. Come back here whenever you want to try a number.20 min · interactive16Questions readers askThirty-two questions that come up when people first understand how a language model works, answered briefly, each with the chapter where the long answer is.14 min17Whiteboard it cold, and what to read nextThe whole book on two pages: the picture, the numbers, the formulas and the words. Then the plan for the books that follow, on the inside of the transformer and on serving with vLLM and SGLang.12 min