Skip to the lesson
ML Explorer Reju Sam John

ML Explorer · Level 5 · Modern AI

Large language models, in miniature

Most large language models today are transformers trained to predict the next token on a very large amount of text. Train a tiny one here, live, watch its writing go from noise to words, and then check how many of the sentences it writes are true.

About 12 minutes · The model trains in your browser in about a minute on a laptop; every number is computed in your browser, and the formulas are in The maths

Overview

In Predicting the next token a model counted which character followed the last few, and wrote text by sampling. Here the job is the same, but the model is the causal transformer of Transformers, built on Attention: it can look back at up to 32 characters, and it learns its probabilities by training rather than counting.

The text it learns from describes the small world invented for this level: eight made-up people, the food each one likes and the place each one lives. Its facts are rules of this page, not claims about anyone. Because the facts are fixed, every sentence the model writes can be checked.

You'll learn

  • that a large language model is trained on one job, predicting the next token, and writes by sampling one token at a time
  • how a tiny one learns: its held-out loss falls and its samples turn from noise into words
  • why fluent output can be false: the model samples likely text, and nothing checks it against the facts
  1. Same job, more context

    The model predicts the next character, as the counting models did, but from up to 32 characters before it.

    The picture shows one sentence of the invented world. Under each character is the character that follows it: that is what the model must predict there. One piece of text gives one prediction per position.

    Look at the outlined column, the prediction after "tom likes ". The food depends on the name, which starts 10 characters back. The counting model with n = 5, which uses the last 5 characters "ikes␣", cannot see it: in the training text it gives a only 0.26, because two of the eight people like apples. This model can look at every earlier character in its window of 32, through attention, so it can learn to use the name.

    The model is one transformer block, as in Transformers, with two attention heads, but causal: each position may look only at itself and the positions before it. Its 28 tokens are the characters of the invented world, letters, the space and the full stop, as in Tokens: text as numbers.

    The maths

    Take a window of characters x1,…,xT+1x_1, \dots, x_{T+1} with T = 32. The model reads x1,…,xTx_1, \dots, x_T and gives, at each position t, a probability for every character of the vocabulary V\mathcal{V} (here ∣V∣=28|\mathcal{V}| = 28). The loss for the window is the mean cross-entropy of the true next characters:

    L=−1T∑t=1Tlog⁡p(xt+1∣x1,…,xt).\mathcal{L} = -\frac{1}{T}\sum_{t=1}^{T} \log p(x_{t+1} \mid x_1, \dots, x_t).

    The causal mask is what makes this one sum honest. In the attention softmax, position t leaves out every position s > t, so its prediction depends on x1,…,xtx_1, \dots, x_t only and never on the answer xt+1x_{t+1}. One forward pass therefore gives all 32 predictions at once, and one backward pass trains all of them. A batch's loss is the mean over its windows.

  2. Train it

    Press Train. The line is the held-out loss; under the picture, a new sample appears every 100 steps.

    Play steps through the lesson, and Train trains the model. Each step takes 8 windows of 33 characters at random from the training text of 2,000 sentences (42,030 characters) and makes one Adam update (from optimisers) with η = 0.003. The run is 2,500 steps: about a minute on a laptop, about two minutes or more on a slow phone. Every 100 steps the page measures the model on held-out text and writes a sample of 60 characters at τ = 1.

    The line is in bits per character on the 300 held-out sentences of Predicting the next token, never used for training. The dashed lines are the counting models of that lesson for n = 1, 3 and 5, measured on exactly the same 6,240 characters (from the 6th character on, so the n = 5 model always has its 5 characters of context): 1.99, 0.47 and 0.39 bits.

    These numbers are for the whole run. The model starts at 4.95 bits, about the cost of a uniform guess (4.81). It is below the n = 1 line from step 200, and below the n = 3 line from step 800 on. It first dips below the n = 5 line at step 2,400, and ends just above the n = 5 counting line: 0.393 bits against 0.391. In this short run it does not beat counting: on this small, regular world counting with n = 5 already does well, and the previous lesson's longer contexts do better still. What the samples show is the change from noise to the words of this world. Of the complete words in each sample shown (the first and last may be cut off), 0% are words of the invented world at step 0; from step 700 on at least 90% are, and from step 1,100 on all of them are.

    Each sample starts after ". " and uses its own seed, so watching the run or not gives the same samples. With reduced motion, Train computes the run without drawing it, then shows the end. Reset goes back to the same starting weights, and the next run is the same as the last.

    The maths

    Training works in natural logarithms (nats). Bits are nats divided by ln 2:

    H=−1N∑i=1Nlog⁡2pi=1Nln⁡2∑i=1N(−ln⁡pi),H = -\frac{1}{N}\sum_{i=1}^{N} \log_2 p_i = \frac{1}{N \ln 2}\sum_{i=1}^{N} \big(-\ln p_i\big),

    where pip_i is the probability the model gave the true character at scored position i, and N = 6,240. The held-out text is read in windows of 32, so a scored character has between 1 and 32 characters before it in its window; the counting lines use their last n. Each sample character is drawn from the model's probabilities given the last 32 characters, as in step 3.

  3. Prompting

    Type the start of a text; the bars give the probability of each next character. Continue adds one, drawn at the temperature τ.

    The bars use the model as it is now: train it in step 2 first, or they come from random weights. Try "zoe likes ", then "zoe lives ", then a name the world does not have.

    Continue draws one character from the bars and adds it to your text, so pressing it again and again writes a continuation, one character at a time. τ = 1 samples from the model's own probabilities, a smaller τ sharpens them, and τ = 0 always takes the likeliest character (greedy). The dashed outlines show τ = 1.

    Capitals become lower case, and the box takes up to 80 characters. A character outside the 28, such as a digit or an emoji, is shown as a dashed chip with a ? and skipped; the model sees the last 32 known characters. What you type stays in this page; it is not stored or sent anywhere.

    The maths

    At the last position the model gives a score zvz_v (a logit) for every character v, and the probabilities at temperature τ are

    qτ(v)=softmax⁡ ⁣(zτ)v=ezv/τ∑u∈Vezu/τ.q_\tau(v) = \operatorname{softmax}\!\left(\frac{\mathbf{z}}{\tau}\right)_v = \frac{e^{z_v/\tau}}{\sum_{u \in \mathcal{V}} e^{z_u/\tau}}.

    This is the same as the previous lesson's p(v)1/τp(v)^{1/\tau}, normalised, since log⁡p(v)\log p(v) and zvz_v differ by the same constant for every v. τ = 1 gives the model's own probabilities.

  4. Fluent is not the same as true

    After "ana likes ", any food makes a fluent sentence, but only one is true in this world. The bars give the exact probability, averaged over the 8 people, that the model writes a true one.

    For each of the 8 people and both prompts, "<name> likes " and "<name> lives ", the bars give the probability that the sentence the model samples next is true, false (a real food or place of this world, but the wrong one) or anything else. They are exact, multiplied out from the model's probabilities, not counted from samples. Write 16 sentences samples one sentence per prompt and marks each one ✓ true, ✗ false or – other; the counts add up press by press.

    These numbers are for the stated run at τ = 1. The mean probability of the true food rises from 0.19 at step 500 to 0.36 at step 1,000, 0.69 at 1,500, 0.74 at 2,000 and 0.85 at 2,500; a false food keeps 0.11. The true place rises more slowly: 0.13, 0.28, 0.34, 0.41 and 0.45, and at the end a false place gets more, 0.51. The model gives each person's own "by" or "in" 0.94 on average, but of the two places that share that word, the true one gets only 0.50, about half.

    So the model learned the shape of the sentences before it linked each name to its facts: at step 500 it already gives 0.72 to a well-formed food sentence, true or false, but only 0.19 to the true one. This is what is often called "hallucination": fluent output that is false. Here you can see why it happens in this model. The model gives probabilities to characters; sampling draws characters according to those probabilities; nothing checks the sentence against the facts table. A false place is a perfectly likely-looking sentence.

    Temperature changes the odds, at the end of the run (true · false · anything else):

    • τ = 0: greedy writes one sentence per prompt, always a fact sentence: the true food for 7 of the 8 people and the true place for 4, a false one for the rest.
    • τ = 0.5: food 0.93 · 0.07 · 0.00; place 0.49 · 0.51 · 0.00.
    • τ = 1: food 0.85 · 0.11 · 0.04; place 0.45 · 0.51 · 0.04.
    • τ = 1.5: food 0.62 · 0.16 · 0.21; place 0.31 · 0.39 · 0.30.

    From τ = 0.5 up, a higher τ takes probability away from the true sentence. For foods, each step up in τ makes false sentences more likely; for places, going from τ = 1 to 1.5 sends probability to broken text instead. Longer training raises the true share, as the steps above show, but sampling from a model never guarantees a true sentence.

    The maths

    The model writes a sentence one character at a time, so the probability of a whole continuation is the product of its characters' probabilities, each given everything before it:

    p(apples.∣ana likes )=qτ(a∣ana likes )×qτ(p∣ana likes a)×⋯×qτ(.∣ana likes apples).p(\text{apples.} \mid \text{ana likes }) = q_\tau(\mathrm{a} \mid \text{ana likes }) \times q_\tau(\mathrm{p} \mid \text{ana likes a}) \times \dots \times q_\tau(. \mid \text{ana likes apples}).

    The true probability is this product for the person's own food (or place); the false probability is the sum of the products for the other foods (or the other places, each with its own "by" or "in"). Ending with the full stop makes these different sentences, so the sums do not overlap, and anything else gets 1−ptrue−pfalse1 - p_{\text{true}} - p_{\text{false}}. The bars show the mean of each over the 8 people.

  5. Scale

    The same recipe, made very much larger, is a large language model.

    This model has 11,324 weights and biases; the bars show where they are. It trained on 42,030 characters of text, and over the run it made 640,000 next-character predictions (2,500 steps × 8 windows × 32 positions).

    Real large language models use the same idea at far larger scale: subword tokens rather than characters, many blocks rather than one, far longer windows, far more weights and far more text, with the same loss. Their sentences come from the same process as here: probabilities, then sampling.

    The maths

    With d = 32, two heads of width 16, a per-position network of 64 units and 28 tokens, the count part by part is: token embeddings 28×3228 \times 32, position embeddings 32×3232 \times 32, attention 4×32×32=40964 \times 32 \times 32 = 4096 (queries, keys, values and the output map), the per-position network 64×32+64+32×64+32=419264 \times 32 + 64 + 32 \times 64 + 32 = 4192, three layer norms with 2×322 \times 32 each, 192 in all, and the output layer 28×32+28=92428 \times 32 + 28 = 924:

    28×32+32×32+4096+4192+192+924=1132428 \times 32 + 32 \times 32 + 4096 + 4192 + 192 + 924 = 11324
  6. After pretraining

    Training to predict the next token on a large amount of text is called pretraining. The models people talk to are usually trained further.

    Instruction tuning continues training on examples of requests paired with good replies, so the model's likeliest continuation of a request is a reply. Preference tuning then trains the model towards replies that people preferred when shown two or more.

    Both change the probabilities the model gives, and can make true replies likelier; neither adds a step that checks each sentence against the facts. The model in this lesson stops after pretraining, in miniature.

    The maths

    Instruction tuning usually keeps the next-token cross-entropy of step 1, applied to the example replies. Preference tuning uses a different loss, built from pairs of replies of which people preferred one; the details vary from method to method.

Quick check

What is the model trained to do?

Right. The loss is the cross-entropy of the next character at every position, so training raises the probability of whatever came next in the training text.

There is no table inside the model, only weights. Any link between a name and its food has to come out of predicting the next character, and at the end of this run the link for places is still weak.

Nothing in the loss says true or false. It rewards giving probability to what followed in the training text; the false sentences in step 4 come from the same probabilities.

It is scored on held-out text it never saw, and its samples are drawn at random from its probabilities. A sample can repeat a training sentence, but it is not read from the text: the false sentences of step 4 never occur in it.

Why can a fluent sentence from the model be false?

Right. At the end of the run a false place gets 0.51 of the probability at τ = 1, more than the true one.

A model has no aims. It gives probabilities and a random draw picks from them; a false sentence is just a likely string of characters.

Every likes and lives sentence in the training text is true in the invented world. The false ones come from probabilities that have not linked each name to its facts.

At τ = 0 the model writes its likeliest sentence, and at the end of the run that is the true place for only 4 of the 8 people.

Lesson complete

  • The large language models people talk to are mostly causal transformers trained to predict the next token; the causal mask lets one window train every position at once.
  • It writes by sampling one token at a time from its probabilities, and temperature reshapes them.
  • Its sentences are likely, not checked: a fluent sentence can be false, and here the model learned word shapes before it linked names to facts.

Common misconceptions

  • "A lower loss means the model tells the truth." The held-out loss here ends just above the n = 5 counting line, yet a false place still gets more probability than the true one.
  • "Greedy decoding gives the correct answer." It gives the likeliest one; at the end of this run greedy writes the true place for only 4 of the 8 people.
  • "The model looks facts up." It has only weights; what it gives a fact is a probability learned from the training text.
Next lesson: Generative models →

Further reading

  1. Jurafsky & Martin, Speech and Language Processing (3rd ed. draft), Chapter 7, "Transformers and Pretraining", and Chapter 8, "Post-training".
  2. Vaswani et al., "Attention Is All You Need", arXiv 1706.03762 (2017), which defines the transformer, including the causal mask used for predicting the next token.

About the data. The world is invented: eight made-up people (ana, ben, eli, mia, noa, sam, tom, zoe), six foods and four places, with a fixed table of who likes which food and lives where. Its facts are rules of this page, not claims about anyone. All the text is made in your browser by the mulberry32 random number generator: the training text is 2,000 sentences from seed 261 and the held-out text 300 sentences from seed 262, by the rule of Predicting the next token (for each sentence, one uniform draw u and a person chosen uniformly: "<person> likes <their food>." if u < 0.4, "<person> lives by/in the <their place>." if u < 0.8, otherwise "<person> and <friend> are friends."). The model's starting weights use seed 292, and the training windows are drawn with seed 291. The sample at step s uses seed 293 + s; the sentences of the k-th press of Write 16 sentences use seed 294 + k (from k = 0); Continue draws from one stream with seed 295.

If this helped, you can .