ML Explorer · Level 5 · Modern AI
Transformers
A transformer block is attention plus a small network, each wrapped in a shortcut and layer normalisation. Build one, train it live to reverse eight digits, and watch its attention maps turn into a mirror image.
Overview
In attention, each position took a weighted average over the positions it was allowed to see, with weights from comparing a query with every key. A transformer puts attention inside a block with a few more parts, and stacks blocks. This lesson builds one block and trains it on a task you can check by eye: given 8 digits, such as 4 1 9 1 7 1 3 0, give them back in reverse order, 0 3 1 7 1 9 1 4.
You'll learn
- the parts of a transformer block, and why positions must be added
- what several attention heads, shortcuts and layer normalisation do
- how attention maps show which positions a trained model reads from
-
One block, part by part
Each digit becomes a list of 32 numbers, one list per position. The block changes these lists in two stages: attention, which mixes information between positions, then a small network, which works on each position on its own.
Hover over a box, or tab to it, to read what it does. The line along the bottom is the shortcut path: each stage reads a normalised copy of the lists, and its result is added back to them, so the input flows through the block unchanged plus what each stage adds.
The block here puts layer normalisation (LN) before each stage. Step 4 says what that does, and how the original design differed.
The maths
Digit at position (, here ) starts as a row of the embedding matrix plus a row of the position embedding , both of width :
Then, with the matrix of all the as rows,
where MHA is multi-head attention (step 3), LN is layer normalisation applied to each position's row (step 4), is and is . The output at each position is a probability for each digit:
Only MHA mixes positions; every other part acts on each position's row on its own, with the same weights at every position.
-
Attention ignores order
Take away the position embeddings, and attention with no mask (as here) cannot tell where a digit sits. Press "Shuffle the input": each output simply moves with its digit.
The block drawn here is untrained, with every position embedding set to 0, so its outputs are arbitrary; what matters is how they move. For each position the stage shows the most likely output digit and its probability. Shuffle the input, and the outputs come out shuffled in exactly the same way (the dashed lines show where each digit came from). So, with no mask, the output at a position depends only on the digit there and on which digits the sequence holds, not on their order.
That rules out reversing. The example 4 1 9 1 7 1 3 0 has the digit 1 at positions 2, 4 and 6, so without positions those three get the same output, yet the reversed sequence needs 3, 7 and 9 there. This is why the original transformer, and the block here, add a position embedding to each digit's embedding before the first block (some later models put position into the attention scores instead). With positions put back, the read-out shows that a shuffle changes more than the order of the outputs.
The maths
A shuffle is a permutation matrix that reorders the rows of , with . The parts that act on each row on their own just follow the rows. For attention, the queries, keys and values of are , , , so the scores become . Softmax works row by row, so the weights become , and
So without positions, . If two positions hold the same digit, swapping them leaves unchanged, so their outputs must be equal. The page's gap is the largest difference between the two sides; it is not exactly 0 only because the sums are added in a different order. With a causal mask (the next lesson uses one) this no longer holds, because each position then sees a different set of positions.
-
Several heads
This block has 2 attention heads. Each head has its own weights for Q, K and V, so each makes its own map of who looks at whom.
Each head works with 16 numbers per position instead of 32. The two heads' results, 16 numbers each, are joined into 32 and mixed by one more matrix, WO. The maps show the two heads on the example, for the model as it is now; Train runs the same training as step 5.
In each map, row is output position , the columns are the input positions (labelled with their digits), and a brighter cell is a larger weight. Each row's weights add up to 1, and a dot marks its largest. Several heads let one block gather from several places at once, each with its own way of matching.
The read-out counts, over all 4,000 rows of the 500 test sequences, how often a row's largest weight falls on its mirror position, the input position the reversed answer needs (step 5).
The maths
With , head (, here ) has its own matrices, :
with softmax taken along each row. Here the rows of the matrices are positions, so the weight matrices multiply on the right; step 1 wrote one position as a column, with the matrix on the left. The heads' outputs are placed side by side and mapped back to width by the matrix :
There is no mask: every position may look at every position, before and after it. The maps are and .
-
Shortcuts and normalisation
A shortcut (residual connection) adds what came into a stage to what the stage produces, so the stage only has to learn a change to it. Layer normalisation rescales each position's numbers before a stage reads them.
The shortcut also gives the gradient a direct path backwards through the block, the same fix for shrinking gradients met in going deeper.
The charts show the 32 numbers at one position of the example as they enter the untrained block, and the same numbers after layer normalisation. At position 1 they have mean −0.16 and variance 0.41; after it, mean 0 and variance 1 (to three decimal places; just under 1, because of a small ε). Each position is rescaled on its own. Then a learned scale and shift are applied, which start at 1 and 0. Move the slider to see other positions.
Here, layer norm comes before each stage, and the shortcut path itself is never rescaled (pre-LN). The original transformer (Vaswani et al., 2017) put it after the add instead, computing LayerNorm(x + stage(x)). Both are in use; this page uses the first.
The maths
For one position's row of numbers, with the population variance (divide by , not ):
with and entry-by-entry multiplication. Here and are learned vectors, not momentum or eigenvalues. With and the output has mean 0 and variance . For a shortcut , the derivative with respect to is : the passes the gradient back unchanged, however small the stage's own part is.
-
Train it to reverse
Press Train. The block learns from 2,000 sequences of 8 random digits, each with its reverse as the target. Watch the two maps for the example.
Play steps through the lesson, and Train trains the model. Training runs Adam (from optimisers) with η = 0.001 on batches of 40 sequences, for 250 steps: 5 passes through the training sequences. The loss on the training sequences starts at 2.57, a little above the 2.30 of giving every digit probability 1 in 10, and ends at 0.01. The run takes about 7 seconds on a laptop and can take 20 seconds or more on a slow phone. Reset goes back to the same starting weights, and the next run is the same as the last.
Every 25 steps the page checks 500 test sequences the model never trains on. At the random start it gets 11.2% of the test digits right, near the 10% of guessing. From the check at step 125 on, every output position is right on all 500 test sequences, and at the end the lowest position is still 500 of 500.
To give output position the model must read input position , its mirror. In the maps that is the anti-diagonal, from top right to bottom left. At the start, the largest weight of a row falls on the mirror position in 17.7% of the 4,000 test rows for head 1 and 16.0% for head 2, not far from the 12.5% of picking one of 8 positions at random. After the run it does so in 3,998 of the 4,000 rows for head 1 and 4,000 for head 2, with mean weight 0.78 and 0.89 on the mirror position. Both heads learned the same pattern here; nothing told them to split the work.
The maths
For one sequence the loss is the mean over the 8 positions of the negative log-probability of the right digit (natural log):
and each step averages it over a batch of 40 sequences, then moves every weight with Adam (, , , , Adam's own ε, not layer norm's). The predicted digit at a position is the one with the largest probability. The mirror share counts the rows of whose largest weight is in column .
-
What we built
The model is one block with 9,386 numbers, every one adjusted by training. The table counts them part by part.
Most of them sit in the attention weights and the small network. Real systems use the same idea at far larger scale: many blocks stacked one on another, each much wider. The original transformer's base model stacked 6 blocks to read its input and 6 to write its output (the writing blocks also attend to the reading blocks' output), each 512 numbers wide with 8 heads (Vaswani et al., 2017).
This block lets every position look both ways, which suits a task where the whole input is given at once. To write text one token at a time, a model may only look back; the next lesson adds that mask and trains a tiny language model.
The maths
With vocabulary size , , , and a network of width , the count is
for the digit and position embeddings, the first layer norm, of both heads, , the second layer norm, the small network, the final layer norm and the output layer. The attention matrices have no bias terms. The total is 9,386.
Quick check
Lesson complete
- A transformer block is multi-head attention and a small per-position network, each read through layer normalisation and added back by a shortcut.
- Attention with no mask ignores order, so the block here, like the original transformer, adds a position embedding to each token's embedding.
- Trained to reverse 8 digits, the block here got every test digit right, and both heads put their largest weight on the mirror position: the anti-diagonal in the maps.
Common misconceptions
- "Attention can see the order of the tokens by itself." With no mask and no positions, shuffling the input only shuffles the outputs.
- "Each head always learns something different." Here both heads ended on the same mirror pattern; the heads are free to split the work, not made to.
- "Layer norm normalises across the batch." It rescales each position's own numbers, separately for every position and every sequence.
- "An attention map explains why the model gave its answer." It shows where one head's weights went. That the output then copies the right digit is what the test accuracy shows.
Further reading
- Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser & Polosukhin (2017), "Attention Is All You Need", arXiv:1706.03762, the paper that introduced the transformer.
- Ba, Kiros & Hinton (2016), "Layer Normalization", arXiv:1607.06450.
- Prince, Understanding Deep Learning (2023, free online), on transformers.
About the data. Each sequence is 8 digits, each for a uniform draw from the mulberry32 random number generator; the target is the same digits in reverse order. The 2,000 training sequences use seed 281 and the 500 test sequences seed 282; the example in the maps is the first test sequence. The starting weights use seed 283: the digit and position embeddings are 0.5 times a standard normal draw, each weight matrix has normal draws with variance 1 divided by its number of inputs, the layer norm scales start at 1, and the shifts and biases at 0. The order of the training sequences in each pass uses seed 284, and the shuffles in step 2 are Fisher–Yates shuffles with seed 285.
If this helped, you can .