Skip to the lesson
ML Explorer Reju Sam John

ML Explorer · Level 5 · Modern AI

Transformers

A transformer block is attention plus a small network, each wrapped in a shortcut and layer normalisation. Build one, train it live to reverse eight digits, and watch its attention maps turn into a mirror image.

About 10 minutes · Every weight, map and accuracy is computed in your browser; the formulas are in The maths

Overview

In attention, each position took a weighted average over the positions it was allowed to see, with weights from comparing a query with every key. A transformer puts attention inside a block with a few more parts, and stacks blocks. This lesson builds one block and trains it on a task you can check by eye: given 8 digits, such as 4 1 9 1 7 1 3 0, give them back in reverse order, 0 3 1 7 1 9 1 4.

You'll learn

  • the parts of a transformer block, and why positions must be added
  • what several attention heads, shortcuts and layer normalisation do
  • how attention maps show which positions a trained model reads from
  1. One block, part by part

    Each digit becomes a list of 32 numbers, one list per position. The block changes these lists in two stages: attention, which mixes information between positions, then a small network, which works on each position on its own.

    Hover over a box, or tab to it, to read what it does. The line along the bottom is the shortcut path: each stage reads a normalised copy of the lists, and its result is added back to them, so the input flows through the block unchanged plus what each stage adds.

    The block here puts layer normalisation (LN) before each stage. Step 4 says what that does, and how the original design differed.

    The maths

    Digit iti_t at position tt (t=1,…,Tt = 1, \dots, T, here T=8T = 8) starts as a row of the embedding matrix E\mathbf{E} plus a row of the position embedding P\mathbf{P}, both of width d=32d = 32:

    xt=Eit+Pt\mathbf{x}_t = \mathbf{E}_{i_t} + \mathbf{P}_t

    Then, with X\mathbf{X} the T×dT \times d matrix of all the xt\mathbf{x}_t as rows,

    ht=xt+MHA⁡(LN⁡1(X))t,yt=ht+W2 ReLU⁡(W1LN⁡2(ht)+c1)+c2\mathbf{h}_t = \mathbf{x}_t + \operatorname{MHA}\big(\operatorname{LN}_1(\mathbf{X})\big)_t, \qquad \mathbf{y}_t = \mathbf{h}_t + \mathbf{W}_2\, \operatorname{ReLU}\big(\mathbf{W}_1 \operatorname{LN}_2(\mathbf{h}_t) + \mathbf{c}_1\big) + \mathbf{c}_2

    where MHA is multi-head attention (step 3), LN is layer normalisation applied to each position's row (step 4), W1\mathbf{W}_1 is 64×3264 \times 32 and W2\mathbf{W}_2 is 32×6432 \times 64. The output at each position is a probability for each digit:

    p^t=softmax⁡(WoutLN⁡f(yt)+bout)\hat{\mathbf{p}}_t = \operatorname{softmax}\big(\mathbf{W}_{\text{out}} \operatorname{LN}_f(\mathbf{y}_t) + \mathbf{b}_{\text{out}}\big)

    Only MHA mixes positions; every other part acts on each position's row on its own, with the same weights at every position.

  2. Attention ignores order

    Take away the position embeddings, and attention with no mask (as here) cannot tell where a digit sits. Press "Shuffle the input": each output simply moves with its digit.

    The block drawn here is untrained, with every position embedding set to 0, so its outputs are arbitrary; what matters is how they move. For each position the stage shows the most likely output digit and its probability. Shuffle the input, and the outputs come out shuffled in exactly the same way (the dashed lines show where each digit came from). So, with no mask, the output at a position depends only on the digit there and on which digits the sequence holds, not on their order.

    That rules out reversing. The example 4 1 9 1 7 1 3 0 has the digit 1 at positions 2, 4 and 6, so without positions those three get the same output, yet the reversed sequence needs 3, 7 and 9 there. This is why the original transformer, and the block here, add a position embedding to each digit's embedding before the first block (some later models put position into the attention scores instead). With positions put back, the read-out shows that a shuffle changes more than the order of the outputs.

    The maths

    A shuffle is a permutation matrix Π\boldsymbol{\Pi} that reorders the rows of X\mathbf{X}, with ΠTΠ=I\boldsymbol{\Pi}^{\mathsf{T}} \boldsymbol{\Pi} = \mathbf{I}. The parts that act on each row on their own just follow the rows. For attention, the queries, keys and values of ΠX\boldsymbol{\Pi}\mathbf{X} are ΠQ\boldsymbol{\Pi}\mathbf{Q}, ΠK\boldsymbol{\Pi}\mathbf{K}, ΠV\boldsymbol{\Pi}\mathbf{V}, so the scores become ΠQKTΠT\boldsymbol{\Pi}\mathbf{Q}\mathbf{K}^{\mathsf{T}}\boldsymbol{\Pi}^{\mathsf{T}}. Softmax works row by row, so the weights become ΠAΠT\boldsymbol{\Pi}\mathbf{A}\boldsymbol{\Pi}^{\mathsf{T}}, and

    (ΠAΠT)(ΠV)=ΠAV(\boldsymbol{\Pi}\mathbf{A}\boldsymbol{\Pi}^{\mathsf{T}})(\boldsymbol{\Pi}\mathbf{V}) = \boldsymbol{\Pi}\mathbf{A}\mathbf{V}

    So without positions, block⁡(ΠX)=Π block⁡(X)\operatorname{block}(\boldsymbol{\Pi}\mathbf{X}) = \boldsymbol{\Pi}\, \operatorname{block}(\mathbf{X}). If two positions hold the same digit, swapping them leaves X\mathbf{X} unchanged, so their outputs must be equal. The page's gap is the largest difference between the two sides; it is not exactly 0 only because the sums are added in a different order. With a causal mask (the next lesson uses one) this no longer holds, because each position then sees a different set of positions.

  3. Several heads

    This block has 2 attention heads. Each head has its own weights for Q, K and V, so each makes its own map of who looks at whom.

    Each head works with 16 numbers per position instead of 32. The two heads' results, 16 numbers each, are joined into 32 and mixed by one more matrix, WO. The maps show the two heads on the example, for the model as it is now; Train runs the same training as step 5.

    In each map, row tt is output position tt, the columns are the input positions (labelled with their digits), and a brighter cell is a larger weight. Each row's weights add up to 1, and a dot marks its largest. Several heads let one block gather from several places at once, each with its own way of matching.

    The read-out counts, over all 4,000 rows of the 500 test sequences, how often a row's largest weight falls on its mirror position, the input position the reversed answer needs (step 5).

    The maths

    With U=LN⁡1(X)\mathbf{U} = \operatorname{LN}_1(\mathbf{X}), head hh (h=1,…,Hh = 1, \dots, H, here H=2H = 2) has its own d×dkd \times d_k matrices, dk=d/H=16d_k = d/H = 16:

    Qh=UWhQ,Kh=UWhK,Vh=UWhV,Ah=softmax⁡ ⁣(QhKhTdk)\mathbf{Q}_h = \mathbf{U}\mathbf{W}^{Q}_h, \quad \mathbf{K}_h = \mathbf{U}\mathbf{W}^{K}_h, \quad \mathbf{V}_h = \mathbf{U}\mathbf{W}^{V}_h, \qquad \mathbf{A}_h = \operatorname{softmax}\!\left(\frac{\mathbf{Q}_h \mathbf{K}_h^{\mathsf{T}}}{\sqrt{d_k}}\right)

    with softmax taken along each row. Here the rows of the matrices are positions, so the weight matrices multiply on the right; step 1 wrote one position as a column, with the matrix on the left. The heads' outputs are placed side by side and mapped back to width dd by the Hdk×dH d_k \times d matrix WO\mathbf{W}^{O}:

    MHA⁡(U)=[ A1V1    A2V2 ] WO\operatorname{MHA}(\mathbf{U}) = \big[\, \mathbf{A}_1 \mathbf{V}_1 \;\; \mathbf{A}_2 \mathbf{V}_2 \,\big]\, \mathbf{W}^{O}

    There is no mask: every position may look at every position, before and after it. The maps are A1\mathbf{A}_1 and A2\mathbf{A}_2.

  4. Shortcuts and normalisation

    A shortcut (residual connection) adds what came into a stage to what the stage produces, so the stage only has to learn a change to it. Layer normalisation rescales each position's numbers before a stage reads them.

    The shortcut also gives the gradient a direct path backwards through the block, the same fix for shrinking gradients met in going deeper.

    The charts show the 32 numbers at one position of the example as they enter the untrained block, and the same numbers after layer normalisation. At position 1 they have mean −0.16 and variance 0.41; after it, mean 0 and variance 1 (to three decimal places; just under 1, because of a small ε). Each position is rescaled on its own. Then a learned scale γ\boldsymbol{\gamma} and shift β\boldsymbol{\beta} are applied, which start at 1 and 0. Move the slider to see other positions.

    Here, layer norm comes before each stage, and the shortcut path itself is never rescaled (pre-LN). The original transformer (Vaswani et al., 2017) put it after the add instead, computing LayerNorm(x + stage(x)). Both are in use; this page uses the first.

    The maths

    For one position's row x\mathbf{x} of dd numbers, with the population variance (divide by dd, not d−1d - 1):

    μ=1d∑j=1dxj,s2=1d∑j=1d(xj−μ)2,LN⁡(x)=γ⊙x−μs2+ε+β\mu = \frac{1}{d}\sum_{j=1}^{d} x_j, \qquad s^2 = \frac{1}{d}\sum_{j=1}^{d} (x_j - \mu)^2, \qquad \operatorname{LN}(\mathbf{x}) = \boldsymbol{\gamma} \odot \frac{\mathbf{x} - \mu}{\sqrt{s^2 + \varepsilon}} + \boldsymbol{\beta}

    with ε=10−5\varepsilon = 10^{-5} and ⊙\odot entry-by-entry multiplication. Here β\boldsymbol{\beta} and γ\boldsymbol{\gamma} are learned vectors, not momentum or eigenvalues. With γ=1\boldsymbol{\gamma} = \mathbf{1} and β=0\boldsymbol{\beta} = \mathbf{0} the output has mean 0 and variance s2/(s2+ε)s^2/(s^2 + \varepsilon). For a shortcut x+f(x)\mathbf{x} + f(\mathbf{x}), the derivative with respect to x\mathbf{x} is I+∂f/∂x\mathbf{I} + \partial f / \partial \mathbf{x}: the I\mathbf{I} passes the gradient back unchanged, however small the stage's own part is.

  5. Train it to reverse

    Press Train. The block learns from 2,000 sequences of 8 random digits, each with its reverse as the target. Watch the two maps for the example.

    Play steps through the lesson, and Train trains the model. Training runs Adam (from optimisers) with η = 0.001 on batches of 40 sequences, for 250 steps: 5 passes through the training sequences. The loss on the training sequences starts at 2.57, a little above the 2.30 of giving every digit probability 1 in 10, and ends at 0.01. The run takes about 7 seconds on a laptop and can take 20 seconds or more on a slow phone. Reset goes back to the same starting weights, and the next run is the same as the last.

    Every 25 steps the page checks 500 test sequences the model never trains on. At the random start it gets 11.2% of the test digits right, near the 10% of guessing. From the check at step 125 on, every output position is right on all 500 test sequences, and at the end the lowest position is still 500 of 500.

    To give output position tt the model must read input position 9−t9 - t, its mirror. In the maps that is the anti-diagonal, from top right to bottom left. At the start, the largest weight of a row falls on the mirror position in 17.7% of the 4,000 test rows for head 1 and 16.0% for head 2, not far from the 12.5% of picking one of 8 positions at random. After the run it does so in 3,998 of the 4,000 rows for head 1 and 4,000 for head 2, with mean weight 0.78 and 0.89 on the mirror position. Both heads learned the same pattern here; nothing told them to split the work.

    The maths

    For one sequence the loss is the mean over the 8 positions of the negative log-probability of the right digit (natural log):

    L=−1T∑t=1Tln⁡p^t(yt),yt=iT+1−t\mathcal{L} = -\frac{1}{T}\sum_{t=1}^{T} \ln \hat{p}_t(y_t), \qquad y_t = i_{T+1-t}

    and each step averages it over a batch of 40 sequences, then moves every weight with Adam (η=0.001\eta = 0.001, β1=0.9\beta_1 = 0.9, β2=0.999\beta_2 = 0.999, ε=10−8\varepsilon = 10^{-8}, Adam's own ε, not layer norm's). The predicted digit at a position is the one with the largest probability. The mirror share counts the rows tt of Ah\mathbf{A}_h whose largest weight is in column T+1−tT + 1 - t.

  6. What we built

    The model is one block with 9,386 numbers, every one adjusted by training. The table counts them part by part.

    Most of them sit in the attention weights and the small network. Real systems use the same idea at far larger scale: many blocks stacked one on another, each much wider. The original transformer's base model stacked 6 blocks to read its input and 6 to write its output (the writing blocks also attend to the reading blocks' output), each 512 numbers wide with 8 heads (Vaswani et al., 2017).

    This block lets every position look both ways, which suits a task where the whole input is given at once. To write text one token at a time, a model may only look back; the next lesson adds that mask and trains a tiny language model.

    The maths

    With vocabulary size ∣V∣=10|\mathcal{V}| = 10, T=8T = 8, d=32d = 32, Hdk=32H d_k = 32 and a network of width m=64m = 64, the count is

    ∣V∣d+Td+2d+3d(Hdk)+(Hdk)d+2d+(2dm+m+d)+2d+(∣V∣d+∣V∣)|\mathcal{V}| d + T d + 2d + 3 d (H d_k) + (H d_k) d + 2d + (2 d m + m + d) + 2d + (|\mathcal{V}| d + |\mathcal{V}|)

    for the digit and position embeddings, the first layer norm, WQ,WK,WV\mathbf{W}^Q, \mathbf{W}^K, \mathbf{W}^V of both heads, WO\mathbf{W}^O, the second layer norm, the small network, the final layer norm and the output layer. The attention matrices have no bias terms. The total is 9,386.

Quick check

Why does a transformer need position embeddings?

Right. Attention with no mask and no positions treats the input as a collection with no order; equal tokens even get equal outputs. Adding a position embedding makes the same digit at different positions look different.

Attention reaches every position in one step, near or far. What it lacks without positions is any sign of where a token is.

Without them the reversal task cannot be solved at all: the digit 1 at positions 2, 4 and 6 of the example would get the same output, though the answer needs 3, 7 and 9 there.

The token embeddings already do that: equal tokens have equal embeddings. Positions add where each token sits.

What does a residual connection add?

Right. The stage then only has to learn a change to its input, and the gradient has a direct path backwards through the add.

The shortcut has no weights at all; it is a plain addition. The parameter table has no row for it.

Nothing random is added; the shortcut is the same addition every time.

That is layer normalisation, the other part each stage is wrapped in.

Lesson complete

  • A transformer block is multi-head attention and a small per-position network, each read through layer normalisation and added back by a shortcut.
  • Attention with no mask ignores order, so the block here, like the original transformer, adds a position embedding to each token's embedding.
  • Trained to reverse 8 digits, the block here got every test digit right, and both heads put their largest weight on the mirror position: the anti-diagonal in the maps.

Common misconceptions

  • "Attention can see the order of the tokens by itself." With no mask and no positions, shuffling the input only shuffles the outputs.
  • "Each head always learns something different." Here both heads ended on the same mirror pattern; the heads are free to split the work, not made to.
  • "Layer norm normalises across the batch." It rescales each position's own dd numbers, separately for every position and every sequence.
  • "An attention map explains why the model gave its answer." It shows where one head's weights went. That the output then copies the right digit is what the test accuracy shows.
Next lesson: Large language models, in miniature →

Further reading

  1. Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser & Polosukhin (2017), "Attention Is All You Need", arXiv:1706.03762, the paper that introduced the transformer.
  2. Ba, Kiros & Hinton (2016), "Layer Normalization", arXiv:1607.06450.
  3. Prince, Understanding Deep Learning (2023, free online), on transformers.

About the data. Each sequence is 8 digits, each ⌊10u⌋\lfloor 10u \rfloor for a uniform draw uu from the mulberry32 random number generator; the target is the same digits in reverse order. The 2,000 training sequences use seed 281 and the 500 test sequences seed 282; the example in the maps is the first test sequence. The starting weights use seed 283: the digit and position embeddings are 0.5 times a standard normal draw, each weight matrix has normal draws with variance 1 divided by its number of inputs, the layer norm scales start at 1, and the shifts and biases at 0. The order of the training sequences in each pass uses seed 284, and the shuffles in step 2 are Fisher–Yates shuffles with seed 285.

If this helped, you can .