One transformer block

❓ Attention is half a floor. Mix-clip-mix is the other half. How do look, think, and a skip around each become one block?

One block: tokens talk, then each token thinks alone, then add the skip.

Whiteboard: one block is talk, think, and a skip around each.

The last math page kept attention’s mix soft. This page puts that looking step next to mix-clip-mix, and adds the original signal back so it does not die.

Attention moves information between tokens. The rest of the block processes each token by itself. Talking is not the same as thinking.

You can pass notes around a table (attention). That does not decide what to do with a note. Each person still has to read, recall, and write. In the transformer, that second job is mix-clip-mix applied to each position independently. People call it a feed-forward network, or an MLP (multilayer perceptron). In this course those names mean only: mix, clip, mix, stacked, once per token, same knobs for every token.

Two jobs, plus a skip

Whiteboard: add the input back. The skip is a path around the clip.

One block is two jobs.

  1. Multi-head self-attention (the last story page), with the no-peek mask.
  2. A position-wise mix-clip-mix: two mixes with a clip in between, typically stretching the list to four times as long and then mixing back.

Around each job: add the input back and re-center the numbers.

Adding the input back is a residual connection, also called a skip. If the job did nothing useful yet, the block can still be the identity: (x+0=x). Slopes used in training have a path that does not vanish through fifty clips.

Re-centering is layer normalization (LayerNorm): subtract the mean of the list, divide by its spread, then scale and shift with a couple of learned knobs. Think of it as a volume knob so the next table of knobs does not see wild magnitudes. The 2017 paper put it after the add. Many later models put it before the job. Same intent.

In symbols, the paper’s version is

[
x \leftarrow \mathrm{LayerNorm}\big(x + \mathrm{Sublayer}(x)\big).
]

Modern chat models rearrange the norm and swap the clip for a smoother curve. The skeleton does not change.

Chat models use this writer block, with the no-peek mask, and they do not include the extra “look at a separate reader stack” the original translation model needed.

If we deleted the skip, what would fifty stacked clips tend to do to the first list?
Kill it. The skip is a path around the clip.

Lanes up a high-rise

Picture each token as a lane running up a high-rise. At every floor two things happen.

On the communication floor, lanes look sideways. fluffy can add something into creature. That is attention.

On the office floor, each lane goes into its own cubicle. The same cubicle design is used for every lane. Inside, the list is stretched into a wider list of neurons — chapter 5’s mix-clip-mix outputs; a neuron is “active” when that number is positive. Most stay quiet. A few fire. Those firings write directions back into the lane — maybe a “sport” direction when the incoming list already means a famous basketball player. Then the lane is added to what it was before it entered the cubicle, so the old information is still there.

Then the next floor.

A useful cartoon, even if you never run the experiment: knock out looking and you can still get fluent local English with a weak grip on what the sentence was about; knock out the think-alone nets and the English itself falls apart. Treat that as intuition, not as a theorem. The architectural moral stands: two jobs, stacked.

Data through one chat-model block

Let (x) be the table of token lists, shape (n \times d) (n tokens, d numbers each).

  1. (Often) LayerNorm (x).
  2. Compute masked multi-head attention. Output has the same shape as (x).
  3. Add: (x \leftarrow x + \mathrm{Attn}(\cdot)). Skip.
  4. LayerNorm.
  5. Mix-clip-mix on each row independently: stretch, clip, mix back.
  6. Add: (x \leftarrow x + \mathrm{FFN}(\cdot)). Skip.

In, out, same shape. That is why you can stack 12 or 96 of them. The skip is why stacking does not immediately destroy the first list from chapter 4.

Does the feed-forward net let it look at animal?
No. That looking already happened. This net only rewrites the numbers inside it.

What this page leaves out

None of those change the two-job story.

Draw a box with two rooms. Label them talk and think. Draw a skip arrow around each room. Write on the skip arrow: +.

If you can do that from memory, you have the transformer architecture. The “large” in large language model is how many times you repeat the box, and how long the lists are.

What this still cannot do

One floor is not a city.
How many times is the block stacked, and how does the last list become a next piece?

The next page is the stack, the un-embedding, and the pick.

← ScaleStack →