What this means when you build
❓ The rooms explain the weights. Context, sampling, tools, and the harness: where does a failure actually live?
You started able to call a model. You should now be able to draw what happens after Send.
A string becomes tokens. Tokens become lists of numbers. Those lists climb a stack of blocks. In each block they talk (no-peek attention) and think (mix-clip-mix), adding the result back to themselves. The last list becomes a pile of chances for the next token. A picker chooses. Your loop repeats. Sometimes the chosen tokens are a tool call, and you run code and stuff the result back into the only working memory the model has: the window.
Nothing in that paragraph requires a second substance called understanding. Until tools do the exact work, the system is pulling a coherent thread out of the statistics of conventional wisdom it compressed. It is also, unexpectedly, enough to be useful.
Map of knobs you already use
| Knob you turn | Where it lives on the machine |
|---|---|
| System prompt | Tokens at the left of the window. Not a different brain. |
| Chat history | More tokens. When you drop history, you wipe working memory. |
| Retrieval / file paste | Extra tokens the last position can look at. Quality depends on what you retrieved, not on “the model knowing.” |
| Context window | Sequence length (n). Looking costs about (n^2). Past (n), the early tokens are simply gone. |
| Temperature / top-p | A policy on the pile of chances after softmax. The learned knobs inside are unchanged. |
| Stop sequences / max tokens | Your loop. The model would happily continue. |
| JSON / schema / constrained decode | Hiding some of the tokenizer’s list at pick-time, or retrying. |
| Tools / function calling | Model emits tokens; your loop executes; results return as tokens. |
| Fine-tuning | Changing knobs, usually a small slice of them. Expensive compared to prompting; persistent. |
| “The model is an expert at X” | Either it saw X in pre-training, or you put X in the window, or you fine-tuned. There is no fourth mechanism. |
If a behavior you want is not on this table, you do not yet have a lever. Inventing a poetic name for it will not create one.
What to do with the picture
Act now. On the next production prompt, print the tokens, sketch the last-list story, and classify the last bug with chapter 11’s list. Change one lever on the table, not three.
Watch. If you are choosing a vendor or a local model, watch window length, tokenizer family, and whether you need knobs changed or only window. Ignore parameter-count bragging until you know the tokenizer and the later training rooms.
Ignore. You can ignore the research frontier of new looking-variants until your bottleneck is (n^2) or quality at long window. You can ignore the 2017 separate-reader diagram if you only call chat models. You cannot ignore next-token-plus-loop and still debug.
What is still not understood
Honesty, because the sources are honest.
No one can fully narrate the inner features of a production model. Interpretability is a research field, not a debugger on your laptop. We do not have a clean theory of why next-token prediction on internet text yields so much structure — one bet is that language is more regular than it looks; that is a bet. We do not know the ceiling of scaling. We do not have a reliable way to make a model know that it does not know.
None of that returns you to the black box. The algorithm is on the table. The learned content of a 100-billion-knob function is not.
When your app fails next week, will you blame a ghost — or name a stage: tokenizer, window, looking-table, picking, knobs, or your loop?
If you want the teachers later
This course compressed a pile of excellent teachers. If you want to go deeper, one order that matches what you just learned:
- Grant Sanderson’s 3Blue1Brown videos on neural nets, for the pictures.
- Andrej Karpathy’s Let’s build GPT and tokenizer lecture, if you want to feel it in code.
- Tom Yeh’s by-hand attention, if you want more napkins.
- The 2017 paper Attention Is All You Need, which is now readable.
- Stephen Wolfram’s essay on ChatGPT, for the philosophical aftertaste.
You should not need those to retell the core. If you cannot retell it, reread the looking step and the one block, with the pencil.
The core, one last time:
Tokens become lists of numbers. Attention mixes across positions. Mix-clip-mix thinks per position. Skip connections keep the signal. The last list becomes the next token. Everything you ship is that loop, plus whatever you put in the window, plus whatever you run when the tokens ask.
What this still cannot do
The loop is now a picture. It is not a proof that the picture is complete.
Take a failure from your own app. Is it the tokenizer, the window, the picker, the weights, or your loop?
If you cannot classify it, the black box is still there. Reread the looking step and the one block with a pencil. Then try the failure again.