The last few words, not the whole book

Classic attention builds a score between the current token and every earlier token, then mixes. That is dense, quadratic in sequence length, and a poor match for sixteen-neighbor streets.

Z1T swaps that for a gated convolutional attention: each position looks at a short recent window (their pictured kernel is four tokens) plus a running summary of what came before. The sparse projections that make the gates can live on Z1. The exponentials, the pooling, the leftovers that are not a local squash — those go to the companion processor.

The companion on today’s boards is an FPGA, a reconfigurable digital chip sitting next to Z1. In principle it could be a graphics chip or some other accelerator. Their published numbers use the FPGA.

A token therefore walks a two-kitchen line. The sampling chip does the local squashes and sparse projections. The digital chip adds the skip-around connections, does pooling, and — when you actually need a word — the big vocabulary multiply that turns a vector into logits. The headline energy ratio in the post leaves that last vocabulary multiply out, because it is dense and they are measuring the body of the model.

The night board does not need the whole book scored again for each next line. It needs the last few words, plus a running summary of the shift.