architectures, worked out by hand

Let's break it down

Each breakdown takes one architecture behind modern AI and walks through a full forward pass, every matrix, every shape, every sum, with a practice panel beside each step so you do the math yourself.

For anyone who enjoys learning architectures from scratch. Finish one and you should be able to close your eyes and picture how it works, which makes every variant after it far more intuitive.

Breakdowns

• 01

  Attention, by hand

  Attention Is All You Need · Vaswani et al., 2017

  The original Transformer, from token embeddings and sinusoidal positional encoding through multi-head attention, the encoder and the decoder to output probabilities.

  • encoder
  • decoder
  • notebooks
  • worksheet
  Start →
• 02

  Next up…

  More architectures are on the way. Star the repo to follow along.

How each one works

• Read the step. Plain language first, then the exact math, with the matrix shape tracked as it changes.
• Do it yourself. Fill in the numbers in the practice panel and check them against the model.
• Take it home. Download a notebook or printable worksheet and run a whole pass on your own.