Lessons
The reading room
Decoded: Attention Is All You Need, the paper that introduced the Transformer
Paper: Attention Is All You Need Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser and Illia Polosukhin. Advances in Neural Information Processing Systems 30 (NIPS 2017), 2017.
How the Transformer replaces recurrence with attention: scaled dot-product attention one operation at a time, why the scores are divided by the square root of d_k, how several heads attend in parallel, how a mask keeps the decoder from seeing the future, and how sine waves tell the model where each word sits. Read along with the paper; every mechanism is drawn from its text and moves one step at a time, with original, labelled illustrative examples beside it.
Read the lesson →
Decoded: TAGE, the branch predictor that keeps several histories at once
Paper: A case for (partially) TAgged GEometric history length branch prediction André Seznec and Pierre Michaud. Journal of Instruction-Level Parallelism, vol. 8, 2006.
How the TAGE branch predictor picks the longest history that recognises a branch, how it learns which table entries deserve to stay, and how a geometric series of history lengths lets a few tables see hundreds of bits of context. Read along with the paper; every mechanism is drawn from its text and moves one step at a time.
Read the lesson →
About
I am an electrical and computer engineering graduate. Like many people, I have a hard time understanding and retaining what I read in research papers, so I built an agent that turns a paper into an interactive guide: the paper sets the order, and every mechanism moves in front of you one step at a time.
Tell me which papers you want broken down next.
How a lesson works
- The paper is the guide. A lesson starts where the paper starts and stops where it stops.
- Prerequisites fold open only if you need them. Short refreshers sit at the top of every lesson.
- Every figure moves one step at a time. Step forward, step back, and watch each piece change.
Get in touch
Recommend a paper, ask a question, or request a correction or removal. E-mail hello@decodedbyjustin.com with the lesson URL, the passage or figure concerned, and the reason for your request. For a rights concern, explain your connection to the work. Please send only information needed to review the request.
There is no contact form or tracking. The contact button opens your e-mail app. See Sources and rights for the policy and privacy details.
Contact Justin