Distill was a peer reviewed journal dedicated to clear explanations of machine learning, and for five years it demonstrated something that the rest of the field has still not caught up with: that a research explanation can be interactive, rigorous and beautiful at the same time. In July 2021 the editorial team announced a break, and it has not returned. What remains is an archive of around thirty articles, all free, several of which are still the best explanation of their subject that exists anywhere. The central innovation was the interactive diagram.
Reading about how a convolutional network builds up representations is one thing. Dragging a slider and watching the features change is another, and the second produces understanding that survives. Feature Visualization from November 2017 explores how neural networks build up their understanding of images, and it does so by letting you look at what individual neurons respond to. The Building Blocks of Interpretability from March 2018 combines visualisation techniques to show what a network is attending to.
Attention and Augmented Recurrent Neural Networks from September 2016 is a visual guide to attention that predates the transformer and still explains the mechanism more clearly than most current material. The Multimodal Neurons work from March 2021 documented neurons in artificial networks responding to concepts across modalities, echoing findings in biological brains. The two Graph Neural Networks articles from September 2021 are the last major publications and remain a strong introduction to a subject that most courses handle badly. The interpretability material is the part I would insist on.
Mechanistic interpretability is currently one of the most active areas in AI safety research, and a great deal of it traces its intellectual lineage directly to these articles. Reading them is not a history exercise, it is reading the foundations of a live research programme. The Circuits work in particular reframed how people think about what is happening inside a network, and the visual approach is why the ideas landed. Peer review is worth flagging because it distinguishes this from the enormous volume of AI blog content.
Articles were submitted publicly through GitHub, reviewed, and revised. The result is that you can trust the explanations in a way you cannot trust a Medium post, and the combination of rigour with genuine design quality is what made Distill influential out of proportion to its size. Now the limitations, and the first one is decisive. Distill stopped in 2021.
That means nothing here covers large language models at scale, instruction tuning, reinforcement learning from human feedback, diffusion models in their current form, agents, retrieval augmented generation, or any of the material that defines current practice. The editorial team was candid about why they stopped, including the difficulty of sustaining volunteer editorial labour at that quality bar, and the reasoning is worth reading if you care about scientific publishing. For a learner it means one thing: this is an archive. Use it for foundations and go elsewhere for anything current.
Second, the corpus is small. Around thirty articles is an afternoon of browsing and a week of careful reading, not a course. Third, there is no sequence and no guidance on where to start. My suggested order would be attention, then feature visualisation, then building blocks of interpretability, then graph neural networks if the subject is relevant to you.
Fourth, difficulty varies sharply. Some articles are accessible to anyone with basic neural network knowledge and others assume real research background. There is no labelling to warn you which is which. Fifth, and practically, some of the interactive elements have aged.
Browsers have moved on and a few visualisations behave less smoothly than they did. Most still work well. On what to read instead for current material. Lilian Weng's Lil'Log covers frontier topics at survey depth.
Jay Alammar's illustrated posts carry the visual explanation tradition forward for transformers and language models. The Anthropic interpretability publications continue the Circuits research directly. None of them match Distill's interactivity, which remains its unique contribution. My four point six is for articles that were genuinely excellent and remain useful, marked down only because the publication is frozen and the corpus is small.
If Distill were still running it would be a five. As an archive of foundational explanations that you can read for free in an afternoon, it is close to essential, and the interpretability articles in particular deserve to be read by anyone who wants to understand what is actually happening inside these models.