Back to index
OtherAround 330 pages across 16 chapters, four to six weeks writing the code yourself·Around $50 in print, with frequent Manning ebook discounts

Grokking Deep Learning (Andrew Trask)

3.8

Still the clearest path to understanding backpropagation by writing it, and it is now eight years old and shows it. Read it for the first half and get your modern material elsewhere.

What We Liked

  • Building everything in NumPy means backpropagation becomes concrete rather than trusted
  • Trask explains gradients from several angles until one of them lands, which is genuinely rare
  • No framework, so nothing important happens in a function call you cannot read
  • Very low prerequisites, and it does not assume calculus you have forgotten
  • The chapter that derives backpropagation is worth the book on its own

What Could Be Better

  • From 2019, and the architecture coverage stops well before anything current
  • Later chapters are visibly weaker than the early ones and feel rushed
  • Deliberate repetition in the early chapters becomes tedious if you are following easily
  • The federated learning material at the end is thin and feels like an author interest
  • Nothing on transformers, attention, or how anybody actually trains models now

Detailed review

There is a specific kind of confusion that afflicts people who learned deep learning through a framework. They can build models, tune them, and get decent results, and if you ask them what happens when they call the backward function they produce a vague answer about chain rules and gradients flowing. The knowledge is nominal. They have never seen it happen.

This book exists to fix exactly that, and its method is uncompromising. You build everything in NumPy. No PyTorch, no TensorFlow, no Keras. You write forward passes, you compute gradients by hand, you implement the update rule, and you assemble a small deep learning library over the course of the book.

It is slow and it is the only way this particular understanding gets built. Trask's explanation of backpropagation is the best I have encountered at an introductory level, and the reason is that he does not explain it once. He comes at it repeatedly from different directions. As the chain rule.

As credit assignment, where each weight gets blamed in proportion to how much it contributed to the error. As a computational procedure you can trace by hand through a tiny network. As code you type. One of these lands for most readers, and having four attempts substantially raises the odds.

Compare this with the standard treatment, which is a derivation, a diagram of a network with arrows pointing backwards, and an assumption that you now understand. The absence of a framework is the central design decision and it is right for the goal. When you use a library, the interesting part is invisible by design, which is exactly what a library is for and exactly wrong when you are learning. Writing the gradient computation yourself and watching it fail because a matrix is transposed, then fixing it, then watching the loss actually decrease, produces a kind of knowledge that reading cannot.

I did this and the difference in how I debug training problems was noticeable afterwards, because I could reason about what was happening to the gradients rather than guessing at hyperparameters. The prerequisites are genuinely low and honestly stated. Trask does not assume you remember calculus and builds up what is needed. This makes it accessible to a working programmer with no mathematical background, which is a real audience and one that is badly served elsewhere.

Now the problems, and they are enough to keep the score down. The book is from 2019 and deep learning has changed more since then than in the decade before. It covers feedforward networks, some convolutional material, and recurrent networks. Attention gets barely a mention.

Transformers, which are the basis of essentially everything anyone currently cares about, are not in here. Modern training practice, the optimisers and normalisation and scheduling that make large models trainable, is largely absent. A reader who works through this and expects to understand contemporary systems will find they have a solid grasp of the era before the one they live in. The foundations remain foundations, and the gap is large.

The quality also drops noticeably in the second half. The early chapters are patient and careful, plainly written and rewritten until they were right. The later ones on recurrent networks and beyond feel rushed by comparison, with less of the multi angle explanation that makes the beginning so good. The federated learning material at the end reads as an author interest rather than something the book needed, and it is too brief to be genuinely useful on a topic that deserves proper treatment.

The repetition that helps struggling readers actively hinders comfortable ones. Trask restates and re-explains constantly in the early chapters, which is correct for his target reader and tedious if you got it the first time. Skimming is the answer and it feels wrong to skim a book you are meant to be working through carefully. And it is worth being clear about what this book does not attempt.

There is no guidance on practical model building, no advice on when a particular architecture is appropriate, nothing about datasets or deployment or evaluation. It is a book about mechanism, entirely, and it leaves you understanding the machinery and unable to apply it to a real problem. That is a defensible scope and it means you need at least one other resource before you can do anything. My 3.8 reflects a book with an outstanding core surrounded by material that has aged badly.

The chapters that derive and implement backpropagation are close to the best explanation available anywhere and I still recommend them without reservation. The rest is a document from an earlier stage of the field. Buy it cheaply, read the first two thirds properly, and get everything about modern architectures from something written in the last two years.

[ final ]

The verdict.

Buy it for chapters three to nine, which teach backpropagation better than anything else at this level. Treat the rest as optional and get your modern architectures from a newer source.