Back to index
OtherAround 400 pages across 27 chapters, four to eight weeks if you type the code·Around $50 in print, or included with an O'Reilly subscription

Data Science from Scratch, Second Edition (Joel Grus)

4.1

Deliberately reimplements linear algebra, statistics, gradient descent and a dozen models in bare Python. Slow, impractical, and the fastest way to stop treating machine learning libraries as magic.

What We Liked

  • Implementing gradient descent yourself removes the mystery permanently
  • Grus writes with actual wit, which is scarce in technical books and helps at 400 pages
  • Covers a genuinely broad sweep from linear algebra through to neural networks and NLP
  • Second edition modernised the Python properly, including type annotations throughout
  • The from scratch constraint forces honesty about how simple most of these algorithms are

What Could Be Better

  • The 2019 publication date shows badly in the deep learning and NLP chapters
  • Breadth means several chapters are too compressed to teach the topic properly
  • You learn nothing about the tools you will actually use professionally
  • Assumes comfortable Python, so it is a second or third book at minimum
  • The final chapters read like a survey rather than a serious treatment

Detailed review

Grus set himself a constraint that sounds perverse and turns out to be pedagogically sharp. Build everything in plain Python. No numpy, no pandas, no scikit-learn. Write the vector operations, write the matrix code, write gradient descent, write the models.

The predictable objection is that nobody would ever work this way, and the predictable defence is that this is a book about understanding rather than production, which is true but understates the case. The stronger argument is that most machine learning algorithms are much simpler than their library interfaces suggest, and the only reliable way to discover that is to write one. Gradient descent is the clearest example. It is explained everywhere, usually with a ball rolling down a hill and a partial derivative or two, and people come away with a picture rather than an understanding.

Implementing it, watching it fail to converge because your learning rate is wrong, and fixing that yourself produces knowledge of a different quality. The same holds for k nearest neighbours, naive Bayes, decision trees, logistic regression and a simple neural network. Each is a couple of dozen lines when you strip away the interface, and seeing that changes your relationship with the library version permanently. The early chapters on linear algebra, statistics and probability are also better than their length suggests.

Building vector and matrix operations by hand makes the subsequent numpy code legible in a way that reading numpy documentation does not. When you later call a dot product, you know exactly what it is, because you wrote it. Grus is also genuinely funny, which matters over four hundred pages of implementation. Technical humour usually lands badly and his does not.

The second edition modernised things properly rather than cosmetically, moving to current Python and adding type annotations throughout, which is a nice secondary lesson in writing readable code. The problems are mostly about age and ambition. This is a 2019 book, and 2019 is a long time ago in this field. The deep learning chapter builds a small network from scratch, which is worthwhile, and then stops well short of anything resembling current practice.

The natural language processing chapter is the one that has aged worst, since it predates the transformer era becoming the whole subject. A reader who arrives wanting to understand modern language models will find almost nothing relevant, and should go to Raschka or Karpathy instead. Ambition is the other issue. Twenty seven chapters covering everything from linear algebra to network analysis to recommender systems to databases and MapReduce means several topics get a compressed treatment that gestures at the subject without teaching it.

The later chapters in particular read like a survey. I would have preferred fifteen chapters done thoroughly, and I suspect the book was shaped by a desire to be comprehensive that the from scratch constraint could not support. The obvious limit is that you finish knowing how these things work and unable to use the tools anyone actually uses. That is by design and it is worth stating plainly, because people do occasionally treat this as a practical introduction to data science.

It is not. You need pandas, you need scikit-learn, and you need to know how a real workflow is structured, and none of that is here. Read this alongside something applied and the two reinforce each other well. Read it alone and you will have unusual insight and no employable skills.

It also assumes real Python fluency. The code uses comprehensions, generators, type hints and functional patterns without ceremony, which is correct for the audience and means this cannot be an early book. Come to it after a proper introduction and ideally after some exposure to the libraries, so you have something to demystify. My 4.1 is for a book that does one thing very well and several things adequately.

The core implementation chapters are excellent and I recommend them without reservation. The score comes down for the dated later material, for spreading itself too thin across too many topics, and for a deep learning treatment that a reader in the mid 2020s will find quaint. Buy it for the middle third and treat the rest as a bonus.

[ final ]

The verdict.

Excellent for the specific job of demystifying the libraries. Read it alongside a practical course, not instead of one.