Back to index
OtherAround 750 pages, a year of serious study rather than a read·Free PDF from Trevor Hastie's Stanford page, print around $80

The Elements of Statistical Learning (Hastie, Tibshirani, Friedman)

4.2

A superb reference and a punishing textbook. Buy the idea that you will read it cover to cover and you will quit by chapter four. Use it properly and it is the deepest treatment available.

What We Liked

  • Genuinely rigorous, with the derivations that gentler books push into the references
  • Free PDF from the authors, currently the twelfth printing of the second edition
  • The best available treatment of boosting, written partly by the people who developed it
  • Chapter three on linear methods for regression is worth returning to repeatedly
  • Works as a lifelong reference in a way that most textbooks do not

What Could Be Better

  • Second edition dates from 2009, so nothing from the deep learning era is here
  • Extremely dense, with a steep jump in difficulty compared with its own introductory sibling
  • No code, no labs and exercises that range from useful to research problems
  • The neural network chapter is a historical curiosity at this point
  • Wrong book entirely for anybody who has not already worked through something gentler

Detailed review

There is a rite of passage where somebody who has enjoyed An Introduction to Statistical Learning decides they are ready for the grown up version, downloads this, and discovers around page ninety that the two books are not the same kind of object at all. I did this myself. The gentler book teaches. This one states, derives and expects you to keep up.

Understanding that distinction before you start will save you a great deal of misplaced frustration, because the book is excellent and most of the complaints about it come from people using it wrongly. What it is, is the most thorough single treatment of classical statistical learning in print. Supervised learning framed properly, linear methods for regression and classification, basis expansions and smoothing, kernel methods, model assessment and selection, model inference and averaging, additive models and trees, boosting, neural networks as they were understood in 2009, support vector machines, prototype methods, unsupervised learning at length, random forests, ensemble learning, undirected graphical models and high dimensional problems. The coverage is enormous and the treatment is deep.

Chapter three, on linear methods for regression, is a good test of whether the book is for you. It covers subset selection, ridge regression, the lasso, least angle regression, principal components regression and partial least squares, and it does so with the geometry and the derivations rather than a description and a function call. If you work through it carefully you will come out understanding regularisation at a level that no course will give you. If you skim it you will get nothing at all.

That is the book in miniature. The boosting material deserves particular mention because Friedman developed gradient boosting and the chapter reads like it. The statistical view of boosting, why it works, what the loss function is doing, how it relates to additive modelling, all laid out by somebody who worked it out. Given how much of practical machine learning still runs on gradient boosted trees, this remains directly useful rather than merely historical.

That the authors made it free is worth saying plainly. The full PDF sits on Hastie's Stanford page, the current version being the twelfth printing of the second edition, and there is no signup or friction. Three of the most cited statisticians alive gave away their major textbook. That deserves more credit than it gets.

Now the limitations, and the largest is simply time. The second edition came out in 2009. AlexNet was 2012. The transformer paper was 2017.

Everything that defines the current field happened after this book stopped being updated, and the neural network chapter reads as a document from a period when neural networks were one method among many rather than the method. If you come here for deep learning you will get a treatment of two layer networks and back propagation that was reasonable at the time and is close to useless now. Use it for the classical material, which has not dated at all, and go elsewhere for anything involving a GPU. The density is the other issue, and it is not a flaw so much as a design choice with consequences.

There is no gentle ramp. The notation is heavy and assumed. The exercises range from genuinely instructive to problems that would occupy a graduate student for a week, with little signposting about which is which. There is no code, which for a book about methods that are almost always implemented in software is a real gap, though the community has produced solution sets and implementations that fill it partially.

Practically, my advice is this. Do not read it front to back. Read ISLR properly first, including the labs, so that you have the conceptual map. Then treat this as what it actually is, which is a reference of unusual depth.

When you find yourself needing to understand why the lasso produces sparse solutions and ridge does not, or what is really happening inside a boosting iteration, or how degrees of freedom generalise to non parametric fits, open the relevant chapter and work through it slowly. Used that way it is superb, and I have gone back to it for over a decade. Used as a first course it will beat you. My 4.2 balances a genuinely outstanding piece of work against a scope that stopped expanding sixteen years ago and a pitch that makes it unsuitable for most of the people who download it.

The classical statistical learning in here is not going to be superseded. It is just no longer the whole of the field, and the book cannot tell you that itself.

[ final ]

The verdict.

Keep it on the shelf and open it when you need to understand something properly. Read ISLR first, and do not treat this as a course.