Back to index
OtherA week or two of evenings, longer if you run the notebooks·Free to read online, pay what you want e-book, paperback around $47, hardcover around $65

The Hundred-Page Language Models Book

4.4

The best short book on how language models work. Burkov does the hard editorial work of deciding what to leave out, and the read first buy later model means you can verify that before spending anything.

What We Liked

  • Ruthlessly concise, which makes it finishable, and a finished book teaches more than three abandoned ones
  • Builds up historically from count based models through recurrent networks to transformers, so the design choices have context
  • PyTorch implementations are included rather than gestured at, so you can run and modify the ideas
  • All chapters remain readable on the site under a read first buy later principle, so you can judge it honestly before paying
  • Follows the same format as his machine learning book, which many people found to be the clearest short introduction available

What Could Be Better

  • A hundred pages on this subject means real omissions, and depth on any single topic is limited
  • Assumes Python and enough mathematical comfort to follow linear algebra and probability without a refresher
  • The fast moving end of the field, especially agents, evaluation and production practice, gets little space
  • Print pricing is high for the page count if you want a physical copy
  • It is a book, so there are no exercises with feedback and no cohort to compare notes with

Detailed review

Brevity is underrated in technical writing. Most books on this subject are eight hundred pages, and most readers stop at page ninety. Burkov's format is a deliberate response to that, and the reason it works is that the difficulty is not in explaining things, it is in deciding what not to explain. He is good at that decision, and it showed in the hundred page machine learning book which many people, including me, still consider the clearest short introduction to the field.

This one applies the same discipline to language models. The structure is historical and I think that is the right call. You start with machine learning foundations, then count based language models, then recurrent networks, then transformers, then the modern practices of fine tuning and prompting. Learning it in that order means that when the attention mechanism arrives you already understand the problem it solves, because you have watched recurrent models struggle with exactly that problem.

A book that opens with transformers gives you the answer without the question, and readers of those books tend to end up with a description of an architecture rather than an understanding of why it exists. The implementations matter too. There is PyTorch code and there are notebooks, so the ideas are runnable rather than decorative, and modifying a small working implementation is one of the fastest ways to find out whether you actually understood the explanation. The obvious cost of a hundred pages is omission.

You will not come out of this an expert in anything specific. Coverage of the frontier is thin, so agents, evaluation methodology, serving, and the operational realities of running these systems get very little space, and that is where a lot of the current professional work sits. The correct way to read this is as a foundation you build on, not a destination. It also asks something of the reader.

The explanations are intuitive where they can be, and there is still mathematics, and you need Python and reasonable comfort with linear algebra and probability to get full value. Someone entirely new to programming should start elsewhere. The commercial model is worth praising. All chapters stay available on the site under an explicitly stated read first buy later principle, so you can read the whole thing, decide whether it earned your money, and then pay.

That is a rare degree of confidence in one's own work and it should be more common. The e-book is pay what you want through Leanpub, and physical editions are priced at a premium that is high per page and typical for self published technical print runs. Hardcover buyers can request a PDF, which is the right way round. My 4.4 reflects an excellent, honest, well judged short book with real limits that it is upfront about.

If you have been using language models for a year without understanding them, this is the most efficient way I know to fix that, and you should plan for what you read next before you finish it.

[ final ]

The verdict.

Buy it if you want the shortest honest route from I use these models to I understand these models. Read the free chapters first, since the whole point of the format is that you can. Then follow it with something deeper on whichever part you now want to actually work in.