Stanford's CS324 was among the first university courses built specifically around large language models, taught in Winter 2022 with the lecture notes written as the quarter progressed and published openly. Fourteen topics: introduction, capabilities, harms one and two, data, security, legality, modelling, training, parallelism, scaling laws, selective architectures, adaptation, and environmental impact. The notes are still online and still free. The structure tells you something about how the field was thought about at that moment, and I mean that as praise.
Two full lectures on harms, plus dedicated treatment of security, legality and environmental impact, in a fourteen lecture course. That is a third of the syllabus on consequences. Compare that to the average practical LLM course today, which spends fifty hours on API calls and prompt patterns and ten minutes on a slide about responsible use. The CS324 framing, that a language model is a technical artefact and a social one and that you cannot understand it as only the first, has aged extremely well.
The data chapter is the piece I would send people to first, and I have. Where training data comes from, how it is filtered, what the filtering choices imply, and how provenance affects model behaviour is a subject that has since become a major legal and commercial issue, and this chapter explains the mechanics clearly and without hysteria. Anyone forming an opinion about copyright and model training should read it before they form the opinion. The modelling, training, parallelism and scaling laws chapters cover the engineering.
Parallelism in particular is useful because model size is often discussed as if it were a number in a config file, and understanding data, tensor and pipeline parallelism explains why training large models is a distributed systems problem before it is a machine learning problem. The scaling laws chapter is a good short treatment of the relationship between compute, data, parameters and loss, and the reasoning survives even though the specific numbers have been revised since. Adaptation covers fine tuning and prompting as they were understood at the time. Now the obvious problem.
This is from early 2022. GPT-3 was the reference frontier model. Instruction tuning was emerging rather than standard. Reinforcement learning from human feedback existed and was not yet the default alignment approach.
ChatGPT did not exist. Nothing here covers agents, tool use, retrieval augmented generation as it is now practised, mixture of experts at current scale, reasoning models, or multimodal systems. The capabilities chapter, which discusses what these models can do, is now a historical document about a much weaker generation of models. If you read this expecting a current picture you will be misled, and it is important to be clear about that.
What I would argue is that the conceptual scaffolding survives even where the details do not. The way to think about a language model as a distribution over sequences, the framing of capability evaluation, the analysis of where harms come from, the economics of scale: those hold. Read it as a course in how to think about language models rather than a course in what current language models are, and it is one of the better free resources here. Second criticism: no assignments in the public material.
The notes are notes. There is no code, no problem sets, no way to check whether you understood. That makes it reading rather than study, and reading produces a weaker kind of knowledge. Third, the notes vary.
They were written live during the quarter and some are polished essays while others are closer to structured bullet points. Fourth, prerequisites. The modelling and training chapters assume real machine learning background. Someone who has not trained a neural network will find them heavy going.
On what to read alongside. Stanford CS336, Language Modeling from Scratch, is the direct modern successor for the engineering and it is in this catalogue. It is far more current and far more demanding, and it will have you building components rather than reading about them. For current capabilities and techniques, Lilian Weng's writing and the vendor documentation are better.
CS324's enduring value is the breadth and the seriousness about consequences, and I would keep it on a reading list for the data and harms chapters alone. My four point zero reflects genuinely good writing with a real framework, discounted substantially because a course about a fast moving field written in early 2022 cannot avoid being dated. Free, short, and worth the weekend if you read it for the right reasons.