Gwern.net is not a course, not a tutorial and not a reference, and it belongs on a list about learning because a handful of its essays did more to shape how practitioners think about large models than most published papers did. It is a personal website that has been running for well over a decade, written by one person, covering an unusually wide range of subjects, and maintained with a level of care that essentially nothing else on the internet matches. The scaling hypothesis writing is the reason most people arrive. Before it was obvious that making models bigger with more data and more compute would keep producing better results, this site laid out the argument for why that might be the case, what it would imply, and what evidence would count for and against.
That is a genuinely difficult thing to do well, arguing for a position before the evidence is conclusive, and it holds up considerably better than most contemporaneous commentary. Reading it now is instructive twice over, once for the argument itself and once as an example of how to reason carefully about a technology under uncertainty. The citation practice is the site's defining feature and the thing that separates it from ordinary blogging. Claims are sourced.
The sources are real and specific and often linked to archived copies so they do not rot. Where the evidence is weak, that is usually stated. Where the author is speculating, that is usually marked. This sounds like a low bar and it clears it by such a distance that reading a few essays here recalibrates what you expect from anything else you read online.
Essays are revised rather than abandoned. Pages carry their revision history, get updated as new evidence arrives, and sometimes record where the original argument turned out to be wrong. That practice of treating a piece of writing as a living document instead of a dated post is unusual and it is the right way to do this. It also means the version you read today may differ from the version someone quoted at you last year.
The site design supports deep reading in ways that are worth noticing. Popup previews on links and citations so you can check a reference without losing your place, extensive cross referencing between essays, a table of contents on long pieces, and a general assumption that you are going to read for an hour rather than skim for thirty seconds. Now the honest problems, and the first one turns most people away immediately. The subject range is extremely wide and a great deal of it has nothing to do with machine learning.
Statistical analysis of highly specific topics, discussions of nootropics and self experimentation, essays on anime, long pieces on darknet markets, genetics and heritability, and a lot else besides. Some of this is fascinating. Some of it is idiosyncratic in a way that will put you off. If you arrive at the front page expecting a machine learning resource you will conclude within two minutes that you are in the wrong place, and the AI writing really is worth finding.
The essays are long and dense. Not long in the sense of padded, long in the sense of thorough, with a great deal packed into each paragraph and an assumption that you can follow a statistical argument without hand holding. These are not casual reading and skimming them produces almost nothing. A single worldview runs through the whole site.
There is a consistent set of assumptions about rationality, quantification, what counts as evidence and how to weigh it, and it is coherent and it is a position rather than a neutral stance. It is mostly not flagged as such, because the author is writing from inside it. Read accordingly. Where an argument feels unusually convincing, check whether it is convincing on the evidence or convincing because it shares your priors, and note that the same applies in reverse.
Some of the older AI material has been overtaken. The field moved fast and essays written before certain developments are now historical documents in places. The site's revision practice mitigates this and does not eliminate it, and the older pieces retain value as records of what a well informed person believed at the time and why, which is genuinely useful for calibrating how much anyone should trust current predictions. There is no structure.
No curriculum, no suggested reading order, no beginner path. You find things by following links or by searching, and a lot of the value is in essays you would never have thought to look for. That is charming if you have time to wander and unhelpful if you have a specific question. My four out of five is for writing of a standard that essentially does not exist elsewhere on the open internet, on a site whose breadth and density mean most people who would benefit from it never get past the front page.
Start with the scaling material, read it slowly, and follow whatever catches you. It is not a place to learn a skill. It is a place to watch someone think carefully in public, which is its own kind of education.