Back to index
OtherAround 15 lectures, roughly 25 hours with the reading·Free on YouTube, paid for credit through Stanford Online

Stanford CS330: Deep Multi-Task and Meta Learning

4.1

A specialist course that explains the theory behind why pre-training and few shot prompting work at all. Narrow, mathematically demanding, and genuinely illuminating if you are ready for it.

What We Liked

  • Chelsea Finn originated much of this material, so the treatment is authoritative
  • Connects meta learning theory directly to why large pre-trained models adapt so well
  • Covers transfer learning and domain adaptation properly rather than as a footnote
  • Strong robotics and reinforcement learning applications that are hard to find elsewhere
  • Public lecture recordings and course pages from multiple offerings are freely available

What Could Be Better

  • Very much a specialist course, and most engineers will not use this directly
  • Assumes solid deep learning and reinforcement learning foundations from the first lecture
  • The classical meta learning material has been partly overtaken by scale, which the course acknowledges unevenly
  • Assignment infrastructure is not available outside the enrolled version
  • Different offerings vary considerably, so the recording you find may not match the current syllabus

Detailed review

CS330 sits in an odd position right now, and it is worth being honest about that before recommending it. The course is about learning from multiple tasks: how to share structure across them, how to build models that adapt to new tasks from a handful of examples, and how transfer learning and domain adaptation actually work. Chelsea Finn is one of the people who defined this area, and her model agnostic meta learning work is a direct part of the syllabus. The awkwardness is that the field partly answered its own question in an unexpected way.

Classical meta learning spent years designing clever algorithms to make models adapt from few examples, and then it turned out that training an enormous model on an enormous corpus produces few shot adaptation almost as a side effect. That does not make the meta learning literature worthless, but it does change what the course is for. Taken as theory, the material explains why in context learning works. When a language model picks up a pattern from three examples in a prompt, it is doing something the meta learning literature has a precise vocabulary for, and having that vocabulary changes how you think about prompting, fine tuning and adapter methods.

That is the strongest reason for an applied engineer to take this course. The content itself covers multi-task architectures and the optimisation problems that come with them, including negative transfer and gradient conflict, then moves through black box meta learning, optimisation based methods including MAML, metric based approaches, and on into meta reinforcement learning. Recent offerings have added self supervised pre-training for few shot learning and material on transfer learning, domain adaptation and domain generalisation, which is a sensible modernisation. The robotics and reinforcement learning applications are a genuine differentiator.

Finn's research group works on robot learning, and the examples reflect that. If you are working anywhere near embodied systems or sample efficient control, this is one of very few courses covering the ground at this level. Prerequisites are unforgiving. You want a deep learning course behind you, a reinforcement learning course, and comfort with the mathematics of optimisation.

The course opens at graduate pace and does not slow down. Access is the practical limitation. Recordings from past offerings are on YouTube and the course sites for several years are public with slides and reading lists. Assignments and the supporting infrastructure are for enrolled students.

The offerings also vary noticeably year to year as the field shifts, so the recording you find and the syllabus you read may not correspond, which makes self study slightly more effortful than it should be. Who should skip this? Anyone whose work is building applications on top of existing models. You will find it interesting and you will not use it. The opportunity cost against a course on retrieval, evaluation or systems is real.

Who should take it? Researchers, people working on personalisation and adaptation, robotics engineers, and anyone whose job involves making models work in a new domain with limited data. For that group this is close to essential and there is no real substitute. My 4.1 is a strong score with a narrow audience attached to it. Judge the fit before the quality, because the quality is not in doubt.

[ final ]

The verdict.

For researchers and engineers working on adaptation, personalisation or robotics. If you build applications on top of existing models, this is interesting but not necessary.