Back to index
OtherA full semester, roughly 26 lectures plus assignments and a project·Free

CMU 10-708: Probabilistic Graphical Models

4.2

A harder, more research-facing companion to Koller's course, with student-scribed notes that are unusually good. This is graduate material and it does not pretend otherwise.

What We Liked

  • Student-scribed lecture notes are detailed and often clearer than the slides
  • Covers modern topics including deep generative models and variational autoencoders
  • Lecture videos are publicly available on YouTube
  • Treats the subject as active research rather than settled curriculum

What Could Be Better

  • Assumes a strong background in probability, linear algebra and optimisation
  • The Spring 2019 offering is the most recent well-archived one and it is showing its age
  • No autograded assignments or interactive feedback of any kind
  • Note quality varies because different students scribed different lectures

Detailed review

Probabilistic graphical models occupy an odd position in machine learning education right now. They were central to the field for two decades, they explain a great deal about how to reason under uncertainty, and then deep learning arrived and a generation of practitioners learned to train networks without ever encountering a factor graph. That is a loss, and it becomes visible the moment somebody needs to reason about latent structure, about combining sources of evidence with different reliability, or about what a variational bound is actually bounding. Eric Xing's 10-708 is one of the two canonical university treatments of this material, the other being Daphne Koller's Stanford course, and the two are different enough that doing both is not redundant.

Koller's is the better first encounter. It is more carefully paced, more thoroughly motivated, more concerned that you understand why conditional independence matters before it shows you an algorithm that exploits it. Xing's is faster, harder, and pointed at research. It assumes you can handle measure-theoretic hand-waving, that you are comfortable with expectations and KL divergences, and that a page of derivation does not frighten you.

The payoff for that assumption is that it reaches further. The course covers representation, exact inference through variable elimination and the junction tree algorithm, approximate inference via loopy belief propagation, variational methods and Monte Carlo sampling, parameter and structure learning, and then it keeps going into territory the older curricula do not reach: variational autoencoders, generative adversarial networks, normalising flows, and the general project of connecting probabilistic modelling to deep networks. That bridge is the reason to take this course specifically. Understanding a VAE as a graphical model with an inference network doing amortised variational inference, rather than as an architecture you copy from a repository, is a genuine conceptual upgrade, and 10-708 builds that understanding from the foundations rather than asserting it.

The student-scribed notes are the course's unexpected strength. Each lecture has notes written up by students and reviewed, and they are frequently clearer than the slides because a student writing them has just been confused by the same thing you are about to be confused by, and they explain the step the lecturer skipped. Note quality varies, which is inevitable with many authors, but the good ones are very good and the whole set is freely available. Lecture videos went to YouTube, so the course is genuinely accessible.

The limitations are the ones every archived university course has. The Spring 2019 offering is the most recent one with a well maintained public site, and 2019 in generative modelling terms is a while ago. Diffusion models, which now dominate generative imaging and have a beautiful probabilistic formulation that would sit perfectly in this syllabus, arrived after. Large language models get no treatment.

The course is not stale in its foundations, because variable elimination and variational inference do not age, but the modern applications section has been overtaken. There is no autograding and no feedback. You can read the assignments, you can attempt them, and nobody will tell you if you are wrong. For material this mathematically demanding, that is a real handicap, and it is why I would push a newcomer to Koller first, where the structure is more supportive.

Four point two. Excellent, demanding, freely available graduate material with notes that are better than they had any obligation to be, slightly dated at the modern end.

[ final ]

The verdict.

Do Koller's course first if graphical models are new to you. Come here afterwards for the harder inference material and the bridge into modern deep generative modelling, which is where this course earns its place.