Almost everything in machine learning is an optimisation problem wearing a disguise. Training is minimising a loss. Regularisation is constraining a solution. Support vector machines, lasso, logistic regression, portfolio problems, control, signal processing, all of it reduces to the same question of finding the best point in a feasible set.
Convex optimization is the branch of that subject where the problem is tractable and the theory is complete, and this course is where you learn it. Stephen Boyd is a rarity. He is a genuinely significant researcher who is also an outstanding teacher, and the lectures are a pleasure to watch in a way that mathematics lectures usually are not. He is funny, he is direct, he tells you which results matter and which are technical bookkeeping, and he repeatedly grounds abstract material in applications from engineering, finance and statistics.
There is no performance of difficulty. He wants you to understand it. The textbook, Convex Optimization by Boyd and Vandenberghe, is freely available as a PDF from the Stanford site and is the standard reference in the field. It is genuinely well written.
Definitions are motivated, proofs are clean, and the exercise sets are extensive with solutions available. That a book of this quality is free is one of the better things about academic publishing. The central skill the course teaches is recognition. Given an arbitrary problem, is it convex, and if not can it be transformed into something convex.
That sounds narrow and it is the thing that transfers. Once you can spot convexity you start seeing structure in problems you previously would have thrown a generic solver at, and you understand why some formulations behave well and others do not. I use this more than any other piece of mathematics I learned. The tooling closes the loop.
CVX, and its Python descendants, let you write a convex problem in something close to mathematical notation and solve it. Being able to state a problem and get an answer in ten lines makes the theory feel real rather than ornamental, and the course uses this well in the assignments. Now the difficulty, which is genuine. This is a graduate course at Stanford and it assumes it.
Comfortable linear algebra, including eigenvalues, matrix norms and decompositions. Real analysis, including convergence and continuity arguments. Multivariable calculus. If any of those are shaky, this course will find the gap immediately and you will spend more time patching foundations than learning optimisation.
The problem sets are heavy. Substantially heavier than the lecture load suggests. A single set can absorb a weekend, and the mathematics is not the kind you can rush. Anyone planning to fit this around a full time job should budget realistically and expect it to take longer than ten weeks.
There is no machine learning framing. Boyd teaches optimisation, and the applications are drawn from across engineering rather than curated for a machine learning audience. That is a strength intellectually and it does mean the connection to your actual work is left for you to make. If you want immediate applicability this will feel abstract for a long time.
The recordings are old and it shows. Camera on a lecturer at a board, no editing, dated audio. The content is timeless and the production is not. Modern deep learning is a genuine caveat.
Neural network training is aggressively non-convex, and the elegant guarantees this course provides do not apply directly. What does carry over is enormous: duality, the geometry of constraints, why regularisers behave as they do, gradient method intuition, and the vocabulary for reasoning about optimisation at all. Anyone who claims convex optimisation is irrelevant to deep learning has usually not studied it. My four point five is for the definitive treatment of foundational mathematics, taught superbly, with a free canonical textbook and public materials, marked down only because the prerequisites are steep, the workload is heavy and the payoff is deferred rather than immediate.
This is a course you will be glad you did for the next twenty years.