Back to index
OtherCohort format: 6 prerecorded lectures, 6+ live office hours, weekly guest speakers, about two live hours a week plus notebook work·Price only shown when a cohort opens for enrolment; includes free re-enrolment in future cohorts and over $2,000 in partner credits

Systematically Improving RAG Applications (Jason Liu)

4.5

The best course I know of for fixing a RAG system that works in a demo and disappoints in production. It is a method, not a tutorial, and it assumes you already have something deployed. Much of the thinking is free on Liu's blog, so the fee buys structure, office hours and community.

What We Liked

  • Teaches a repeatable improvement process (measure, segment, fix, repeat) rather than a list of tricks
  • Strong on retrieval evaluation with precision, recall and MRR, and on building synthetic eval sets
  • Jason Liu built search and recommendation systems at Facebook and Stitch Fix and created the Instructor library, so he speaks from practice
  • Free re-enrolment in future cohorts makes the price easier to justify
  • Office hours with real systems being debugged live are, by student accounts, the best part
  • 4.8 out of 5 from 88 reviews

What Could Be Better

  • You need a deployed RAG system for the course to make sense, so it is useless as a first RAG course
  • No price shown between cohorts, which is frustrating when you need budget approval
  • Prerecorded lectures with office hours feel thinner than a fully live cohort
  • Plenty of the core ideas are in Liu's free written guide, so disciplined self-learners can get far without paying
  • The course page mentions both 12 and 20+ notebooks, a small sign the materials are being updated faster than the page

Detailed review

Retrieval augmented generation has a well-known shape: it demos beautifully and then underwhelms the moment real users start asking real questions. Jason Liu's course exists to deal with that second phase, and it is the most useful material on the topic I have seen. The core idea is what Liu calls the RAG flywheel. Instead of tweaking prompts and chunk sizes by feel, you build synthetic evaluation sets with LLM-generated questions and answers, measure retrieval separately from generation using precision, recall and MRR, segment your incoming queries to find which kinds are failing and how often, and then fix the segments that matter most.

The later material covers fine-tuning embeddings, collecting much more user feedback than most products bother with, building specialised indexes for documents, tables and images, combining BM25 with embeddings and metadata filters, and routing queries to whichever retriever handles them best. None of this is exotic. What is valuable is the order and the discipline, which is exactly what most teams lack. Liu has the background to teach it.

He spent years building search and recommendation systems at Facebook and Stitch Fix, has consulted for many AI startups on retrieval, and wrote Instructor, the structured output library that a lot of Python developers use. Students on the page include engineers from OpenAI, Anthropic, Google and Meta, and an Anthropic solutions engineer is quoted saying the frameworks turned what they already knew into practice. That is the right audience and the right claim. This course will not teach you what RAG is.

The prerequisites say outright that you should already have a RAG system deployed. The format is six prerecorded lectures, at least six live office hours where people bring their own problems, weekly guest speakers, a set of Python notebooks with walkthroughs, and a Slack community you keep for life. Free re-enrolment in later cohorts is a genuinely generous policy and means one payment can cover you as the material evolves, which it clearly does, given that the page mentions 12 notebooks in one place and more than 20 in another. My main frustration is the price, or rather its absence.

Between cohorts the page shows nothing, so you cannot build a budget case until enrolment opens. The other thing to know is that Liu publishes a lot of this thinking for free in a long written guide on his own site. If you are disciplined, that guide plus your own logs will take you a long way. The course earns its fee through the office hours and the guest speakers, and you get the most from it if you arrive with a struggling production system and specific questions.

[ final ]

The verdict.

Read Liu's free guide on improving RAG first. If you recognise your own system in it and want structured help and live debugging, the course is worth your employer's money. If you have not shipped RAG yet, start somewhere else.