Comet has been in experiment tracking for a long time, which in this field means it has watched the ground shift underneath it. Tracking loss curves and hyperparameters was the central problem when models were trained from scratch by every team. A great deal of current work involves calling a model somebody else trained and worrying about whether the output is any good, which is a different problem needing different tools. Opik is Comet's answer to that shift and it is the more interesting part of what they now document.
Taking the classical product first, the experiment tracking documentation is competent. Integrations with the common training frameworks are documented with working examples, logging metrics and parameters and artifacts is straightforward, and the comparison and visualisation features are explained clearly. If you are training models and need tracking, this will get you set up without difficulty. It is also difficult to describe as differentiated.
Several tools do this well, the documentation cannot tell you why to prefer this one, and the older pages read as maintained rather than actively developed. Opik is where the attention should go. Evaluating language model output is the problem an enormous number of teams are genuinely stuck on right now. You have built something.
It produces text. Sometimes the text is good. You have no systematic way to know whether your last change made things better or worse, and your evaluation process is one person reading outputs and forming an impression. The documentation takes this problem seriously and treats it as an engineering discipline rather than a matter of taste.
The tracing material is the most immediately practical part. Modern language model applications are multi step. There is retrieval, there are tool calls, there are several model invocations, and when the final answer is wrong the interesting question is which step went wrong. Tracing gives you that visibility and the documentation explains instrumentation clearly with examples that work.
Anyone debugging an agent by adding print statements should read this section and stop doing that. The evaluation documentation covers building datasets, defining metrics, using a model as a judge, and running evaluations as part of a development workflow rather than as an occasional manual exercise. The mechanics are well explained. Where it is weaker is judgement.
Choosing what to measure is the hard part of evaluation, and the docs are much better at showing you how to compute a metric than at helping you decide which metric reflects whether your application is actually working. Model as judge in particular gets thorough mechanical treatment and lighter treatment of its failure modes, which include the judge sharing biases with the model being judged and scores that look precise while measuring something other than what you care about. That gap matters because a confidently wrong evaluation is worse than no evaluation. Opik being open source and self-hostable is documented properly, and I want to credit that.
Teams with data that cannot leave their infrastructure have a real path here rather than a footnote, and the self-hosting documentation is substantial enough to follow. In a category where most tools are hosted only, this is a genuine differentiator and it is treated as one. The navigation problem is real. Two products with different audiences and different maturity levels share a documentation set, and working out which pages apply to which product takes more attention than it should.
Someone arriving specifically for evaluation has to filter past a lot of experiment tracking material that is irrelevant to them. The category is crowded and getting more so. Several tools now address language model evaluation and observability, with meaningful differences in approach and pricing. The documentation cannot help you compare, and you should not treat any single vendor's material as an evaluation of the space.
Three point eight, weighted heavily toward Opik. The evaluation and tracing material addresses a problem people genuinely have, the open source and self-hosting story is real, and the tracing documentation in particular will save people time immediately. Held back by a documentation set serving two audiences at once, by evaluation guidance that is stronger on mechanics than on judgement, and by an older half that is adequate rather than compelling.