Let me start with the thing that is genuinely excellent, because it is the reason to look at this at all. You add two lines to a training script. From that point on, every hyperparameter your script uses, every metric it logs, the exact state of your code including uncommitted changes, the Python environment, the command line arguments, the console output and any plots you produce are captured automatically and made comparable across runs. You did not instrument anything.
You did not decide what to log. The framework hooked into the libraries you were already using and worked it out. For anyone who has tried to reconstruct what configuration produced a model from three weeks ago, the value is immediate and large. The automatic capture is the differentiator against tools that require you to log things explicitly, and the documentation demonstrates it well.
The comparison interface deserves specific credit. Selecting several runs and seeing parameters diffed, metrics overlaid and code differences highlighted is exactly the workflow that experiment tracking exists to enable, and it is well built. Reading the documentation on it is enough to get you using it properly rather than treating it as a log dump. The remote execution model is the genuinely clever piece of design.
An agent process runs on a machine with hardware. You take a previously logged experiment, change some parameters, and enqueue it. The agent recreates the environment from what was captured, pulls the code at the right commit, and runs it. This closes the loop between tracking and execution in a way most tools do not attempt.
Reproducibility stops being a discipline you have to maintain and becomes a property of the system, which is a much better place for it to live. Full self-hosting is a real advantage for a certain audience. Everything, including the web interface and the storage backend, can run on your own infrastructure. For regulated environments where experiment metadata and model artifacts cannot leave the premises, that is not a preference but a requirement, and it is documented properly rather than mentioned.
Now the problems, and scope is the main one. This project covers experiment tracking, data versioning, pipeline orchestration, hyperparameter optimisation, model serving and reporting. Each of those areas has dedicated tools that do it more thoroughly. The documentation is correspondingly spread thin, and the difference in depth between the tracking material, which is excellent, and the serving material, which is adequate, is stark.
A team that adopts the whole platform because it is convenient ends up with a mediocre orchestrator and a mediocre serving story bundled with an excellent tracker, and the documentation naturally presents the integrated whole as the goal. My advice is to take the good part and be sceptical about the rest. The automatic capture has a dark side that the documentation does not treat seriously enough. When it captures the right things it is magic.
When it captures something you did not want, or fails to capture something you assumed it had, working out why means understanding the hooks it installed into libraries you did not know it was patching. That is a genuinely opaque debugging experience. There is also the matter of what gets captured by default, which can include more of your environment than you intended, and in a corporate setting that is worth checking before you point it at anything sensitive. The commercial boundary is fuzzier than it should be.
Some capabilities are open source, some are in the hosted free tier, some are paid, and the documentation does not consistently label which is which. You can get some way into a design before discovering that the feature you planned around is on a plan you are not on. Clear labelling in the documentation would fix this and it is not there. Terminology inconsistency is a smaller but persistent irritation.
The same concept appears under different names across components, partly reflecting the platform's growth by accretion, and it makes searching the documentation less effective than it should be. You learn the mapping eventually and it is friction that should not exist. There is also the deeper limitation common to all tools in this space. You will learn to use this platform.
You will not learn what makes an experiment worth tracking, how to design a comparison that answers a question, or how to structure a research process so that the results mean something. Those are the skills, and the tool is the easy part. My three point four is for genuinely best-in-class automatic capture, a comparison interface that works the way you want it to, an elegant remote execution model, and real self-hosting. Marked down for spreading across too many product areas and documenting each less well as a result, for opacity when the automation misbehaves, for unclear commercial boundaries and for inconsistent terminology.
Take the tracking. Think twice about the rest.