Everyone wants to talk about models and nobody wants to talk about labels, which is unfortunate given that label quality determines model quality far more reliably than architecture choice does. Label Studio is the most widely used open source tool for the job and its documentation is a reasonable, workmanlike guide to operating it. The coverage is broad. Installation across Docker, pip and various deployment shapes.
Database and persistent storage configuration. Project creation and management. Import and export, including sync with S3, Google Cloud Storage, Azure and Databricks. Machine learning backend integration.
REST API, Python SDK and webhooks. Frontend customisation. This is the boring half of the problem and the documentation takes it seriously, which I appreciate more than I expected to. The labelling configuration system is the heart of the product.
You describe your labelling interface in an XML dialect, and that description determines what annotators see and what shape your output takes. Template coverage is genuinely good, spanning image classification, object detection, segmentation, named entity recognition, text classification, audio transcription, time series, ranking, and a long tail of less common tasks. Starting from a template and modifying it is the practical path and the documentation supports that path well. The XML itself is where I have a complaint.
It is a capable little language with real depth, and it is taught almost entirely by example. When your task does not match a template closely, you end up reading tag reference pages and guessing at how they compose. A proper explanation of the model behind the configuration, rather than a gallery of instances of it, would save people a lot of trial and error. The ML backend documentation earns its place.
Connecting a model so that it pre-annotates your data and your humans correct rather than create is the single largest efficiency gain available in any labelling operation, often several times over. The docs explain the interface, the setup and the workflow adequately. The active learning material, where the model chooses what to label next, is lighter than the idea deserves. The API and SDK coverage is good enough that you can drive the whole thing programmatically, which is how any serious project ends up running it.
Projects created from code, tasks pushed in, annotations pulled out, webhooks firing on completion. This matters because manual clicking does not survive contact with a real dataset. Now the two substantial problems. The first is commercial clarity.
Label Studio community edition is open source and free. HumanSignal sells Starter Cloud and Enterprise tiers, and features like fine grained role management, annotation agreement metrics, review workflows, advanced project analytics and various security controls live in the paid tiers. The documentation covers all of it together, and while there are comparison pages and some labelling of enterprise features, the boundary is not consistently marked in the places you encounter it. The result is a predictable and irritating experience where you read about a capability, plan around it, and discover later it is not in the edition you installed.
Anyone evaluating this should check the edition comparison page before they read anything else, and should verify rather than assume. The second problem is bigger and it is about what is absent. This documentation teaches you to operate a labelling tool. It teaches you almost nothing about labelling.
How to write an annotation guideline that two people interpret the same way. How to run a pilot round and fix your taxonomy before you have labelled ten thousand items with a broken one. How to measure inter-annotator agreement and what a poor score is telling you. How to handle genuinely ambiguous cases rather than forcing them into a category.
How to audit a completed dataset. What to do when two annotators disagree consistently and both are being reasonable. This is the craft, and it is the difference between a dataset that trains a good model and one that bakes in confusion. It is not really the tool vendor's job, and its absence from the primary resource that new labelling teams read still leaves those teams badly prepared.
Notably, the agreement metrics that would help you measure this are themselves an enterprise feature. Scaling guidance is the smaller gap. Documentation for a handful of annotators on a modest project is fine. Guidance for hundreds of thousands of tasks, many concurrent annotators, and the database and storage behaviour under that load is thinner than someone planning a large operation would want.
My three point six is for competent, practical operational documentation on a genuinely useful open source tool, marked down for the persistent ambiguity about what is free and for the near total absence of the annotation craft that determines whether any of this produces data worth training on.