Active learning is the idea that a model should choose what gets labelled next. It has a long literature, a real mechanism, and a published caution that is much less widely cited than the technique itself.
The idea, and the only reason it works
You have a large pool of unlabelled examples and a budget for labelling a small fraction of them. Random sampling spends that budget in proportion to how common each kind of example is, which means most of it goes on cases the model already handles. Active learning instead trains a model on what has been labelled so far, uses it to rank the unlabelled pool by how informative each item would be, and sends the top of that ranking to the annotators.
The mechanism is entirely dependent on one condition being true: examples must differ substantially in how much they teach. In a pool where every example is equally informative, active learning cannot beat random and only adds machinery. In a pool that is 95% near-duplicate easy cases with a thin tail of hard ones — which describes most production traffic — the difference is large, and that is the regime worth building for.
Four acquisition strategies
