Multi-Label Classification

In standard multi-class classification, each example belongs to exactly one of several mutually exclusive categories, such as classifying an email as either 'spam' or 'not spam.' Multi-label classification removes the mutual exclusivity assumption, allowing zero, one, or many labels to apply to the same example. Two widely used approaches handle this problem differently. Binary relevance trains one independent binary classifier per label, predicting whether each label applies or not without regard to the others; it is simple and scales well, but it ignores potential correlations between labels, such as the fact that 'thriller' and 'action' movie tags often co-occur. Classifier chains address this limitation by training a sequence of binary classifiers where each one receives the predictions of prior classifiers in the chain as additional input features, capturing label dependencies at the cost of being sensitive to the chosen label order and to error propagation along the chain. Label powerset is another approach that treats each unique combination of labels as its own single class, which captures label correlations directly but suffers when the number of possible label combinations grows too large relative to available training data. Evaluation also differs from single-label classification: standard accuracy is a poor fit for multi-label problems since a prediction can be partially correct, so metrics like Hamming loss, which measures the fraction of individually mispredicted labels, and label-based or example-based F1 scores are used instead. A common misconception is treating multi-label classification as equivalent to running a standard multi-class classifier with more categories; the mutual exclusivity assumption baked into typical multi-class softmax outputs makes that approach structurally wrong for problems where labels can co-occur.