Distance Metrics in Machine Learning

A distance metric turns raw feature values into a single number representing how far apart two points are, which many algorithms rely on directly. Euclidean distance, the straight-line distance familiar from geometry, is a common choice in many introductory examples and works well when features are continuous, roughly on comparable scales, and the notion of similarity matches physical closeness. Manhattan distance, also called taxicab distance, sums the absolute differences along each dimension rather than taking the square root of squared differences; it suits grid-like movement constraints and is less sensitive to outliers in individual dimensions than Euclidean distance. Cosine distance measures orientation between nonzero vectors rather than their magnitudes, making it useful in some text and recommendation systems where a document or user profile's overall direction, such as topic balance, matters more than its raw length or intensity. Mahalanobis distance generalizes Euclidean distance by accounting for the correlations and differing variances between features, effectively normalizing the space so that features are compared fairly regardless of their original scale, which is useful in anomaly detection and multivariate outlier analysis. Hamming distance counts the number of positions at which two equal-length strings or categorical vectors differ, making it suited for categorical data, error-correcting codes, and genetic sequence comparison rather than continuous numeric data. A common misconception is that Euclidean distance is always the safe default; in high-dimensional spaces, all pairwise Euclidean distances tend to become similar, a phenomenon sometimes called the curse of dimensionality, which can make nearest-neighbor comparisons less informative; checking distance distributions and task performance can reveal when that matters.