Silhouette Score and Cluster Evaluation

Clustering usually has no target labels, so internal metrics assess structure using only the data and assigned groups. For observation i, let a(i) be its average distance to other points in its own cluster. Let b(i) be the smallest average distance from i to points in any other cluster. The silhouette coefficient is (b-a)/max(a,b), ranging from -1 to 1 when the distances are defined. A value near one suggests that the observation is much closer to its own cluster than to another. A value near zero suggests a boundary position, and a negative value suggests another cluster may be closer. The overall silhouette score averages the coefficients across observations. A high average can support compact, separated clusters under the chosen distance, but it does not prove the groups are meaningful. A large cluster can dominate the average, and a few poorly assigned points may be hidden. Inspect per-cluster and per-point values, cluster sizes and visualizations. The score also depends on the distance metric and feature scaling. Davies-Bouldin compares within-cluster scatter with between-cluster separation; lower is generally better under its definition. Calinski-Harabasz compares between-cluster dispersion to within-cluster dispersion; higher is generally better. These indices have different scales and preferences, so do not treat a larger raw number in one metric as comparable to another. Their assumptions can also favor certain cluster shapes or balances. Internal metrics are not substitutes for external validation when labels become available, nor for evaluating downstream utility. A silhouette score can favor a partition that is geometrically clean but operationally meaningless. Conversely, useful clusters with irregular shapes may receive a modest score. Compare candidate methods on consistent data and preprocessing, inspect stability under resampling and parameter changes, and ask domain users whether the groups support a real decision. If clustering is used to serve people, assess coverage and harms as well as geometric separation.