Tiếp theoHướng dẫn tiếp theo
Giải mã rủi ro Bayes tối thiểu
Ngôn ngữ AI
HƯỚNG DẪN cơ bản
Empirical risk minimization (ERM) chooses a hypothesis by minimizing its average loss on a finite training sample as a proxy for expected loss under an unknown data distribution.
Nghe hướng dẫn nàyAI thuật lại · ~3 min
It matters because fit to that sample can differ from performance on new data, which is why model complexity and independent evaluation are part of the learning problem.
Empirical risk minimization formalizes what most machine learning training actually does. The true risk of a model is the expected value of some loss function over the entire, usually infinite and unknown, distribution of possible data. Since that distribution is inaccessible, ERM substitutes it with the empirical risk: the average loss computed over a finite training sample drawn from that distribution. Learning becomes an optimization problem: search over a hypothesis class (all linear functions, or all neural networks of a given architecture) for the one that minimizes this average sample loss, typically via gradient descent or a closed-form solution. For a fixed hypothesis and representative independent samples, average sample loss can estimate expected loss. To make a guarantee that holds across a whole hypothesis class, additional conditions on the class and sample are needed; uniform-convergence results such as VC bounds make those assumptions explicit. The catch is that minimizing training loss too aggressively, especially with a hypothesis class that is complex relative to the sample size, can drive empirical risk toward zero while true risk stays high; the model has memorized noise specific to the training sample rather than learning the underlying pattern. This is overfitting, and it is why practitioners use regularization, cross-validation, and held-out test sets: not because ERM is wrong, but because minimizing empirical risk alone provides no guarantee about true risk without additional constraints or enough data relative to model complexity. A common misconception is that ERM refers to a specific algorithm; it is a general principle that many learning procedures use, sometimes with regularization or other constraints and different loss functions or hypothesis classes.
Training a spam filter by adjusting its parameters to minimize the fraction of misclassified emails in a labeled training set, hoping this generalizes to new incoming email.
Fitting a linear regression line by minimizing mean squared error across the observed data points, which is literally ERM with squared-error loss.
A neural network's training loop that repeatedly computes gradient updates to reduce average cross-entropy loss over mini-batches drawn from the training set.
A model that achieves near-zero error on training data but performs poorly on new data, illustrating the gap between empirical risk (training loss) and true risk (expected loss on unseen data), i.e. overfitting.
ERM remains a widely used training principle across supervised learning, often combined with regularization or other constraints, from simple regressions to large neural networks, and this is unlikely to change since it is a general mathematical framework rather than a specific technique that could be superseded. Active research continues on refining the gap between empirical and true risk for very large, over-parameterized models, where classical overfitting intuitions sometimes fail to predict observed generalization behavior. Work on distributionally robust optimization and other risk formulations extends ERM's ideas to handle shifts between training and deployment data, but these remain extensions of the same underlying principle.
Bản tin miễn phí
Ba câu chuyện AI đã được xác minh vào mỗi buổi sáng các ngày trong tuần, được viết bằng tiếng Anh đơn giản. Miễn phí mãi mãi, không có quảng cáo.
Một email mỗi ngày trong tuần. Hủy đăng ký trong một cú nhấp chuột. Chúng tôi không bao giờ bán hoặc chia sẻ địa chỉ của bạn.
Tự kiểm tra
Phản hồi tức thì về mọi câu trả lời và chứng chỉ có thể chia sẻ với ID có thể xác minh sau khi bạn vượt qua khóa học.
Hỗ trợ giáo dục AI miễn phí. AI Understanding là tổ chức phi lợi nhuận 501(c)(3) — không bao giờ có quảng cáo, không có tường phí. Hãy quyên góp
Empirical risk minimization (ERM) chooses a hypothesis by minimizing its average loss on a finite training sample as a proxy for expected loss under an unknown data distribution. It matters because fit to that sample can differ from performance on new data, which is why model complexity and independent evaluation are part of the learning problem.
ERM substitutes the unobservable true risk with the empirical risk, the average loss measured on the finite training sample actually available.
True risk is an expectation over the full, typically unknown and infinite data distribution, so it cannot be computed directly; only a finite sample is available.
The guide cites the law of large numbers and VC-theory uniform convergence bounds as the theoretical basis for empirical risk converging to true risk with more data.
Overfitting occurs when a model memorizes training-sample-specific noise, achieving very low empirical risk while its true risk on unseen data stays high.
Regularization adds a penalty, such as an L2 norm term, to the ERM objective, accepting somewhat higher training loss in exchange for a hypothesis that generalizes better.
Tiếp tục học hỏi
Đã chọn thêm hướng dẫn cho chủ đề này
Tiếp theoHướng dẫn tiếp theo
Giải mã rủi ro Bayes tối thiểu
Ngôn ngữ AI