InoteveraGaidhi rinotevera
Kududzira kweMinimum Bayes Risk
Mutauro AI
Basics GUIDE
Empirical risk minimization (ERM) chooses a hypothesis by minimizing its average loss on a finite training sample as a proxy for expected loss under an unknown data distribution.
Teerera kugwaro iriAI-yakarondedzerwa · ~3 min
It matters because fit to that sample can differ from performance on new data, which is why model complexity and independent evaluation are part of the learning problem.
Empirical risk minimization formalizes what most machine learning training actually does. The true risk of a model is the expected value of some loss function over the entire, usually infinite and unknown, distribution of possible data. Since that distribution is inaccessible, ERM substitutes it with the empirical risk: the average loss computed over a finite training sample drawn from that distribution. Learning becomes an optimization problem: search over a hypothesis class (all linear functions, or all neural networks of a given architecture) for the one that minimizes this average sample loss, typically via gradient descent or a closed-form solution. For a fixed hypothesis and representative independent samples, average sample loss can estimate expected loss. To make a guarantee that holds across a whole hypothesis class, additional conditions on the class and sample are needed; uniform-convergence results such as VC bounds make those assumptions explicit. The catch is that minimizing training loss too aggressively, especially with a hypothesis class that is complex relative to the sample size, can drive empirical risk toward zero while true risk stays high; the model has memorized noise specific to the training sample rather than learning the underlying pattern. This is overfitting, and it is why practitioners use regularization, cross-validation, and held-out test sets: not because ERM is wrong, but because minimizing empirical risk alone provides no guarantee about true risk without additional constraints or enough data relative to model complexity. A common misconception is that ERM refers to a specific algorithm; it is a general principle that many learning procedures use, sometimes with regularization or other constraints and different loss functions or hypothesis classes.
Training a spam filter by adjusting its parameters to minimize the fraction of misclassified emails in a labeled training set, hoping this generalizes to new incoming email.
Fitting a linear regression line by minimizing mean squared error across the observed data points, which is literally ERM with squared-error loss.
A neural network's training loop that repeatedly computes gradient updates to reduce average cross-entropy loss over mini-batches drawn from the training set.
A model that achieves near-zero error on training data but performs poorly on new data, illustrating the gap between empirical risk (training loss) and true risk (expected loss on unseen data), i.e. overfitting.
ERM remains a widely used training principle across supervised learning, often combined with regularization or other constraints, from simple regressions to large neural networks, and this is unlikely to change since it is a general mathematical framework rather than a specific technique that could be superseded. Active research continues on refining the gap between empirical and true risk for very large, over-parameterized models, where classical overfitting intuitions sometimes fail to predict observed generalization behavior. Work on distributionally robust optimization and other risk formulations extends ERM's ideas to handle shifts between training and deployment data, but these remain extensions of the same underlying principle.
Tsamba yemahara
Nhatu dzakasimbiswa dzeAI nyaya mangwanani ega ega pakati pevhiki, dzakanyorwa neChirungu chakajeka. Yemahara zvachose, hapana ads.
Imwe email zuva rega rega revhiki. Bvisa kunyoresa kamwe chete. Hatimbotengese kana kugovera kero yako.
Zviedze iwe pachako
Mhinduro yepakarepo pamhinduro yega yega, uye chitupa chinogovaniswa chine ID inobatika kana wapedza kosi.
Tsigira dzidzo yemahara yeAI. AI Understanding ndeye 501(c)(3) isingabatsiri - hapana ads, hapana paywall, zvachose. Ita chipo
Empirical risk minimization (ERM) chooses a hypothesis by minimizing its average loss on a finite training sample as a proxy for expected loss under an unknown data distribution. It matters because fit to that sample can differ from performance on new data, which is why model complexity and independent evaluation are part of the learning problem.
ERM inotsiva iyo isingaonekwe njodzi yechokwadi neiyo empirical njodzi, avhareji yekurasikirwa inoyerwa pane inogumira kudzidziswa samuenzaniso iripo.
Njodzi yechokwadi inotarisirwa pamusoro pezere, kazhinji isingazivikanwe uye isingaperi kugovera data, saka haigone kuverengerwa zvakananga; muenzaniso wekupedzisira chete unowanikwa.
Nhungamiro inodudza mutemo wenhamba huru uye VC-theory yunifomu yekuchinjika miganho sehwaro hwedzidziso hwe empirical njodzi ichichinjika kunjodzi yechokwadi ine data rakawanda.
Kuwedzeredza kunoitika kana modhi inobata nemusoro kudzidziswa-sample-chaiyo ruzha, kuwana yakaderera empirical njodzi nepo njodzi yayo yechokwadi pane isingaonekwe data inoramba yakakwirira.
Regularization inowedzera chirango, senge L2 yakajairwa temu, kuchinangwa cheERM, kubvuma kurasikirwa nekudzidziswa kwepamusoro mukutsinhana nefungidziro inoumba nani.
Ramba uchidzidza
Mamwe madhairekitori akasarudzirwa nyaya iyi
InoteveraGaidhi rinotevera
Kududzira kweMinimum Bayes Risk
Mutauro AI