A Data Science Central Community
Why does the regularization term prevent overfitting? Is there any statistical interpretation for the regularization term?
Tags: regularization
I'm not sure how rigorous you want the answer to be, but the regularization term in the cost function that you are trying to minimize controls the 'complexity' of the model. Overfitting occurs when the model you are fitting to the data is too complex (e.g. fitting a polynomial function of degree 99 to 100 datapoints). So when the regularization term is large, the complexity term (eg. the L1 norm) has more importance in the cost function and thus minimizing the complexity term is put at a 'higher priority' in the overall minimization problem. So the regularization term affects how complex the model is which affects how much the model overfits.
© 2021 TechTarget, Inc. Powered by
Badges | Report an Issue | Privacy Policy | Terms of Service
Most Popular Content on DSC
To not miss this type of content in the future, subscribe to our newsletter.
Other popular resources
Archives: 2008-2014 | 2015-2016 | 2017-2019 | Book 1 | Book 2 | More
Most popular articles