Subscribe to DSC Newsletter

Why does the regularization term prevent overfitting? Is there any statistical interpretation for the regularization term?

Tags: regularization

Views: 253

Reply to This

Replies to This Discussion

I'm not sure how rigorous you want the answer to be, but the regularization term in the cost function that you are trying to minimize controls the 'complexity' of the model.  Overfitting occurs when the model you are fitting to the data is too complex (e.g. fitting a polynomial function of degree 99 to 100 datapoints).  So when the regularization term is large, the complexity term (eg. the L1 norm) has more importance in the cost function and thus minimizing the complexity term is put at a 'higher priority' in the overall minimization problem.  So the regularization term affects how complex the model is which affects how much the model overfits.


On Data Science Central

© 2021   TechTarget, Inc.   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service