Subscribe to DSC Newsletter

Why does the regularization term prevent overfitting? Is there any statistical interpretation for the regularization term?

Tags: regularization

Views: 251

Reply to This

Replies to This Discussion

I'm not sure how rigorous you want the answer to be, but the regularization term in the cost function that you are trying to minimize controls the 'complexity' of the model.  Overfitting occurs when the model you are fitting to the data is too complex (e.g. fitting a polynomial function of degree 99 to 100 datapoints).  So when the regularization term is large, the complexity term (eg. the L1 norm) has more importance in the cost function and thus minimizing the complexity term is put at a 'higher priority' in the overall minimization problem.  So the regularization term affects how complex the model is which affects how much the model overfits.

RSS

On Data Science Central

© 2019   AnalyticBridge.com is a subsidiary and dedicated channel of Data Science Central LLC   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service