Regularization - AnalyticBridge2020-04-03T18:09:36Zhttps://www.analyticbridge.datasciencecentral.com/forum/topics/regularization?feed=yes&xn_auth=noI'm not sure how rigorous you…tag:www.analyticbridge.datasciencecentral.com,2013-10-01:2004291:Comment:2756202013-10-01T15:58:45.574ZAlex Minnaarhttps://www.analyticbridge.datasciencecentral.com/profile/AlexMinnaar
<p>I'm not sure how rigorous you want the answer to be, but the regularization term in the cost function that you are trying to minimize controls the 'complexity' of the model. Overfitting occurs when the model you are fitting to the data is too complex (e.g. fitting a polynomial function of degree 99 to 100 datapoints). So when the regularization term is large, the complexity term (eg. the L1 norm) has more importance in the cost function and thus minimizing the complexity term is put at a…</p>
<p>I'm not sure how rigorous you want the answer to be, but the regularization term in the cost function that you are trying to minimize controls the 'complexity' of the model. Overfitting occurs when the model you are fitting to the data is too complex (e.g. fitting a polynomial function of degree 99 to 100 datapoints). So when the regularization term is large, the complexity term (eg. the L1 norm) has more importance in the cost function and thus minimizing the complexity term is put at a 'higher priority' in the overall minimization problem. So the regularization term affects how complex the model is which affects how much the model overfits.</p>