Comments - Explaining variability in logistic regression - AnalyticBridge2019-12-11T03:56:46Zhttps://www.analyticbridge.datasciencecentral.com/profiles/comment/feed?attachedTo=2004291%3ABlogPost%3A75409&xn_auth=noHi,
Let me try to explain wh…tag:www.analyticbridge.datasciencecentral.com,2010-08-23:2004291:Comment:768762010-08-23T16:26:16.865ZArunhttps://www.analyticbridge.datasciencecentral.com/profile/Arun
Hi,<br />
<br />
Let me try to explain what you're asking for, and why you can't get it.<br />
<br />
In logistic regression, a measure of 'how much variation has been explained' by the independent variables is <b>not applicable</b>! Any such measure would be misleading given the nature of logistic regression - dichotomous.<br />
A pseudo-Rsquare is also a measure of the <b>Deviance</b> of the model from actual, just as AIC or SC is!<br />
<br />
Try reading up more on why there is <b>no error term</b> in a logistic regression, and…
Hi,<br />
<br />
Let me try to explain what you're asking for, and why you can't get it.<br />
<br />
In logistic regression, a measure of 'how much variation has been explained' by the independent variables is <b>not applicable</b>! Any such measure would be misleading given the nature of logistic regression - dichotomous.<br />
A pseudo-Rsquare is also a measure of the <b>Deviance</b> of the model from actual, just as AIC or SC is!<br />
<br />
Try reading up more on why there is <b>no error term</b> in a logistic regression, and you'll end up understanding why you can't measure a goodness of fit by knowing variance explained as in linear regression.<br />
Remember, in Linear Reg, variance is a constant, while in Logistic Reg, it a function of the probability function you're modeling - a variable variance... can you see why there's a problem understanding the variance explained now??<br />
<br />
Hope this helps.<br />
<br />
Thanks,<br />
Arun Thanks Biswajit for your comm…tag:www.analyticbridge.datasciencecentral.com,2010-08-06:2004291:Comment:756882010-08-06T06:20:20.279ZJai Shanker Singhhttps://www.analyticbridge.datasciencecentral.com/profile/JaiShankerSingh
Thanks Biswajit for your comments<br />
<br />
Hosmer and Lemeshow Goodness of Fit statistic is more useful in assessing the significance of the Logistic Regression than telling us about how much variability of the dependent variable is being explained by the independent variables like R2 in Linear Regression.<br />
<br />
What I am looking for is a number which would tell us how much of the variability of the dependent variable is being explained by the independent variables and how much is not<br />
<br />
Thanks
Thanks Biswajit for your comments<br />
<br />
Hosmer and Lemeshow Goodness of Fit statistic is more useful in assessing the significance of the Logistic Regression than telling us about how much variability of the dependent variable is being explained by the independent variables like R2 in Linear Regression.<br />
<br />
What I am looking for is a number which would tell us how much of the variability of the dependent variable is being explained by the independent variables and how much is not<br />
<br />
Thanks Hi
You can use Hosmer and Lem…tag:www.analyticbridge.datasciencecentral.com,2010-08-03:2004291:Comment:754972010-08-03T18:35:30.818ZBiswajit Palhttps://www.analyticbridge.datasciencecentral.com/profile/BiswajitPal
Hi<br />
You can use Hosmer and Lemeshow Goodness of Fit statistic in order to measure the discriminating power of the model. It tests whether the predicted and observed values for the dependent variable are same or different. In SAS the option “LACKFIT” in the model statement generates this.<br />
Another method is representation in a confusion matrix which leads to ROC Curve.<br />
Please let me know whether it provided you any relevant insight or not.<br />
Thanks<br />
Biswajit
Hi<br />
You can use Hosmer and Lemeshow Goodness of Fit statistic in order to measure the discriminating power of the model. It tests whether the predicted and observed values for the dependent variable are same or different. In SAS the option “LACKFIT” in the model statement generates this.<br />
Another method is representation in a confusion matrix which leads to ROC Curve.<br />
Please let me know whether it provided you any relevant insight or not.<br />
Thanks<br />
Biswajit As part of the output you wil…tag:www.analyticbridge.datasciencecentral.com,2010-08-02:2004291:Comment:754272010-08-02T20:49:33.189ZRalph Wintershttps://www.analyticbridge.datasciencecentral.com/profile/RalphWinters
As part of the output you will get a predicted probability of being in the class designated by 0. You also have the original classes of 0 or 1. So take the original classes for each observations as the x values (0,1,0,0,1 etc.) and run a linear regression against the predicted values (.03, .22, .98, .21 etc.) and use the r2 of the result.<br />
<br />
-Ralph Winters
As part of the output you will get a predicted probability of being in the class designated by 0. You also have the original classes of 0 or 1. So take the original classes for each observations as the x values (0,1,0,0,1 etc.) and run a linear regression against the predicted values (.03, .22, .98, .21 etc.) and use the r2 of the result.<br />
<br />
-Ralph Winters Hi,
I am running a logistic…tag:www.analyticbridge.datasciencecentral.com,2010-08-02:2004291:Comment:754252010-08-02T20:02:50.624ZJai Shanker Singhhttps://www.analyticbridge.datasciencecentral.com/profile/JaiShankerSingh
Hi,<br />
<br />
I am running a logistic regression in SAS<br />
<br />
Ralph,<br />
<br />
Thanks for your comments but can you please elaborate what you have said
Hi,<br />
<br />
I am running a logistic regression in SAS<br />
<br />
Ralph,<br />
<br />
Thanks for your comments but can you please elaborate what you have said You did not say what package…tag:www.analyticbridge.datasciencecentral.com,2010-08-02:2004291:Comment:754172010-08-02T17:56:26.007ZRalph Wintershttps://www.analyticbridge.datasciencecentral.com/profile/RalphWinters
You did not say what package you are running. For a quick linear regression type of R2 you can take all of the predicted values and regress them against the observed 0 and 1's.<br />
<br />
-Ralph Winters
You did not say what package you are running. For a quick linear regression type of R2 you can take all of the predicted values and regress them against the observed 0 and 1's.<br />
<br />
-Ralph Winters