Subscribe to DSC Newsletter

In campaign measurements, it's possible to get a larger lift at the overall level compared to all the individual decile level lifts or vice versa, because of the differences in sample size across the deciles, and across Test & Control. In such scenarios, how do you calculate the overall lift? Which methods are commonly used in the industry?




Tags: and, campaign, control, lift, measurment, response, test

Views: 885

Reply to This

Replies to This Discussion

decile level lifts are calculated or if i may say lifts are calculated at the decile level only because thats why lift makes sense. after you have scored your population, and have divided them into groups: deciles, twentiles or whatever quintilies, and have sorted descending, then lift would be the ratio of model vs actual and you would contact only upto those deciles using the model where the lift is atleast 1.
overal lift=cumulative lift? just add up the actuals and the models as you go down the deciles and take the ratios, thats cumulative or overall lift for you. that better be more than 1 too or the model is overall useless :)
Tthanks arup. guess, i have confused everyone with my short quick question. I know about lift :)

I just wanted to know how the cumulative lift can be used when your results are biased because of different sample sizes in the test and control groups. this question has more to do with measurement rather than modeling. also, this problem is commonly known as Simpson's paradox (an apparent paradox in which the successes in different groups seem to be reversed when the groups are combined)

maybe we need to make some modifications in the way we calculate overall lift in such scenarios?
data, what's the overall lift?

'difference in sample size across deciles' - a decil = 10%, what difference?

it there are enough data then lift is sensitive to good/bad distribution, not to sample size. random sampling keeps the distibution. so you can use simple average across multiple samples ...

pls. explain your idea more :)


On Data Science Central

© 2021   TechTarget, Inc.   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service