Subscribe to DSC Newsletter

I was once asked to perform a study for NBC. They were running advertising using a weekly mix of TV shows at various times, to promote their Internet website. I was asked to identify which TV shows worked best at driving new users to the web site. Each week, they were using pretty much the same mix of 10-12 shows out of 20 different shows. GRT data (number of eyeballs who have seen a particular TV show at a particular time) was collected, as well weekly number of new users, over a 6-month time period. No A/B testing was possible, so the problem was quite complicated.

Here’s how I solved the problem.

I computed a distance for all pairs of weeks {A, B}, based on which TV shows were used in the weeks in question, as well as the total GRT of each week (high GRT results in many new users unless the TV ad is really bad). I then looked at pair of weeks that were very similar (in terms of GRT’s and show mixes), and close in time (no more than 2 months apart). I focused on pairs where week A had one more show, say show S, compared with the week B. Week B did not have S, but instead had an extra show not shared with A, say show T. Actually, it was a bit more complicated than that, I allowed not just one, but two different shows. But for the sake of simplicity, let’s stick with my example: Show S in week A and not in week B, and show T not in week A but in week B. If week A had more new users than B, I would adjust the performance index of shows S and T as follows: P(S) = P(S) + 1, P(T) = P(T) - 1. The converse was done if A had fewer new users than B.

To identify the best performing shows, all was left to do is to rank shows by their performance index. The Law & Order TV show was found to be the winner, and was used more often in the mix of TV shows used to promote the web site. 

Views: 2665


You need to be a member of AnalyticBridge to add comments!

Join AnalyticBridge

Comment by Tarek Soukieh on December 2, 2013 at 2:50pm

Interesting, you simulated an A/B testing. How did you calculate the performance index? and did you use ANOVA or any modeling?

Thanks for sharing.


On Data Science Central

© 2021   TechTarget, Inc.   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service