Understanding Polling Statistics in Light of Indian Elections

MM brings to you a guest post by Dr. Prasenjeet Ghosh, a friend and classmate of mine from the Wharton class of 2009. In an informal, extremely unscientific poll, Dr. Ghosh was voted as the student most likely to be the next Wharton Professor, and thus it seemed natural to ask him to share how scientific polls are really done! Dr. Ghosh will be sharing educational and informational articles with the MM readers on topics of everyday interest… topics that we encounter everyday, but don’t quite know the science/art behind them.  This is hopefully the first of many posts on such relevant topics! Pls welcome Dr. Ghosh, feel free to ask any questions on the subject matter, and be nice :) -Amad

The Parliamentary Elections are in full swing in India, and much is at stake for this young, and dynamic nation. However unlike the previous elections where the entire voting was concluded in a span of couple of days, the 2009 elections will be conducted in five phases segmented into different geographic regions that will span over a month long period. Phases 1, 2 and 3 were completed on April 16, April 22-April 23, and April 30, while Phases 4 and 5 will be conducted on May 7 and May 13, 2009. The results of the election will be announced on May 16, 2009.

The spanning of elections over a month can have interesting consequences, most important of which is that it can influence the actual outcome of the elections. For example, the exit and opinion polls from places that have already conducted the elections (here Phases 1, 2 and 3), can influence the voting patterns for Phase 4 and 5. Fortunately the Election Commission of India realized this potential problem and issued guidelines according to which,

“No result of any opinion or exit poll conducted at any time shall be published, publicized or disseminated in any manner by print, electronic or any other media at any time till the last round of voting was cast.”

This order will be applicable to all the states and territories in which the elections are going to held in different phases.

Often people misinterpret opinion and exit polls and use them interchangeably; however they are very different polls. Exit poll, as the name suggests is the poll carried out when the voters exit from the polling booth. In other words, it is a survey after the voter has already cast his vote. Contrast this with opinion poll, which is a survey conducted before the voter has actually cast his vote, and measures how the voter might actually vote. Clearly exit polls are more relevant for predicting the actual winner, however they are available only after the voting has already been done, and therefore of little use to the political party to do any course correction (if needed). Opinion polls in contrast offer feedback on how the election campaign is going and serves as a good metric for what needs to changed/continued in the current campaign.

It is interesting to note that despite the official prohibition by Indian government to disclose opinion and exit polls, such polls galore on the internet, created by many enthusiastic and computer savvy Indian voters. However, very surprisingly, most of these polls lack the statistical veracity and sophistication needed to interpret their results meaningfully. For instance, many of the polls might claim that Congress is going to win 55% of the seats, BJP 40%, and Independents the remaining 5%, but conveniently fail to describe the confidence interval in their predictions, or how many voters did they actually survey to get to the result. Confidence intervals, and number of voters sampled are important considerations in creating the poll result.

Some common knowledge of statistics might help the savvy voter to interpret these numbers more meaningfully. A confidence interval is an interval estimate of certain statistical parameter. In our example, such statistical parameter is the fraction of the votes Congress will get (here 55%). But instead of quoting just one number, the confidence interval might be an interval like 50% to 60%. Think of confidence interval as margin of error in your statement. Mathematically, margin of error is exactly half the width of the confidence interval. Thus, in our example, margin of error is plus or minus 5%, which is half the width of the confidence interval of 10% (=60%-50%). Going forward, we will use the words confidence interval and margin of error interchangeably.

Typically confidence intervals are associated with a particular confidence level (say 50%, 90%, 95% etc) which reflects the certainty by which you are making a claim.  Assume that in our example, the confidence level is 95%. Then an ideal poll website should claim that based on its polling surveys, it finds with 95% confidence level that Congress will get anywhere from 50% to 60% of the total votes. This is a complete, and statistically satisfying statement.

However such satisfying statements come at a price. The price manifest in terms of number of voters sampled. Higher the confidence level, greater the number of voters that need to be sampled. The intuition is simple, as you sample more and more voters, you can infer some statistical parameter with more certainty. Infact statistical theory shows that for a margin of error of plus or minus 3% and a 95% confidence level, one needs to sample approximately 1000 voters. In other words, if the poll website makes a statement that with 95% confidence, Congress would win 55%  of votes with margin of error of 3%, they need to sample approximately1000 voters (1067 to be precise). If fewer than 1000 voters have been sampled, then either the margin of error is higher, or their confidence level is lower. If these criteria are satisfied, then what they are really saying is that with 95% confidence, Congress will win anywhere from 52%-58% of the total votes.

So next time you see a poll that claims that their opinion and exit polls are predicting a certain political party to claim a certain % of votes, stop and ask yourself three questions before interpreting the results: (a) What is their margin of error, (b) What is their confidence level, and (3) How many voters did they actually survey.

Dr. Prasenjeet Ghosh has a PhD in Chemical Engineering and an MBA from the Wharton School of Business. He is a keen student of mathematics and statistics, and manages a R&D group at a major oil & gas corporation.