 |
 |
|||
 |
|
     |
||
| |
|
|||
|
|
|
||
Statistically Speaking
On Election Polls: Part III 
by Dr. Romulo A. Virola 1
Secretary General, NSCB
While elsewhere, investors envy how much richer Googlers Sergey Brin and Larry Page are about to become five years from their Stanford University days, many candidates during our May 10 elections have to start thinking how much poorer they shall have become, less than five days from now. And some of them will curse Pulse Asia and SWS!
But should they? Should election polls be banned, especially near election time?
I recall past attempts to ban exit polls and to prohibit under the Fair Elections Practices Act the publication of election survey results 15 days before a national election and 7 days before a local election. This is premised on the theory that the poll results inform but also influence the voters. Some also aver that these “weapons of mass distraction” detract from discussion of issues and convert the elections into a horse-race with the media getting into a leader fixation mode. But our Supreme Court ruled on May 5, 2001 that such prohibitions violated our freedom of speech! You agree?
Let us find out how other countries treat election polls.
In Taiwan, releasing results of opinion polls 10 days before election day is prohibited under the Election and Recall Law.
In India, the Election Commission issued guidelines in 1998 and 1999 prohibiting the dissemination of exit polls and opinion polls one month before the elections but was rebuked by the Supreme Court which ruled that the Commission was not empowered to do so.
In Canada, the Canada Elections Act was amended in 1993 to prohibit the dissemination of results during the last three days of an election campaign but this was declared to be unconstitutional by the Supreme Court of Canada in 1998.
And in a 1996 survey conducted by the Foundation for Information, an Amsterdam-based organization for the protection of the right to obtain and make use of information, the European Society for Opinion and Marketing Research (ESOMAR) and the World Association for Public Opinion Research (WAPOR), 30 of 78 countries surveyed had some restrictions on the publication of polls; 47 had absolutely no restrictions and one had no answer. Some of those countries with restrictions include, would you believe, France, Italy, Spain and New Zealand. In Japan and Thailand, questions about the royalty are to be avoided by opinion polls; while in Albania, China, Hongkong, Croatia, Indonesia, Mexico, North Korea, Turkey, Venezuela and some Middle Eastern countries, questions on foreign and defense policies and armed conflicts within the country are not permitted.
In our country, despite the Supreme Court decision, many, especially those supporting the “losing” candidates during the elections next week, continue to question the wisdom of publishing election survey results. True, election polls can influence voters in different ways:
But isn’t this what democracy and genuine freedom are all about? As the Foundation for Information says, “…the freedom to carry out and publish the results of public opinion polls is a fundamental democratic right. It is just as fundamental as the freedom of the media to publish comments and opinions on the issues of the day.” And our own Supreme Court has declared that “to sustain the ban on survey results would sanction the censorship of all speaking by candidates in an election, on the ground that the usual bombasts and hyperbolic claims made during the campaign can confuse the voters and thus debase the electoral process.”
Hallelujah! Indeed, a ban on election and exit polls is totally inconsistent with democratic processes and our constitutional freedom of speech!
As I have said before, election polls, when conducted scientifically and objectively, are powerful tools for measuring the pulse of the voters. Using a sample of less than 2000 respondents, it is possible to predict with reasonable degree of precision, the winners in the coming elections. This is what we statisticians call statistical inference. From the opinion of a small group of Filipino voters, professional statisticians can make valid inferences about the opinion of the entire population of Filipino voters!
So, yes, Virginia, we should not ban election polls. But all of us, especially the media, should watch out for the illegitimate ones! And, what distinguishes a good election survey from a bad one? Let us count the ways!
Transparency. The pollsters must be willing to share and discuss the methodology with the public. They must publish information on the sponsorship, the margin of error, exact formulation of the questions asked, geographic distribution of respondents and sample size at the very least. I therefore hope that before the media release any survey results, they will make it a pre-condition to be given access to the methodology, the soundness of which can be evaluated by trained statisticians. It will be also reassuring to know the caliber and integrity of the statisticians behind the surveys.
Representativeness of the sample. If you do not thoroughly mix the sugar in your kapeng Barako, the first sip will taste differently from the last few sips. Same thing with surveys. If you interview only those who idolize Manny Pacquiao, the result will be different when you interview only those who, Monday thru Friday, 8:30 to 9:00 in the evening, will allow you only over their dead bodies, to watch anything other than Marina! And if you interview only those who come from your bailiwick, sure, you will “win” the election by a mile! In doing a survey, while we cannot guarantee a representative sample, we must want an election poll to have respondents who collectively represent the Filipino voters!
Use of a random sample. The respondents must be selected randomly, to allow measures of confidence on the results, expressed in terms of margins of error. You should not do a survey only among those who attend your rally! So, the pollster must tell us exactly how the respondents were selected. The common practice of the media to solicit public opinion thru telephone calls or texting definitely does not involve random samples and does not pass standards of statistical rigor. These phone-in polls are as unscientific as polls can be! And even if they are conducted for fun, they destroy the credibility of legitimate polls and they hurt everyone, including those who are favored by the voodoo poll results, who are lulled into a false sense of expectation. It must be said however, that a random sample does not necessarily produce correct results all the time. A random sample can still be unrepresentative of the population, and the results from such samples fall in those margins of error.
Unbiased estimation. If the survey exercise is repeatedly done, the estimated proportion of voters who will vote for a particular candidate is expected to be close to if not equal to the actual proportion. What this requires is that if 1200 sample respondents are distributed into 300 for Metro Manila, 300 for the rest of NCR, 300 for Visayas and 300 for Mindanao, they must be weighted differently to account for the fact that these four areas have different numbers of voters. And the weights must be right!
Sufficiency of sample size. To attain acceptable margins of error, sample sizes must be large enough. If the contest is close, larger sample sizes are needed to achieve meaningful and statistically significant results. Did you notice that both the SWS and Pulse Asia had been using bigger samples than usual? However, for election polls, large is not really that large. With all the criticisms raised against opinion polls, obviously there are not many trained statisticians in our midst. It also means that some basic statistical truths are neither obvious nor intuitive. Let me reiterate what I wrote in Part II – less than 2000 respondents are more than sufficient for election polls in our country of 84.2 million people and 43,536,028 registered voters for the 2004 elections! I hope I won’t have to hear any more gasps how in hell 1200 Filipinos could speak for all of us! What I would like to call these instant mami of statisticians is endearingly unprintable!
Proper design of questionnaire. Illegitimate pollsters use improper questions to get the biased results they wanted in the first place. The phrasing of the question, the way the interviewer asks the questions and the sequencing of the questions all can be abused in illegitimate surveys. To these illegitimate pollsters, opinions expressed by respondents are lost in translation.
Treatment of the undecided/ refusals. Several studies have been conducted in different countries which conclude that it is not usually correct to randomly allocate the undecided/refusals. A study by Durand, Blais and Vachon entitled Accounting for Biases in Election Surveys: The Case of the 1998 Quebec Election, wherein the last six polls of the campaign showed the Parti Quebecois (PQ) clearly ahead by 5 percentage points over the Liberal Party and while the PQ won the election, its share of the votes was smaller than was indicated in the polls. A study of the nonresponse showed a consistent tendency for the Liberal supporters to refuse to answer surveys and the undersampling of Liberal supporters because of the absence of respondents living in institutional households. In another study in Mexico, the conclusion was that the refusals were afraid to reveal their oppositionist views. In short, there is information to be mined behind those undecided votes! And while it is not easy to decide how to treat the undecided, one should be careful in interpreting results of surveys where the proportion of the undecided is big enough to sway the results in the opposite direction.
Indeed, it is amazing, albeit not totally surprising, that some supposedly intelligent individuals cannot understand all this. I thought the dinuguan taste test was just about as simple an explanation could get!
But I will not give up. Just for fun, let us play a game. Let us keep track of the Pulse Asia results released today and the SWS results to be released on Saturday and let us compare them with the final election results. Specifically, let us count the number of winning candidates predicted by the two pollsters and compare them with the actual winners. Then we will see who did better! And hopefully, we will also know better how to deal with these opinion polls next time. And don’t you worry, we promise to forget what you said about SWS and Pulse Asia!
Certainly, legitimate poll surveys do not aim to influence election results; but even if they do not, I would not be surprised, if in fact, they do influence some voters. When they do, I say this is part and parcel of the right to informed choices, a right guaranteed in a true democracy.
Opinion polls are about providing information, sharing knowledge. They are not about subverting free choice nor engendering fear among the losers. Those who know how to convert the information supplied by opinion polls into knowledge, then to wisdom, shall be the winners!
And the media will play a major role if sound opinion polls are to serve their purpose. Media must carefully assess the survey methodology and integrity of pollsters before publishing their results. No matter how boring the election has become, media must not fall victim to the biases of illegitimate pollsters. Five years into the Third Millennium, media must enlighten itself with the enormous power of statistics, and not be trapped in the mindset of losers. Like some politicians! Like Dugong !
Meanwhile, on Monday, let us vote wisely. When the votes are out, let us accept the result of the electoral process and let us give our all-out support to the winners! For the sake of our country!
Reactions and views are welcome thru email to the author at ra.virola@nscb.gov.ph.
_______________
1 Secretary General of the National Statistical Coordination Board (NSCB) and Chairman of the Statistical Research and Training Center (SRTC). He holds a Ph. D. in Statistics from the University of Michigan in Ann Arbor, USA and has taught mathematics and statistics at the University of the Philippines. He is also a past president of the Philippine Statistical Association.
Posted 06 May 2004.
 |
 |
|||||||
  E-mail the webmaster |
Terms of Use |
Home • Top of Page | ||||||
1997-2013,
National Statistical Coordination Board |
||||||||
Read