NSCB - Statistically Speaking... On Election Polls: Part IV - WHO DID BETTER - SWS OR PULSE ASIA? by Dr. Romulo A. Virola

On Election Polls: Part IV
WHO DID BETTER - SWS OR PULSE ASIA?
by Dr. Romulo A. Virola ¹
Secretary General, NSCB

In Part III, I asked the readers to play a game to check this out! In the first 47 minutes of our game, while other people added va-va-voom, bioweapon and designer baby to their vocabulary, ours was wretchedly enriched by such un-Third Millennium phrases as Honest Mistake, Noted, Filibuster, Shut up, etc. Now that the Kuya Cesaresque canvass of votes has come to a climax, and after the President, much to the relief of many, has finally been proclaimed and inaugurated in the historic city of the delicious danggit and the original otap, we can rest and tell.

Read PART I - PART II - PART III

Quote We will tell how the two survey firms fared in predicting the winners of the presidential, the vice-presidential and the senatorial races and in predicting the percentages of the votes the winners got. Our assessment will focus at the national level (due to space and data limitations), although we will also peek a tiny bit on the NCR as requested by a reader. SWS and Pulse Asia will be evaluated on the basis of their last surveys prior to the election and the exit poll conducted by SWS on election day. We shall refer to them as the SWS, Pulse Asia and Exit Poll, respectively.

We will also take the view that winning is far more important than the actual percentage of votes garnered. Thus, our point system awards 50 points for each winning candidate predicted, based on the forecast percentage of votes and ignoring the statistical ties in the rankings. In addition, for each of the winning candidates, we will award 20 points if the difference between the predicted and the actual percentage of votes is at most 1 percentage point; 18 points if it is between 1 to 2 percentage points; 16 points if it is between 2 to 3 percentage points; and so on, with 2 points awarded if the difference is between 9 to 10 percentage points and 0 if beyond 10 percentage points. To get their overall rating, we will divide the total points earned by the maximum possible score of 980 points. Don’t ask me why, though.

As basis for the actual number of votes, we will use the Comelec Senatorial Canvass Report No. 20 with 176/176 COCs Canvassed and the Congress Joint Canvass Committee Tabulation with 180/180 COCs Canvassed. To derive the actual percentage of votes received, we shall use as denominator the number 35,438,327 based on the NSCB estimate (refer to ‘A Statistical Analysis of Voters’ Registration and Participation” at www.nsch.gov.ph ) of 81.4% turnout rate from 43,536,028 registered voters. Incidentally, the SWS exit poll estimated an 81.3% weighted turnout rate, tantalizingly close to our estimate. The percentages of votes in the Exit Poll were also adjusted for the non-100 % turnout (the Exit Poll’s percentages excluded the NO Answer and for the presidential race, the Exit Poll percentages summed up to 100%).

Before we go to the figures, let me say at the outset that I would have expected the SWS survey of May 1-4 to do better than the Pulse Asia Survey of April 26-29 just on the basis of its being closer to the actual election date. By extension, if the SWS Exit Poll is as methodologically sound as the pre-election polls and if everything went as planned, the Exit Poll should have done better than either of the other two polls. However, the methodological details of the Exit Poll conducted by the SWS are not very clear to me. Also, there are three points worth mentioning that may have some bearing on the quality of the Exit Poll. First, the interview success rate is nowhere near the rate for the pre-election polls. Out of a target of 10,520 respondents, only 5,963 were successfully interviewed by the Exit Poll for a completion rate of 56%. Second, because of the time constraint on the Exit Poll, it is possible that some bias may have been introduced if the respondents successfully interviewed were those who voted or came home early. Third, the Exit Poll would have benefited from last minute changes in voter intentions, such as for example, the effect of the Maalaala Mo Kaya episode on Pia Cayetano aired by Tita Charo on May 6.

Now, for the verdict:

All three surveys fared very well at the national level in predicting the winners: SWS was perfect, Pulse Asia and the Exit Poll each had only one mistake. Pulse Asia had Maceda instead of Cayetano, while the Exit Poll had Barbers instead of Lim. But Pulse Asia did much better than SWS in predicting the percentages of votes received.
In the presidential race,

All three surveys were within one percentage point of the actual percentage of votes received by GMA;
SWS and Pulse Asia underestimated the percentage of votes for FPJ by 3.2 and 2.2 percentage points respectively; but the Exit Poll’s underestimation was quite off at 5.6 percentage points (the FPJ voters may have come home late and were missed by the Exit Poll);
For the other three candidates, all three surveys were less than two percentage points away; and
In the language of one reader, the Exit Poll for NCR was, like our very own Paeng Nepomuceno, the only four-time World Cup Champion bowling 10 straight kanals! In the Comelec count for NCR, GMA had 20% while FPJ had 28%; the Exit Poll (adjusted percentages) had it the other way around, 28 % for GMA and 21% for FPJ!

In the vice-presidential race,

The percentage estimates of SWS and the Exit Poll for De Castro were within one percentage point of the Comelec count; Pulse Asia missed by more than three percentage points;
As with FPJ, all three underestimated the share of the KNP candidate Legarda – SWS by one percentage point, Pulse Asia by three percentage points and the Exit Poll, by four percentage points; and
The Exit Poll also overestimated De Castro’s share in NCR: 33% instead of 28%.

In the senatorial race,

They all succeeded in predicting the first three positions;
They all predicted that Biazon would barely come in;
They all predicted that among those who had some but not great chances of winning, both Osmena and Mercado, would probably not come in;
Both the SWS and the Pulse Asia (pre-Maalaala Mo Kaya episode) underranked Cayetano; and
The SWS and its Exit Poll underranked Lim, partly explainable by the fact that the Comelec count showed a very close race for the 8 th to 13 th places . The Exit Poll almost consistently underranked Lim, and in particular, in vote-rich Region 3 ( the Lim voters, like the FPJ voters, may have come home late!).

Overall, the ratings, which are based only on the winning candidates at the national level as shown on Tables 07-12-1 and 07-12-2 ( thus, factoring out the kanal of the Exit Poll in NCR and see NOTE at the end of the article ) are

SWS May 1-4 Survey 90 %
Pulse Asia April 26-29 Survey 90 %
SWS Exit Poll on May 10 82 %

If these surveys were the work of my students at the UP School of Statistics, I would give the following grades: SWS – 1.5; Pulse Asia – 1.5 and Exit Poll – 2.0. The grades would probably be lower if I examined the regional levels as well.

And for the lessons:

Conduct the last pre-election poll a week before the election;
SWS should improve its Exit Poll design, especially for NCR ( on-site interviews might be better than home interviews); and
I will leave the lessons of Juday and Maalaala Mo Kaya to the losers.

Finally, I would rather that the SWS and Pulse Asia hired two different firms to do their field operations to make them truly independent of one another. And their fight more interesting! And after all is said and done, it must be NOTED that the most important lesson to be learned in the election and exit polls is that candidates and non-candidates alike should trust truly the power of statistics. Only then could we hope for the tragic comical teleserye on The Philippine Elections to end happily, just like “Sana’y Wala Nang Wakas” for Ara and Christian! And no replay please!

Note: Yesterday, July 11, the Philippine Daily Inquirer carried an article on the SWS admission of errors in the Exit Poll, specifically for NCR, Davao Region, Socksargen and ARMM.

TABLE 07-12-1
	SWS		PULSE ASIA		SWS EXIT POLL		COMELEC
	May 1-4		April 26-29		May 10
	% Votes		% Votes		% Votes		% Votes
ARROYO	37		37		36.6		36.4
POE	30		31		27.6		33.2
LACSON	11		11		8.1		9.9
ROCO	6		7		4.9		5.9
VILLANUEVA	4		5		4.1		5.6

DE CASTRO	43		46		43.1		42.6
LEGARDA	39		37		35.8		40.1
AQUINO	3		3		3.3		2.8
PAJO	0.4		0.1		0.2		0.1
	% Votes	Rank	% Votes	Rank	% Votes	Rank	% Votes	Rank
ROXAS	42	1 / 2	48.9	1	45.5	1	54.7	1
REVILLA	42	1 / 2	43.6	2	41.5	2	44.6	2
PIMENTEL	32	3 / 4	38.1	3/7	30.9	3/4	38.2	3
MADRIGAL	30	6 / 8	36.0	3/9	30.1	5	37.4	4
GORDON	29	9	33.7	4/14	28.5	6/8	35.9	5
CAYETANO	27	10 /11	28.4	10/17	30.9	3/4	35.4	6
SANTIAGO	32	3 / 4	35.3	3/10	28.5	6/8	34.4	7
LIM	27	10 /11	36.3	3/9	22.8	14/15	31.8	8
ENRILE	31	5	33.9	3/13	25.2	11/12	31.6	9
ESTRADA	30	6 / 8	32.9	4/14	28.5	6/8	31.3	10
LAPID	30	6 / 8	31.3	6/16	27.6	9	31.0	11
BIAZON	26	12	30.7	7/16	25.2	11/12	30.01	12

BARBERS	25	13 /15	29.5	8/16	26.8	10	29.98	13
MACEDA	25	13 /15	30.2	7/16	22.8	14/15	28.1	14
OSMENA	25	13 /15	29.8	7/16	23.6	13	28.0	15
MERCADO	22	16	27.9	10/17	17.9	16	23.4	16
Notes: 1. The SWS Exit Poll percentages were derived by multiplying the SWS published percentages by 81.3%, the SWS estimated turnout rate. 2. The Comelec percentages were derived by dividing the votes received by 35,438,327, the NSCB-estimated number of actual voters.

TABLE 07-12-2
	SWS May 1-4		PULSE ASIA April 26-29		SWS EXIT POLL May 10
	Winner Points	% Votes Points	Winner Points	% Votes Points	Winner Points	% Votes Points
ARROYO	50	20	50	20	50	20
DE CASTRO	50	20	50	14	50	20

ROXAS	50	0	50	10	50	2
REVILLA	50	16	50	20	50	14
PIMENTEL	50	8	50	20	50	6
MADRIGAL	50	6	50	18	50	6
GORDON	50	8	50	16	50	6
CAYETANO	50	4	0	8	50	12
SANTIAGO	50	16	50	20	50	10
LIM	50	12	50	12	0	4
ENRILE	50	20	50	16	50	8
ESTRADA	50	18	50	18	50	16
LAPID	50	20	50	20	50	14
BIAZON	50	12	50	20	50	12

SUB-TOTAL	700	180	650	232	650	150
SUB-RATING	100 %	64%	93%	83%	93%	54%

GRAND TOTAL	880		882		800
OVERALL RATING	90%		90%		82%

Reactions and views are welcome thru email to the author at ra.virola@nscb.gov.ph.

_______________
¹ Secretary General of the National Statistical Coordination Board (NSCB) and Chairman of the Statistical Research and Training Center (SRTC). He holds a Ph. D. in Statistics from the University of Michigan in Ann Arbor, USA and has taught mathematics and statistics at the University of the Philippines. He is also a past president of the Philippine Statistical Association.

Read PART I - PART II - PART III

Posted 12 July 2004.


E-mail the webmaster	Terms of Use	Home • Top of Page
	1997-2013, National Statistical Coordination Board Makati City, Philippines