Stirring the electoral soup

The opinion pollster's job has been likened to that of a chef tasting soup: it is more important to stir thoroughly than to take a large spoonful. And, even though they didn't agree on the best method, the six organisations producing polls in the run-up to the general election on May 5th were all doing their best to stir well.

They had to decide how heavily to discount the voting intentions of people who claim to have voted Labour last time (simple arithmetic shows that large numbers of them must be misremembering), whether to remind respondents of the names of the three main parties (otherwise, some have been known to forget about the existence of the Liberal Democrats) and how to correct for those unwilling to admit their true voting intentions (some find it embarrassing to say they support an unpopular party).

In the event, the eve of poll predictions were all remarkably close - and remarkably accurate - putting Labour on 36-38%, the Conservatives on 32-33% and the Liberal Democrats on 21-24%. After a poor performance in the previous few elections, the industry will see this as a vindication. In 2001 the pollsters overestimated Labour's share of the vote by around 6% and in 1992 only Gallup predicted a Conservative lead, and then of only 0.5% (the voters made it 8%).

Since then they have introduced a bewildering array of corrections in an attempt to avoid another disaster. For example, in 1992 MORI sent its interviewers out to named constituencies, telling them to interview certain "quotas" of young, old, male, female and so on. But a constituency is a big place, meaning that their interviewers had latitude consistently to choose houses with short driveways rather than those with long, and houses in urban areas rather than rural. The resulting bias led to a substantial underestimate of Conservative support. MORI now sends its interviewers to census enumeration areas, which are very much smaller than constituencies - and it reckons that this single change has increased their finding of Conservative support by as much as 2 percentage points.

does my vote count?

Although there is no agreement between the various polling organisations about which corrections to use, they all adjust for a wide range of factors, and reweight and even discard many responses. For example, MORI completely discounts responses from anyone self-described as less than certain to vote, while the other pollsters, apart from YouGov, all attempt to weight responses by likeliness to vote. YouGov doesn't attempt to correct for unwillingness to admit a preference for an unpopular party (it only polls on the internet and reckons that we tell the truth to our computers); the others, except MORI, all do. And all - again except for MORI - weight responses by past voting record, attempting to correct for all those people who "remember" voting Labour.

"MORI's preliminary analysis puts the pollsters collectively out by 1.3%, which is pretty good," says Dr Paul Baines, a researcher from Middlesex University who is currently on secondment to MORI. "This is not a fluke: it's because of the effort they put in."

Although the opinion polls were very accurate, they were used for predictions of the number of seats for the various parties that were not so impressive. But, says Dr Baines, predicting the makeup of Parliament is not the pollster's job. "The polls are about vote share, not seat predictions," he says. Media predictions of a Labour majority between 100 and 160 were predicated on a uniform swing - "but the pollsters all know that uniform swing doesn't operate, because the marginals are hotly contested."

Even if it wasn't their job to make predictions, some pollsters couldn't resist. "I have Bob Worcester [Chairman of MORI] on tape offering me any money I like on a three figure Labour majority," says Professor Leighton Vaughan Williams, Director of the Betting Research Unit at Nottingham Trent University. Professor Vaughan Williams, guided by the betting markets, was predicting a majority of around 60 seats, but, sadly for him, he couldn't take up the offer, as it was made on air and on-air gambling is illegal.

The pollsters may have played their part of chef to perfection, and stirred well before tasting. But once the favourable geographical distribution of Labour's voters and the ferocity of the first past the post system were factored in, the electorate looked a little less like soup, and a little more like stew.

If you have anything to say about this or any other topic that might be of interest to Plus readers, e-mail plus@maths.cam.ac.uk. Let us know if you are happy for your email and our response to be published in Plus. (We may edit emails before publication.)