The previous piece on opinion polls looked at who commissions opinion polls and what might be their motivations. This second piece is about how opinion polls are conducted and how reliable they are.
Let us start by going back to the General Election of 1992 again – when all the opinion polls about the election were wrong and the Conservatives won the election with a modest absolute majority. One of the first defences of the pollsters was to suggest that respondents had simply lied about their intentions. Maybe they had – but if this were true then why have polling, why have market research at all?
This gave rise to the phenomenon of the ‘Shy Tories’ – people who intended to vote Conservative but would not admit to it. Two reasons for this were put forward. The first was that voters were ashamed to admit to voting Tory. Dismayed by Labour’s defeat, some left wing commentators used this line to find someone to blame for it. The second was that many Conservative voters kept their politics private, and/or were not happy to share their voting intentions with polling organizations. If this were the case, it was not a noticeable feature at previous elections, so perhaps marks the start of some distrust of ‘the establishment’.
A long term response of polling organizations to this is the use of MRP. Mentioned previously, this (and similar methods) allow pollsters to adjust their results (eg for under/over representation of particular demographics) before publication. These are professional organizations and the results will be adjusted according to standardised criteria – but how many people think that the results of an opinion poll on the tv or newspapers are anything but a straight count of responses?
Next, how many respondents to ask? Without going into standard deviations etc it is accepted that just over 1000 respondents is the minimum for the result to have a fair reliability and margin of error. Note that opinion polls in the recent Makerfield (Andy Burnham) by election were way out. Opinium had a sample size of only 543, which may well explain it.
Please note the careful use of the word ‘respondent’ in the previous paragraph. The number asked will always be greater than the number of respondents. The “ won’t says” and the “dont knows” are a problem for the pollsters and are rarely reported. The figures can be found buried in the published spreadsheets if you know where to look – and in the Opinium/Makerfield poll above there were don’t know/wont vote respondents totalling 20% of the already small sample. The poll was commissioned by ForwardDemocracy and StopReformUK.
During the1992 UK election, a largely unnoticed feature of opinion poll results was the very large number of don’t knows – another factor as to why the polls got it wrong, presumably.
Number of respondents apart, who to ask? The elusive representative sample is the aim, but unachievable. Internet samples are cheapest, of course. But to get to respondents the pollsters have to access potential respondents who are a) internet users and b) interested enough to have signed up to a panel or to newsletters etc. This leads to a self selected sample. Nonetheless, this method is widely used, often alongside (more expensive) face to face interviews in order to compensate for demographic mismatches and to ‘balance’ the sample. The organizations, in other words, try to approximate to a representative sample.
And now, what to ask. This sounds simple – ‘Who are you going to vote for?’ No. The standard question starts ‘If there were a general election tomorrow….’ Market researchers will tell you that the use of the word ‘If’ immediately turns this into a hypothetical question – to be avoided whenever possible.
Then there is the issue of prompts. ‘Which of the following would you vote for?’ may be followed by a list of political parties, but how many? Is the Monster Raving Loony party included? No. More seriously, few pollsters use Yourparty (Corbyn’s vehicle) or Restore. Yet it is well known that respondents will tend to select a response from a list rather than to offer their own alternative. So it is likely that opinion polls understate the level of support for emergent smaller parties.
Of course, the pollsters do not need to use prompts, and some do not. So respondents will offer their own thoughts, often by way of the name of a party leader rather than a political party.
So what to make of the polls? Indicators of opinion, perhaps. Unbiassed? No, because figures may be adjusted for demographics according to assumptions which may themselves have bias. Forecasts, no. Unbiassed? No, simply because not all polls are published and one reason for this is that they don’t show the ‘right’ outcome. Reliable? Varies. Unbiassed? Ask who has commissioned and/or paid for the poll. Your author tends to disregard polls on the basis that they are at least in part designed to shape opinion, rather than reflect it.
Some circumstantial evidence for this exists. Opinion polling on ‘controversial’ subjects (typically where the populus may well disagree with the government) is thin on the ground. Such subjects might be capital punishment, wars, perhaps abortion, immigration. Look, for instance, at the yougov graphic from a month or two ago. Respondents are asked about levels of immigration – but not about immigration policy. You gov do also poll respondents on how strict immigration enforcement policy is, but again, not on policies such as right to remain or deportation. Likewise, yougov survey on reproductive rights and the right of women to have an abortion. They find (recently) that 90% agree with this right, but do not go on to ask under what circumstances. Many would see a difference between a very young girl having an early abortion and a woman terminating after 30 weeks.
Polling and market research is an imprecise science and should be treated as such. Political opinion polling have many funders and a variety of motives. Read with a moderate dose of scepticism.
© Hongkonger 2026
