After ICM’s May Polling for The Guardian Found 3 million More UKIP voters Than in April – An Industry Methodology Rethink Is In Order.

In late April, I attended the Political Studies Association briefing for Westminster lobby journalists ahead of this year’s local elections with the great and good of the British political media to listen to the in depth preview from Professors Colin Rallings and Michael Thrasher of Plymouth University.

Providing an overview of the current political climate, and mainly focusing on the surging support for the UK Independence Party – Professor John Curtice – Professor of Politics at the University of Strathclyde noted that there was a pollster in the room. Professor Curtice said that (after the pending local elections) that “we would soon see whether Survation’s (General Election) 16% or ICM’s 9% support figure for UKIP was the more correct figure!”

That hurt a bit, as ICM, who have been polling for nearly 25 years now, were the most accurate pollster in the 2010 General Election – so I got the idea – ouch!

After UKIP scored 23% of the national equivalent vote share in those local elections (Rallings & Thrasher projection) We’ve been vindicated on this subject (again).  But what’s the reason for this variance between pollsters predicting a surging 4th party in British politics?

Survation’s Unusual Methodology

We’ve taken a lot of stick from commentators and competitors as we have, since 2011 generally had the UK Independence party with a higher percentage than the other members of The British Polling Council.

Anthony Wells, on his UK Polling Report once devoted a whole page of analysis to discuss this. YouGov’s Peter Kellner, to show that even not prompting UKIP in a European Election (as we and ComRes did earlier in the year) makes sense, performed detailed analysis to argue their case, essentially saying that we are wrong as the 2009 European Elections results bore this out.

After the PCC Elections in November 2012 where UKIP scored over 15% in 6 regions in England & Wales  I asked seasoned commentator Mike Smithson of Politicalbetting.com over Twitter if he still thought the unprompted Telephone polls such as Ipsos Mori (showing 3% UKIP national support just before in October 2012) had UKIP’s support level right.

Ben Page, Chief Executive of Ipsos Mori, the UK’s second largest market research firm – swiftly tweeted at me to indicate that he thought indeed they had, referencing the 2010 General Election result as evidence.

It’s not 2010 any more…

It’s that mindset that’s core to my belief that the industry needs a big rethink of it’s methodology – which is essentially unchanged in a decade. All of the polling companies in the UK (including in some aspects our own – I’ll get to that) have methodologies conceived in the days of 3-party politics, and when a Coalition was something your grandparents might just remember that Churchill and Attlee made work.

So what should change exactly?

1) For Westminster “state of the parties” questions polling companies need to stop not mentioning UKIP in the first instance as this serves to undercook the correct level of support – for all four parties – as UKIP takes 2010 supporters from the Conservatives, Labour and the Lib Dems in different amounts. We would expect other firms to begin taking our lead on this, or run the risk of continued inaccuracy. We would argue that the doubling of ICM’s support level for UKIP in one month was due to UKIP simply being understated in their previous polls. The party seem to have reached a tipping point where even though the ICM phone interviewer does not initially mention UKIP, the respondent is more aware that the party can still be selected and this week that’s happened in dramatic style. We discussed this phenomenon at length last year.

This isn’t simply a national poll prediction issue. Lord Ashcroft’s marginals research  – 19,000 interviews in 213 marginal constituencies published in March this year did not prompt for UKIP. We would argue that not attempting an accurate capture of UKIP puts some subsequent conclusions under question – for example the negative conclusions about the 2015 prospects of the Lib Dems. If UKIP were then understated, and UKIP takes more votes from the Conservatives than the Lib Dems in Lib Dem / Tory marginals – how could you decide how many seats the Lib Dems would lose to the Tories in 2015?

2) A rethink of the readjustment method for people who say they “don’t know” or “refuse” to indicate their current voting intention. This polling technique is useful for assessing how undecided voters will actually behave when they come to make a decision in the end. It allocates a proportion of these responders back to the party they voted for in the 2010 General Election. The problem with it now in May 2013 is that this boosts Labour, Conservatives and particularly (usefully) the Lib Dems (who have the most 2010 voters now undecided) but gives UKIP – who performed poorly in the 2010 general election no extra boost.

To give an idea of this method (which most companies use including ourselves) in action, ICM had UKIP at 19% in their poll published today before this step, which reduced their percentage to 18%. This becomes much more dramatic in byelection & constituency polling. We estimate that this cut 2-3% from the true level of UKIP support in the Survation, Ashcroft Polling and Populus’ Eastleigh polling.

3) There are further polling methodology hangovers from the 3-party politics of the last decade that need to change. We are aware that other firms are also now being forced to urgently review these amid this new era of 4-party politics but we’ll leave the less-obvious thoughts for now – do contact us if you have interest!

Damian Lyons Lowe

Chief Executive
Survation Ltd
damian.lyonslowe@survation.com

Survation conduct regular bespoke telephone and online research in constituencies and for local, national and European elections. Survation is a member of the British Polling Council and abides by its rules.