Tuesday 10 November 2009

Our Survey Said. . .


Following my earlier blog posting, I've now received a reply from the office of Peter Lilley in response the questions I raised following last weeks commons debate.

1. Can you please point me towards the Pew survey giving the UK 15% of people in the UK "take (climate change) seriously or are seriously concerned" figure you quoted. I can only find data from October relating to Pew's USA surveys. I would be interested to see the data showing how this percentage figure is much higher in most other countries.

2. Is the Science Museum survey that Mr Lilley referred to in his speech last week the "Prove It" survey that is available for people to enter online or can he point me towards other survey results that, as he stated in the house, resulted from questionnaires being completed by people who had actually walked through the physical exhibition space at the museum. I want to make sure that I get this clear as the Daily Telegraph reported the same "6:1" ratio when it reported that the online survey had been hijacked by sceptics.

And the answers? Here they are paraphrased with my comments.

1. That 15% Pew figure
The 15% figure quoted by Mr Lilley did not come from a Pew survey. It came from a Daily Telegraph article reporting the results of the 2009 HSBC Climate Confidence Monitor that has been commissioned for the past three years by the HSBC Climate Partnership. The survey questions 1,000 people in each of 12 countries worldwide and shows recent trends in public attitudes towards climate change in developed and developing countries.

If you look at the interactive version of the results, you can see that, in 2009, 15% of those surveyed "agree or strongly agree that climate change and how we respond to it are among the biggest issues that they worry about today". This is down from 22% in 2007 and 26% in 2008. Although the question posed is slightly different from the "take (climate change) seriously or are seriously concerned (about climate change)" reported by Mr Lilley (which ignores the existence and relative significance of other important issues that people have to worry about), we should indeed perhaps be concerned by this downward trend and ask ourselves if messages about climate change are being communicated effectively.

It is important however to look at the full content of the interactive report rather than simply cherry-picking results. The same report throws up some strange paradoxes and inconsistencies in the UK results. According to the report, only 9% of people in the UK agree or strongly agree that people or organisations who should be doing something about climate change are doing enough. 29% agree or agree strongly that the UK should go further than existing emission reduction targets.

The report concludes that "Consumers feel strongly about a deal (at Copenhagen) being reached. People in all countries support a budgetary commitment to tackling climate change despite the increased importance of competing priorities. We are seeing a demand for carbon dioxide emission reductions across the globe. Respondents everywhere believe that all countries must reduce their emissions regardless of whether they are in emerging or developed regions. A global call to action is clear"

2. That Science Museum "Prove It" Survey
Mr Lilley has confirmed "I suspect the Science Museum figs came from Wattsupwiththat blog last week". I'd never heard of this blog so I checked it out. The entry relating to the Science Museum survey is tailed by comments about how "robovoting" can be used to boost the "count me out" result. Once comment by "lihard" attributes 1,000 of the "I'm not convinced" votes to their own robovoting script.

It seems that the online poll was poorly designed, was open to abuse from all sides and made no attempt to check the validity of names and email addresses entered or whether they were from UK respondents. In other words, the poll has been exposed as worse than useless. It should never have been cited by an MP in a commons debate to suggest that the UK public are not convinced by the scientific evidence presented to them.

So, basically, ignore the online Science Museum Poll. It's broken. And ignore Mr Lilley's assertion that the poll demonstrates a sceptical public

1 comment:

  1. I've been contacted by the Science Museum who have stressed that the poll website has now been modified to prevent "robovoting" or multiple votes from the same computer or network. You can find their statement here:

    http://www.sciencemuseum.org.uk/about_us/press_and_media/press_releases/2009/10/Prove%20It%20Results.aspx

    However, this statement doesn't address the concerns I raised about concerted efforts to skew the vote by people who haven't visited the exhibition. It's still possible to vote without looking at the evidence presented online.

    So, as a useful online poll, it still has limitations and certainly should not be used by MPs to illustrate or justify points they make about a "sceptical public".

    I would still love to see the results of a poll taken as people leave the exhibition.

    ReplyDelete