Building trust in statistics

16/12/2008


The UK Statistics Authority (UKSA) has done its bit this week in keeping the government honest, at least when it comes to its use of statistics. Within 24 hours of the Home Office releasing unverified and selective statistics, despite the protests of government statisticians, Sir Michael Scholar (chair of the UKSA) wrote a public letter to the Prime Minister's Office criticising the government's actions. The quick response from the independent body responsible for ensuring the quality of official statistics generated a lot of media attention, not only highlighting the mistakes made in this instance, but also showing the power the UKSA may wield to stop the misuse of statistics.

Hand holding kife

Only complete and verified figures can show whether knife crime is falling.

The Home Office issued a factsheet on Thursday 11 December highlighting a reduction in knife crime in areas taking part in the Tackling Knives Action Programme. The factsheet claimed, among other things, that "the number of teenagers admitted to hospital for knife or sharp instrument wounding in nine [...] police force areas fell by 27 per cent according to new figures published today." Sir Michael stated, in his letter to Downing Street, that these statistics were not only published before the government statisticians had had the opportunity to fully check the quality of the figures, but also despite the statisticians who produced them and the National Statistician Karen Dunnell trying to stop their "premature, irregular and selective release".

The UKSA was created in April this year in order to rebuild public trust in government statistics. A survey of public confidence in statistics in March 2008 found that only one in five people think figures are compiled without political interference, and the UK came last in a 2007 survey within European countries designed to evaluate people's trust in their governments’ statistics. The UKSA does not produce its own statistics, instead it overseas the Office for National Statistics, monitoring and reporting on all official figures produced for the UK. Official statistics are produced according to the National Statistics Code of Practice covering the collection and analysis of data, confidentiality, presentation and public release of statistics. As an independent body reporting directly to Parliament (rather than government ministers), the UKSA evaluates the quality of official statistics, and through its online publication hub provides a source of official statistics that aims to be "accessible, clear and free from political bias".

The government's release of the knife crime figures breached several National Statistics Code guidelines at once. According to the Code, figures should be released only when a project has been completed, they should be released in full, and they should be released separately from political comment regarding policy issues. The government's knife crime campaign, however, will not be completed until March 2009, and the published figures, according to Sir Michael, were unchecked and selected to make a political point. The government's press release, in which the figures were mentioned, clearly advocates the government campaign, and the release date was chosen to coincide with the launch of the No to knives campaign endorsed by Gorden Brown. What's more, the figures were released against the advice of the statisticians involved in producing them and the National Statistician who, according to the Code, should be consulted before publication of any figures. While opposition politicians were quick to detect the whiff of unashamed spin, Sir Michael, in his letter to Downing Street, pointed out that actions like these are likely to further undermine the public's trust in statistics.

Yesterday Home Secretary Jacqui Smith apologised for the release of the statistics, admitting that officials had been "too quick off the mark with the publication of one number" relating to knife crime. Sir Michael said he had received a letter from the prime minister's office which acknowledged some errors. Not the unreserved apology statisticians would have wished for, but nevertheless a high-profile acknowledgement of the UKSA's concerns, and an embarrassing episode for the government. Perhaps this strong public action by Sir Michael and the UKSA will reassure the public and help to rebuild trust in statistics, despite the government's actions. At the very least, debates like these will remind the public that not all statistics are bad and that some simple checks can go a long way when it comes to assessing their quality.