Statistics and uncertainty

Based on the talk by Mark Pont from the Office for Statistics Regulation.

Statistics aren't certain facts but are often presented as if they were. You want to be clear on the caveats but you don't want to undermine trust.

Always ask yourself, or the researchers you are working with: are there uncertainties involved in the mathematical or scientific research that should be highlighted to the reader?

Communicating uncertainty is hard, but it is very important to be honest about the limitations of any statistics or results. Here are some recommendations from Mark's talk, and from a very useful review on presenting uncertainty, produced by Full Fact in 2021:

• Be transparent about the quality and limitations of the data.

• Be specific about what exactly is uncertain – for example is the uncertainty due to insufficient data? The purpose of presenting uncertainty is to avoid your audience drawing misleading conclusions, not to make them think that nothing can be trusted.

• Not every number needs to have the word "estimate" around it. Intersperse this occasionally, to clarify and give the narrative that these numbers are not absolute facts. Rounding numbers is also helpful as it avoids spurious accuracy.

• Indicate uncertainty in existing data using numerical ranges in brackets, after the main value. For example, say "unemployment is estimated at 3.9% (between 3.7% and 4.1%)" at least when you introduce the figure.

• In case of future predictions, use verbal expressions to indicate the general direction of travel, but supplement these with numerical probability ranges, and wherever possible access to underlying data. Do say, for instance, "global warming is likely (66% chance) to reach 1.5°C between 2030 and 2052 if it continues to increase at the current rate".

• Take care when using large numbers and jargon. If jargon is necessary, remember to explain it. When using large numbers, remember that the difference between 1 million and 1 billion is clearer if the latter is expressed as 1,000 million.

Charts and visualisations

Based on the talk by Martin Ralphs from the Office for National Statistics.

• Simple is (usually) better than complex. Charts should work on their own, without having to read the stuff around it. This is particularly useful if someone is likely to reuse your chart or graphic elsewhere.

• One main message per visual: you only have 10-15 seconds to get the message across.

• Have a good descriptive title that conveys your main message. If a statistical description is necessary it could sit beneath this main title, or be in the caption.

• Try to bring uncertainty into any graphics: Uncertainty is your friend in statistics. Try to embrace it and embed it.

• Accessibility really matters.

Accessibility

Based on the talk by Hannah Thomas from the Government Analysis Function.

Digital accessibility is all about making content published online easy to access and use for all users, regardless of impairment, medical condition or disability.

• All non-text content should have a text alternative that serves the equivalent purpose. For example, what is the message of your chart? Answering this question will make your chart better. Include a fully descriptive text alternative directly under your chart, before information on sources, notes, etc.

• Do not use colour alone to communicate a message. If you can't differentiate between the elements in your charts legend, you won't be able to differentiate between the categories of your data.

• Use 3 to 1 contrast ratio for adjacent colour elements in visualisations.

Here is some guidance specifically on the use of colour and on accessibility more generally.

Useful resources

• Uncertainty
  • Guidance on writing about and presenting statistics from the Office for National Statistics
  • Uncertainty Toolkit for Analysts in UK Government (and one page summary)
  • Guidance on communicating quality, uncertainty and change from the Government Analysis Function
  • How to communicate uncertainty from FullFact
• Accessibility and visualisations (all from from the Government Analysis Function)
  • Guidance on charts, tables and the use of colour
  • Guidance on infographics
  • Guidance on communicating uncertainty
• Trustworthiness
  • Five rules for evidence communication by Michael Blastland et. al, Nature, November 2020
  • Transparent communication of evidence does not undermine public trust in evidence by John R. Kerr et al, PNAS Nexus, December 2022
  • Communicating the coronavirus crisis, plus.maths.org, March 2020 – Our interview with David Spiegelhalter where he explained trustworthiness and intelligent transparency.
• Other resources
  • How to write numbers – a free online course aimed at journalists from the Royal Statistical Society
  • Resources for journalists from the Winton Centre for Risk and Evidence Communication