Weather forecasters have it hard, especially in England where the weather is such a hot topic. Whenever they get it wrong, we deem them incompetent. How come that in the age of space travel we still can't get a decent weather forecast?

English weather is hard to predict.

The answer is that the weather really is very hard to predict. That's not because we don't know how it works — we do. We understand the laws of nature that govern all the factors that make the weather, and, as physicists like to boast, these laws can be beautifully simple. But this doesn't mean that the outcomes of the laws, the weather we actually see, can't still be very complex. "[Meteorology] is the study of 'happenings', if you like," explains John Barrow, mathematician and cosmologist at the University of Cambridge (you can watch our interview with him below). "It's quite different from studying the laws that govern the happenings."

The idea that even simple rules can give rise to unpredictable behaviour wasn't fully appreciated until the late 1970s, with the advent of personal computers. "There were computers before of course, but they were incredibly expensive, and they were the domain of large research groups, doing blockbusting problems like exploding stars, modelling the economy, or bombs," explains Barrow. With personal computers mathematicians could for the first time study complicated problems, which you can't analyse with pencil and paper. "You can't solve these problems in 30 minutes, under examination conditions, so they don't get studied at school," says Barrow. "At the end of the 1970s and in the 1980s you saw the beginning of the study of complexity, the study of chaos."

We've all got our own ideas of what chaos is, but in mathematics the word has a precise meaning. Imagine you are trying to predict the path a ball on a billiard table will take in response to a push. The rules at play simple: the ball's acceleration is equal to the force applied divided by its mass (that's Newton's second law of motion) and when it hits a side, the angle of reflection is equal to the angle of incidence (strictly speaking you also need to include the effect of friction, but that's not too hard to do).

The trouble is that in your average pub setting you can't easily measure the exact amount of force applied to the ball, the exact angle with which it hits a side, and so on. As you make your calculations, this small initial uncertainty can snowball, so that pretty soon your prediction may become so uncertain as to be useless. That "sensitivity to ignorance", as Barrow calls it, is the hallmark of mathematical chaos. It's also known as the butterfly effect: the idea that the small air disturbance caused by a butterfly's beating wing can develop into a tornado half way around the world.

"The weather is also [chaotic]," explains Barrow. "It's not that we don't understand the weather and all the bits of physics that go into it. We do, and we have very good equations to actually predict how the weather will change. But we don't know the state of the weather now, in every position, with perfect accuracy. We might have weather stations every 50 or 100 miles, but the scope for little variations in-between is great enough that when you run your computer simulation, if you interpolate a little bit differently between the weather stations, you can end up getting a very different forecast for tomorrow and certainly for next week."

Order in chaos

At this point you may raise an interesting question. If the weather is so hard to predict in the short term, how come we can all make reasonably accurate longer-term predictions? We all know that, in the UK, the winter will be colder than the summer, and that autumn and spring are likely to be windier.

This is Hurricane Isabel, which devastated parts of the US in 2003, as seen from the International Space Station. Forecasters saw this one coming, but in general the weather is very difficult to predict.

The point is that, when we say that the summer is warmer than the winter, we are not talking about individual day-to-day fluctuations in temperature, but about the average temperature measured over time and over a whole region. Averages can show a predictable pattern, even if the numbers that make them up vary unpredictably.

A more astonishing example comes from the air molecules in your room. The motion of an individual one is completely unpredictable, not only because of sensitivity to ignorance, but also because there is genuine randomness involved. Yet, you don't notice this crazy motion. You can even come up with things like Boyle's law, which says that the pressure of a gas, times the volume it is contained in, divided by its temperature is always the same: it's a constant, despite the fact that individual molecules are moving randomly. Pressure and temperature are again average quantities: the temperature is the average speed of the molecules, and the pressure is the average force on a unit of area of the container's walls. "It's possible that, even though the individual molecular motions are chaotic, the average behaviour is entirely predictable and changes very slowly," says Barrow.

There are many processes that, like the weather, exhibit chaos, but not all these processes are stable in the sense that average behaviour is predictable. It's obviously useful to know which ones are. "At the moment there are a few processes we are particularly interested in," says Barrow. "One is the long-term evolution of climate. The other, over shorter timescales, is the evolution of an economy." The question is whether these are locked in a long-term stable pattern or whether they are going to spin out of control.

Chaos is one way in which complex outcomes can arise from simple laws. "If someone comes along and asks me, as they often have, 'Is the world simple or is it complicated?', I think the interesting answer is, if you look at the world at the level of the laws of nature, it's simple," says Barrow. "But for people for study the outcomes of the laws, the world is complicated."

Further reading

To find out more about how the laws of nature relate to their outcomes see Stuff happens: Laws versus outcomes.

About this article

This article is part of our Stuff happens: the physics of events project, run in collaboration with FQXi. Click here to see more articles and videos about the laws of nature and their outcomes.

Marianne Freiberger is Editor of Plus. She interviewed John D. Barrow in December 2015.