John Conway – discovering free will (part I)

On August 19, 2004, John Conway was standing with his friend Simon Kochen at the blackboard in Kochen’s office in Princeton. They had been trying to understand how a particular, odd, detail of the world worked. Using a thought experiment involving quantum particles they were investigating the interaction of three important consequences from quantum physics and relativity.

Simon Kochen and John Conway explaining their Free Will Theorem (photo: Princeton University, Office of Communications, Denise Applewhite)

“We were having great difficulty explaining how this particular experiment worked,” says Conway. “We stood back from the blackboard and asked ourselves, ‘What have we proved?’” After a long pause... Conway realised they had proved something entirely unexpected. “I said, ‘We’ve proved that if we have free will, then so do the particles.’ ” With this discovery, and this particular choice of words, Conway and Kochen created one of the most controversial theorems of their careers: The Freewill Theorem.

The story of Conway and Kochen’s theorem gives us a fascinating tour through the history of the quantum world and the intrepid scientists that have explored it. And the terrain they discovered is very strange indeed.

A cloudy and fitful quantum reality

Quantum physics is famously weird, counterintuitive and hard to understand; there’s just no getting around this. So it is very reassuring that many of the greatest physicists and mathematicians have also struggled with the subject. The legendary quantum physicist Richard Feynman famously said that if someone tells you that they understand quantum mechanics, then you can be sure that they are lying. And Conway too says that he didn’t understand the quantum physics lectures he took during his undergraduate degree at Cambridge.

The key to this confusion is that quantum physics is fundamentally different to any of the previous theories explaining how the physical world works. In the great rush of discoveries of new quantum theory in the 1920s, the most surprising was that quantum physics would never be able to exactly predict what was going to happen. In all previous physical theories, such as Newton’s classical mechanics or Einstein’s theories of special and general relativity, if you knew the current state of the physical system accurately enough, you could predict what would happen next. “Newtonian gravitation has this property,” says Conway. “If I take a ball and I throw it vertically upwards, and I know its mass and I know its velocity (suppose I’m a very good judge of speed!) then from Newton’s theories I know exactly how high it will go. And if it doesn’t do exactly as I expect then that’s because of some slight inaccuracy in my measurements.”

Instead quantum physics only offers probabilistic predictions: it can tell you that your quantum particle will behave in one way with a particular probability, but it could also behave in another way with another particular probability. “Suppose there’s this little particle and you’re going to put it in a magnetic field and it’s going to come out at A or come out at B,” says Conway, imagining an experiment, such as the Stern Gerlach experiment, where a magnetic field diverts an electron’s path. “Even if you knew exactly where the particles were and what the magnetic fields were and so on, you could only predict the probabilities. A particle could go along path A or path B, with perhaps 2/3 probability it will arrive at A and 1/3 at B. And if you don’t believe me then you could repeat the experiment 1000 times and you’ll find that 669 times, say, it will be at A and 331 times it will be at B.”

Albert Einstein during a lecture in Vienna in 1921

This was a terrible shock to many physicists. Einstein himself was appalled at the thought of physical reality playing out by chance, and famously said he couldn’t believe that God plays dice with the universe. Instead Einstein, and others, believed that some greater theory would come along to complete the picture drawn by quantum physics (similar to the way Einstein’s theories of general and special relativity boil down to Newton’s classical mechanics when working at the scale of physical world we live in). This bigger theory would again be deterministic, predicting exactly how physical systems would evolve, but would also imply all the results of quantum physics.

The key to this idea was that this larger, complete theory would identify variables that determined the future of physical systems, variables that are hidden in quantum physics. One way to imagine these hidden variables is that they might act like a tiny clock hidden inside quantum particles, each clock running according to the rules of this larger hidden variable theory. Then at the level of quantum physics we would have no idea what time was shown on these hidden clocks, but the position of the clock hand would determine how the particle would behave.

“Einstein said: here’s this little particle, maybe there’s something we can’t see,” says Conway. “Perhaps it’s got a little clock ticking inside it. Let’s say the clock has a period of one hour and if, at the time you perform the experiment, the clock’s in the first 20 minutes then the particle goes to A, and if it’s in the last 40 minutes it goes to B. The particle can see the clock, but we can’t.”

Einstein's (so-called) attack on quantum theory

On 6 May 1935, the headline "EINSTEIN ATTACKS QUANTUM THEORY" appeared in the New York Times. “Einstein was furious,” says Conway. “He dashed off a letter to the paper that very day.” Einstein’s response made it clear that someone (most likely his colleague Boris Podolsky) had leaked their results early. And although he criticised the paper for publishing the article before the research had been through the peer review process – “It is my invariable practice to discuss scientific matters only in the appropriate forum and I deprecate advance publication of any announcement in regard to such matters in the secular press” – he did not dispute the original headline’s claim.

Werner Heisenberg in 1933 (photo: German Federal Archive)

The research in question is now known as the EPR Paradox, named after Einstein and two of his assistants, Boris Podolksy and Nathan Rosen. The result challenged one of the great discoveries of the 1920s, Werner Heisenberg’s uncertainty principle. Heisenberg discovered that there are certain pairs of physical properties that can’t be measured simultaneously: if you measure the position of a particle, you can’t simultaneously measure its momentum (and therefore speed) precisely, and vice versa.

But Einstein was appalled by the “cloudy fitfulness” this view of reality provided. Eventually he, working with Podolsky and Rosen, seemed to discover a loophole. “Imagine you have two particles,” says Conway. “You can measure their joint position reasonably well, and their momentum is zero.” Then whatever happens to this physical system the momentum of one particle will always be equal to the negative momentum of the other particle, as their joint momentum must always be zero. (This is due to conservation of momentum, one of the few classical principles to survive the transition to the quantum world.)

But suppose you split the particles up, one particle travelling one mile west and the other travelling one mile east. “Then you can measure the position of the westward particle very accurately, and measure the momentum of the eastward particle very accurately,” says Conway. “And because they are negatives of each other, you can measure both the position and momentum of each particle as accurately as you like. This contradicts Heisenberg’s uncertainty principle.”

If instead you believe Heisenberg’s principle is true, then the only other option as far as Einstein was concerned, was that the two particles were instantaneously exchanging information. But this directly contradicts one of the fundamental principles of Special Relativity: that nothing, including signals or information, travels faster than the speed of light. Conway says: “They had discovered that quantum mechanics implied a result that appeared to contradict relativity theory, and everybody believed relativity theory,” says Conway.

The EPR Paradox at first appeared to challenge quantum physics and convinced Einstein that the theory was incomplete. But unlike Einstein, the majority of physicists had accepted the cloudiness of the quantum world and did not believe that the particles in the EPR Paradox had a definite value of position and momentum before they were measured; the probabilistic nature of quantum physics meant that these observations could have a range of values and this value was only fixed when the measurement was made. Over the following decades attempts to explain the EPR Paradox would make quantum theory richer and much, much, stranger. These new results would also make it possible to experimentally test the predictions of quantum theory – and rule out the possibilities of hidden variables, and a deterministic universe, once and for all.

Next: The Kochen-Specker Theorem – a first hint of free will...