This article is the third in a four-part series on quantum electrodynamics. You can read the previous article here and the next one here. You might also want to read our interview with Freeman Dyson.

Where do great ideas come from? One way to find out is to ask someone who's had quite a few himself, so when we met the legendary Freeman Dyson we did. "It's true of almost every great idea that you really don't know afterwards where it came from," he said. "Our brains are random, that's of course a wonderful trick of nature for being creative. They don't have to be programmed, they can invent things just by random chance. So all really good ideas are accidental; there's some random arrangement of things buzzing around in somebody's head, and it suddenly clicks." (You can read extracts from our interview with Dyson here.)

At the end of the 1930s theoretical physics was in bad need of some such random clicks. Physicists had attempted to apply the new theory of quantum mechanics to electromagnetism, one of nature's fundamental forces. The result was quantum electrodynamics (QED), a theory describing the interaction of matter and light. The trouble was that almost any calculation you'd care to make using QED returned infinity as an answer — for all practical purposes the theory was useless.

When Dyson boarded the Queen Elizabeth from England to New York in 1947, to become a student at Cornell University, he was already acquainted with quantum field theory. He had been inspired by his Cambridge teacher Nicholas Kemmer and a rare German text book, Gregor Wentzel's Quantum theory of fields, which he still keeps in his office at the Institute for Advanced Study in Princeton. "It was a precious treasure that book, there were only two copies in England at that time, and I had one of them."

But quantum field theory (QFT) wasn't particularly popular in the US. "[US physicists] considered it like an Italian opera; extravagant and irrational entertainment. America was very empirical, so the people I worked for, Hans Bethe and Richard Feynman, had no use for it."

QFT did indeed make some esoteric predictions. Heisenberg's famous uncertainty principle implied that particles, such as photons or electrons, would be constantly created out of nothing, popping in and out of existence like bubbles in a bubble bath. During their briefest of lifetimes these virtual particles would affect other particles and their interactions. In fact, the electrostatic repulsion between two like charges was explained in this way: an electron would emit a virtual photon which would in turn be absorbed by another electron, the interaction pushing the two particles apart.

Virtual reality: the Lamb shift

All this was theory, but in 1947 experimentalists observed effects of virtual particles. "There were experiments in Columbia University, which were the driving force," says Dyson. "Isidor Isaac Rabi and Willis Lamb and various other people, using the new technology of microwaves which came up during the war, [were able to] measure atomic levels very accurately. They got, for the first time, a really accurate picture of the hydrogen atom, which was supposed to be the simplest atom." What they found was that the energy levels of the hydrogen atom were slightly different from what old-fashioned theory, not taking account of virtual particles, predicted. "That was the famous Lamb shift," says Dyson. "Everybody was excited about that; for the first time a real discrepancy [between experiment] and theory." (There is a nice discussion of the energy spectrum of the hydrogen atom here. The Lamb shift represents a small correction to this spectrum.)

People immediately suggested that the difference might be due to the effect of virtual particles affecting goings-on inside the hydrogen atom. And when Hans Bethe performed a quick, and very approximate, calculation on a train ride back from a conference, he found that this might indeed be the case.

This explained the Lamb shift as far as the physics was concerned. But the mathematics was still an "unholy mess" in the words of the historian of physics David Kaiser (who has written an excellent book including an introduction to this topic).

The trouble was that in calculations the effect of virtual particles would always come out to be infinity. For example, the effective mass and the effective charge of an electron, which you could measure in experiments, were actually made up of two contributions: the so-called bare mass and bare charge, and , and the contributions to mass and charge from the virtual particles, written as and

and

It was the contribution terms and that turned out to be infinite. Thus, any calculation trying to take account of virtual particles would necessarily blow up too. Something was horribly wrong.

"All the giants from the old times were still around, people like Werner Heisenberg and Erwin Schrödinger and Paul Dirac and Robert Oppenheimer," remembers Dyson. "[They] all thought that we needed radically new physics. They all had their theories of changing the whole basis of physics, introducing completely new ideas."

"But they were all wrong. Everyone of them turned out to be wrong."

God is great!

The idea that saved the day, without throwing out QED, was surprisingly simple: stand back and work with what you see. Since you can never actually catch a bare particle on its own, without its virtual cousins, throw away that Platonic idea and work with effective quantities instead. "The infinities only lay in the artificial separation of the bare particle from the physical particle," explains Dyson. "[But] the mass of a bare particle has no meaning from an operational point of view. So forget about that and only calculate things you can observe. It works like magic."

Simple as it was, the idea of swapping bare for effective quantities had already been around since the late 1930s, suggested by Hendrik Kramers and Victor Weisskopf. But it wasn't as easy to put into practice as it may seem: rearranging complicated calculations, you were juggling with infinities that could still blow up in your face if you weren't careful. And all the while you had to comply with Einstein's special relativity which imposed constraints on the calculations.

It was Julian Schwinger working in the US and, independently, Sin-itiro Tomonaga in Japan, who finally cracked in the late 1940s the calculations that exactly described the Lamb shift. Schwinger's calculation inspired Rabi to exclaim that "God is great!"

Yet, more than a whiff of unholiness remained. The calculations were still incredibly complicated, making QED no more useful than it was before. As Dyson recalls, Schwinger's personality didn't help. "He was this young prodigy, and he came and gave a talk here at Princeton, explaining his calculation. And Oppenheimer said 'You know when other people give talks it's to tell you how to do it. But when Julian Schwinger gives a talk, it's to tell you that only he can do it.'"

What is more, Schwinger and Tomonaga's calculations were still simplified. The particular infinites they had managed to tame were a result of the fact that a single virtual particle comes with a potentially unbounded amount of energy (see the previous article). To make things easier, they had considered interactions that involved only one or two virtual particles, managing to get the corresponding infinities under control. Whether the method would work once you considered the potentially unlimited amount of virtual particles was anyone's guess.

The great idea was due to another prodigy working at the same time, Richard Feynman, and it was Dyson who put it on a secure footing. We'll explore it in the next article.

---

Further reading

David Kaiser's book Drawing theories apart explores the development of Feynman diagrams and contains an excellent introduction to the topics discussed in this article.

---

About this article

Marianne Freiberger and Rachel Thomas are Editors of Plus. They interviewed Freeman Dyson and David Kaiser in February 2013. They are hugely grateful to Jeremy Butterfield, a philosopher of physics at the University of Cambridge, and Nazim Bouatta, a Postdoctoral Fellow in Foundations of Physics at the University of Cambridge, for their many patient explanations and help in writing this article.