This article first appeared on the FQXi community website. FQXi are our partners in our Information about information project. Click here to read other articles related to John Wheeler's concept of "it from bit".

David Wolpert (Santa Fe Institute)

David Wolpert has his tongue mostly in his cheek when he says that his theory puts limits on religions. "There cannot be two gods, both of which can perfectly observe everything, or both of which can make anything happen that they want," he says. In fact, there cannot even be two gods who both have perfect memory. In short, it is mathematically impossible to have a polytheistic universe with all-knowing gods.

Such statements (made with "very heavy air quotes") are the consequence of limits that Wolpert has derived on the ability of any so-called "inference device" (a machine which can observe and/or control the world around itself). Rather than analyzing various pantheons, Wolpert is interested in the mathematical restrictions on what information a device can have concerning a system and the implications those restrictions have for the laws of physics. He has discovered constraints that look remarkably similar to those that physicists have long been familiar with from quantum theory, for instance, Heisenberg's Uncertainty Principle. Wolpert's findings could help physicists finally understand the origin of this famous fuzziness in the quantum realm. It could as arise from unavoidable formal restrictions on what can be known.

Wolpert's fascination with foundational questions began when he was studying for a PhD in physics at the University of California, Santa Barbara. "Probably everybody, as a grad student in physics, has this dream that they will figure out the ultimate theory of everything," says Wolpert, who is now at the Santa Fe Institute in New Mexico.

In 1998, while still a graduate student, Wolpert attended a workshop on complex systems at Santa Fe, where he met John Wheeler, an eminent physicist at Princeton University, who had already championed the idea of linking information with the quantum. In his just published autobiography, Wheeler had said, "I suggest that we may never understand this strange thing, the quantum, until we understand how information may underlie reality." Despite his formidable stature as a physicist, Wheeler was interested in what Wolpert had to say. "I was sort of taken aback that he wanted to go on a walk and know more about what I, a grad student, thought about these things," says Wolpert.

Inference Devices

Wolpert can't recall exactly what he told Wheeler—but it was likely about his early attempts to understand how we can make predictions about physical reality. Wolpert realized that any attempt to "know" reality involves either observing physical quantities, remembering past values of such quantities and/or predicting or controlling their future values. He found that all these processes—different ways of having "semantic information" about the state of the universe—share some common mathematical features. He called any device that implements those features an "inference device" (see the article here).

Are there clouds in the sky? You may think you know the answer, but can you know someone else's answer to the same question? (Image by Chatennoir - CC0)

Inference devices are physical machines that obey the normal rules of mathematics and logic. The only a priori restriction on them is that they exist in the same physical universe as the system they want to know about. In Wolpert's model, for an inference device to know something about the state of a system they must be able to correctly answer YES or NO to any binary question concerning the state of that system. The question could be something as simple as: Are there clouds in the sky? If its answer is always correct, no matter what the actual state of clouds in the sky, then the device knows whether there are indeed clouds in the sky.

Wolpert proved that it's impossible for one inference device (device A) to both know its own answer to an arbitrary question and to also know the answer to the same question by a different inference device B. The same holds true for B. This limitation holds no matter what the laws of physics actually are—the limits apply in any universe.

Surprisingly, when this and similar limitations are expressed in terms of probability distributions, they offer intriguing connections to quantum mechanics, the theory that governs the atomic realm, and which appears to impose laws that strain our everyday sense of logic. For instance, according to Heisenberg's uncertainty principle, it is impossible to precisely measure pairs of certain properties of a quantum particle—its position and momentum, say—simultaneously. Formally, this is very similar to limitations on how accurately inference devices A and B can simultaneously know one another's states.

To understand this connection between inference devices and quantum rules, suppose that two inference devices A and B are both asked a question about the state of one another, as before. But instead of analyzing whether the precise answers A and B give are always correct, instead calculate,"What is the probability that A is correct?" and "What is the probability that B is correct?"

One can take this kind of analysis one step further and calculate the best possible values of the probabilities of A knowing B's answer and B knowing A's answer, simultaneously. "You can actually prove that that product of probabilities obeys something that looks very much like the uncertainty principle of quantum mechanics," says Wolpert.

In other words, there's a limit to how well A can know B's answer and B can know A's answer simultaneously (just think of A and B as gods, and you have your argument against a polytheistic universe).

Physicist Philippe Binder of the University of Hawaii in Hilo says that Wolpert's work is "very good and of foundational importance." Writing in the journal Nature soon after Wolpert published his work on inference devices, Binder said that Wolpert's results "slam the door" on scientific determinism, as first articulated by Pierre Simon Laplace in the nineteenth century.

Destroying Demons

Laplace (1749-1827)

Laplace introduced the idea of a hypothetical super-intelligent "demon" that can predict the future state of the universe if it has complete knowledge of the current state. But, by using Wolpert's framework, one can model the current avatar of any such demon as an inference device, while the future avatar of the demon must be regarded as another inference device. (This is because the state of the universe changes from one moment to another and inference devices can only be defined within the context of one given state of the universe.) Wolpert's work shows that the current avatar of the demon cannot know with absolute certainty the future avatar's knowledge about the state of the universe. In other words, Laplace's demon cannot predict without uncertainty. (This is true even if one assumes a classical, finite, non-chaotic universe, as Laplace did.)

The work has implications for our ability to come up with the kind of theory of everything that Wolpert longed for as a graduate student. As Binder pointed out in his essay, Wolpert's work suggests that "the entire physical Universe cannot be fully understood by any single inference system that exists within it"—including any particular human. So, at best we can hope to a have a "theory of almost everything."

FQXi member Paul Davies, of the Arizona State University in Tempe, Arizona, says that Wolpert tackles the question of how something that is a part of the universe can understand the whole. "It is getting one level deeper to the nature of reality," says Davies.

Wolpert is now using a $50,000 FQXi grant to investigate whether the intriguing connection he uncovered between inference devices and the uncertainty principle in quantum mechanics can be tightened and even extended. If it can, that would help physicists understand the origin of one of the basic principles that governs reality.

Inference devices could also help to reveal the links between Claude Shannon's theory of "syntactic information" and thermodynamics. The best example of that relationship is the Landauer principle—due to physicist Rolf Landauer—which says that you cannot reset even a single bit of information (whether it is a 0 or a 1) without releasing heat. In separate work, Wolpert has extended that work beyond just erasing a bit, but an arbitrary stochastic process evolving a system forward in time. Now he wants to figure out whether this result has any bearing on his work on inference devices, which involves semantic rather than syntactic information. "Information has ultimately got to involve both," says Wolpert.

But even a monotheistic God may have to accept some limitations when it comes to such information. With his tongue back in his cheek, Wolpert says that God can get the universe rolling, but can't interfere with its functioning afterwards. "Or, after someone else gets the universe going, you can interfere, but you can't do both," says Wolpert. Deism is allowed, he says, but not the traditional Abrahamic God.