Albert Einstein's impact on our understanding of the Universe is so widely regarded by both the physics community and the general public, that his name is now a synonym for genius. He even once confessed: "I am no Einstein." So it is reassuring and fascinating to gain an insight into how he explored new scientific territory, how he responded to evidence that disagreed with his theories, and that heading down dead-ends is an important part of even the greatest researcher's work.

This insight was made possible by the Einstein Papers Project, a joint initiative by Caltech and the Hebrew University of Jerusalem to make all of Einstein's scientific and personal writings publicly available online. A full archive of his writings and documents from his early life to 1923, with English translations, has just been released, allowing all of us a chance to look over Einstein's shoulder as he developed his revolutionary theories about the photoelectric effect, and special and general relativity.

Last year we went along to a conference that brought together cosmologists and philosophers to discuss some of the big questions in cosmology. Physicist Cormac O'Raifeartaigh, from the Waterford Institute of Technology in Ireland, told us about his recent and unexpected discovery of an unpublished paper by Einstein in the digital archive. The paper contained a model of the Universe that was dramatically different from any of the others that Einstein was known to have studied. And although it wasn't the model that Einstein, or the rest of the scientific community, eventually settled on, the discovery is important as it sheds light on how Einstein's thinking about the Universe changed, as he was dealing with some of the big questions in cosmology at the time. O'Raifeartaigh believes the manuscript dates from 1931, "an extremely interesting moment in cosmology, when the first evidence for an expansion [of the Universe] has emerged,".

The story starts in 1917 when Einstein published his first model of the Universe. This model, like all models of the Universe, is a solution to the field equations in his theory of general relativity. Einstein's first model was for a static universe, one that did not change in size (indeed in his model it had a constant radius R) and did not change in density. "When Einstein put together his famous static model there was no evidence for an expanding universe," says O'Raifeartaigh. "Even the idea would have seemed wacky, so everyone assumed the Universe was static, including Einstein."

In the following years cosmologists discovered there was a problem with Einstein's model: it wasn't stable, the slightest nudge would set the Universe expanding or contracting. Another advance came from astronomy; with Hubble's observation of the recession of the galaxies in 1929, many theorists began to consider the idea that the Universe was expanding and that the stars and galaxies are continually moving away from each other. (You can read more about our expanding Universe in What happened before the Big Bang?)

Einstein's next published model of the Universe appeared in 1931 and is now known as the Friedmann-Einstein model. This model is of an expanding, evolving universe, where the density of matter changes. "That's how we found the manuscript," says O'Raifeartaigh. "We were looking at that particular model [the Friedmann-Einstein model]... and we found a very obvious [numerical] mistake. I had wondered was the mistake made on the part of the journal, rather than Einstein?" So began O'Raifeartaigh's detective work.

Cormac O'Raifeartaigh pondering the effects of gravity

While hunting for drafts of the 1931 paper on the Einstein Online Archive, O'Raifeartaigh and his colleagues came across a manuscript which had the same title and same opening paragraph, but then went off in a completely different direction. (You can see the original manuscript here, and a translation in the paper by O'Raifeartaigh and his colleagues.) In fact, the manuscipt contained a completely different model of the Universe by Einstein that was unknown. "I nearly fell off my chair," says O'Raifeartaigh. "It was like finding a play by Beckett featuring Godot, except Godot turns up in the first act."

The model in this newly discovered paper had the Universe expanding but its density staying the same – the ratio of matter to empty space in the Universe would stay constant. This type of model is now known as a steady state model. "This fits very much with what we know," says O'Raifeartaigh. "Einstein was actually extremely hostile to the idea of a dynamic universe." And on actually learning about the theoretical and observational evidence for expansion, his first instinct was to say, "Well maybe the Universe is dynamic, in the sense that it's not static, but maybe it's not changing." In this sense this paper represents a middle step in Einstein's thinking about how to move from his initial idea of a static universe, to one that was expanding and evolving.

To enable the density to stay constant, Einstein's steady state model had a continuous creation of matter occurring in empty space. "It had puzzled me until now that nobody in the 1930s had come up with the idea of continuous creation of matter in an expanding universe," says O'Raifeartaigh. Other physicists, such as Svante Arrhenius, William MacMillan and even Paul Dirac had previously suggested the idea of matter being created out of nothing. "So it comes as no surprise that Einstein considered this. It shows you the simple logical development of great minds thinking about empirical cosmology as it unfolded over the twentieth century."

The continuous creation of matter would also form the bedrock of the famous and controversial steady state model independently proposed by Cambridge physicist Fred Hoyle 20 years later. "Hoyle's model was perfectly consistent [that is, mathematically correct]. It was a perfectly viable model of the Universe, but it turns out that's not the way the Universe is," says O'Raifeartaigh. Soon after evidence began to mount that the amount of matter in the Universe stays the same as it expands: the density goes down. The distribution of galaxies today is very different to the distribution of galaxies many years ago. Hoyle's model became controversial because the stronger the evidence became the more Hoyle and his collaborators fought it. "Usually theoreticians like to explore different models of the world. They're not supposed to fall in love with a particular model and stick to it through thick and thin, but Hoyle was not for turning."

Einstein's steady state model is also interesting to physicists as it uses the cosmological constant – something that he'd previously introduced into his equations to enable a static Universe – to instead represent some latent energy in empty space that was responsible for the creation of matter. This is particularly remarkable given the current understanding today of dark energy and how it may explain the value of the cosmological constant. (You can read more about the cosmological constant in Lambda marks the spot – the biggest problem in theoretical physics.)

Einstein's exploration of the steady state model lay undiscovered for over 80 years because it was never published. His particular steady state model contained a technical flaw that led to a universe empty of matter. Rather than pursue the idea further, it seems that he then dropped the idea and moved on to evolving models. "I think it confirms that all of the great masters were really in the business of exploring different models," says O'Raifeartaigh. "Like good detectives they're open to every line of enquiry."

You can now read Einstein's papers yourself! As well as the Einstein Papers Project featuring digital version of the original documents, English translations of the papers from 1879-1923 have just been published online. And you can read all about O'Raifeartaigh's discovery in his excellent blog, in the paper by O'Raifeartaigh and his colleaguesas well as in Nature and Scientific American. Happy reading!

About this article

Rachel Thomas is Editor of Plus. She interviewed Cormac O'Raifeartaigh at the Cosmology and the constants of nature conference at the University of Cambridge in March 2014.