This figure illustrates how the sequence of averages of rolls of a die (red) converges to the expected value of 3.5 (green) as the number of rolls grows.
When you roll a fair die you have an equal chance of getting each of the six numbers 1 to 6. The expected value of your die roll, however, is 3.5. But how can this be? That number isn't even on the die!
In probability theory the expectation or expected value is an idealised average that reflects the probability of the possible outcomes of something. In our die example, each of the six numbers has a probability of of being rolled. This means that if you roll the die lots and lots of times, you should see a 1 in roughly of all the rolls, a 2 in roughly of all the rolls, a 3 in roughly of all the rolls, and so on. So if you have rolled the die times, then each of the numbers comes up roughly times.
The number you get when averaging all the outcomes of the rolls is therefore roughly equal to
The strong law of large numbers says that the larger the number , the closer the actual average gets to 3.5. The number 3.5 is, in a sense, the average you'd get if you'd rolled the die an infinite number of times.
The same idea works more generally. Suppose your die is not fair, so the six numbers don’t all have the same probability of coming up. Suppose the probability of a 1 is , the probability of a 2 is , and so on. The average outcome of a large number of rolls is then roughly
This is the idea behind the general definition of expectation. If a random variable has possible outcomes up to , with corresponding probabilities up to , then the expected value of the outcome is
More generally, if you are looking at a sample of events that occur according to some probability distribution, the expected value is the same as the average (technially called the mean) of that distribution.
This seemingly impossible result does not apply to every averaging situation. It only applies when one averages the probabilities. If we were to consider the probability of rolling a particular number (un-averaged) of the die it would still remain 1/6. So it is incorrect to claim that the expected value is anything specific, when it lays between 1 and 6.