This seemingly impossible result does not apply to every averaging situation. It only applies when one averages the probabilities. If we were to consider the probability of rolling a particular number (un-averaged) of the die it would still remain 1/6. So it is incorrect to claim that the expected value is anything specific, when it lays between 1 and 6.