Outer space: You guessed it

If you are a flower then April is allegedly the cruellest month, but if you are a student of any sort then I'm sure you would have picked June. June is a month of exams, exams and more exams for school and college students everywhere. One of the more common forms of exam paper that is devised to aid quick marking is the multiple choice question paper. You have to pick the right answer from a suite of alternatives.

My most memorable encounter with this format was when I first saw the written part of an American driving test. It asked what you should do if you started to feel drowsy whilst driving on the freeway. The options were (a) speed up so as to get to your destination sooner; (b) do an emergency stop; or (c) wait until you arrive at a place where it is safe to stop, then pull in and rest until refreshed before continuing safely on your journey.

Other tests can be more demanding though. One of the features of all multiple choice tests is that they need to be as immune as possible to random guessing. You don't want to give a passing mark to a candidate who just guesses at random. One way to discourage a guessing strategy is to penalise wrong or blank answers with negative marks. Let's see how we might do that.

Suppose that there are five answers to choose from and we give one mark for the correct answer but subtract $Q$ marks for any of the four wrong answers, or if no answer is chosen. If you guess answers at random then there is a 1 in 5 chance that you will choose any one of them and your expected average score on each question will be $$(1/5 \times 1) + 4 \times (1/5 \times Q);$$ that is, a 1 in 5 chance of guessing the right answer and scoring 1, and a 4 in 5 chance of guessing wrongly and scoring $Q.$ Ideally, you might want the reward for guessing to be zero. This will be achieved if you make the penalty mark $Q$ equal to $-1/4.$ More generally, it is easy to check for yourself by the same argument that if we had $N$ answers to choose from then the expected score from a strategy of random guessing will be zero if we choose a penalty of $Q = -1/(N-1).$

We can also work out the variance, which measures by how much the score of a question differs from the zero score, on average, if you're guessing the answer. Our simple statistical process gives an outcome of $+1$ with probability $1/N$ and an outcome of $Q = -1/(N-1)$ with probability $(N-1)/N.$ The variance is the average value of the square of the outcome minus the square of the average, which we have designed to be zero — so it's just the average value of the square of the outcome. This makes the variance in the score per question $$1^2 \times 1/N + (1/(N-1))^2 \times (N-1)/N = 1/(N-1),$$ which is the negative of our penalty mark $Q.$ If the test had a total of T questions in it, then the average total score would be zero and the variance around that would be $T/(N-1).$

You might like to think of other ways to do the test scoring. Suppose we want to be harsher. You might be very keen to discourage guessing and give a bigger penalty for a wrong answer than for a blank answer – you could even require the expected average score from pure guessing to be negative, rather than zero as we have done, by making $Q$ more negative than $-1/(N-1).$ Alternatively you could have different scores for different wrong answers according to how "wrong" they were. Choosing 1814 for the battle of Waterloo, rather than 1815, deserves a bit more credit than choosing 1914. Or you could allow candidates to choose a subgroup of the possible answers. You get maximum credit for choosing the right one, a bit less for choosing a pair that contains the correct answer and less still for a trio containing the correct one. If you finish your summer maths exam an hour early then you could see how the expected results would turn out for the random guessers under some of these new scenarios.

---

About the author

John D. Barrow is Professor of Mathematical Sciences at the University of Cambridge, author of many popular science books and director of the Millennium Mathematics Project of which Plus is a part.