How effective are the vaccines — and how do we know this?

With all the questions in the news we asked experts Matt Keeling and Sam Moore, epidemiological modellers at Warwick University and members of the JUNIPER modelling consortium, how effective COVID-19 vaccines are and how we know this. Here is what we learned.

When we say that a vaccine is x% effective (or has an efficacy of x%), we mean that the vaccine stops x% of potential infections among those vaccinated.

Let's take numbers from the report on the trial of the Oxford/AstraZeneca vaccine, which is quoted as being 70% effective on average, as an example. In this trial 5807 people were given the vaccine and 5829 people were given a placebo (see here to find out why there's always a group of people given a placebo, as well as a group given the actual vaccine).

Out of the people given the placebo 101 developed COVID-19 symptoms. Since 101 is 1.7% of 5829 (101/5829=0.017), this means that 1.7% of those who were given the placebo caught COVID-19. Now imagine that the 5807 people who did receive the vaccine had not been vaccinated. Assuming that the 1.7% is a representative figure, we would then expect 1.7% of the 5807 to have caught COVID-19 — that's 98.7 people.

New vaccines are tested in randomised controlled trials.

In reality, though, the 5807 people did receive the vaccine and only 30 caught COVID-19. So rather than the expected 98.7 cases, only 30 were observed and hence it is assumed that the vaccine prevented 98.7-30= 68.7 from becoming ill with infection. Since 68.7 is 69.6% of 98.7, the study suggests that the vaccine protected nearly 70% of people. The figure of 70%, then, is a good estimate of the effectiveness of the vaccine.

A figure arrived at through a calculation like this, involving the results of a particular study, is only an estimate of course — if you repeated the study, you might well get slightly different results. Luckily, though, there are statistical techniques that help you account for this uncertainty and help you extrapolate the result of your study to a whole population of people over time. This then helps you to get a more reliable estimate of the effectiveness of the vaccine, which also comes with a confidence interval, that is, with a quantification of how confident you are that your estimate is correct (find out more here). Estimates of vaccine efficacy become more precise, and hence the confidence intervals become smaller as data is collected on more individuals with and without the vaccine.

We used the Oxford vaccine as an example because, as you have probably heard in the news, it presented a curious result. Scientists discovered that when people were first given half a dose of the vaccine and then, for their second jab, a full dose, then the vaccine was 90% effective (although here the study size was small and hence the confidence intervals wide). But when they were given full doses for both jabs, the vaccine was only 62% effective.

The figure of 70% calculated above came from lumping all vaccinated people together, regardless of what combination of doses they received, and seeing how many of those caught COVID-19. The figure of 62% comes from doing the same calculation only considering those that received full doses, and the figure of 90% comes from doing the same calculation only considering those that received half dose followed by a full one.

Return to our COVID-19 vaccines FAQ

About this article

Matt Keeling

Matt Keeling is a professor at the University of Warwick, and holds a joint position in Mathematics and Life Sciences. He is the current director of the Zeeman Institute for Systems Biology and Infectious Disease Epidemiology Research (SBIDER). He has been part of the SPI-M modelling group since 2009.

Sam Moore is a postdoctoral research associate who has been working on vaccination modelling for Covid-19 after joining SBIDER within the University of Warwick at the start of the pandemic earlier this year.

Sam Moore

Both are members of JUNIPER, the Joint UNIversity Pandemic and Epidemic Response modelling consortium. It comprises academics from seven UK universities who are using a range of mathematical and statistical techniques to address pressing question about the control of COVID. The universities are Cambridge, Warwick, Bristol, Exeter, Oxford, Manchester, and Lancaster. You can see more content produced with JUNIPER here.

Marianne Freiberger and Rachel Thomas are Editors of Plus.