Harold Shipman, a family doctor in Yorkshire, is currently serving a life sentence for murdering 15 of his patients, and is likely to have murdered over 200 more. How did he manage to kill so many and remain undetected?
As part of its investigation, the Shipman Inquiry commissioned a study to see if a monitoring system could have detected such an alarming death rate, and the results have just been presented to the inquiry and published in The Lancet. The researchers, Paul Aylin, Nicky Best, Alex Bottle and Clare Marshall, from Imperial College, London, used a statistical monitoring method developed for quality control in industry in the 1930s.
Quality control?
[Image DHD Photo Gallery]
"The basic idea of the method is [to] monitor an industrial process which, when stable, still shows some intrinsic variability," Dr Best told the inquiry. "If some external factor causes the process to become unstable for some reason then we would expect that to lead to a different pattern of variation, in particular a greater variability than the stable process."
Statistical Process Control (SPC) charts are used to monitor, over time, whether a process is in or out of control. The first step is to decide what is to be monitored. This study focused on doctors and practices with at least 1,000 patients on their books, in each of five geographical areas in the UK. The researchers monitored the number of patients who died between 1993 and 1999.
Next they had to decide what they meant by the "process" being in control, that is, what was an acceptable mortality rate. "Most industrial processes are well characterised in the sense that, when a process is in control, the only source of variation is random," the researchers said. "By contrast, health outcome processes are far more complex. Even when in control, the process is subject to many non-random sources of variation, particularly changes in case mix."
The researchers used statistical techniques to transform a doctor's or practice's death rate, taking into account such factors as the mix of patients' ages (a rough estimate of the case mix) and allowing for an acceptable level of variation deduced from historical data. They expected that the transformed annual death rates would follow a normal distribution which, when the process was in control, would have mean 0 and variance 1. If a death rate was unacceptably high, the mean of the distribution would be shifted above 0.
In the early 1990's, Shipman was recorded as having around 10 more deaths among his patients per year than would have been expected, corresponding to a mean of 1.7. By 1998, when he recorded nearly 40 excess deaths, this mean had risen to 6.6.
By plotting the differences between the observed and expected annual death rate for a doctor, and letting the differences accumulate over time, it becomes apparent becomes apparent when a doctor's death rate is growing out of control. The researchers set an alarm threshold on the chart, which, when crossed, could trigger an investigation. The alarm threshold was chosen so that the chance of successfully detecting out of control doctors is high, but the number of false alarms is acceptably low.
When testing for a very high death rate, where the average is four times higher than expected, Shipman triggered the alarm along with 11 other doctors. Health authorities have been notified about the other 11 doctors, but the researchers believe that, almost certainly, the raised mortality rates will be either statistical or medical false alarms, and not medical true alarms. Probably those doctors and practices with higher death rates will turn out to have an unusual mix of patients - because of a local cancer hospice, for example - or data quality issues may emerge.
The study has shown that it is possible to monitor death rates in such a way that doctors with excessive numbers of patients dying will be detected, particularly if records are improved. However, they highlighted that "performance monitoring should be used as a starting point for an audit and learning opportunity, rather than recrimination". A commitment must be made to investigate any alarms adequately and sensitively, and to use monitoring as an opportunity to improve the quality of care for all patients.
Shipman's actions betrayed the unquestioned trust that we used to have in our doctors. Perhaps if a monitoring system like this one helps to protect against such a tragedy in the future, our trust in doctors might in some way be restored.