It’s been over four months since the NHS Covid-19 contact-tracing app launched across the UK, and since then the health services have been short of updates on the performance of the technology, to say the least.
Now, some statistics have been revealed to the public, finally shedding light on the scope of the app’s contribution to the fight against Covid-19 – and despite the technology’s initial shortcomings, the results are encouraging.
The Department of Health and Social Care (DHSC) announced that the app has been downloaded 21.63 million times, a steady increase since the technology was released in September. In total, over 1.7 million users across England and Wales have been advised to isolate by the app, after 825,388 positive test results were logged in. Researchers calculated that this has potentially prevented up to 600,000 positive cases.
The estimation is the result of an analysis carried out by scientists from the Alan Turing Institute and Oxford University who, based on the available data, were asked to find out whether a link could be established between the use of the app and reduced case numbers.
“It was almost a year ago now that we set up the theory behind the app,” Michelle Kendall, research fellow at the University of Warwick, who participated in the latest Turing/Oxford analysis of the app, tells ZDNet. “We knew that about half of all infections come from people who aren’t at the time showing symptoms. From that, we saw that to stay one step ahead of the virus we needed fast contact tracing. In theory, the app could play a really big role in this.”
“So, it’s been great to see these preliminary results matching up with that,” she continues. “The number of cases we had since September could have been far higher without the app.”
Based on the API jointly released by Google and Apple to help national health services build secure digital contact-tracing tools, the NHS Covid-19 app taps Bluetooth to quickly warn users that they have been in close contact with somebody who later tested positive to the virus.
Devices that are actively running the app regularly generate random IDs that are exchanged whenever two smartphones that have downloaded the technology come into prolonged contact. This means that all contacts that are deemed “risky” by the app’s algorithm can be easily warned if one user then gets infected.
According to the DHSC, the app is currently the fastest way to notify users that they are at risk of having contracted the virus, sending alerts to close contacts as soon as 15 minutes after a positive result has been entered into the app.
But speed is not the only factor at play in determining the technology’s efficiency. The app was also designed to be accurate: a major update to the technology’s algorithm last October was presented by the health services as assurance that the technology would only warn users who were genuinely at risk of infection. The idea was to avoid potential false positives delivered by the app, which might see users who were not at risk being asked to self-isolate.
But although in theory the app has been shown to work, the technology’s real-world impact remained to be proven; and the Turing/Oxford team was tasked with finding out whether the tool is effectively helping reduce the number of COVID-positive cases.
Proving that one factor is causing an event not to happen is an academic challenge, explains Lucie Abeler-Dörner, scientific manager at Oxford University, who also took part in the analysis: “It’s always difficult to know how many cases exactly you have averted, because those are figures that haven’t happened.”
What’s more, the team could only use limited data. To protect privacy, the app’s system is decentralized, meaning that interactions between smartphones are only registered on the device. The Bluetooth “handshakes” that occur between users are entirely anonymous and cannot be traced back to a specific person.
The app, however, does provide data on how many people use it in each postcode area, how many notifications are sent out to contacts of infected individuals, and how many of those later register a positive test with the app.
Combining this with surveys of how well people adhere to quarantine allows researchers to gather an idea of the number of cases that can be avoided thanks to infected users being warned in the app. But the method is flawed: it is impossible to know, for example, whether users were only prompted to take a test and self-isolate by the technology, or if they also got a call from manual contact-tracing services.
There are many other external factors that have been shown to influence the rate of infection in certain areas, and that also need to be accounted for. Those include the level of poverty, for example; but also differences in local restrictions that were introduced through the government’s tier system.
As Abeler-Dörner explains, the only way to really understand the impact of the app would be to organize an entirely controlled clinical trial. “It would be interesting to follow a set of people, with their permission, and to look at their contacts to understand how many really got infected or not as a result of using the app,” she says.
“There were a lot of factors that could have influenced the analysis,” she continues. “We spent most of the time trying to figure out how to account for everything else that might play a role, and did the best as possible given the data we had. It’s a very high-level analysis, a bird-view kind of research, that rather looks at the big picture based on that available data.”
The researchers came up with mathematical models to compare areas with similar demographics and ongoing interventions, and concluded that in places that had higher levels of app uptake, there were fewer COVID-19 cases.
Unsurprisingly, the correlation was even stronger after the app was updated to improve accuracy. On average, estimated the analysts, every 1% increase in app usage in a given area resulted in cases reducing by 2.3%. Nationwide, that is almost 600,000 cases avoided – almost a third of the 1.9 million people who were infected with Covid-19 between October and December.
Although there are too many uncertainties to exactly verify the results, Abeler-Dörner argues that the conclusions of the analysis effectively show the success of the app. “To be honest, we didn’t know what to expect, because it’s a new technology,” she says. “But the results were within the range of figures that we thought might be possible. We are very pleased because if the app averted another 600,000 cases, that’s a big chunk of the total infections.”
To further improve the app’s impact will require even more users to download the technology. With close to 22 million downloads, the NHS Covid-19 app ranks similarly to neighboring technologies like Germany’s CoronaWarn app, which is used by over 25 million residents – and much better than some countries like France, where the national contact-tracing app has been downloaded 13 million times.
Statistics released by the DHSC showed solid engagement with other features of the app. For example, the venue check-in feature was used over 103 million times, and users reported Covid-19 symptoms in the symptom checker 1.4 million times.
The numbers, however, have to be put into perspective. The Turing/Oxford researchers found that only 16.5 million UK users regularly use the NHS app, which represents less than half of the eligible population with compatible smartphones. This is much less than the 21.63 million downloads reported by the DHSC, and the discrepancy shows that a proportion of users uninstall the app or turn off its contact-tracing capabilities.
“It’s been really quiet since the app launched, and a lot of people might have concluded that it is not working properly,” says Abeler-Dörner. “So it is important to say it is working properly, and that it can actually make a huge difference in your local community.”
Although it is yet to be peer-reviewed, the Turing/Oxford analysis is hoped to help further boost take-up of the app across the country. The DHSC has also said that data on the NHS Covid-19 app will be published weekly alongside NHS Test and Trace data from 18th February.
