Each day we’re inundated with new data about the spread of COVID-19. But how do we know which numbers tell the real story?
Caroline Chen covers health care for ProPublica and is currently reporting on the coronavirus pandemic. She talked with Krys Boyd about how we should interpret the numbers to get a sense of where we’re headed.
Be sure to read her recent ProPublica story and listen to the complete Think conversation. Below are edited highlights:
Krys Boyd: What are some examples of numbers that can sound comforting or alarming if we don’t put them in the right context?
Caroline Chen: All of the numbers – cases, deaths, recoveries, hospitalizations – it all depends on how you’re being told about them, whether they’re being taken out of context and how they’re being presented to you.
Sometimes there are people who take these numbers and they want to give a specific narrative, and that can be really dangerous and very concerning to me when I see that happen. But even with the best of intentions, if you’re not looking at something with the appropriate context, you can just be a little bit confused because you’re looking at too small of a time frame.
This frankly can be a little bit bewildering because there’s so many numbers being thrown around. I don’t think there is any one specific number that is particularly confusing or particularly misleading on its own. It’s more of a question of how they’re being used.
Boyd: Why are seven-day rolling averages considered more accurate than any spikes or declines we might see for any one particular day?
Chen: A seven-day rolling average takes each day’s number and averages with the six days before it. … This is important because, you can understand that, say for case counts, they might wiggle a little bit up or down every day.
All of the numbers - cases, deaths, recoveries, hospitalizations - it all depends on how you're being told about them, whether they'e being taken out of context and how they're being presented to you.
And particularly what you’re seeing with case counts is that they dip on Sundays and Mondays. And this is because weekend counts drops. Sometimes it’s because labs don’t report on weekends or sometimes even local county health departments might be closed or have fewer people working on weekends.
So the counts drop on weekends – and this is what I was just talking about with regards to too small a time frame. If you just looked at a chart on Sunday, you might think, “Things are dropping – things are getting better!” And that’s why having these rolling averages helps smooth things out.
Boyd: Epidemiologists and public health experts are paying attention to the positivity rate among people tested. A lot of experts say you want a positivity rate under 5%, and if it’s over 10% testing positive, that’s a really bad sign.
Chen: One of the things that I had gone around asking epidemiologists and experts is, “What metrics should I track if I want to know what’s going on in my state?” And I asked a lot of people this, like, “I want you to give me one, golden number – just boil it down for me.” And everybody was like, “Sorry, there’s not one number – you’ve kinda need to look at a number of things in tandem.”
Most consistently what I got back was: You want to look at cases, where that’s trending – up or down – along with case-positivity rates and deaths. …. [Case-positivity rates] measure the percentage of total tests conducted that are coming back positive and it gives you a sense of how much testing is being done overall in a region. In New York, in March, the positivity rate was really high – it was probably like over 25%. And that would have told you that there was not enough testing happening. This is where that 5% comes in – that’s a WHO [World Health Organization] guideline. We want that to be below 5%, and that gives you a sense of confidence that enough testing is happening. …
The positivity rate gives you a sense of how much can I trust the trend I’m seeing in case counts. When it climbs above 10%, you start to get a little bit more worried – are we actually doing enough testing?
Boyd: If someone tests positive and then goes back to get retested and tests positive again – how do we know that person won’t be counted as two cases instead of one?
Chen: Health departments get information from the labs, which include people’s names and contact information because they need this for contact-tracing work. Of course, they know how to handle this information sensitively – they’re not, like, going out and posting people’s names. But they have information.
And so what health departments try to do is they try to remove duplicates so that they won’t double count. Now, I did press some people on this, and they said, “Look, is it possible that in some cases somebody gets double counted because they misspell somebody’s name? It is possible.” … This isn’t going to skew the numbers in a large way.
The takeaway is there are often many ways of why there might be a scenario in which this might increase the number by one or two when it shouldn’t, or cause a slight overcount or a slight undercount. I’m trying to think about what is the big picture in my state: Are we overall trending upwards? Are we overall trending downwards? …
I try not to get too distracted by arguments for why things can be overcounted or undercounted because we can always come up with those arguments.
