Messy, incomplete U.S. data hobbles pandemic response
The nation’s decentralized, underfunded reporting system hampers efforts to combat the coronavirus.
How many people have been infected at this point? No one knows for sure, in part because of insufficient testing and incomplete reporting. How many fully vaccinated people have had breakthrough infections? The Centers for Disease Control and Prevention decided to track only a fraction of them. When do inoculated people need booster shots? American officials trying to answer that have had to rely heavily on data from abroad.
Critically important data on vaccinations, infections, hospitalizations and deaths is scattered among local health departments, is often out of date and hard to aggregate at the national level, and it is simply inadequate for the job of battling a highly transmissible and stealthy pathogen.
“We are flying blind,” said Ali Mokdad, an epidemiologist at the University of Washington’s Institute for Health Metrics and Evaluation who spent two decades working for the CDC. “With all our money, with all our know-how, we have dropped the ball. … We don’t have the data. We don’t have the good surveillance system to keep us informed.”
The dearth of timely, comprehensive data impaired the ability of the nation’s top public health officials and infectious-disease experts to reach a consensus on the need for booster shots. The experts looked at conflicting data from Israel, Britain and the United States and came up with a bewildering set of recommendations. The debate seemed to confuse more than clarify arguments for the necessity of an additional shot.
“We are pulling data in from all different sources,” said a senior administration official who spoke on the condition of anonymity because the person was not authorized to discuss the issue publicly. “We’re trying to put it all together to see … what is the vaccine efficacy? And there’s this wide divergence. It’s not reconcilable.”
Data is key to an effective pandemic response — and the lack of proper data has hobbled the U.S. response again and again. The lack of testingand then of standardized reporting of cases and deaths left U.S. officials slow to grasp the scale of the crisis when the virus began to spread. Insufficient data also meant supplies to fight the pandemic arrived too late in hard-hit cities. State and federal officials made decisions about travel restrictions and policies on reopening with an incomplete picture of what was happening.
Many places were forced to shut down before they had substantial outbreaks, former Food and Drug Administration commissioner Scott Gottlieb told The Washington Post, and when the virus finally arrived in various communities, some resisted a return to restrictions.
“Early on, CDC couldn’t even tell us how many people were being hospitalized for covid,” Gottlieb said.
Multiple factors underlie this data deficit. First and foremost: The United States does not have a national health system such as Israel’s or Britain’s,and in a pandemic, U.S. authorities must rely on a vast and decentralized public health infrastructure that is notoriously underfunded and full of holes. As a result, there is no simple way to track infections or outcomes across the population.
Another obstacle to data aggregation may be the siloed computer systems and the self-interest of medical institutions. Some hospital systems want to hang onto their data, said Michael Kurilla, director of the division of clinical innovation at the National Institutes of Health’s National Center for Advancing Translational Sciences.
“They don’t necessarily want to give up all that data because they see that as a potential future revenue stream,” Kurilla said.
‘A largely 19th-century system’
The CDC compiles national statistics by collecting data from every state and locality, but these jurisdictions often have different ways of counting tests, infections and even deaths. The data may not be submitted to the CDC for days or weeks. Many smaller jurisdictions still share that data via fax, an outdated technology.
“We’re still operating on a largely 19th-century system,” Kurilla said. “Who exactly is to blame is really hard to point a finger at. There are systems where things are done on paper, some information is being faxed, so it’s being transcribed. There isn’t any way to seamlessly upload information.”
The Biden administration recently unveiled a pandemic preparedness planto “fundamentally transform our capabilities to protect the nation.” One element would be the modernization of digital health data, with standardized software that would enable jurisdictions to share and analyze data.
The data problem has been recognized by federal officials and outside experts for many years, Biden’s science adviser, Eric Lander, said in an interview Wednesday.
“It’s a question that pertains to the whole health-care and public health system. In the United States, our data systems are not interoperable. They don’t talk to one another,” Lander said.
The task of gathering and analyzing data is too laborious at present, and it compromises situational awareness in a crisis, he said.
“If it takes weeks to clean the data … it means you’re going to be running weeks behind the war that you’re fighting. That’s just no way to take on a pandemic,” Lander said.
Solving this problem won’t happen overnight but should not take more than two or three years, he estimated. The underfunding of public health departments is the core of the problem, he said. Giving them access to affordable, standardized software for handling data “is going to be useful not just in the next pandemic — it’s going to be useful in the next flu season; it’s going to be useful in the next measles outbreak.”
Empirical rigor over speed
The CDC is charged with making sense of the patchwork of state data and regularly issues reports on outbreaks. But critics say the CDC operates too slowly, as if it were an academic institution and not a first responder in a crisis. A long-standing criticism is that the agency focuses heavily on retrospective studies and does not share the results quickly — even with other health agencies. The CDC traditionally has emphasized empirical rigor over speed, an aspiration that is in conflict with the demands of a rapidly evolving health emergency.
“They’re out there putting [out] reports from three months ago, and you can’t do that in a pandemic when things move so fast,” said Walid Gellad, an associate professor of medicine at the University of Pittsburgh School of Medicine who criticized as premature the Biden administration’s initial push for vaccine booster shots for all adults.
“In a span of three months, we had super-high cases, then the lowest cases we’ve ever seen, and now we’re back up again. You can’t use old data in a health emergency that is changing as quickly as covid,” he said.
In response to questions from The Post, the CDC said it has shared the results of numerous vaccine-effectiveness studies over the past two months that helped shape the discussion on the need for booster shots. CDC spokeswoman Kristen Nordlund said in a statement that those studies showed that vaccines are effective at reducing the risk of severe disease, hospitalization and death, but also that their protection may decline over time and that they may be less able to protect against the delta variant. She added that the CDC is publishing all of its data on vaccine effectiveness in one place Thursday.
“Even highly effective vaccines often become less effective over time, and tracking this can take time,” Nordlund said. “We relied on the data from colleagues in Israel and the U.K. because the epidemiology of their outbreaks, experience with the delta variant, and use of boosters preceded what happened in the United States.”
Many at the CDC recognize that the agency needs to move faster. In August, CDC Director Rochelle Walensky announced plans to develop a new forecasting and outbreak-analytics center to process data in real time to better predict disease threats. The CDC expects the center to be in operation early next year.
CDC data played only a small part in the booster decision, senior administration officials said in interviews, in part because Israel vaccinated its population faster than the United States vaccinated its own and began experiencing a delta wave several weeks sooner, giving Israel a data set that covered a longer period.
But administration officials and outside experts said the CDC should have shared its own findings on vaccine effectiveness more quickly, rather than waiting until its results were publication-ready in the late summer and early fall.
The CDC also was criticized for its decision in the spring to stop tracking all breakthrough infections and instead follow only those that resulted in hospitalization. The agency has said it could compile more accurate and complete data from its studies tracking thousands of people who are regularly tested and monitored to see whether they develop breakthrough infections. Critics of the decision say policymakers need as much real-time information as possible about new variants that may cause even minor breakthrough infections, but some experts contend that these more-focused studies offer clearer and more reliable results, and are a better use of the agency’s resources.
But even the results of those focused studies were often hard to come by. As administration officials debated in July and August whether the United States would need to administer boosters, they repeatedly implored the CDC to share what it was learning, said several people familiar with the conversations who spoke on the condition of anonymity because of the sensitivity of the conversations. As a result, officials initially relied on data from Israel and on studies conducted at the Mayo Clinic, in New York state and by Kaiser Permanente.
“There is no way nationally in this giant country to connect who’s been vaccinated and what their outcomes are, and that’s the underlying problem,” Gellad said. “Whenever you have to piece lots of data together, a lot of which is contradictory, it can be confusing.”
Better data by itself would not have resolved all the disagreements among disease experts and policymakers. Experts on infectious diseases will look at identical data sets and reach different conclusions about who needs an additional shot.
Some scientists — including several in the Biden administration — think that data from Israel, Britain and elsewhere showing waning immunity against infection over time strongly supports the need for millions of people to get booster shots.
They argue that these breakthrough cases associated with waning immunity, even if they are not classified as “severe,” translate into shuttered classrooms, lost income and continued widespread transmissionof the virus. As long as infections continue in large numbers, several administration officials and scientists argue, the country cannot crush the pandemic.
Many of these scientists also think that waning immunity is an early warning of an inevitable increase in hospitalizations, an outcome they hoped to avoid by administering boosters early.
But many members of the FDA and CDC advisory committees, who also are scientists and public health experts, took a starkly different approach. They focused on the risk of hospitalization among vaccinated individuals, which has not increased significantly in the United States.
Some of these scientists think the public has unrealistic expectations of vaccines. The fact that hospitalizations are not dramatically increasing among the vaccinated — especially those younger than 65 — indicates that the vaccines are functioning as designed, they say.
They also wonder whether new data from Israel, which showed a rise in “severe” disease, was fully applicable to the United States. Israel uses a different definition of “severe,” basing it on such measurements as oxygen saturation and elevated respiration rate, rather than on hospitalization. And Israel isn’t the United States: It’s a much smaller country, less diverse demographically, and doesn’t have as great a burden of chronic health problems, such as obesity and diabetes.
The data shortfall is not simply an issue for the crafters of national vaccine policy. It’s also a conundrum for individuals trying to figure out their own risk.
Even Mokdad, the IHME epidemiologist who studies the data for a living, is uncertain about his continuing level of protection from vaccines.
“I’m a healthy 59-year-old person,” he said. “I’m not obese, I’m very healthy, I don’t have any chronic condition. The only thing I have against me is age.”
It has been more than six months since he got his second shot.
“Do we know how much immunity I have against hospitalization in the U.S.? No,” he said. “Do we know how much immunity I have against death? We don’t.”
Lena H. Sun contributed to this report.