EBM Focus: Masking Our Confirmation Bias About Preventing COVID

Practice Point: According to the newest and best available evidence to date, wearing surgical masks in the community doesn't prevent COVID or flu-like illnesses. And N95 masks don’t seem better than surgical masks in healthcare settings.

EBM Pearl: It’s human nature to seek out evidence that confirms our point of view, but that doesn’t make it true. It’s not surprising that a benefit seen in case reports or cohort studies diminishes or disappears in randomized trials. As we remove systematic error and better approximate the truth, the magnitude of effect almost always decreases.

I came across a meme the other day that featured an array of pictures from early 2020 of people wearing things like water cooler bottles on their heads, plastic grocery bags on their faces, and even maxi pads duct-taped across their noses and mouths. The entertainment value of these photos is based on the ridiculousness of it all because we know enough about the biophysiology of aerosolized viral spread and the mechanics of these various homemade protection devices to be so certain they won’t work that we would never bother testing them. But masks were engineered to prevent the spread of illness, and we thought they should work in practice, so we wore them believing they would.

Before last month, the best evidence we had on masking was a systematic review of 71 observational studies and a single RCT that suggested that both surgical masks and N-95 masks were effective for preventing COVID and flu-like illnesses. But here comes a systematic review and meta-analysis of 78 randomized trials by the Cochrane group that concluded there was no benefit to wearing surgical masks in the community, and that N95 masks are no better than surgical masks in healthcare settings. How do we explain these very different results from two systematic reviews, considering systematic reviews supposedly sit atop the evidence pyramid? The answer comes down to systematic bias in study design. A perfectly executed systematic review is still only as good as the data analyzed. Garbage in = garbage out. (Or for you non-cynics out there, diamonds in = diamonds out.)

This is where EBM comes in and exactly why we need systematic reviews of high-quality randomized trials evaluating interventions that we think should work: to show us what does work when tested. We also have to affirm the statistical phenomenon wherein the magnitude of effect is diminished as you “move up” the evidence pyramid, reducing systematic bias and better approximating the truth. In this case, the Cochrane review analyzed higher quality evidence than did the previous systematic review of observational studies, and in the absence of any methodological concerns about the Cochrane review, the Cochrane is inherently higher quality evidence with outcomes that are objectively more trustworthy and closer to the truth.

But let’s also make room for nuance here. In many ways, the study of masks really comes down to mask efficacy on an individual level vs mask effectiveness on a population level. We can interpret the Cochrane results in one of two ways: either 1) masks are not effective, or 2) masks can’t be worn long enough or tight enough by enough people for them to work properly all the time and so they lack effectiveness. We want to believe the truth is #2, but is that just our confirmation bias at play?

All in all, what we have is trustworthy evidence that real-world masking doesn't make a difference in the community spread of COVID, but no definitive evidence that perfect masking all the time in the community doesn't work because that hasn’t been studied. But for what it’s worth, my kid is wearing a mask in a high school musical and this EBM die-hard isn't stopping him.

Reference: Cochrane Database Syst Rev. 2023 Jan 30;1(1):CD006207

For more information, see the topic COVID-19 Infection Control and Prevention in DynaMed.