What VAERS data really says about COVID-19 vaccines

I’m sure you’ve seen one of those alarmist posts based on data from VAERS, the FDA and the CDC’s joint vaccine adverse effect reporting system, that conclude the COVID-19 vaccine ‘kills’ as many as 0.4% of its recipients. Advocates for this position state that the data ‘comes from the government itself’. Setting aside for a moment whether that should make one trust the data more or less, this is missing a pretty important point. VAERS is a passive reporting system. And as such, it must be appropriately analysed.

I know a thing or two about VAERS – I’ve written on it, supervised theses on it, visualized it. I’ve published a cadence of papers recently on what VAERS data shows with regard to the COVID-19 vaccines in particular. You can find some of that stuff here. I don’t, however, expect you to take anything on faith alone. So we’re going to go through the exact maths and reasoning that’s behind the the assertion I support, which is that the vaccines are remarkably safe, step by step. In the end, it’s for you and your medical provider to decide what to do, but you (and your clinician) should do it based on a correct understanding of statistics.

The fallacy of reporting rates

Let’s look at an unconnected hypothetical.

Hypothetical:
A drug, X, is administered to millions of people for a frequent indication. The FDA and the manufacturer solicit reports of adverse effects (this is called passive reporting). 4,200 reports are made, of which 1,500 indicate the patient developed agranulocytosis, a condition where the bone marrow fails to produce enough of a certain type of immune cell. Agranulocytosis is serious and may rarely lead to fatal opportunistic infections.

Incorrect reasoning:
Of 4,200 reports, 1,500 reported agranulocytosis. Therefore, over a third (35.7%) of X recipients will develop agranulocytosis. At a risk of a serious side effect over 1:3, the drug is clearly dangerous.

Correct reasoning:
All we know is that of the 4,200 who have reported some symptoms, 1,500 reported agranulocytosis. It is perfectly statistically conceivable – indeed, quite likely – that that’s about all the cases of agranulocytosis there were. We can’t really tell – assuming all reports of agranulocytosis are correct, and 3 million have received drug X, the frequency of agranulocytosis may be as low as 1,500 in 3 million or 1:2,000. That’s a low risk, and the drug may, depending on its effects, be quite safe.

What this highlights is the inherent bias of reporting: people who report are not random samples from the exposed population but rather people who are experiencing, or claim to experience, some adverse effect.

Fixing reporting bias

There are several ways to get rid of this bias. One is to use a metric called the Reporting Odds Ratio. Say I am interested in the rate of anaphylaxis in mRNA vaccines, something examined quite in depth in a recent study of mine. I can build what is called a ‘contingency table’ (machine learning and statistics folks might know this as a ‘confusion matrix’):

From the above, I can calculate the reporting odds ratio, ROR, as . What the ROR tells me is the likelihood of a certain adverse event occurring vis-a-vis the likelihood of the same event occurring with other exposures. A ROR below unity (i.e. 1.0) favours the intervention we’re considering (in this case, mRNA vaccines) while a ROR above unity favours other interventions. Long story short, RORs normalise the risk, taking the rest of the data as the baseline.^[1]

In the case of VAERS, we can look at individual adverse events, and see how often they have been reported in the context of COVID-19 vaccines vis-a-vis other kinds of events, then compare how that fraction looks for all other vaccines taken together. This does not get rid of all of the biases (that’s why more in-depth studies use matching and case-control designs), but it gets the fundamental point right: we compare ‘likes to likes’.

What the evidence says

The evidence from VAERS does not show what you have probably been told it shows. Rather, it shows that while some side effects are a little more frequent, most adverse effects of the COVID-19 vaccines are not significantly more frequent than they are for other vaccines, and some are significantly less frequent. Guillain-Barre syndrome is an example of the latter – with a reporting odds ratio of about 0.25, GBS from COVID-19 vaccines is much less likely than GBS from non-COVID-19 vaccines.

Some other findings:

• Overall, COVID-19 vaccines are some of the safest vaccines ever.
• Autoimmune adverse events are generally less likely with COVID-19 vaccines than with other vaccines.
• There’s no significantly elevated risk of neuroautoimmune adverse events with COVID-19 vaccines.
• mRNA vaccines are not significantly more likely to cause anaphylaxis than non-mRNA vaccines, and they’re safe in people with pre-existing allergies.

Summing up

In law, the eminent British judge Lord Steyn observed, context is everything.^[2] Epidemiology is no different. The context for understanding the risks of a vaccine is not just its benefits, but also how it compares against other similar interventions. When compared against other vaccines, the COVID-19 vaccines authorised in the United States are remarkably safe, and data from VAERS supports this assertion. VAERS is a reporting portal, not an analytical interface. It is easy to look at the right data but come, as many have, to the wrong conclusions. I hope this post has cleared up some of these.