We need to talk about public comments.

In 1943, Norman Rockwell created a series of iconic paintings, each epitomising one of the Four Freedoms articulated by President Roosevelt at his 1941 State of the Union message. Of these, by far, I have always found Freedom of Speech to be the most poignant. I was born into a country where that right did not exist, and even today it is a tenuously practiced right: all too many people believe that freedom of speech might not really guarantee freedom after speech, a witticism that struck uncomfortably close to the truth during the era of Communist repression.

Western democracies, however, run on public comment. Mature democracies have found ways to integrate this. When I was a junior staffer at the European Parliament, I got a full folder of public comments every other day, to which I had to devise a response. They ranged from 100-page comments written by teams of experts with deep understanding of the subject matter to hand-written screeds about implantable tracker chips, mind control satellites and something about… rabbits? To this day, I have no idea what that one was about.

As anyone will tell you, cranks are the price you pay for the occasional good idea and the fundamental right of public representation. For every mind control satellite weirdo, there will be multiple considered opinions, often by interested parties who have the legal and economic wherewithal to put their point across in well-reasoned arguments.

But, of course, that doesn’t mean the cranks don’t get to speak. And we’ve got a bit of a crank problem when it comes to hearings of ACIP, the FDA’s Advisory Committee on Immunization Practices.

The attribution problem

One of the biggest problems when it comes to public comment is attribution. Consider the following tweet I have been forwarded:

Right… except the FDA didn’t say that. Nor did ACIP, which technically is separate from the FDA. Rather, it was part of an oral comment.^[1] The risk is that because public comment is a right, anyone with any credentials (or none) and any understanding (or, again, none) of vaccine adversomics can turn up at a public hearing and air their views. Journalists sympathetic to the anti-vaccine cause, meanwhile, will launder these comments as if they were attributable to the FDA.^[2]

In this age of social media, a lie can become viral before the truth has even opened its browser window (to clumsily paraphrase that old chestnut). In fact, while cases of myocarditis/cardiomyopathy have been reported in conjunction with the COVID-19 vaccine,^[3], only approx. 60 cases of myocarditis per million fully vaccinated individuals are in evidence. This is not higher than other vaccines once we adjust for the fact that most vaccines are administered in childhood, when myocarditis risk is the lowest, while the COVID-19 vaccine has been administered in a targeted manner to populations that are statistically at a higher risk of cardiac side effects.

The source

In fact, the root of the 71x figure and the “killing two for every life saved” appears to be this calculation, from Steve Kirsch’s presentation:

There is a more extensive debunking of this slide by Jeffrey Morris at Covid-19 Data Science than I could endeavour to do. I would like to reflect only on the crucial element, viz. the underreporting factor of 41. The authors claim to have arrived at this by comparing VAERS-reported cases of anaphylaxis against anaphylaxis rates reported in Blumental et al.’s paper.^[4] This touches upon an area within my professional work, namely VAERS analytics.

I do a lot of analytics on VAERS data, and it is far from trivial. I have, in fact, written about anaphylaxis following the COVID-19 vaccine, as well as other aspects of VAERS data on the COVID-19 vaccine. There are three major issues with the excess death calculation:

• It assumes that VAERS underreporting is isotropic, i.e. underreporting of cardiomyopathy will be the same as underreporting anaphylaxis. This is definitely not the case. Reporting burden is constant, but gravity of the symptoms reported shifts, consequently some things are less likely to be underreported than others.
• It assumes that this underreporting factor is homogenous over the set of all vaccines vis-a-vis the COVID-19 vaccine. This, too, is typically not the case.
• Finally, it assumes that the underreporting is homogenous over the set of all types of vaccines. Because some vaccines are more frequently given in institutional settings, while others (such as the COVID-19 vaccine) are the matter of public interest, there can be distortions between types of vaccines. There is a cognitive bias called the frequency illusion or Baader-Meinhof effect, which enhances this effect.^[5]

The result is a figure that, while frightening and attention-grabbing, is far from the truth.

The perils of exploiting public comment

This, then, emerges as a peculiar form of abusing public comment sessions, as a podium to air views that can then be laundered by sympathetic commentators and attributed not to the individual commenter but rather to the organisation convening the meeting, i.e. ACIP or the FDA. Journalists can become unintentional accomplices in this game: the FDA “turning” on the vaccine is a much better story than the fact that, uh… someone said something at a public hearing.

The problem is that this abuses the public hearing system. It endangers the public hearing system. It encourages public hearings to become rubberstamp sessions, held to comply with the letter (but not the spirit) of public law. The strategic use of public comment sessions to launder opinions by individuals as facts asserted by regulators makes decision-makers less likely to listen with an open mind, and in the end, nobody benefits except those intent on abusing the system.

The solution lies not with the regulatory bodies. Limiting who may or may not comment vitiates the ‘public’ nature of public comment, and would just give rise to accusations of bias by the organisers. Rather, those who watch and use public comment materials must become responsible users of the process. The only way to stop the exploitative strategic use of public comment to spread misleading information is to disincentivise it, and the biggest disincentive is of course refusing to blindly repeat what was said, and be clear about attribution.

Which, by the way, ought to be good policy anyway.