On transparency in science: lessons from the Pandemrix vaccine and 'partial' evidence-based practice

Minus any hype or sweeping generalisation, I want to draw your attention to a feature article recently published by Peter Doshi [1] titled: "Pandemrix vaccine: why was the public not told of early warning signs?" complete with some media interest (see here).

The feature is based around the vaccines developed to counter the 2009 flu pandemic that starred the H1N1 influenza virus which was sweeping across the globe at the time. The 2009 flu pandemic was potentially serious insofar as its infection pattern and fatalities linked to infection although in reality did not turn out to be quite the 'global killer' that was prophesied. The rush to bring to market a vaccine to protect lives and counter the rise of H1N1 also brought with it controversy from several perspectives [2]; some later talking points seemingly stemming from the inclusion of a specific adjuvant - AS03 - in some vaccine preparations, and data suggestive of a possible correlation between onset of narcolepsy and use of certain flu vaccines containing the AS03 adjuvant (see here). Narcolepsy - a sleep disorder - was one potential adverse side-effect talked about, but other labels/conditions were also the source of some inquiry too (see here). On the basis of the emerging published data, court cases ensued, some of which have apparently unearthed some "internal reports" suggesting that some manufacturers "were aware of other serious adverse events logged" in relation to certain H1N1 vaccines...

The Doshi feature goes through some of these 'revelations' that did not seem to have been made public at the time. He talks about an 'undisclosed problem' whereby one vaccine in particular, Pandemrix, seemed to have a greater frequency of various adverse events reported according to those internal company reports. Most worrying are the reports of 'fatal outcome' across different vaccine preparations, particularly Pandemrix. He also writes: "And the raw numbers of adverse events were not small" highlighting how in one of the latest internal reports dated March 2010 seen by himself (or at least, the British Medical Journal) over 5000 reports of "serious adverse events for Pandemrix" had been received by the company behind the vaccine.

It's timely that the Doshi feature appears now at the time of writing this post, just days after the Peter Gøtzsche - Cochrane drama (see here) and at the same time as other relevant pieces have been published (see here and see here [3]). The similarities around questions of transparency in science (particularly industry-derived science that has a public health aspect attached to it) are very noticeable. As is also the question of 'how much detail should the public be provided with?' when it comes to science in general and then specifically, science and science communication linked to the potential cost-benefit ratio of particular medicines or medical preparations. It's interesting too that Doshi also references some past debate in Germany [4] about the 2009 flu pandemic where it was apparently reported that "the German interior ministry rejected accusations that the ministry had ordered a less risky vaccine for top officials" in light of some newspaper headlines published at the time. 'All animals are equal, but some animals are more equal than others' perhaps springs to mind...

I don't think the issue of less than full transparency in science and the question 'how much detail should the public be provided with' are going to go away any time soon. Such issues strike to the heart of the matter of how peer-reviewed published science is seemingly only one part of a much bigger picture when it comes to the search for a complete truth. One wonders for example, how many other 'internal reports' lie in [research] file cabinets around the world that could conceivably influence evidence-based decisions on the effectiveness and safety of various medicines and other interventions indicated for various conditions? Without such documents ever seeing the cold light of day, how can science ever truly be said to be complete, transparent or indeed settled?

Such a 'transparency' situation also makes initiatives like OpenTrials - calling for all research to be more 'discover-able' and 'traceable' - all the more vital and important. Issues such as the requirement for pre-registration of all studies via databases such as ClinicalTrials.gov and importantly, the timely release and publication of results (see here) are pressing features. Study pre-registration (especially prospective registration) in particular, should perhaps be pushed a lot more than it currently is, and shouldn't just be reserved for big studies of drugs and medicines either.

Science is great and is getting better all the time. But it still has lots of issues and problems to overcome. A lack of research transparency seems to be a particularly important issue that, whilst starting to receive some much required attention, will if present, always mean that evidence-based practice really should be re-branded as 'partial evidence-based practice'. And that has lots of implications...

To close, given the subject matter included in the Doshi paper, I want to end with a reminder about not making sweeping generalisations. At the same time I'd also champion the words included in another recent article by Jørgensen and colleagues [5] (yes, also including Peter Gøtzsche as an author) responding to the idea that public confidence in public health initiatives such as vaccination may be "undermined" by even having such discussions in public. To quote: "Debates over sources of evidence must take place in public, especially when public health interventions are at stake." This is again, with the idea that science needs transparency - even better in the peer-reviewed domain - and closed door discussions rarely aid such transparency.

----------

[1] Doshi P. Pandemrix vaccine: why was the public not told of early warning signs? BMJ 2018;362:k3948

[2] Doshi P. Influenza: marketing vaccine by marketing disease. BMJ 2013;346:f3037.

[3] Jørgensen L. et al. Challenges of independent assessment of potential harms of HPV vaccines. BMJ 2018;362:k3694

[4] Stafford N. Only 12% of Germans say they will have H1N1 vaccine after row blows up over safety of adjuvants. BMJ 2009;339:b4335.

[5] Jørgensen L. et al. Response to Cochrane editors: Jørgensen, Gøtzsche and Jefferson. BMJ EBM Spotlight. 2018. Sept 23.

----------

Neuropsychiatric disorder onset "temporally related to prior vaccinations"?

"Given the modest magnitude of these findings in contrast to the clear public health benefits of the timely administration of vaccines in preventing mortality and morbidity in childhood infectious diseases, we encourage families to maintain vaccination schedules according to CDC guidelines."

The quote opening this post comes from the paper published by Douglas Leslie and colleagues [1] (open-access) and offers not a conclusion from their study looking at the possibility that "the onset of some neuropsychiatric disorders may be temporally related to prior vaccinations in a subset of individuals" but a caution that cause-and-effect were not 'proven' in their study. Anyone with any knowledge about previous occasions where administration of vaccines have been correlated with specific psychiatric or behavioural outcomes (see here and see here for examples) will recognise how important such a caution is, bearing in mind that vaccines are medicines (albeit preventative) and are subject to similar monitoring for safety and the small possibility of adverse effects as other medicines.

Based on the examination of "the MarketScan® Commercial Claims and Encounters database", a US drug and medical insurance claims database, researchers looked at various diagnoses of interest including OCD (obsessive compulsive disorder), AN (anorexia nervosa), anxiety disorder, tic disorder, major depression, bipolar disorder and ADHD (attention-deficit hyperactivity disorder) in children/young adults aged 6-15 years old. This alongside several other classes of diagnosis including broken bones and 'open wounds'. Participants were matched one-to-one with controls without said neuropsychiatric diagnoses and exposure(s) to various vaccinations - "influenza, tetanus and diphtheria (TD), hepatitis A, hepatitis B, meningitis, and varicella" - were 'tracked'. Interestingly, this is not the lead authors' first foray into using the MarketScan database [2], where autism was the previous topic of analysis (specifically healthcare service use and costs).

Results: bearing in mind that samples sizes varied according to those diagnoses under investigation, the authors report: "Receipt of any vaccine in the previous 6 months was highest for children with AN (21.4%), followed by OCD (15.9%) and tic disorder (15.8%), and was lowest for children with open wounds (10.3%)." This information needs to be treated carefully because - again - it tells us nothing about any cause-and-effect relationship, just correlation and trend; trends that could be there for all-manner of different reasons outside of the variables looked at. Further: "HRs [hazard ratio] associated with receipt of any vaccine were highest for children with AN... followed by OCD."

The authors also looked at specific vaccinations in relation to those neuropsychiatric disorders included in their study. They report: "Influenza vaccinations during the prior 3, 6, and 12 months were... associated with incident diagnoses of AN, OCD, and an anxiety disorder." Conversely: "children with major depression were less likely to have received the influenza vaccine in the previous 3 months" and "children with bipolar disorder were also less likely to have received the influenza vaccine in the previous 3 or 6 months."

OK, it is worth reiterating - yet again - that this was a study looking at possible associations and not necessarily cause-and-effect. Indeed, judging by that last paragraph and the table of HRs produced by the authors (see here) one might easily claim that flu vaccination might potentially shield someone from developing major depression as much as 'cause' AN, OCD and/or anxiety disorder. Such is the nature of such studies and the findings being reported. And indeed someone has actually looked at depressive symptoms (symptoms that is, not depression as a clinical diagnosis) before and after an influenza vaccination and found very little...

I note that with specific regard to the influenza vaccine and the findings that "children with AN, OCD, or a tic disorder were more likely to have received the influenza vaccine in the preceding periods" the authors head into the research talking about narcolepsy and the "AS03-adjuvanted H1N1 vaccine" as a possible template for their findings. This despite not covering the diagnosis of narcolepsy in their study (they could have). I've touched upon this area of research before on this blog (see here) (something that continues to appear in media discussions) and whilst not disputing the findings, do for example, wonder why in the work of Szakács and colleagues [3] ADHD was picked up as a comorbidity present in their "post-H1N1 vaccination (PHV) narcolepsy group" but in the Leslie data the HRs showed little evidence of any relationship. Yes, H1N1 vaccination is not necessarily the same as influenza vaccination reported in the Leslie data (we don't actually know what specific influenza vaccines were administered), but surely if discussions turn to an 'autoimmune' element as a possible mechanistic feature potentially linking vaccination and [some] neuropsychiatric disorder(s), one would expect to see the same/similar pattern of conditions being represented and reported? For balance, I should also point out that other independent study has talked about eating disorders potentially having "immune-mediated mechanisms" connected, particularly those associated with autoimmunity [4] although I don't doubt such a connection is likely complicated and probably not universally applicable.

It's not difficult to find issues with the Leslie paper and no doubt these will be emphasised in any further discussions about the data reported even when the authors stress throughout that "findings do not demonstrate a causal role of vaccination in the pathoetiology of any of these conditions." The strengths of the data - e.g. the use of that administrative database for confirming diagnoses and vaccine exposure - are worth mentioning again in light of other debates on data sources from other 'disappearing' manuscripts in this area. I might also add that by focusing in on various diagnoses but not autism (which again, they could have done) and not a certain vaccine, the authors seem well aware of the history in this area - "the association of the measles, mumps, and rubella vaccine with autism spectrum disorder has been convincingly disproven" - and probably either thought nothing more of it or chose to steer well clear of it (or perhaps a combination of both).

Is there a 'where next' when it comes to the Leslie data? To quote again from the paper: "findings require replication in a larger population-based sample, possibly including assessments of various potentially important host factors, e.g., the individual’s genomic and epigenomic background, the individual’s microbiome, their history of antecedent psychosocial stress, infections, as well as other potentially simultaneously administered vaccinations, the differences in vaccine types, and the route of administration (e.g., intramuscular or intranasal administration of influenza vaccine)." The scientific literature also provides some other avenues for further study in this area [5]. But science does not happen in a social vacuum; any further research needs to be done with care and focus. Mindful of both the important reliance on vaccination in protecting both the individual and population at large and mindful that sweeping generalisations (often emanating from scientific study) need to be minimised.

And on the topic of the research of Douglas Leslie, I'll be coming to another recent paper he published on the important issue of depression in parents of children diagnosed with autism [6] quite soon...

----------

[1] Leslie DL. et al. Temporal Association of Certain Neuropsychiatric Disorders Following Vaccination of Children and Adolescents: A Pilot Case-Control Study. Front Psychiatry. 2017 Jan 19;8:3.

[2] Wang L. et al. Healthcare Service Use and Costs for Autism Spectrum Disorder: A Comparison Between Medicaid and Private Insurance. Journal of autism and developmental disorders. 2013;43(5):1057-1064.

[3] Szakács A. et al. Psychiatric comorbidity and cognitive profile in children with narcolepsy with or without association to the H1N1 influenza vaccination. Sleep. 2015 Apr 1;38(4):615-21.

[4] Raevuori A. et al. The increased risk for autoimmune diseases in patients with eating disorders. PLoS One. 2014 Aug 22;9(8):e104845.

[5] Sadarangani SP. et al. "Let there be light": the role of vitamin D in the immune response to vaccines. Expert Rev Vaccines. 2015;14(11):1427-40.

[6] Cohrs AC. & Leslie DL. Depression in Parents of Children Diagnosed with Autism Spectrum Disorder: A Claims-Based Analysis. J Autism Dev Disord. 2017 Feb 18.

----------

Leslie DL, Kobre RA, Richmand BJ, Aktan Guloksuz S, & Leckman JF (2017). Temporal Association of Certain Neuropsychiatric Disorders Following Vaccination of Children and Adolescents: A Pilot Case-Control Study. Frontiers in psychiatry, 8 PMID: 28154539

Psychiatric comorbidity in post H1N1 vaccination narcolespy?

I can imagine that the paper by Atilla Szakács and colleagues [1] is likely to draw some rather differing opinions about potential importance based on their subject matter and methods/participant numbers looking at the frequency of psychiatric comorbidity among those with narcolepsy including narcolepsy post H1N1 vaccination. The fact that autism - "pervasive developmental disorder not otherwise specified (i.e., atypical autism)" - is mentioned as one of the comorbidity in the post-vaccination narcolepsy group, albeit only present in some 3% of the 31 participants (one person by my reckoning), potentially moves us into some contentious territory.

I can't go back to yesterday because
I was a different person then.

That being said, the finding that nearly half of the post-vaccination narcolepsy cases presented with some kind of psychiatric comorbidity and the lion's share of that comorbidity was represented by a diagnosis of attention-deficit hyperactivity disorder (ADHD) (~30%) is perhaps the bigger headline from this study. Or is it?

OK, it might be worthwhile taking this one step at a time. The report by the European Centre for Disease Prevention and Control (ECDC) [2] provides a good overview on the correlation between the use of the Pandemrix influenza vaccine and risk of narcolepsy in children and adolescents based on European data. The paper by Miller and colleagues [3] put a UK perspective on this, concluding that: "The increased risk of narcolepsy after vaccination with ASO3 adjuvanted pandemic A/H1N1 2009 vaccine indicates a causal association, consistent with findings from Finland". These findings and others suggested that there may be quite a bit more to see when it came to any relationship and, as a result, the door was opened to possible compensation as a vaccine-damage case (see the BBC report on this here). But, and it is an important point, other studies in other parts of the world have not reported a similar connection [4] and indeed, whether influenza infection (including 2009 H1N1 infection) itself could have been a risk factor for narcolepsy [5] has been discussed in the peer-reviewed literature.

The hows and whys of any relationship between the H1N1 influenza vaccine and [some] narcolepsy are still under discussion. The hypocretins have been the source of quite a bit of speculation [6] although the research road linking processes related to these compounds and the possible effects of vaccination has not run so smoothly (see here). Autoimmunity, or rather some of the genetics of autoimmunity, has also been highlighted as being a potentially revealing mechanism of involvement [7] perhaps shadowing some recent work in an unrelated area (see here). Indeed, another publication from Szakács and colleagues [8] hinted as much: "All patients in the postvaccination group were positive for human leukocyte antigen (HLA)-DQB1*0602".

Insofar as narcolepsy not related to vaccine administration, there is already a body of work to suggest quite a few comorbidities might already be elevated in risk following a diagnosis. The paper by Ohayon [9] detailed various somatic and psychiatric/behavioural issues to be over-represented alongside a diagnosis of narcolepsy. Of particular note to the psychiatric side of things were reports that: "major depressive disorder (MDD)... and social anxiety disorder" affected around 20% of people with the diagnosis. Both these conditions were noted in the Szakács study. Modestino & Winchester [10], based on the use of "a retrospective self-report questionnaire indicating the presence of childhood ADHD symptomatology in adults" suggested that "childhood ADHD symptomatology history among adult narcoleptics is common". Ideas about mis-diagnosis of ADHD covering for a condition like narcolepsy have also been suggested elsewhere (see here).

I will reiterate that the Szakács study only looked at 38 children and adolescents, and 31 of those participants fitted that post-vaccination narcolepsy criteria. This is a very small study where even the authors highlight that this means little can be conclusively taken from their findings. Insofar as that autism - atypical autism - finding, it is not beyond the realms of statistical possibility that such detection of a single case is entirely a fluke finding or indeed, something that might be expected given the increasing prevalence rate/estimates (see here). Indeed, I'll draw your attention to an earlier report on this matter [11].

The ADHD findings ("29% had attention deficit hyperactivity disorder (ADHD) inattentive type") are a little more difficult to explain away taking account of my previous chatter on the sample size of the Szakács paper. I've talked before on this blog about the paediatric ADHD prevalence figures/estimates (see here) and their hovering between 5-8% in the general population based on the collected work so far published in this area. Diagnoses like autism do seem to carry a greater risk of ADHD (see here) but given that only one case of autism was detected in the Szakács sample, I don't think we can say that this factor accounts for the results reported.

I draw back from suggesting that ADHD (in any or all its forms) is definitely related to narcolepsy post 2009 H1N1 vaccination or indeed natural infection on the strength of the current data and small (unevenly distributed) participant groups. What however the Szakács findings do perhaps impress, is the need for greater scrutiny of this possible variable; if anything else, to potentially highlight any possible mechanisms of response and possibly offer some further insights into the underlying genetics and biochemistry of [at least some] ADHD.

I close with the official (UK) advice on flu vaccination (see here).

Music: Wild Honey by Hugh Laurie (House).

----------

[1] Szakács A. et al. Psychiatric Comorbidity and Cognitive Profile in Children With Narcolepsy With or Without Association to the H1N1 Influenza Vaccination. Sleep. 2014 Oct 17. pii: sp-00241-14.

[2] European Centre for Disease Prevention and Control. Narcolepsy in association with pandemic
influenza vaccination (a multi-country European epidemiological investigation) Stockholm: ECDC; September 2012.

[3] Miller E. et al. Risk of narcolepsy in children and young people receiving AS03 adjuvanted pandemic A/H1N1 2009 influenza vaccine: retrospective analysis. BMJ. 2013 Feb 26;346:f794.

[4] Duffy J. et al. Narcolepsy and influenza A(H1N1) pandemic 2009 vaccination in the United States. Neurology. 2014 Oct 15. pii: 10.1212/WNL.0000000000000987.

[5] Han F. et al. Narcolepsy onset is seasonal and increased following the 2009 H1N1 pandemic in China. Ann Neurol. 2011; 70: 410–417.

[6] Hungs M. & Mignot E. Hypocretin/orexin, sleep and narcolepsy. BioEssays. 2001; 23: 397-408.

[7] Mahlios J. et al. The autoimmune basis of narcolepsy. Curr Opin Neurobiol. 2013 Oct;23(5):767-73.

[8] Szakács A. et al. Increased childhood incidence of narcolepsy in western Sweden after H1N1 influenza vaccination. Neurology. 2013 Apr 2;80(14):1315-21.

[9] Ohayon MM. Narcolepsy is complicated by high medical and psychiatric comorbidities: a comparison with the general population. Sleep Med. 2013 Jun;14(6):488-92.

[10] Modestino EJ. & Winchester J. A retrospective survey of childhood ADHD symptomatology among adult narcoleptics. J Atten Disord. 2013 Oct;17(7):574-82.

[11] Cohen BI. Possible connection between autism, narcolepsy and multiple sclerosis. Autism. 1998; 2: 425-427.

----------

Szakács A, Hallböök T, Tideman P, Darin N, & Wentz E (2014). Psychiatric Comorbidity and Cognitive Profile in Children With Narcolepsy With or Without Association to the H1N1 Influenza Vaccination. Sleep PMID: 25325473