Publication bias and autism research

The title of the paper by Konstantin Mechler and colleagues - "Defining the hidden evidence in autism research. Forty per cent of rigorously designed clinical trials remain unpublished - a cross-sectional analysis" [1] - provides some discussion today. Drawing on the ideas that publication bias and/or the so-called 'file-drawer problem' - terms that refer to the non-publication of study results potentially skewing the collected scientific opinion in a particular area - might also extend into autism research too, Mechler et al detail results according to their analysis of a particular trial registration database called: ClinicalTrials.gov.

ClinicalTrials.gov is one of the premier 'tell everyone about your trial' databases designed to provide a bit of transparency to science. The idea is that you register (pre-register hopefully) your research study, register what you are going to do and how, and importantly, provide some details about what your are going to be assessing (outcomes). Some of the research that I've been involved with has a mention in this database (see here) with more to come in future times. Once your entry is in ClinicalTrials.gov it kinda stops you from making any major 'alterations' to your study potentially based on the results you get, whilst at the same time also getting quite a few prods to post things like your raw study results for everyone to see and analyse as they wish. And believe me, they are quite a persistent bunch over at ClinicalTrials.gov!

So Mechler and colleagues "searched for all completed randomized controlled clinical trials investigating interventions in ASD [autism spectrum disorder] and their results made public." They looked at how many trials had been submitted and how many reported results. Where no results were available on ClinicalTrials.gov or on other 'scientific databases' or after "enquiries of the responsible parties or sponsors listed", the authors listed the trial as 'not published'.

The good news: 60% of trials (n=30) were published in the peer-reviewed domain. The not-so-good news was that reported in the opening sentence of this post as some 40% of trials (n=20) fell into that not published category. Authors also mention that some 1600 participants were included in those not published trials, inferring that data on quite a few people diagnosed with an ASD and agreeing (themselves or by proxy) to participate in research remain 'missing' in a research sense.

"The results emphasize the serious issue of publication bias. The large proportion of unpublished results precludes valuable information and has the potential to distort evidence for treatment approaches in ASD." This an important point in any area of science but perhaps more so when you have a label like autism and a whole host of 'unknowns' about the aetiology, nature and course of presentation. I would like to think that there were some rational reasons why so many trials were missing results (studies having to be halted/stopped, collaborations breaking down, resources being exhausted, etc) but there is always going to suspicion around the non-publication of such results particularly the idea that 'researchers or funders did not get the results they hoped for'. Other discussions on this topic outside of autism (see here) point to the failure to publish being tantamount to an ethical breach. Strong words indeed.

Although Mechler et al focused on data from the ClinicalTrials.gov initiative I'd perhaps suggest that the issues they discuss probably go a lot deeper as a function of there being quite a few other databases where such trial information is held and listed. Perhaps also just as concerning is the fact that many other pieces of research are not listed anywhere when it comes to trial registration and therefore have even less 'motivation' to publish results or are perhaps more likely to be subject to 'alteration' in terms of methodology or outcome(s). All of which contributes to the possible tarnishing of science and the reporting of said science and paves the way for criticism.

Music, it's not Mother's Day or anything like that but do give her a call from time-to-time and treat your mother right...

----------

[1] Mechler K. et al. Defining the hidden evidence in autism research. Forty per cent of rigorously designed clinical trials remain unpublished - a cross-sectional analysis. International Journal of Methods in Psychiatric Research. 2016. Nov 9.

----------

Mechler K, Hoffmann GF, Dittmann RW, & Ries M (2016). Defining the hidden evidence in autism research. Forty per cent of rigorously designed clinical trials remain unpublished - a cross-sectional analysis. International journal of methods in psychiatric research PMID: 27862603