Questioning Answers: Systematic reviews and autism

Monday, 5 January 2015

Systematic reviews and autism

There were a few reasons why I wanted to bring the commentary from Sven Bölte [1] on the topic of systematic reviews and autism research to your attention. One particular sentence included in the text stuck out for me: "... systematic reviews do not always tell the whole truth either" reflective of how we perhaps should always be a little cautious in the way we interpret science even when faced with the platinum standard that is the systematic review (with or without meta-analysis).

I'm not slaying anything. I don't slay, so put it out of your mind.

I'll hold my hand up and say that I have a professional interest in discussions on the use of systematic reviews - "A systematic review attempts to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a given research question" - with autism in mind. My research history includes more than a smidgen of work looking experimentally at the use of gluten and casein-free diets for [some] autism [2] which has been the subject of quite a few reviews down the years [3]. I'd like to think that I'm objective and dispassionate in my efforts, but as with anyone who devotes a considerable amount of their life largely looking at one specific topic, I'd also like to think the may be something in the whole diet side of things for [some] autism [4] and I haven't been wasting my time (or others' funds).

The Bölte paper is open-access, so it doesn't need any grand discussions from me. I do like the idea that the topic of communicating science is touched upon in the paper and how a 'lack of evidence' can sometimes be mistaken for “No evidence for anything” particularly when most reviews end with the requirement for more rigorous study of the particular field in question. Even better the mention of how the scientific press release should sometimes be cautiously interpreted as per other recent discussions in this area (see here).

Autism is quite a tough condition to scientifically investigate. The catch-all label covers an astounding degree of heterogeneity and elevated risk of quite a bit of comorbidity (the 'autisms'?). Such factors can play havoc with the way that things like randomised controlled trials are organised and conducted and indeed, how evidence is synthesised. It's little wonder that some researchers have talked about looking at new ways of doing research when it comes to autism (see here) with more value placed on concepts like the N=1 (see here) and the issue of responders and non-responders when it comes to things like intervention (see here). Without trying to commit scientific heresy, I'd be minded to agree that there may be 'better' ways of organising/doing autism research that complement the gold standards, and as per Prof. Bölte's assertion that: "many studies that could contribute important information often are not included" in such systematic reviews.

Some music to close: Prokofiev - Dance of the Knights. As well as being the opening music for a certain 'You're fired' program, supporters of our local football (soccer) team might also recognise it as 'grand entrance' music...

----------

[1] Bölte S. The good, the bad and systematic reviews. Autism. 2015; 19: 3-5.

[2] Whiteley P. et al. The ScanBrit randomised, controlled, single-blind study of a gluten- and casein-free dietary intervention for children with autism spectrum disorders. Nutr Neurosci. 2010 Apr;13(2):87-100.

[3] Marí-Bauset S. et al. Evidence of the gluten-free and casein-free diet in autism spectrum disorders: a systematic review. J Child Neurol. 2014 Dec;29(12):1718-27.

[4] Whiteley P. Nutritional management of (some) autism: a case for gluten- and casein-free diets? Proc Nutr Soc. 2014 Oct 14:1-6.

----------