Showing posts with label replication. Show all posts
Showing posts with label replication. Show all posts

Saturday, 16 March 2019

PACE trial for chronic fatigue syndrome (still) being put through its paces: a reply

I'm bringing the paper published by Michael Sharpe and colleagues [1] to your attention today in the interest of balance and peer-reviewed 'right to reply'.

The Sharpe paper concerns the PACE trial, the study which reported that "when added to specialist medical care, cognitive behaviour therapy and graded exercise therapy were more effective in improving both fatigue and physical function in participants with CFS [chronic fatigue syndrome], than both adaptive pacing therapy and specialised medical care alone."

Anyone with a little bit of knowledge about the PACE trial will know that it's a 'contentious' topic within CFS (and ME, myalgic encephalomyelitis) circles. Indeed, the Sharpe paper comes about as a direct result of a reanalysis paper (see here) which reported findings raising "serious concerns about the robustness of the claims made about the efficacy of CBT [cognitive behavioural therapy] and GET [graded exercise therapy]" in the context of CFS/ME. 'Serious concerns' is putting it mildly considering how others have described the PACE trial and some of its tenets (see here).

Sharpe et al, who were authors listed on the original PACE trial paper [2], have had to defend their work/findings before in the peer-reviewed realm (see here). Same as before, the name Carolyn Wilshire is addressed and her teams reanalysis of the PACE trial data [3]. Said data was, I might add, (partially) released only following intervention from the Information Commissioners Office (ICO) here in Blighty (see here). More recent events have similarly reiterated that 'access to raw study data' is something that CFS/ME researchers perhaps need to bear in mind at study conception (see here).

I'm not going to clinically dissect the Sharpe paper in this post because (a) 'interpretation' forms quite a bit of the reply to the Wilshire reanalysis, and (b) your opinion on the scientific quality of the Sharpe reply is most likely going to be shaped by where you stand in terms of the whole CBT/GET for CFS/ME discussion. Indeed, a peer-reviewer of the Sharpe paper also said as much (see here). What I will comment on is how the Wilshire reanalysis paper and the more recent Sharpe reply to the reanalysis paper might further inform research more generally with CFS/ME in mind.

Oh, and it's probably just a coincidence that the Sharpe paper comes out only days after a news headline reads "Online activists are silencing us, scientists say" talking about a familiar topic.

So:

  • Point 1: Design a good trial analysis plan and stick to it. From my 'outsider looking in' perspective, the changes made to "the scoring of the pre-specified outcomes" regarding fatigue and physical functioning in the PACE trial, however innocent they might have been, have created tension. Lots of tension. Such changes, whether agreed by "Trial Data Monitoring and Steering Committees" or not, can be construed in various different ways. It's better not to make such changes in the first place.
  • Point 2: If you are going to study something like physical functioning in relation to CFS/ME, don't just rely on things like questionnaires and Likert scales; use actigraphy too. Self-report is always a good thing but I've never understood why, with the wide range of cost-effective technology out there (available I believe, even in the early 2000s), wearable trackers such as pedometers or similar were not also utilised during such studies (see here). If you're spending £5 million on a trial, a few quid for some pedometers is not exactly going to break the bank and will inevitably bring some further quality data to the table.
  • Point 3: Recovery. As per other discussions (see here), most people would characterise recovery as a complete remission of symptoms and/or return to typical functioning. If you're not going to use this description, don't use the word recovery. Use something else instead. Indeed, use 'partial remission' or 'improvement' if you need to but don't call anything less than the complete remission of symptoms 'recovery'.
  • Point 4: Long-term outcomes. It's probably best to avoid any sweeping statements after the arms of a trial - a "randomised trial" not necessarily a "randomised controlled trial" according to Wilshire et al - have been completed. More so when you're measuring such long-term outcomes via a postal question minus any objective measure(s) (see point 2). It's probably also a good idea to ask patients about their quality of life too and whether that has changed (see here).
  • Point 5: Even if your paper states in no uncertain terms that: "The effectiveness of behavioural treatments does not imply that the condition is psychological in nature" the use of something like CBT for CFS/ME implies that you probably think there is a substantial psychological 'component' to the condition. This is compounded when you're for example, a Professor of Psychological Medicine. If you were pitting CBT in particular against a specific pharmacological or biological intervention 'for CFS/ME' (see here for example), I'd be more inclined to see your view in a more 'rounded sense'. Indeed, if you were to study one or two biological parameters as well as behavioural ones looking for any change following intervention, you might convince more people that psychology is not the primary line you take. And whilst on the topic of psychology and CFS/ME, it's probably also best not to use 'psychobabble' terms like 'deconditioning' in your research. Such terms are pretty much scientifically untestable and, given the recent discussions about the legacy of some adherents to something like psychosomatic research (see here), is likely to be consigned to the scientific dustbin as some later point.

I think I've covered the main points as I see them. Please feel free to agree/disagree as you wish.

End of Line.

Addition: 26 March 2019. Not quite 'End of Line' it seems, as a reply to a reply to a reply emerges [4]. Peer reviewed science is far from slow...

----------

[1] Sharpe M. et al. The PACE trial of treatments for chronic fatigue syndrome: a response to WILSHIRE et al. BMC Psychology. 2019; 7: 15.

[2] White PD. et al. Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial. Lancet. 2011; 377(9768):823-36.

[3] Wilshire CE. et al. Rethinking the treatment of chronic fatigue syndrome—a reanalysis and evaluation of findings from a recent major trial of graded exercise and CBT. BMC Psychology. 2018; 6: 6.

[4] Wilshire CE. & Kindlon T. Response: Sharpe, Goldsmith and Chalder fail to restore confidence in the PACE trial findings. BMC Psychology. 2019; 7: 19.

----------

Thursday, 1 November 2018

"abnormally high extra-axial cerebrospinal fluid (CSF) volume" and autism

"Increased extra-axial CSF [cerebrospinal fluidvolume is a reliable brain anomaly that has now been found in three independent cohorts, comprising both high-risk and normal-risk children with autism spectrum disorder."

So concluded the findings reported by Mark Shen and colleagues [1] following their study results - "case-control MRI study" results - enquiring whether "increased extra-axial CSF volume is found in a large, independent sample of children diagnosed with autism spectrum disorder, whether extra-axial CSF remains abnormally increased beyond infancy, and whether it is present in both normal-risk and high-risk children with autism."

This latest study from Shen et al follows a research theme [2], a quite long running research theme [3] by all accounts, observing that extra-axial CSF volume might be important to at least some autism. Increased extra-axial CSF volume is described as a brain anomaly insofar as representing a larger than expected volume of cerebrospinal fluid 'coating' the brain, or at least filling the extra-axial space sitting on top and around the brain. It's perhaps not surprising that this work has also included the words 'brain enlargement' in discussions given the physical effect that such increased volume might have.

This latest chapter in the extra-axial CSF volume research comes from a familiar research initiative - the UC Davis MIND Institute Autism Phenome Project - "a longitudinal analysis of children diagnosed with autism spectrum disorder and age-matched typically developing children." The Autism Phenome Project (APP) has already been discussed a few times on this blog for various research reasons (see here and see here). The APP also has something of an interest in brain enlargement appearing alongside regression too (see here). This time around "159 children with autism spectrum disorder (132 male, 27 female) and 77 with typical development (49 male, 28 female) underwent MRI scans." Researchers were looking at extra-axial CSF volume as well as things like brain volume and head circumference. Alongside, various other behavioural measures and questionnaires on things like sleep were included.

"The autism spectrum disorder group had an average of 15·1% more extra-axial CSF than controls after accounting for differences in brain volume, weight, age, and sex." Further: "Both extra-axial CSF volume... and brain volume... uniquely contributed to enlarged head circumference in the autism spectrum disorder group." Authors also reported that: "Increased extra-axial CSF volume was associated with greater sleep disturbances... and lower non-verbal ability."

As per the opening quote to this post, this is not the first time that some of those findings have been reported in the science arena and I very much doubt that it will be the last time either. The authors also talk about such MRI findings in light of "normal risk (ie, from simplex families) or high risk (ie, from multiplex families)" for autism and applying "a previously validated machine learning algorithm based on extra-axial CSF volume, brain volume, age, and sex" but I'd like to see a lot more data before venturing further into these aspects. Not least, data covering the question of 'why?'. From the previous studies in this area, some hypotheses have been put forward, for example: "[as] CSF circulates through the developing brain, it removes inflammatory cytokines and proteins secreted by neurons that can otherwise accumulate and have a pathological effect on brain development." Such a hypothesis needs further research but is perhaps complementary to other discussions about 'neuroinflammation' in the context of autism (see here).

We await further investigations.

----------

[1] Shen MD. et al. Extra-axial cerebrospinal fluid in high-risk and normal-risk children with autism aged 2-4 years: a case-control study. Lancet Psychiatry. 2018 Sep 27. pii: S2215-0366(18)30294-3.

[2] Shen MD. et al. Increased Extra-axial Cerebrospinal Fluid in High-Risk Infants Who Later Develop Autism. Biol Psychiatry. 2017 Aug 1;82(3):186-193.

[3] Shen MD. et al. Early brain enlargement and elevated extra-axial fluid in infants who develop autism spectrum disorder. Brain. 2013;136(9):2825-2835.

----------

Wednesday, 23 October 2013

Put it in the (autism science) replicator

Replication is one of the most important processes in those things we call science and the scientific method. Outside of conjuring up images of a certain 'Make it so' Captain with his "Earl Grey, Hot", scientific replication provides the community at large with some degree of reassurance that a research finding was not just a fluke or an artefact of a particular sample of people or a method used or an interpretation of results. As per the recent BBC article on vitamin use: "Looking at any one individual study won't be very revealing to answer the question of whether vitamin supplementation is good for you."
Uncanny likeness @ Wikipedia 

With the issue of replication in mind, I was interested to read the Letter to the Editor from Robinson and colleagues* (open-access) who set about trying to replicate the findings from Skafidas and colleagues** (open-access) and their notion that science might be making some in-roads into the detection of "genetic biomarkers [that] can correctly classify ASD from non-ASD individuals". I posted about the Skafidas study at the time also (see here) and their analysis of single-nucleotide polymorphisms (SNPs) in relation to autism spectrum disorder (ASD).

The Robinson letter reports an attempt to replicate the Skafidas findings based on an independent analysis of data from the Psychiatric Genomics Consortium (PGC) "which includes ~5400 cases, more than three times the number used in the original [Skafidas] report". I'm not on this occasions going to get the fine-toothed comb out on both papers because they're open-access so free for anyone to read.

The conclusions from Robinson et al are pretty clear: "We find no evidence that the implicated SNPs, the classifier or the pathways named in Skafidas et al.1 are associated with ASDs. We therefore conclude that the classifier, as presented, cannot be used in a general way to predict ASDs, and consequently is unlikely to have any translational value."

Obviously such findings are both a blow to autism research and also the original authors who first proposed the classifier model, who I don't doubt probably invested quite a lot of time, effort and funds into getting their experiments done and results published (and published in a Nature journal). The ego also takes a bit of a knock under such circumstances, believe me (see here and here and here).

The Robinson data however re-emphasize the importance of replication in autism research. Perhaps just as important, they also reaffirm that autism is a tremendously difficult set of conditions to study. As is often the case when it comes to a heterogeneous condition like autism (or should that be the autisms) often carrying more than its fair share of comorbidity (see here) including risk of certain somatic conditions (see here), consistent findings are often few and far between. Indeed, that the use of the label autism, whilst providing a way of classifying certain types of behaviour and their impact on a person's life, is not necessarily the best thing for research purposes, as was vocalised through the grudge match that was DSM V vs. RDoC (see here).

The added realisation that outside of no one single SNP being linked to all autism (see here) there may be a significant degree of overlap when it comes to the genetics of the autisms with other developmental and psychiatrically defined conditions (see here) implies that it's going to be some time yet before any genetic biomarkers or test is going to be able to accurately classify autism, sorry the autisms with any great accuracy. Then there is the question of what such a test would accomplish. I've not even mentioned the fact that autism, whilst having a genetic component, is probably not without it's [variable] partner in crime, environment (however you want to define this) when it comes to aetiology. And don't even mention that other area of increasing interest, epigenomics (see here)... which in recent days has seen some interesting papers published (see here and see here).

I suppose in the spirit of all this talk on replication, the last question should be: who next is going to try and replicate the Robinson results? Indeed, does science any longer need the 'Letter to the Editor' in light of the rolling out of PubMed Commons?

----------

* Robinson EB. et al. Response to ‘Predicting the diagnosis of autism spectrum disorder using gene pathway analysis’. Molecular Pyschiatry. 2013: Oct 22. doi: 10.1038/mp.2013.125

** Skafidas E. et al. Predicting the diagnosis of autism spectrum disorder using gene pathway analysis. Molecular Psychiatry. 2013; Sep 11. doi: 10.1038/mp.2012.126

----------

ResearchBlogging.org E B Robinson, D Howrigan, J Yang, S Ripke, V Anttila, L E Duncan, L Jostins, J C Barrett, S E Medland, D G MacArthur, G Breen, M C O'Donovan, N R Wray, B Devlin, M J Daly, P M Visscher, P F Sullivan, B M Neale (2013). Response to ‘Predicting the diagnosis of autism spectrum disorder using gene pathway analysis’ Molecular Psychiatry DOI: 10.1038/mp.2013.125