Differentiating between autism and ADHD the machine learning way (again)

So: "These results support the potential of creating a quick, accurate and widely accessible method for differentiating risks between ASD [autism spectrum disorder] and ADHD [attention-deficit hyperactivity disorder]."

That was a conclusion reached in the paper by Marlena Duda and colleagues [1] (open-access) building on their previous foray into this important research area (see here). Last time around [2] this research group - the Duda/Wall et al research combination - set the scene for boiling down the Social Responsiveness Scale (SRS) from 65 items to something considerably smaller when it came to distinguishing autism from ADHD. This based on the idea that autism and ADHD are not unstrange diagnostic bedfellows (see here).

This time around, researchers set out to "expand upon our prior work by including a novel crowdsourced data set of responses to our predefined top 15 SRS-derived questions from parents of children with ASD (n=248) or ADHD (n=174) to improve our model’s capability to generalize to new, ‘real-world’ data." Mention of the term 'crowdsourced' means that authors utilised various online social media platforms to "to inform the community of the study" and gather responses. Importantly, they note that "diagnoses of ASD or ADHD were provided as parent report."

Results: once again applying various machine learning algorithms to their recently captured data and "mixing these novel survey data with our initial archival sample (n=3417)" authors reported some interesting findings. Taking the two samples - the archival samples and the recent crowdsourced data  - together they reported on the creation of "a classification algorithm that can generalize well to unseen data (AUC=0.89±0.01), even when those data have more natural variablity like the kind seen in our survey sample." This was based on the use of 15 items from the SRS.

But... things were not all smooth sailing in this latest research effort. Bearing in mind the use of those 'parent reported' autism and ADHD participants in this latest study, authors noted that 'real-world' data is not necessarily the same as the very clinical data relied upon on the last research occasion. So: "In the archival sample, the responses for ADHD subjects were more uniform and on average less severe than the ADHD responses in the survey sample."

Still, these are important results albeit requiring 'continued evaluation' as further crowdsourced and other data filter through. Indeed 'adaption' to new data seems to be something that the authors are particularly keen on to "further improve the generalizability of the classifier." I continue to applaud their research in this area as a function of their efforts (see here) to make autism and/or ADHD screening quicker, easier and more cost-effective.

And on that last point. it is timely that such research continues given what is being proposed in certain parts of England when it comes to autism diagnoses (see here). Indeed, the suggestion of "restricting an autism diagnosis to only the most severe cases" as a function of some quite spectacular increasing demand - "The team is supposed to carry out 750 assessments a year. But it is getting almost double that level of demand, with about 25 referrals a week" - reiterates a need to streamline diagnostic services to make screening/diagnosis quicker, easier and more cost-effective.

For those also who have said 'so what' to the increase in cases of autism (yes, someone actually did albeit with caveats), such proposals to potentially restrict autism diagnoses, I would say, are a direct result of such a mindset to 're-think' autism. Although well meaning, if enough people talk about difference over disability for example, purse string holders in the NHS (National Health Service) were eventually bound to ask 'why diagnose?' and 'why offer services?' (services that can cost quite a lot and even for those with 'severe autism' are often not there). As other authors have eloquently argued (see here) and indeed, foretold, mixed in with the current economic situation being put forward all in the name of austerity, low-hanging NHS services fruit like autism screening/assessment were certain to be eventually targeted and the 'difference over disability' framing unfortunately provides ample ammunition for such proposals...

----------

[1] Duda M. et al. Crowdsourced validation of a machine-learning classification system for autism and ADHD. Transl Psychiatry. 2017 May 16;7(5):e1133.

[2] Duda M. et al. Use of machine learning for behavioral distinction of autism and ADHD. Transl Psychiatry. 2016 Feb 9;6:e732.

----------

Duda M, Haber N, Daniels J, & Wall DP (2017). Crowdsourced validation of a machine-learning classification system for autism and ADHD. Translational psychiatry, 7 (5) PMID: 28509905

Diagnosing ME/CFS the machine learning way?

In today's post I want to draw your attention to the findings reported by Diana Ohanian and colleagues [1] (open-access available here) talking about "the use of machine learning to further explore the unique nature"of various conditions/labels including those typically headed under the label of chronic fatigue syndrome / myalgic encephalomyelitis (CFS/ME).

Including one 'Jason LA' on the authorship list, researchers set about looking at "what key symptoms differentiate Myalgic Encephalomyelitis (ME) and Chronic Fatigue syndrome (CFS) from Multiple Sclerosis (MS)."You may be wondering why such a comparative study was undertaken but a quick trawl of the research literature reveals that these different clinical labels may well have some important commonalities [2].

This was an internet-based research project whereby "106 people with MS and 354 people with ME or CFS fully completed the [DePaul Symptom Questionnaire] questionnaire" and based on the responses received "decision trees were used to determine what symptoms differentiated those with MS from those with ME or CFS." Decision trees, as the name suggests, is a statistical technique where binary (0 or 1, no or yes) choices make branches and: "At each branch the computer decides what symptom would best predict classifications, in this case whether someone has MS or ME or CFS." This process continues and continues through the different levels of branches "until the tree reaches a balance between classification accuracy and generalizing to new data." Such a machine learning tool has been previously discussed quite recently on this blog (see here).

Results: "Five symptoms best differentiated the groups." These were: flu-like symptoms, tender lymph nodes, alcohol intolerance, inability to tolerate upright position and next day soreness after strenuous activity. The first two symptoms - flu-like symptoms and tender lymph nodes - were pretty good by themselves at correctly categorising MS or CFS/ME (~80% correct). Indeed, these seemed to be the core differentiators that were examined and as the authors note: "The most important two symptoms that differentiated MS versus ME or CFS existed within the immune domain."

Of course further investigations are warranted to potentially build on these findings. One has however to be slightly cautious about the use of the internet and social media when undertaking such research, especially when very little information about the formal diagnoses of participants is included in the current paper. This is a particular issue when it comes to CFS/ME and the various ways that it can be defined and diagnosed [3].

Still, I can't quibble with the continued rise and rise of machine learning being applied to many areas of medicine, and not before time that it starts to reach ME/CFS. And just before I go, it appears that the research team at DePaul University have been quite busy...

To close, on what retiring Presidents of the USA should do next. I think I would go with George Washington and his whisky business... 🍻

----------

[1] Ohanian D. et al. Identifying Key Symptoms Differentiating Myalgic Encephalomyelitis and Chronic Fatigue Syndrome from Multiple Sclerosis. Neurology (ECronicon). 2016;4(2):41-45.

[2] Morris G. & Maes M. Myalgic encephalomyelitis/chronic fatigue syndrome and encephalomyelitis disseminata/multiple sclerosis show remarkable levels of similarity in phenomenology and neuroimmune characteristics. BMC Medicine. 2013; 11: 205.

[3] Jason LA. et al. Case definitions integrating empiric and consensus perspectives. Fatigue. 2016;4(1):1-23.

----------

Ohanian D, Brown A, Sunnquist M, Furst J, Nicholson L, Klebek L, & Jason LA (2016). Identifying Key Symptoms Differentiating Myalgic Encephalomyelitis and Chronic Fatigue Syndrome from Multiple Sclerosis. Neurology (E-Cronicon), 4 (2), 41-45 PMID: 28066845

'My child is not talking'. Online concerns and internet-based screening for autism?

"Online communities are used as platforms by parents to verify developmental and health concerns related to their child."

That was the starting point for the study results reported by Ben-Sasson & Yom-Tov [1] (open-access available here) who approached an increasingly important issue related to how the Internet and social media in particular, is fast becoming one of the 'go-to' options when it comes to parental concerns about their child's development and the question: could it be autism?

So: "we analyzed online queries posed by parents who were concerned that their child might have ASD and categorized the warning signs they mentioned according to ASD [autism spectrum disorder]-specific and non-ASD-specific domains." The online queries included for study came from "the Yahoo Answers platform" between June 2006 and December 2013. There's a lesson there to reiterate that the Internet is an open platform and what you post is typically in the public domain and hence fodder for many different purposes...

Authors turned up quite a few thousand queries, determining that over 1000 were "posted by parents who suspected their child might have autism". They randomly selected 195 to be used as the basis for this study. I personally don't know why 195 were selected and not rounded up to say 200, but ho-hum. Content analysis - analysing the content of the post! - was undertaken first "to rate a child's risk of ASD as either low, medium, or high". High risk was defined "as concerns related to at least two types of ASD-specific sign, 1 from the RRBI domain and another from the Social and Communication domains" among other things. Then content analysis was used to "identify the types of warning signs noted by parents." From these analyses: "each query received an ASD global risk score and was coded for either presence or absence of each sign domain and its subdomains."

Results: from the 195 queries selected, the vast majority were posted in relation to a boy and most concerned a boy who was aged under 3 years. Contrary to the title of this blog post - 'My child is not talking' - the majority of queries were actually in relation to repetitive and restricted behaviors and interests (RRBI) although concerns related to language were not too far behind in frequency. In relation to those categorisations of low, medium and high risk groups, over half of the queries were labelled as high risk. Interestingly, there were fewer language concerns noted in those allocated to the low risk group than the medium or high risk groups, so perhaps I wasn't so far off with using those 'my child is not talking' words in the title.

But things didn't just stop there for the authors, as the words "test the efficacy of machine learning tools in classifying the child's risk of ASD based on the parent's narrative" are also noted in their paper. Machine learning as in, 'giving computers the ability to learn without being explicitly programmed' according to one definition, is something that has cropped up on the blog before with autism in mind (see here for example). This led to the production of a decision tree - yes or no - "for distinguishing low-risk queries from medium- and high-risk queries." This is interesting but I'd perhaps like to see it tested independently before I say too much more.

In these days of continued austerity and seemingly evermore limited resources when it comes to things like autism assessment and screening for various reasons, this kind of work has an important place. Certainly I don't think posting symptoms on-line with ever replace autism screening, and one has to bear in mind that at least here in the UK, we might have (knowingly or unknowingly) already initiated population autism screening in children (see here) as a consequence of changes to the Healthy Child Program. But with the technological advances being made where machine learning and the connected artificial intelligence are starting to make strides in relation to science and medicine, I don't doubt that one day parents will be typing in their child's symptoms on-line and somehow and somewhere Dr Google or some related system(s) might be talking back...

Music and more bad lip reading applied to Star Wars: No, it's not the future (and watch Chewie holler).

----------

[1] Ben-Sasson A. & Yom-Tov E. Online Concerns of Parents Suspecting Autism Spectrum Disorder in Their Child: Content Analysis of Signs and Automated Prediction of Risk. J Med Internet Res. 2016 Nov 22;18(11):e300.

----------

Ben-Sasson A, & Yom-Tov E (2016). Online Concerns of Parents Suspecting Autism Spectrum Disorder in Their Child: Content Analysis of Signs and Automated Prediction of Risk. Journal of medical Internet research, 18 (11) PMID: 27876688

Machine learning applied to autism screening going big time?

Machine learning, when machines, er.. learn, is of growing interest to the autism research field. The names Wall and Duda have filled quite a few posts on this blog (see here and see here for example) on this topic and their suggesting that applying machine learning algorithms to something like autism screening and detection could cut down on time taken and resources used.

As per the publication of the paper by Daniel Bone and colleagues [1] it appears that others working in autism research are also waking up to the idea that this might be a useful area to investigate. So: "In this work, we fastidiously utilize ML [machine learning] to derive autism spectrum disorder (ASD) instrument algorithms in an attempt to improve upon widely used ASD screening and diagnostic tools." Fastidiously is such a lovely word (particularly in the context of science).

The tools in question were the Autism Diagnostic Interview-Revised (ADI-R) and Social Responsiveness Scale (SRS) (both of which have already been machine learning 'applied') and their scores "for 1,264 verbal individuals with ASD [autism spectrum disorder] and 462 verbal individuals with non-ASD developmental or psychiatric disorders, split at age 10." And the results... well, let's just say that the authors were not disappointed - or at least less disappointed than on previous research occasions [2] - as they reported on created algorithms that "were more effective (higher performing) than the current algorithms, were tunable (sensitivity and specificity can be differentially weighted), and were more efficient (achieving near-peak performance with five or fewer codes)." Indeed: "We present a screener algorithm for below (above) age 10 that reached 89.2% (86.7%) sensitivity and 59.0% (53.4%) specificity with only five behavioral codes." Sensitivity and specificity are important concepts when it comes to something like screening instruments in terms of identifying 'all' those with a specific condition and making sure that no 'not-cases' aren't mistakenly identified as 'cases'. The nearly 90% sensitivity rate presented by Bone et al on the basis of 5 behavioural codes is not to be sniffed at.

The addition of one Cathy Lord to the authorship of the Bone paper also adds an air of inevitability that applying machine learning to autism research (and practice) is going to continue and increase. Not only because of her historical connection to the ADI-R [3] (which is a hefty document in anyone's book) but also given her very prominent role in autism research history. Who knows, I might one day be blogging about more big autism research names talking about Wall/Duda things including autism screening triage by YouTube? The final question is: outside of just behavioural variables, who would be brave enough to talk genetics/epigenetics/biology machine learning as the next step in autism screening and/or assessment?

----------

[1] Bone D. et al. Use of machine learning to improve autism screening and diagnostic instruments: effectiveness, efficiency, and multi-instrument fusion. J Child Psychol Psychiatry. 2016 Apr 19.

[2] Bone D. et al. Applying machine learning to facilitate autism diagnostics: pitfalls and promises. J Autism Dev Disord. 2015 May;45(5):1121-36.

[3] Lord C. et al. Autism Diagnostic Interview-Revised: a revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. J Autism Dev Disord. 1994 Oct;24(5):659-85.

----------

Bone D, Bishop S, Black MP, Goodwin MS, Lord C, & Narayanan SS (2016). Use of machine learning to improve autism screening and diagnostic instruments: effectiveness, efficiency, and multi-instrument fusion. Journal of child psychology and psychiatry, and allied disciplines PMID: 27090613

Introducing MARA: The Mobile Autism Risk Assessment

I just had to post an entry about the latest Duda/Wall combo paper [1] continuing their machine learning voyage through autism screening and assessment (see here) culminating in an important end-point: the MARA - Mobile Autism Risk Assessment.

So, what is the MARA? Well, we are told it is: "a new, electronically administered, 7-question autism spectrum disorder (ASD) screen to triage those at highest risk for ASD."

What seven questions?

"1. How well does your child understand spoken language, based on speech alone? (Not including using clues from the surrounding environment)

2. Can your child have a back-and-forth conversation with you?

3. Does your child engage in imaginative or pretend play?

4. Does your child play pretend games when with a peer? Do they understand each other when playing?

5. Does your child maintain normal eye contact for his or her age in different situations and with a variety of different people?

6. Does your child play with his or her peers when in a group of at least two others?

7. When were your child’s behavioral abnormalities first obvious?"

And how was the study done?

We are told that some 220 participants completed the MARA and then 'participated' in a clinical visit following referral "to see a team of clinicians including a developmental- behavioral pediatrician and child psychologist, from November 2012 through December 2013." Caregivers went to a secure website where MARA and the relevant consents for their children were taken. Although 222 participants is quite a nice cohort number, it reflected less than half of children invited to take part in the study.

Results: bearing in mind the scoring of the MARA - which takes approximately 5 minutes to complete - "with negative scores indicating high risk and positive scores suggesting low risk for ASD", the schedule didn't do bad at all. Those who were eventually assessed to have an ASD (69/222) were generally more likely to "receive a MARA score that was indicative of ASD." And when it came to those all-important sensitivity and specificity values, well, I've seen worse values ("sensitivity = 89.9 % and specificity = 79.7 %") in the autism research literature. Even those who were miss-classified as potentially having an ASD by MARA were more likely to receive other diagnoses related to language, motor or global developmental delay disorder. The authors conclude that, with more research to do, the MARA, in its current form: "demonstrated good ability to distinguish ASD versus other developmental and behavioral concerns."

Of course there still quite a bit more to do research-wise with the MARA before it becomes part and parcel of routine screening. This set within the recent publication of the opinion piece by Albert Siu and the US Preventive Services Task Force (USPSTF) [2] who, in the face of quite a lot of evidence to the contrary, have said 'no' to the universal screening for autism in young children at the moment. It seems that we here in Blighty, knowingly or unknowingly, might have taken a lead on this issue (see here). That also the MARA might have some 'mobile' competition (see here) seems to indicate that telemedicine is starting to take some big strides into the realms of autism screening and assessment. Now, how about coupling such work with something a little more 'objective' too (see here for example)?

----------

[1] Duda M. et al. Clinical Evaluation of a Novel and Mobile Autism Risk Assessment. Journal of Autism and Developmental Disorders. 2016. Feb 12.

[2] Siu A. et al. Screening for Autism Spectrum Disorder in Young Children. JAMA. 2016; 315: 691-696.

----------

Duda, M., Daniels, J., & Wall, D. (2016). Clinical Evaluation of a Novel and Mobile Autism Risk Assessment Journal of Autism and Developmental Disorders DOI: 10.1007/s10803-016-2718-4

Differentiating between autism and ADHD the machine learning way

Five of 65 behaviours measured by the Social Responsiveness Scale (SRS) were "sufficient to distinguish ASD [autism spectrum disorder] from ADHD [attention-deficit hyperactivity disorder] with high accuracy." Further: "machine learning can be used to discern between autism and ADHD."

Machine learning - outside of any visions of the Matrix or the T-1000 comin' at yer - applied to autism usually means one lab based at Stanford University and a familiar name, Dennis Wall. Actually, I should alter that last sentence to include another name, Marlena Duda, who appears as first author on today's blog offering [1] following on from previously authored research in this area (see here).

This time around research attention was directed towards the diagnostic combination that is autism and ADHD (see here) and the application of various machine learning algorithms to "the 65 items in the SRS as features and the diagnosis of either ASD or ADHD as the prediction class." Drawing on SRS data for almost 3000 people diagnosed with autism (n=2775) or ADHD (n=150) held by various autism research initiatives, researchers tested the various machine learning combinations to see if they could differentiate ASD and ADHD.

Yes, they could and with quite a large degree of accuracy it was reported. More than that however, was the finding that most of the algorithms tested were able to do so on the basis of only five behaviours. The sorts of SRS items deemed important were: trouble with the flow of normal conversation, difficulty with changes in routine, appropriate play with peers, difficulty relating to peers, atypical or inconsistent eye contact and also 'regarded by other children as 'odd''. I know that list includes 6 items, but those were the ones "consistently identified as the top ranked features." The Duda paper does include quite a bit more about the hows and whys of the results reached and I would encourage readers to have a more detailed look.

To quote further:

"Behavioral diagnosis of both ASD and ADHD is a time-intensive process that can be complicated by the overlaps in symptomatology. Due to the high demand for the multi-hour clinical assessments necessary for diagnosis, many children are waitlisted for over a year, delaying their diagnosis and thereby delaying the start of behavioral and/or pharmaceutical interventions. Currently, there is no diagnostic instrument that can directly distinguish autism from ADHD, nor does there exist a screening tool that is expressly designed to distinguish risk between the two disorders with high accuracy."

Within that text is everything readers need to know about why these results are potentially so important. Even if used as 'triage' (something that has been mentioned in previous publications by this group) the idea that with further investigation, so few behavioural items could be used, invites quite a bit more research scrutiny in this area. I'm sure we're going to hear more... Indeed, we have [2] (and I'll be posting about this very soon).

----------

[1] Duda M. et al. Use of machine learning for behavioral distinction of autism and ADHD. Transl Psychiatry. 2016 Feb 9;6:e732.

[2] Duda M. et al. Clinical Evaluation of a Novel and Mobile Autism Risk Assessment. Journal of Autism & Developmental Disorders. 2016. 12 Feb.

----------

Duda M, Ma R, Haber N, & Wall DP (2016). Use of machine learning for behavioral distinction of autism and ADHD. Translational psychiatry, 6 PMID: 26859815

Boiling down ADOS for autism detection (again)

Today I want to direct your attention to the paper by Kosmicki and colleagues [1] (open-access) reporting that the use of "machine learning algorithms" could help "streamline ASD [autism spectrum disorder] risk detection and screening."

Regular readers of this blog might have already cottoned on to the fact that any talk about applying "computational and statistical methods" to autism screening and/or diagnosis can really mean only one person and research group: Dennis Wall from Stanford University. To quote from his institutional website on this area of research, the aim is to "evaluate the degree of redundancy of the ADOS and ADIR and if so determine whether a reduced set of uncorrelated features could correctly classify individuals with the same accuracy as the gold-standard diagnostic tests." ADOS and ADI-R by the way, are some, if not the, gold-standard schedules when it comes to the assessment of autism. The idea is that boiling down these respective schedules might save both time and resources when it comes to identifying those where a diagnosis of ASD is indicated. In case you'd like some history about this line of work, look no further than here...

The latest paper from the Wall group continues the research journey looking this time at modules 2 and 3 of the ADOS where previous work looked at module 1 (see here). In case you're not familiar with the concept of modules in ADOS, it's all about selecting the correct module according to verbal fluency (see here) where module 1 is for those who have very little or inconsistent phrase speech and modules 2 and 3 represent increasing phrase speech with also a little more focus on the use of age-appropriate props.

The results? Based on the development of 'classifiers' for each module, several machine learning algorithms were developed and tested (see here). One of the algorithms, ADTree, is by the way, the same classifier used in the previous module 1 ADOS work [2]. But ADTree did not perform best on this occasion: "The logistic regression classifier based on analysis of archival records from ADOS module 2 consisted of nine items, 67.86% fewer than the complete ADOS module 2, and performed with 98.81% sensitivity and 89.39% specificity in independent testing." Further: "The SVM module 3 classifier based on analysis of archived ADOS module 3 records consisted of 12 items, 57.14% fewer than the complete ADOS module 3, and performed with more than 97% sensitivity and specificity in testing."

The authors conclude: "These results support the notion that fewer behaviors when measured using machine learning tools can achieve high levels of accuracy in autism risk prediction."

Anyone who has either professional or personal experience of undertaking an ADOS will know that this is a highly specialised assessment schedule which often requires some time to complete. It's nothing like as time-consuming as the ADI but still, significant efforts and resources are needed to carry out the assessment and do so with skill and reliability (and maintain those all-important reliability stats). Wall et al have really started to shake the establishment when it comes to ADOS (and ADI) when asking just how much of the schedule is really needed to assess for autism/ASD. This on top of their other work talking about assessment 'triage' via YouTube videos using, horror of horrors, non-clinical raters (see here). I'm not saying that these approaches are ready for clinical practice; quite a bit more replicative work is required [3] including crossing geographical boundaries. But something like the idea that "mobile health approaches that ultimately enable individuals to receive more expedient care than is possible under the current paradigms" is a tantalising prospect.

So: Start by The Jam.

----------

[1] Kosmicki JA. et al. Searching for a minimal set of behaviors for autism detection through feature selection-based machine learning. Transl Psychiatry. 2015 Feb 24;5:e514.

[2] Wall DP. et al. Use of machine learning to shorten observation-based screening and diagnosis of autism. Transl Psychiatry. 2012 Apr 10;2:e100.

[3] Bone D. et al. Applying Machine Learning to Facilitate Autism Diagnostics: Pitfalls and Promises. J Autism Dev Disord. 2014 Oct 8.

----------

Kosmicki JA, Sochat V, Duda M, & Wall DP (2015). Searching for a minimal set of behaviors for autism detection through feature selection-based machine learning. Translational psychiatry, 5 PMID: 25710120

An observation-based classifier for rapid detection of autism risk

"Keep clear of the moors"

Among the many researchers and research groups admired on this blog for their contribution to the world of autism research, the name Dennis Wall is fast becoming a real favourite. Aside from mention of the words 'systems biology' in his profile at Stanford University, I'm particularly interested in the way the Wall research group are looking at trying to apply machine-learning approaches to things like autism assessment.

I've covered a few of their past research reports with regards to instruments like the Autism Diagnostic Interview (ADI) and the Autism Diagnostic Observation Schedule (ADOS) previously (see here and see here respectively). More recently was the work suggesting that YouTube videos and non-expert raters might be a useful resource for autism triage (see here). That last report certainly set the cat among [some] pigeons...

Today I'm talking about another paper from the Wall laboratory by Marlena Duda and colleagues [1] (open-access) and the suggestion that: "reductions in the process of detecting and monitoring autism are possible". The ADOS was once again the focus of the study following on from their previous 'preliminary' foray [2].

The paper is open-access but here are a few choice details:

• If I'm reading the paper correctly, this was a follow-up study to the previous Wall paper [2] testing the accuracy of the "observation-based classifier (OBC)" which I think was previously called/included the ADTree algorithm. This time around "a cohort of archival score sheets of over 2600 subjects, including more than 280 assessments of non-spectrum controls" were included in the study derived from ADOS and ADOS-2 algorithms. ADOS-2 by the way, represents the revised algorithms used to score ADOS reported by Gotham and colleagues [3]. I've talked about the Gotham paper before on this blog and how it seemed to 'predict' diagnosis of autism in DSM-5 (see here).
• The aim was to test whether a boiled down version of the ADOS / ADOS-2, the OBC, that: "presently contains eight behaviors... that are often impacted in children with autism, including eye contact, imaginative play and reciprocal communication" might be able to distinguish autism from not-autism and "shorten screening and diagnostic processes overall and potentially enabling more families to receive care far earlier and during timeframes when interventions have the most positive benefits".
• Results: "The OBC was significantly correlated with the ADOS-G (r=−0.814) and ADOS-2 (r=−0.779) and exhibited >97% sensitivity and >77% specificity in comparison to both ADOS algorithm scores". These figures aren't bad at all, if a little down on the previous Wall data [2]. The authors add: "Less than 5% of all tested cases were misclassified by the OBC and 78% of the misclassified individuals were given a low OBC score".

Obviously there is much more investigation needed in this area of autism research before one might start shortening ADOS assessments (or indeed doing away with trained ADOS raters altogether). The issue of comorbidity is, for example, something that needs to be included in any further study and whether that might interfere with any results obtained [4]. I might also add that ADOS is only part of the diagnostic assessment for autism and does not replace reasoned clinical opinion.

I am however drawn to the authors suggestion that: "use of the OBC as a web-based assessment in advance of a clinical visit may enable clinicians to quickly prioritize patients according to symptom severity, scheduling shorter, more immediate diagnostic appointments for individuals that can be clearly identified as on or off the autism spectrum, and allowing longer time periods for deeper evaluation of children that exhibit clinically challenging symptoms". Certainly with the numbers of children/adults seemingly coming through the various referral systems, this kind of triage might yet hold some usefulness. And it seems other groups are getting in on the computer-assisted act [5]...

Music to close and The Marcels with Blue Moon. "Keep clear of the moors" as we were once told...

----------

[1] Duda A. et al. Testing the accuracy of an observation-based classifier for rapid detection of autism risk. Translational Psychiatry. 2014; 4: e424.

[2] Wall DP. et al. Use of machine learning to shorten observation-based screening and diagnosis of autism. Transl Psychiatry. 2012 Apr 10;2:e100.

[3] Gotham K. et al. The Autism Diagnostic Observation Schedule: revised algorithms for improved diagnostic validity. J Autism Dev Disord. 2007 Apr;37(4):613-27.

[4] Leyfer OT. et al. Overlap between autism and specific language impairment: comparison of Autism Diagnostic Interview and Autism Diagnostic Observation Schedule scores. Autism Res. 2008 Oct;1(5):284-96.

[5] Hashemi J. et al. Computer Vision Tools for Low-Cost and Noninvasive Measurement of Autism-Related Behaviors in Infants. Autism Research and Treatment. 2014. 935686.

----------

M Duda, J A Kosmicki, & D P Wall (2014). Testing the accuracy of an observation-based classifier for rapid detection of autism risk Translational Psychiatry, 4 : 10.1038/tp.2014.65

Put it in the (autism science) replicator

Replication is one of the most important processes in those things we call science and the scientific method. Outside of conjuring up images of a certain 'Make it so' Captain with his "Earl Grey, Hot", scientific replication provides the community at large with some degree of reassurance that a research finding was not just a fluke or an artefact of a particular sample of people or a method used or an interpretation of results. As per the recent BBC article on vitamin use: "Looking at any one individual study won't be very revealing to answer the question of whether vitamin supplementation is good for you."

Uncanny likeness @ Wikipedia 

With the issue of replication in mind, I was interested to read the Letter to the Editor from Robinson and colleagues* (open-access) who set about trying to replicate the findings from Skafidas and colleagues** (open-access) and their notion that science might be making some in-roads into the detection of "genetic biomarkers [that] can correctly classify ASD from non-ASD individuals". I posted about the Skafidas study at the time also (see here) and their analysis of single-nucleotide polymorphisms (SNPs) in relation to autism spectrum disorder (ASD).

The Robinson letter reports an attempt to replicate the Skafidas findings based on an independent analysis of data from the Psychiatric Genomics Consortium (PGC) "which includes ~5400 cases, more than three times the number used in the original [Skafidas] report". I'm not on this occasions going to get the fine-toothed comb out on both papers because they're open-access so free for anyone to read.

The conclusions from Robinson et al are pretty clear: "We find no evidence that the implicated SNPs, the classifier or the pathways named in Skafidas et al.1 are associated with ASDs. We therefore conclude that the classifier, as presented, cannot be used in a general way to predict ASDs, and consequently is unlikely to have any translational value."

Obviously such findings are both a blow to autism research and also the original authors who first proposed the classifier model, who I don't doubt probably invested quite a lot of time, effort and funds into getting their experiments done and results published (and published in a Nature journal). The ego also takes a bit of a knock under such circumstances, believe me (see here and here and here).

The Robinson data however re-emphasize the importance of replication in autism research. Perhaps just as important, they also reaffirm that autism is a tremendously difficult set of conditions to study. As is often the case when it comes to a heterogeneous condition like autism (or should that be the autisms) often carrying more than its fair share of comorbidity (see here) including risk of certain somatic conditions (see here), consistent findings are often few and far between. Indeed, that the use of the label autism, whilst providing a way of classifying certain types of behaviour and their impact on a person's life, is not necessarily the best thing for research purposes, as was vocalised through the grudge match that was DSM V vs. RDoC (see here).

The added realisation that outside of no one single SNP being linked to all autism (see here) there may be a significant degree of overlap when it comes to the genetics of the autisms with other developmental and psychiatrically defined conditions (see here) implies that it's going to be some time yet before any genetic biomarkers or test is going to be able to accurately classify autism, sorry the autisms with any great accuracy. Then there is the question of what such a test would accomplish. I've not even mentioned the fact that autism, whilst having a genetic component, is probably not without it's [variable] partner in crime, environment (however you want to define this) when it comes to aetiology. And don't even mention that other area of increasing interest, epigenomics (see here)... which in recent days has seen some interesting papers published (see here and see here).

I suppose in the spirit of all this talk on replication, the last question should be: who next is going to try and replicate the Robinson results? Indeed, does science any longer need the 'Letter to the Editor' in light of the rolling out of PubMed Commons?

----------

* Robinson EB. et al. Response to ‘Predicting the diagnosis of autism spectrum disorder using gene pathway analysis’. Molecular Pyschiatry. 2013: Oct 22. doi: 10.1038/mp.2013.125

** Skafidas E. et al. Predicting the diagnosis of autism spectrum disorder using gene pathway analysis. Molecular Psychiatry. 2013; Sep 11. doi: 10.1038/mp.2012.126

----------

E B Robinson, D Howrigan, J Yang, S Ripke, V Anttila, L E Duncan, L Jostins, J C Barrett, S E Medland, D G MacArthur, G Breen, M C O'Donovan, N R Wray, B Devlin, M J Daly, P M Visscher, P F Sullivan, B M Neale (2013). Response to ‘Predicting the diagnosis of autism spectrum disorder using gene pathway analysis’ Molecular Psychiatry DOI: 10.1038/mp.2013.125

Fatigue severity and serum leptin levels in chronic fatigue syndrome

In the very complicated world of medical research and science, the days of one chemical, one metabolite, or one gene driving and sustaining ill-health and particular diseases or conditions seem to be all but long past. Sure, there are conditions which on the surface seem to be driven by only one factor, but more often than not is the realisation that we humans are very complicated creatures indeed.

Leptin @ Wikipedia  

I was therefore interested to read the paper by Elizabeth Stringer and colleagues* (open-access) describing the results from a small cohort of women diagnosed with chronic fatigue syndrome (CFS) looking at potential biological correlates which might accompany day-to-day changes in the severity of fatigue experienced by participants.

Yes, I'm back with CFS to add to my ramblings about gut bacteria, mitochondrial disorder, amino acids. Bear with me...

The paper is open-access but a few pointers might be useful:

• It was an interesting methodology the authors adopted which saw 10 women diagnosed with CFS and 10 asymptomatic age- and BMI-matched controls asked to monitor their fatigue-related behaviours over the course of 25 days.
• This self-report was accompanied by a professionally taken daily blood draw (yes, 25 days of giving a blood sample!) which were subsequently analysed for various cytokines - 51 in all.
• The self-report data and pattern of cytokine levels were analysed, correlated and networked (using a machine learning algorithm).
• Results: "Six participants with CFS and one healthy control demonstrated significant positive correlations between fatigue and leptin". Leptin by the way is a hormone normally implicated in the in-and-out process of energy expenditure, so potentially relevant to a condition like CFS which is characterised by fatigue.
• Buoyed by their leptin results, the authors also report that with the help of that Weka’s LibLINEAR algorithm, they were able to use the suite of cytokine results to distinguish 'high' and 'low' fatigue days for the CFS group with 78.3% accuracy compared with just above chance level in the asymptomatic control group. "The CFS model correctly identified 77.8% of the low fatigue days and 78.9% of high fatigue days".
• Ergo cytokines and inflammation seem to be not only tied into CFS pathology but might actually be overlap with the ebb and flow of clinical symptoms on a day-to-day basis.

You can perhaps see how this study might be an important one for CFS. Given the connection between leptin (energy) and CFS, you might be saying to yourself that this sounds all very logical so why did no-one look at the possible connection before? Well, they did, or rather Cleare and colleagues** did and concluded: "we found no evidence of alterations in leptin levels in CFS" despite some potential effects from low dose hydrocortisone therapy on leptin levels under placebo-controlled conditions.

This is not by any means the first time that immune function has cropped up on the CFS research radar (see this post) and probably won't be the last either. I don't however want to speculate too much more on these results without them being subject to appropriate replication with a larger patient set and that all-important diagnostic criteria being standardised. The XMRV story (see here) still lingers in the mind, as do other controversies on the CFS landscape such as Ampligen and Rituximab.

To close, a song about a dirty old town.

----------

* Stringer EA. et al. Daily cytokine fluctuations, driven by leptin, are associated with fatigue severity
in chronic fatigue syndrome: evidence of inflammatory pathology. Journal of Translational Medicine. 2013; 11: 93.

** Cleare AJ. et al. Plasma leptin in chronic fatigue syndrome and a placebo-controlled study of the effects of low-dose hydrocortisone on leptin secretion. Clin Endocrinol (Oxf). 2001; 55: 113-119.

----------

Stringer, E., Baker, K., Carroll, I., Montoya, J., Chu, L., Maecker, H., & Younger, J. (2013). Daily cytokine fluctuations, driven by leptin, are associated with fatigue severity in chronic fatigue syndrome: evidence of inflammatory pathology Journal of Translational Medicine, 11 (1) DOI: 10.1186/1479-5876-11-93