ADOS via YouTube for autism 'triage'?

The paper by Vincent Fusaro and colleagues [1] and some accompanying press attention about this study caught my eye recently. Based on an analysis of 100 home videos posted on YouTube, some tagged with words leading to the assumption of the presence of autism, undergraduate researchers used scoring measures from the gold-standard autism assessment schedule ADOS (Autism Diagnostic Observation Schedule) to quite accurately grade whether children appearing in the videos could be classified as autistic or not.

Testing, testing, 1, 2.. @ Wikipedia 

Having quite recently attended a conference where autism screening and assessment was a big talking point, including some chatter specifically about the ADOS and it's relationship to DSM-5 (see here for some background), it is timely that this paper was published when it was. That also one of the authors is Dennis Wall - he of the machine-learning work on ADOS (see here) and ADI (see here) - was also of some interest. This is a man who seems to be looking very closely at the ADOS (and ADI).

The Fusaro paper is open-access so there is little point in me just repeating the results. A few aspects however caught my eye:

• The focus on "non-clinical raters" is interesting. Anyone who knows a little bit about the ADOS will know about the training required to administer it and indeed, CPD to maintain those all-important inter-rater reliability figures. The Fusaro results suggested that: "Raters were purposely given minimal instructions to code only when the video clearly depicted a behavior and/or contained opportunities for the child to exhibit the behavior in question, and otherwise to code the behavioral item as not applicable (N/A)". The mean rater agreement in the Fusaro paper is quoted at 73.3% "despite the diversity of environmental contexts represented in the videos". That's not bad at all given that a typical ADOS relies on specific cues and scenarios to score behaviours. 
• That being said: "Because two of the raters helped to locate videos appropriate for this study, they were not always naïve to the diagnoses". So, not everyone scoring the videos was completely in the dark about whether autism was mentioned or not when it came to some of the videos examined. Remember also that these videos were tagged with words pertinent to the presentation of autism meaning that parents or caregivers already potentially recognised some of the symptoms being displayed. Fair-dos to the authors though as they relied on a professionally-trained ADOS practitioner completely blinded to the suggestion of autism or not in videos to score a selection of videos and showing agreement "between each non-clinical and clinical rater [in] 71.3%".
• Raters only scored behaviours according to module 1 of the ADOS. Module 1 ADOS is normally reserved for those children who have little or no phrase speech. Given that children in the videos ranged in ages between 1 and 15 years and were only included for analysis if the video was under 10 minutes long, I'd be minded to say that we have to be a little bit careful when talking about accuracy of the results obtained.

That all being said, I don't want to take anything away from the Fusaro results and their conclusion: "it is possible for non-clinical raters to correctly detect the presence of autism with high inter-rater reliability and >94% accuracy". In these days of increasing pressures to recognise and diagnose autism as early as possible, anything that might potentially highlight children where further observations and assessments might be required should be welcomed. That also there is a rather large bank of undergraduate students around the world studying psychology or related degrees who might be able to provide a lay first-sweep assessment of children perhaps requiring more formal inquiry is an untapped resource that, at the very least, is worthy of more detailed research inspection.

And just in case you'd like some more discussion about this study, have a look here... complete with a response from the Dr Fusaro too.

Music y'say. Well, alrighty... Heart of Glass by Blondie.

----------

[1] Fusaro VA. et al. The Potential of Accelerating Early Detection of Autism through Content Analysis of YouTube Videos. PLoS ONE. 2014; 9: e93533.

----------

Fusaro, V., Daniels, J., Duda, M., DeLuca, T., D’Angelo, O., Tamburello, J., Maniscalco, J., & Wall, D. (2014). The Potential of Accelerating Early Detection of Autism through Content Analysis of YouTube Videos PLoS ONE, 9 (4) DOI: 10.1371/journal.pone.0093533