ADTree reloaded: classifying autism based on 7 ADI-R items?

A lengthy quote to begin this post:

"Deploying a variety of machine learning algorithms, we found one, the Alternating Decision Tree (ADTree), to have high sensitivity and specificity in the classification of individuals with autism from controls. The ADTree classifier consisted of only 7 questions, 93% fewer than the full ADI-R, and performed with greater than 99% accuracy when applied to independent populations of individuals with autism, misclassifying only one out of the 1962 cases used for validation".

Interested? The quote comes from this recent paper by Dennis Wall and colleagues* (full-text) following up some related research from this group based on another gold standard autism assessment schedule, the ADOS (see here).

I'm not going to dwell too much on the study details aside from saying:

• The ADI-R (Autism Diagnostic Interview - Revised) is probably one of the more time and resource-intensive interview questionnaires used for the assessment of autism, coming in at 93 questions long and asking about current behaviour and 'most abnormal' (aged 4-5 years) if appropriate.
• An abbreviated ADI-R would therefore be quite a useful measure in these austere times; particularly one which could maintain the same level of accuracy as completion of the full schedule.
• Similar to their previous study, Wall and colleagues applied various machine learning algorithms (n=15) to ascertain whether any might be able to determine which ADI-R items are most relevant to diagnosis based on the AGRE dataset.
• Once again, the Alternating Decision Tree (ADTree) model performed best: perfect sensitivity (1.0), a low false-positive rate (0.013) and "overall accuracy of 99.9%".
• Seven items of the ADI-R comprised the ADTree model: (i) comprehension of simple language at most abnormal (4-5 years), (ii) reciprocal conversation (regarding the ability to facilitate the flow of conversation), (iii) use of imaginative / pretend play at most abnormal (I have post about pretend play coming up fairly soon), (iv) social imaginative play with peers at most abnormal, (v) direct gaze at most abnormal, (vi) group play with peers at most abnormal (spontaneous games or activities) and (vii) age when abnormality was first evident.
• Testing accuracy was again carried out on participant data from the Boston Autism Consortium (AC) and the Simons Simplex Collection (SSC) from where the quite compelling data for the success of the ADTree model was derived.

So the ADI-R has been boiled down to 7 pertinent items. The ADOS boiled down to 8 distinguishing module 1 items. I think most people would stand up and take note of these findings even if further replication is still required (based on different geographical groups for example). Don't get me wrong, there is still a large degree of skill required to deliver the ADI-R and ADOS and maintain your reproducibility and diagnostic prowess so I don't think this combined data will be putting people out of work just yet; certainly not with the number of people estimated to be coming through the diagnostic process. That and the fact that these are assessment instruments and so are subservient to a final clinical opinion for an autism diagnosis or not.

Aside from the grand findings I am interested in the types of behaviours which are noted to be important for diagnosis. All very 'social-communicative' (sounds familiar) and not at all heavy on the 'restricted and repetitive behaviour' side of things. Indeed, looking back at the ADOS ADTree paper, I might be wrong but only one element, 'functional play with objects' seems to have any strong relation to issues with repetitive behaviours. This could be that we aren't asking the right questions about this area of behaviour, but I would hedge my bets that more likely is the stress on the social-communicative side of presentation as being key to diagnosis. I think I might have to look at this further in future posts.

Finally, I have previously talked about the lack of instruments to appropriately and accurately assess 'change' in autism (as a function of maturation or intervention or anything else). Y'know with these combined data, I think we might have the outline of something really quite useful...

----------

* Wall DP. et al. Use of artificial intelligence to shorten the behavioral diagnosis of autism. PLoS ONE. 2012; 7: e43855.

----------

Dennis P. Wall, Rebecca Dally, Rhiannon Luyster, Jae-Yoon Jung, & Todd F. DeLuca (2012). Use of artificial intelligence to shorten the behavioral diagnosis of autism PLoS ONE : 10.1371/journal.pone.0043855