The proposed changes to the DSM-IV diagnostic criteria for pervasive developmental disorders (PDDs) include shifting from a multicategorical model to a single diagnostic category of autism spectrum disorder (ASD), replacing the three-domain model with a two-domain model, relaxing the criteria for age at onset, and adding symptoms not previously included in DSM-IV, such as sensory interests and aversions. Although these changes are based on empirical data (e.g., 1, 2), little is known about the sensitivity and specificity of the new criteria. In particular, it is unclear whether the revised criteria will inadvertently narrow the definition of PDDs. This is of major significance to families concerned that their affected children might not meet the proposed criteria for ASD and might therefore lose necessary services.
To date, various empirical studies have found support for a two-domain ASD symptom model (3–5). One difference from the original model is that communication deficits are subsumed under social impairments. Mandy and colleagues (6) tested this model, including sensory behaviors as part of the restricted and repetitive behavior criterion, and found that this model has an excellent fit. In contrast, the original DSM-IV model did not meet statistical criteria for an acceptable fit. Although this work confirms the conceptual validity of the proposed changes to DSM-IV, it tells us little about the sensitivity of the new criteria.
Because of the newness of the proposed criteria, only a handful of studies have examined the DSM-5 criteria, and all have examined slightly different versions of the criteria under consideration. McPartland and colleagues assessed the sensitivity and specificity of the proposed DSM-5 criteria by using the DSM-IV field trial checklist items and found DSM-5 to perform quite poorly (7). Using existing data from parent questionnaires, the Autism Diagnostic Interview–Revised (8), and the Autism Diagnostic Observation Schedule (9), Mattila et al. (10) examined an early draft of the criteria (2010) and found that only 46% of children with PDD diagnoses were identified as meeting the ASD criteria. Notably, when the authors used criteria more similar to the current DSM-5 criteria, approximately 96% of the children with PDD diagnoses were classified correctly.
The poor sensitivity of the early draft criteria and the remarkable increase in sensitivity with the new draft are likely explained by Mattila and colleagues’ stringent interpretation of the 2010 criteria. For example, sensitivity was improved when they required “routines AND/OR rituals” instead of “routines AND rituals.” Furthermore, unlike the early draft, the improved model included “unusual sensory behaviors” and the removal of an onset criterion of 36 months. This revision, which has been implemented in the latest DSM-5 draft, increased sensitivity, particularly in the “high-functioning” subgroup (i.e., children with full scale IQs of 70 or higher).
In another study, Frazier et al. (11) mapped items from the Social Communication Questionnaire (12) and the Social Responsiveness Scale (13) to DSM-5 criteria and found that 19%–22% of children with DSM-IV PDD diagnoses did not meet the proposed criteria. Notably, these analyses were based on criteria from DSM field trial phase 1, which required a greater number of symptoms than the currently proposed criteria. When the authors required fewer symptoms within each criterion (as in the current DSM-5 proposal), the sensitivity was comparable to that of DSM-IV and there was a slight improvement in specificity. This pattern of results was similar across many of the subgroups, such as girls, verbal youth, and multiplex families. Nevertheless, while the study group of Frazier et al. was large (N=14,744), the methods of the study limit the interpretability of their findings. For example, analyses included items based on past behavior (“When she/he was 4 to 5, did she/he smile back if someone smiled at her/him?”), whereas the proposed DSM-5 social communication criteria relate to current functioning and behavior.
Though specificity is an important focus of the proposed revisions, it is not yet clear that it will improve with the DSM-5 criteria. Frazier and colleagues’ recent analyses of the proposed criteria suggest improved specificity for the DSM-5 criteria over DSM-IV (11), particularly with a relaxed version of the DSM-5 criteria, using one less symptom per domain. However, these results were obtained from siblings of affected children, of which only about 30% had a caregiver-reported non-PDD diagnosis. Additional evidence from children with non-PDD diagnoses is necessary to make claims about DSM-5’s specificity.
The proposed change to a single ASD category, as well as the requirement that there must be a history of restricted and repetitive behaviors, has led some to believe that DSM-5 will make it more difficult for some individuals with PDDs to qualify for a diagnosis. The comprehensive review by Wing et al. of the proposed criteria articulates some of these concerns, explaining that DSM-5 could inadvertently exclude subgroups of affected people, including very young children, girls, and those with diagnoses of Asperger’s disorder (14). The introduction of “social communication disorder” in DSM-5 raises additional concerns that children currently diagnosed with PDDs will be misclassified with this disorder if they do not meet the DSM-5 requirement for restricted and repetitive behavior.
In sum, from the existing empirical work, the sensitivity of the proposed DSM-5 criteria remains unclear. In addition, relatively little attention has been paid to questions about specificity. Thus, before the proposed diagnostic changes go forward, it is critical to use the recent availability of large and well-characterized groups of children with PDDs and non-PDD diagnoses to attempt to shed light on these issues.
In the current study we sought to provide additional insights into DSM-5’s sensitivity and specificity by assigning individual items from well-established autism diagnostic measures to the proposed criteria and then using symptom counts to estimate how many children with previous DSM-IV diagnoses of PDDs or non-PDD disorders (e.g., language disorder, attention deficit hyperactivity disorder) would meet the DSM-5 ASD criteria. We applied these same methods to the DSM-IV criteria and to slight variations of the DSM-5 criteria, including those of Frazier et al. (11). We also completed domain-specific analyses to examine whether any children with clinical diagnoses of PDDs might meet the criteria for DSM-5 social communication disorder.
This study was not a field trial for DSM-5. It used previously collected data to evaluate DSM-5 criteria in groups of children with DSM-IV clinical diagnoses.
Participant data were obtained from three sources: 1) proband data from the Simons Simplex Collection, a genetic consortium study focusing on families having just one child with an ASD (“simplex” families) (15), 2) the Collaborative Programs of Excellence in Autism (subsequently referred to as “Collaborative Programs”), a multicenter study of ASD (4), and 3) the University of Michigan Autism and Communication Disorders Center data bank (subsequently referred to as “University of Michigan”), which consists of research participants and patients clinically referred for assessment of ASD (16). All groups have been previously described in detail (4, 15, 16). Institutional review board approval was obtained at each site, and written informed consent was obtained from the participants’ legal guardians.
+
DSM-IV Diagnostic Confirmation
All study participants had previously undergone diagnostic testing that included the Autism Diagnostic Interview–Revised, the Autism Diagnostic Observation Schedule, and cognitive or developmental testing. Clinical best-estimate diagnoses were determined by experienced clinicians (e.g., psychologists, psychiatrists) on the basis of all available information from the parent interview and child assessment.
+
Operationalizing of DSM Criteria
For the study analyses, we relied primarily on the Autism Diagnostic Interview–Revised, a 96-item parent report measure. It includes items assessing current and past behaviors and covers a wide range of ASD-related impairments (e.g., use of idiosyncratic language).We also used the Autism Diagnostic Observation Schedule, a clinician-based measure of ASD impairments. These two instruments were particularly well suited for the current study because they include items based on current behavior and they take into account developmental level in their design. This is consistent with the DSM-5 criteria, which operationalize symptoms differently for individuals of different ages in order to account for the effect of development on ASD symptoms (17–20).
As a first step in our analyses, items from the Autism Diagnostic Interview and the Autism Diagnostic Observation Schedule were mapped onto the DSM-5 criteria. Before the items were assigned to each criterion, the subjects were divided into age by language groups. Groups for children under the age of 4 and children over the age of 10 were created to be consistent with the age-based routing rules of the Autism Diagnostic Interview. Children were assigned to language groups depending on which module of the Autism Diagnostic Observation Schedule was administered. After consensus was reached among all study authors about item assignments for the DSM-5 criteria, this process was repeated for DSM-IV criteria (item assignments are available in supplemental Tables S1 and S2 in the data supplement accompanying the online version of this article).
For each item included in the DSM-IV and DSM-5 item maps, a score of 1, 2, or 3 indicated the presence of a symptom, whereas a score of 0 indicated the absence of a symptom. DSM-IV and DSM-5 guidelines were then followed to determine whether each participant met or did not meet the DSM-5 criteria for ASD and the DSM-IV criteria for autistic disorder, Asperger’s disorder, and/or pervasive developmental disorder not otherwise specified (PDD-NOS). Initially we established classifications, such as meeting versus not meeting the DSM-5 ASD criteria, by extracting symptom information from only the Autism Diagnostic Interview. We then established classifications by using information from both the Autism Diagnostic Interview and the Autism Diagnostic Observation Schedule, i.e., allowing evidence of symptoms to come from parent report, direct observation, or both. Unfortunately, it was not practical to attempt to establish classifications by using only information from the Autism Diagnostic Observation Schedule, because there are no relevant items for certain subdomains (see online supplemental Tables S1 and S2). However, because there are adequate numbers of items on both instruments that assess DSM-5 criteria A1 and A2, we were able to examine sensitivity and specificity when symptoms in these domains were required from both measures.
To ensure that the creation of both the DSM-5 and DSM-IV item assignments agreed with other clinicians’ interpretations of the criteria, these were reviewed by two psychologists and one psychiatrist who were not otherwise involved in the design or execution of the current study. All have extensive experience with ASD diagnosis and the study instruments. As a result of their feedback, two items were reassigned and one item was removed from the DSM-5 criteria (for details, see online supplemental Tables S1 and S2). The majority of the study authors and the independent experts noted some overlap between DSM-5 criterion A1 and criterion A3. For example, whereas a poor-quality social overture or initiation could be considered evidence of “abnormal social approach” (A1), it could also reflect “difficulties adjusting behavior to suit different social contexts” (A3). In general, however, the group agreed that the items were easier to map onto the DSM-5 criteria than DSM-IV criteria.
Analyses were restricted to participants ages 2 to 17 for whom Autism Diagnostic Interview and Autism Diagnostic Observation Schedule data and DSM-IV clinical diagnoses were available. All statistical analyses were run by using SPSS 17.0 (SPSS, Chicago).
Analyses examined the sensitivity and specificity of the proposed DSM-5 ASD criteria and the DSM-IV PDD criteria in the three study groups, individually and combined, and in specific subgroups of children (i.e., children with DSM-IV Asperger’s disorder or PDD-NOS diagnoses, girls, young children). For each clinical diagnosis of a PDD, McNemar’s test was used to compare the proportions of non-PDD children who, according to the clinical best-estimate diagnosis, were correctly classified by DSM-5 and by DSM-IV. Domain-specific analyses were also conducted to explore whether children who did not meet the DSM-5 criteria for ASD might meet the proposed criteria for social communication disorder.
Demographic data and mean IQs for the study groups are displayed in Table 1. The participants ranged in age from 2 to 17 years, 11 months. They represented a wide range of nonverbal and verbal ability; approximately 30% across all three groups had nonverbal IQs under 70. The nonverbal IQ of the PDD group from the University of Michigan was significantly lower than the nonverbal IQs of both other groups. The majority of the participants were Caucasian and male, but the three groups had significantly different male-to-female ratios.
+
DSM-IV PDD and DSM-5 ASD Classifications Compared With Best-Estimate Diagnoses
As outlined in Table 2, on the basis of parent-reported symptoms only, in children with a clinical best-estimate diagnosis of any PDD, the sensitivity of the proposed DSM-5 criteria ranged from 0.89 to 0.93 (Table 2, “ASD, One Symptom” column). On the basis of either parent report or clinical observation, DSM-5 sensitivity ranged from 0.97 to 0.99 for any PDD. In every sample, sensitivity was highest for children with a DSM-IV clinical diagnosis of autism. Not surprisingly, given that the subjects in the Simons Simplex Collection were the only group in which participants’ initial eligibility partially depended on scores from the Autism Diagnostic Observation Schedule and Autism Diagnostic Interview, sensitivity was highest in that group. Overall, the sensitivity values of the DSM-5 and DSM-IV criteria were similar (see Table 2).
Sensitivity was also examined within the separate PDD diagnostic groups. DSM-5 sensitivity based on parent report only in the subjects with a clinical diagnosis of Asperger’s disorder or PDD-NOS ranged from 0.76 to 0.94 (see Table 2), while DSM-5 sensitivity in those with autism ranged from 0.93 to 0.95. The sensitivity of the DSM-5 criteria was also examined within PDD phenotypic subgroups based on sex, IQ, and age. As shown in Table 3, the sensitivity for girls ranged from 0.88 to 0.93. For those in the “high-functioning” range of cognitive ability (nonverbal IQ>70), DSM-5 sensitivity was between 0.86 and 0.91, while among those with a nonverbal IQ of ≤70, sensitivity ranged between 0.93 and 0.97. In children under 4, sensitivity ranged between 0.90 and 0.98.
Table 2 includes specificity values for the Collaborative Programs and University of Michigan study groups (the Simons Simplex Collection was restricted to children with PDDs). In the Collaborative Programs group, on the basis of parent-reported items DSM-IV specificity was as high as 0.72 for the autistic disorder criteria and as low as 0.36 for the PDD-NOS criteria. In the University of Michigan group, DSM-IV specificity was 0.20 for PDD-NOS, 0.30 for Asperger’s disorder, and 0.48 for autistic disorder. In contrast, when the DSM-5 ASD criteria were applied, specificity was 0.49 in the University of Michigan group and 0.63 in the Collaborative Programs group.
When evidence of impairments in social reciprocity and nonverbal behavior was required from both parent report and clinical observation, the specificity of the DSM-5 criteria improved (Table 2). This improvement was most clinically meaningful in the University of Michigan subjects, of whom approximately 26% had non-PDD diagnoses. In this group, DSM-5 specificity increased to 0.62. Specificity in the Collaborative Programs sample increased to 0.69 with the requirement that symptoms be evident on both instruments. On the other hand, this requirement led to a decrease in sensitivity across all groups, most strikingly for children with clinical diagnoses of PDD-NOS or Asperger’s disorder (see Table 2). As in the study by Frazier et al. (11), requiring one less subdomain from either domain, with either the parent or clinical report, provided the best balance of sensitivity and specificity, although specificity remained low.
McNemar’s chi-square tests were used to investigate whether DSM-5’s proportion of correct classification of the non-PDD cases was significantly different than that of DSM-IV. When parent-reported items were used, the proportion of individuals with non-PDD diagnoses who were correctly classified by DSM-5 but misclassified by DSM-IV as having PDD-NOS was significantly higher (χ2=117.55, df=1, p<0.001) than the proportion who were misclassified by DSM-5 and accurately classified by DSM-IV (34.9% versus 5.9%). Misclassification by DSM-IV of a non-PDD condition as Asperger’s disorder was also significantly higher (χ2=161.54, df=1, p<0.001) than misclassification by DSM-5 (29% versus 11%).
The DSM-5 domains were examined individually to assess how many children might meet the criteria for social communication disorder and to better understand why some were misclassified in relation to their clinical diagnoses (see Figure 1). In the Simons Simplex Collection (N=2,130), eight subjects who had clinical diagnoses of PDDs failed to meet the DSM-5 criteria for ASD because they did not exhibit enough symptoms in the restricted and repetitive behavior domain, and 178 did not meet the criteria because they did not exhibit enough symptoms in the social communication domain. Similarly, in the Collaborative Programs group (N=1,021); 14 subjects did not meet the criteria in the DSM-5 restricted and repetitive behavior domain, while 45 did not meet the DSM-5 criteria in the social communication domain. In the University of Michigan group (N=1,992), 53 children did not meet the DSM-5 criteria for restricted and repetitive behavior, while 97 did not meet the social communication criteria. In total, 75 of 5,143 subjects met the criteria in the social communication domain only.
In this study we explored the proposed DSM-5 criteria for ASD in three groups of children with clinical DSM-IV PDD or non-PDD diagnoses. The majority of children with clinical diagnoses of PDDs met the DSM-5 ASD criteria according to item scores on the Autism Diagnostic Interview–Revised and the Autism Diagnostic Observation Schedule. Notably, application of the DSM-5 criteria demonstrated adequate sensitivity across all groups, as well as in phenotypic subgroups, including young children, girls, and children denoted as cognitively “higher functioning.” These results, together with those of Frazier et al. (11), provide evidence that the proposed criteria would likely be able to correctly classify a phenotypically wide range of children with ASD. What is more, the results of the current study indicate that the specificity of the DSM-5 criteria is improved when compared with the DSM-IV criteria for Asperger’s disorder and PDD-NOS. Overall, the accuracy of nonspectrum classification made by DSM-5 was better than that of DSM-IV. Thus, although there is much room for improvement with respect to specificity, the proposed criteria appear to meet the stated goal of the DSM-5 committee to create criteria that better distinguish ASD from nonspectrum disorders such as language disorders, intellectual disability, attention deficit hyperactivity disorder, and anxiety disorders. Our results further indicate that requiring evidence of clinician-observed social communication deficits, in addition to parent-reported deficits, can increase the specificity of the proposed criteria. However, the inevitable tradeoff between specificity and sensitivity occurred when evidence was required from both parent report and direct observation.
Given concerns that the restricted and repetitive behavior requirement might lead to reduced identification of children previously diagnosed with ASDs under DSM-IV and possible misclassification as social communication disorder, we examined why some children with PDDs did not meet the DSM-5 criteria. In all three groups, most children who did not meet the criteria did so because they did not demonstrate the required impairments in social and communication functioning, and not because they did not meet the restricted and repetitive behavior criteria. In fact, few children did not meet the restricted and repetitive behavior requirement in DSM-5. These results suggest that few children with ASDs are likely to be misclassified as having social communication disorder and lend further support to the addition of the restricted and repetitive behavior criterion.
Finally, the process of matching individual symptom items to the diagnostic criteria revealed potential challenges in the interpretation of DSM-5 criterion A3. In addition to the reduced number of items (especially on the Autism Diagnostic Observation Schedule) that could be applied to A3, it was also sometimes difficult to determine whether an item should be placed in A3 (“difficulties adjusting behavior to suit different social contexts”) or A1 (“abnormal social approach”). Although the difficulty in assigning specific items may have partially resulted from the fact that the study measures were based on the DSM-IV criteria and therefore not designed to map directly onto the DSM-5 criteria, it will be critical to ensure that the final wording of the DSM-5 criteria lends itself to being clearly and reliably interpreted by ASD diagnosticians.
Replication of our findings in other study groups (including adults), by means of both retrospective data analysis and prospective field trial methods, is needed. Two of our study groups may not be fully representative of children typically referred for assessment of ASDs. Our groups may represent extremes in terms of ASD phenotypes: on the one hand, clinical cases at the University of Michigan with complex presentations, and on the other, clearer cases of ASDs in the Simons Simplex Collection.
The results obtained here may not reflect the proposed criteria’s true sensitivity and specificity. Using archival data and symptom counts is not comparable to clinical diagnosis. As the study instruments were largely based on the DSM-IV criteria, it is likely that behaviors that might fit into the DSM-5 criteria are not currently captured by these methods. In spite of the breadth of the Autism Diagnostic Interview, analyses of existing data cannot begin to approximate a field trial. Conducting evaluations in real time and making determinations about whether a child meets the DSM-5 criteria on the basis of all information gathered during that evaluation is the only way to assess the true sensitivity and specificity of the DSM-5 criteria. Nevertheless, though in practice it would be inappropriate to make diagnoses solely on the basis of symptom counts, our use of these methods allows comparisons with other researchers’ analyses of the DSM-5 criteria.
To our knowledge, this study is the most comprehensive assessment to date of the newly proposed DSM-5 ASD criteria. Based on symptom extraction from previously collected data, our findings indicate that the majority of children with DSM-IV PDD diagnoses would continue to be eligible for an ASD diagnosis under DSM-5. Additionally, these results further suggest that the revisions to the criteria, when applied to records of children with non-PDD diagnoses, yield fewer misclassifications. Our findings also contribute to literature that supports the use of both parent report and clinical observation for optimal classification accuracy.
The principal investigators of the Simons Simplex Collection were A. Beaudet, R. Bernier, J. Constantino, E. Cook, E. Fombonne, D. Geschwind, D. Grice, A. Klin, D. Ledbetter, C. Lord, C. Martin, D. Martin, R. Maxim, J. Miles, O. Ousley, B. Peterson, J. Piggot, C. Saulnier, M. State, W. Stone, J. Sutcliffe, C. Walsh, and E. Wijsman. Approved researchers can obtain a subset of the data described in this study at the Simons Foundation Autism Research Initiative (SFARI) web site (https://base.sfari.org).