Three-Year Predictive Validity of DSM-IV Attention Deficit Hyperactivity Disorder in Children Diagnosed at 4–6 Years of Age

DSM-IV criteria for attention deficit hyperactivity disorder (ADHD) were based on reviews of research (1, 2) and a field trial (3, 4), but the paucity of preschool children in research studies raises concerns about the validity of the diagnosis at younger ages. It is particularly important to study the validity of ADHD in younger children because of recent increases in the use of psychoactive medication among preschoolers (5, 6). Because all treatments confer some risk, it is essential to know if ADHD is associated with significant and persisting functional impairment in younger children to determine if the risks inherent in treating are less than the risks inherent in not treating (7).

In 1995, we initiated a study of the validity of ADHD in children first diagnosed at 4–6 years of age (8). The diagnosis showed strong concurrent validity: children who met DSM-IV symptom criteria for ADHD exhibited greater social and academic impairment, more placements in special education, and more unintentional injuries in wave 1 than comparison children, even when demographic factors, intelligence, and co-occurring conduct and internalizing problems were controlled (8). A similar study (9) found that preschool children who met symptom criteria for ADHD exhibited more concurrent social skills problems than comparison children.

It is clear that the diagnosis of ADHD in elementary school predicts continuing symptoms and impairment (10–13). There are currently no published data on the persistence of impairment in younger children diagnosed with ADHD, but two longitudinal studies have shown that preschoolers rated as highly hyperactive showed persistent symptoms over time (14, 15). In the present study, we determined if ADHD has predictive validity by measuring if 4–6-year-old children who meet full diagnostic criteria for ADHD continued to exhibit symptoms and impairment over time. To avoid circularity, the impairment measures assessed in the three waves after the initial diagnosis of ADHD were not used in our operational definition of cross-situational impairment in making the initial diagnosis.

Method

Participants

Two cohorts of 3.8–7.0-year-old children were recruited in consecutive years at two sites. In Chicago, child psychiatry clinic attendees with complaints of ADHD were recruited as probands. In Pittsburgh, 42.4% of ADHD probands were recruited from a child psychiatry clinic, with the remainder recruited through advertisements. Probands recruited from clinics or advertisements did not differ significantly on demographic or impairment measures (8).

All participants lived with their biological mother and were enrolled in structured educational programs: 36% in preschool, 43% in kindergarten, 21% in first grade, and 1% in second grade. Five (two in Pittsburgh, three in Chicago) potential probands were excluded because they had previously received clinical diagnoses of pervasive developmental disorder, mental retardation, or seizure disorder. In addition, comparison children were recruited from the same schools as probands or from schools in similar neighborhoods. Comparison children had never been referred for mental health problems but were not excluded if they met criteria for a mental disorder other than ADHD. Comparison children were selected to approximately match probands in terms of gender, ethnicity, and age. Of the 310 eligible participants, 259 parents (120 in Chicago, 139 in Pittsburgh) gave written informed consent after the study was explained; all children gave oral assent.

Psychoactive medications taken by the children were grouped into three classes: stimulants, antihypertensives (clonidine or guanfacine), and mood medications (selective serotonin reuptake inhibitors, imipramine, divalproex, or risperidone). The percentages of children meeting wave 1 symptom criteria for ADHD who were taking medications during the year preceding each assessment were 17.0%, 41.1%, 44.6%, and 48.4% for stimulants; 0.0%, 2.4%, 5.0%, and 5.7% for antihypertensives; and 0.0%, 3.2%, 5.8%, and 11.5% for mood medications. The corresponding percentages for comparison children were 0.0%, 0.8%, 1.7%, and 3.4% for stimulants; 0.0% in all waves for antihypertensives; and 0.0%, 0.0%, 0.0%, and 0.8% for mood medications, respectively. Parents were asked to discontinue medications on the day of the assessment if approved by the child’s physician, and informants were asked to rate children when they were not taking medication.

Measures

Two lay interviewers concurrently conducted independent interviews of the mothers and children. Intelligence was estimated by averaging Stanford-Binet Intelligence Scale Short Form (16) scores from waves 1 and 2. Four children were excluded because their average intelligence scores were <70. The Diagnostic Interview Schedule for Children (17) was administered to the parent in each wave. Information was obtained on DSM-III-R diagnostic criteria for ADHD, oppositional defiant disorder, conduct disorder, anxiety disorders (simple phobia, social phobia, agoraphobia, separation anxiety, and panic, avoidant, and overanxious disorders), major depression, and dysthymia during the last 6 months. In addition, questions used in the DSM-IV field trials (4) to assess DSM-IV symptoms of disruptive behavior disorders not in DSM-III-R were asked. The test-retest reliability of the subtypes of ADHD with this version of the Diagnostic Interview Schedule for Children is kappa=0.51–0.64 (4).

Information on symptoms of the disruptive behavior disorders was obtained from teachers by mail using the DSM-IV version of the Disruptive Behavior Disorder Rating Scale (18). As in previous studies (18), symptoms rated “pretty much” or “very much” were scored as present. As in the DSM-IV field trials, symptoms were considered present if reported by either parent or teacher (19). In wave 1, impairment was assessed in two ways to make the diagnosis of ADHD. First, the parent was asked in the Diagnostic Interview Schedule for Children if the child’s ADHD symptoms had caused problems 1) at home or with friends or 2) at school. Second, parents and teachers completed the Impairment Rating Scale (unpublished measure by Fabiano GA et al.), in which the child’s need for treatment in a variety of areas is rated using 7-point subscales ranging from “no problem, definitely does not need treatment” (score=0) to “extreme problem, definitely needs treatment” (score=6). Parents rated the child’s need for treatment because of problems in relations with peers, siblings, and parents; academic progress at school; self-esteem; and impact on the family. Teachers rated the child’s need for treatment because of problems in relations with classmates and teachers, academic progress, classroom disruption, and self-esteem. Both respondents rated the child’s overall need for treatment. In two groups, Impairment Rating Scale ratings of ≥3 on at least one subscale optimally differentiated clinic and nonclinic children for both parents and teachers (unpublished, by Fabiano GA et al.). Test-retest stability (different teachers, 1 year apart) for the six Impairment Rating Scale subscales is r=0.39–0.63 (p<0.001) (unpublished, by Fabiano GA et al.).

Diagnoses

Children met full diagnostic criteria for ADHD (full ADHD) if they met DSM-IV symptom criteria and exhibited cross-situational impairment. Cross-situational impairment was defined as 1) parent report of problems at home or with peers in the Diagnostic Interview Schedule for Children or a parent rating of ≥3 on any Impairment Rating Scale subscale (excluding the school setting) and 2) parent report of school problems in the Diagnostic Interview Schedule for Children or a teacher rating of ≥3 on any Impairment Rating Scale subscale. All children met the age-at-onset criterion. In wave 1, 96 of 129 children who met symptom criteria for ADHD also exhibited cross-situational impairment. However, 29 children met symptom criteria for ADHD but displayed impairment in only one setting during wave 1. Therefore, we created a situational ADHD group to evaluate the validity of the cross-situational impairment criterion. The remaining 126 comparison children did not meet symptom criteria for ADHD in wave 1. Four children who met symptom criteria for ADHD in wave 1 but did not display impairment in any setting were conservatively classified in the comparison group (N=130).

Although 29.4% of the situational ADHD group met symptom criteria for the combined type, 76.0% of the full ADHD group met symptom criteria for the combined type (χ²=14.99, df=2, p<0.001, N=129). Therefore, differences between the full and situational ADHD groups necessarily also reflect differences in subtypes.

Longitudinal Assessments

Of the 255 children assessed in wave 1, 91.4% were reassessed in all four annual assessments, with no significant differences in retention among the three groups (full ADHD, situational ADHD, and comparison groups). Because only 10.2% had fewer than two waves of teacher reports and our data analyses allowed for missing data, there was little opportunity for bias due to attrition. The percents of interviews of parents conducted by interviewers who were unaware of previous assessments across waves 2–4 were 70.9%, 65.8%, and 60.0%, respectively. The corresponding percentages of youth assessments conducted by blind interviewers were 92.5%, 85.3%, and 74.1%. We controlled for nonblind interviews in all analyses of parent and youth reports. This was not relevant to teacher reports because different teachers completed questionnaires each year.

Child functioning during waves 2–4 was assessed by using impairment measures not used in the wave 1 diagnosis. Ratings of the child’s lowest level of overall functioning during the past 6 months were obtained on a 1–100 thermometer-like scale independently from the parent and the interviewer who administered the Diagnostic Interview Schedule for Children using the Nonclinician Version of the Children’s Global Assessment Scale (20). In each wave, the parent was also asked if the child had suffered an injury more serious than a scratch, bruise, or bump on the head that the parent attributed to the child’s carelessness, impulsiveness, or poor judgment.

The child’s regular education teacher used 5-point scales to estimate the proportions of classmates who liked and disliked the child (21). The widely used and well-validated social preference score (22, 23) was created by subtracting the “dislike” rating from the “like” rating. The sum of two items (each rated 0–2) from the Social Skills Rating Scale (24) quantified the teacher’s perception of classroom academic functioning: “produces correct schoolwork” and “finishes class assignments within time limits.” Findings based on this measure of classroom performance should be interpreted cautiously because the wording of some symptoms of ADHD (i.e., “makes careless mistakes in schoolwork” and “fails to finish schoolwork”) themselves imply impaired classroom functioning. However, if children who met criteria for ADHD were not impaired on this measure over time, it would raise serious questions about the predictive validity of ADHD. A child self-report scale validated against sociometrics in this age range (25, 26) assessed friendship difficulties from the child’s perspective in each wave (2-week test-retest reliability: r=0.77–0.79) (26). The 15 items cover making friends, keeping friends, and engaging in activities with friends.

Data Analysis

The 12-month stability of impairment measures without published reliability data was assessed from the present group with wave-to-wave correlations for continuous measures and Cohen’s kappa for nominal measures (27). We assessed the association of each class of medication with ADHD symptoms by conducting separate Poisson regressions in general estimating equations (28), treating each class of medication as a time-varying covariate (0=no, 1=yes). Continuous Children’s Global Assessment Scale ratings were reversed to approximate Poisson distributions, and planned comparisons among the three groups were conducted in general estimating equations. Differences in the rates of meeting full criteria for ADHD during waves 2, 3, and 4 were assessed by using binomial longitudinal regression. All longitudinal analyses specified unstructured correlations and used the z statistic (df=1) and robust standard errors. Interactions between time (wave) and groups were tested but were never significant. Cumulative response variables defined by one or more occurrence (unintentional injury and special education placement) were assessed by using logistic regression. Ordinal measures of impairment (teacher ratings of classroom work and social preferences) were analyzed by using mixed-effects ordinal regression in MIXOR (29). All analyses controlled for age, sex, family income (logged), intelligence, total number of oppositional defiant disorder and conduct disorder symptoms, total number of anxiety and depression symptoms, and site. In longitudinal analyses, time (waves) and the presence of blind interviews in each wave also were controlled.

Results

Stability across successive annual assessments of parent Diagnostic Interview Schedule for Children ratings of the sum anxiety and depression symptoms was r=0.79, 0.71, and 0.68, and the 12-month stability of Diagnostic Interview Schedule for Children impairment in home or peer relations was kappa=0.71, 0.70, and 0.56, and kappa=0.60, 0.55, and 0.52 for school impairment. Annual stability of teacher ratings was r=0.52, 0.56, and 0.61 for academic problems, and r=0.54, 0.49, and 0.47 for social preference. Annual stability of Children’s Global Assessment Scale ratings was r=0.67, 0.70, and 0.70 for parents, and r=0.72, 0.76, and 0.79 for interviewers. All correlations were p<0.0001.

In wave 1, the groups did not differ in sex, race/ethnicity, age, or family income (Table 1). Both ADHD groups had lower mean intelligence scores than the comparison group but did not differ significantly from one another. The full ADHD group exhibited more internalizing and conduct symptoms than the situational group in wave 1, which was reported to exhibit more such symptoms than the comparison children.

Similar to findings with older children (30), taking stimulants (z=3.76, df=1, p<0.001) or mood medications (z=2.42, df=1, p<0.02) but not antihypertensives (z=1.13, df=1, p=0.26) was associated with more ADHD symptoms during the same wave, with control for time, site, and blinding status. We interpret this as indicating that children with more ADHD symptoms were more likely to be given prescription medication.

As shown in Figure 1, both the full ADHD group (β=3.19, z=8.87, p<0.0001) and the situational ADHD group (β=1.72, z=4.46, p<0.0001) were more likely to meet full diagnostic criteria for ADHD during waves 2–4 than the comparison children. The full ADHD diagnosis in wave 1 predicted the full diagnosis in wave 4, with a positive predictive value of 0.75 and a negative predictive value of 0.86. The situational group was significantly less likely to meet full diagnostic criteria for ADHD during waves 2–4 than the full ADHD group (β=–1.46, z=–4.28, p<0.0001). Cumulatively, 3.1% of the comparison group met full diagnostic criteria for ADHD at least twice during waves 2, 3, and 4, compared to 79.2% of the children in the full ADHD group (odds ratio=119.7, 95% confidence interval [CI]=39.4–363.4) and 34.5% of the situational group (odds ratio=16.6, 95% CI=4.7–58.2).

The full ADHD group exhibited significantly greater global, academic, and social impairment during waves 2–4 than the comparison children (Table 2 and Figure 2, Figure 3, Figure 4, Figure 5, and Figure 6). The situational ADHD group was also more impaired on the Children’s Global Assessment Scale and was reported to have greater academic and social impairment over time than the comparison children. Similarly, the odds of at least one unintentional injury during waves 2–4 was 7.1 times greater for the full ADHD group (95% CI=2.3–21.8) and 5.1 greater for the situational group (95% CI=1.5–17.8) than for the comparison group. The odds of placement in special education during waves 2–4 for the full ADHD group was 3.1 times greater than for the comparison children (95% CI=1.3–7.5), but the children in the situational group were not more likely to be placed in special education than the comparison children (odds ratio=2.0, 95% CI=0.7–5.3) when other variables were controlled. The full and situational ADHD groups differed significantly on global Children’s Global Assessment Scale ratings but did not differ significantly on any measure of specific aspects of functional impairment.

Discussion

These findings strongly support the predictive validity of DSM-IV criteria for ADHD by showing that 4–6-year-old children who met full diagnostic criteria were likely to continue to meet criteria for ADHD and to experience greater functional impairment than comparison children over 3 years. These enduring differences in impairment were found across parent, teacher, interviewer, and child reports. It is particularly impressive that strong evidence of persisting functional impairment was found in teacher reports because different teachers provided data each year and were blind to diagnoses.

The 3 years covered in this study are developmentally important. Because the present study followed the children from preschool, kindergarten, or first grade well into elementary school, these findings suggest that the great majority of children given the diagnosis of ADHD at 4–6 years of age continue to exhibit ADHD symptoms and impairment into the age range during which ADHD has been extensively studied (4). It will be essential to follow these youth much longer, however, to determine if they continue to be symptomatic and impaired.

Thirty-four percent of situational ADHD children met full diagnostic criteria for ADHD at least twice in waves 2–4 compared to 3.1% of comparison children, and they were more impaired than comparison children on several measures. This suggests that a substantial minority of children who meet symptom criteria for ADHD but do not meet the cross-situational impairment requirement in their initial assessment will later meet full diagnostic criteria for ADHD. This could be limited to younger children because classroom demands on children’s attention and behavior at 4–6 years may not yet reveal classroom impairment that will emerge later, but the validity of the cross-situational impairment requirement has not been tested in children diagnosed at older ages. It is hoped that the present findings will stimulate research on this requirement, which was added in DSM-IV. Indeed, it would be useful to reconsider the appropriateness of this criterion on logical grounds. It is unclear why children who meet all diagnostic criteria for ADHD but display impairment in only one setting (e.g., seriously impaired academic functioning) should not be given the diagnosis of ADHD. Functional impairment must be a diagnostic criterion for all mental disorders, but it may be inappropriate to require impairment in multiple settings only for ADHD.

The present findings showed that children who met criteria for ADHD continued to display impairment over time. We stringently controlled age, sex, income, intelligence, co-occurring psychopathology, site, and nonblind assessments to ensure that impairment was related to ADHD, per se, and not to factors that co-occur with it (8). If ADHD had not accounted for unique variance in predicting future impairment after controlling these variables, there might be no need for the construct of ADHD. For example, Werry et al. (31) hypothesized that lower intelligence might fully explain the impairment associated with ADHD. If this hypothesis had been confirmed, one could dispense with the diagnosis of ADHD. None of the many types of functional impairment associated with ADHD could be fully explained by lower intelligence, concurrent psychopathology, or other factors, however. Indeed, we may have “overcontrolled” these characteristics in our analyses. For example, children who meet criteria for ADHD may score lower on intelligence tests than comparison children wholly or partly because their inattention and activity interfere with testing. If so, controlling for intelligence would artificially reduce the amount of variance in impairment explained by ADHD symptoms. Impressively, however, we were able to demonstrate that DSM-IV ADHD was associated with persisting impairment, even when intelligence was controlled.

Nonetheless, the present study was limited by the impairment measures used. To date, more effort has gone into the study of diagnostic criteria than functional impairment. Because the accurate measurement of functional impairment is at least as important to the diagnostic process as the measurement of symptoms and even more important to treatment planning, this is an area that sorely requires additional research. Similarly, because only one of many possible operational definitions of “cross-situational impairment” was used in this study, further research is need on that criterion. Finally, because this study used a structured diagnostic protocol, it is not clear that the findings of validity generalize to the kinds of unstructured assessments of ADHD often used in clinical practice.

TABLE 1

Enlarge table

TABLE 2

Enlarge table