Pathways of Self-Reported Problem Behaviors From Adolescence Into Adulthood
Abstract
OBJECTIVE: The authors determined the impact of different pathways of psychopathological development on adult outcome in subjects followed from ages 11–18 to ages 21–28. METHOD: Problem behaviors of subjects from a general population sample were assessed through the Youth Self-Report and the Young Adult Self-Report given at four time points (1987, 1989, 1991, and 1997). In addition, DSM-IV diagnoses, information pertaining to signs of maladjustment, and measures of social functioning were obtained at the last assessment. On the basis of the self-report ratings, four contrasting developmental pathways of psychopathology were determined: persistent, decreasing, increasing, and consistently normal. RESULTS: Subjects whose overall level of psychopathology was persistent over time had a higher lifetime prevalence of DSM-IV diagnoses and a poorer general outcome in adulthood than did subjects whose level of psychopathology increased. Subjects whose level of psychopathology returned to normal after high levels of problems in adolescence were only slightly different in terms of outcome from subjects with consistently normal ratings. CONCLUSIONS: 1) People who showed high levels of problems in early adolescence but whose level of psychopathology diminished by adulthood seemed to be as healthy as people who never attained a serious level of psychopathology. 2) An ongoing devious pathway into adulthood had negative effects on many domains of functioning. These two findings are both powerful arguments for early intervention in adolescence.
Knowledge of the natural course of psychopathology from adolescence into adulthood is essential for our understanding of the development of psychopathology. One aspect of psychopathological development we still know little about is the extent to which variations in individual trajectories of maladaptation affect the outcome. For instance, little is known about the effects of the duration of maladapted functioning during adolescence on the outcome in adulthood. It may be that the more consistently a deviating pathway is followed over time, the more unlikely it becomes that a normal pathway can be reclaimed (1). However, there still is little empirical support for this hypothesis. Also, little is known about the level of adulthood adaptation when individuals with high levels of adolescent psychopathology recover during their development. It may be that earlier, temporary breakdowns make individuals more resilient thereby decreasing the probability of later maladaptation (2); conversely, earlier psychopathology may set off vulnerabilities that render individuals prone to future maladaptation (3).
Few longitudinal general population studies exist that used comparable measures of psychopathology from adolescence into adulthood. Achenbach et al. (4) tested the 6-year continuities and predictive paths of syndromes from adolescence (subjects aged 13–16 years assessed with the Child Behavior Checklist [5], a parent rating scale) to adulthood (subjects assessed at age 19–22 with the Young Adult Self-Report [6] and the Young Adult Behavior Checklist [6], a parent rating scale). It was found that several young adult syndromes were strongly predicted by their adolescent counterparts. Ferdinand and Verhulst (7), at an earlier stage of the current study, examined the 8-year stability of psychopathology from adolescence into young adulthood. They found that of the individuals aged 13 through 16 years with Child Behavior Checklist total problem scores in the deviant range, 27.3% had Young Adult Self-Report total problem scores in the deviant range at 8-year follow-up.
These studies elucidated the strength of the continuity of psychopathology from adolescence into young adulthood. However, these studies were less informative with respect to the variations in psychopathological development and its consequences for later adaptation.
The present study focused on the impact of different developmental pathways of psychopathology on outcomes in adulthood by classifying individuals on the basis of prospective self-report information at four assessment points that spanned a 10-year interval. Four groups of individuals were selected on the basis of their developmental trajectories: those who could be regarded deviant at all four assessments (persistent psychopathology), those who showed high levels of problems in adolescence but returned to normal in adulthood (decreasing psychopathology), those who showed high levels of problems only in adulthood but not in adolescence (increasing psychopathology), and those who had low levels of problems at all four assessments (consistently normal).
By assessing DSM-IV diagnoses, other signs of maladjustment (referral to mental health services, suicide attempts, arrests, expulsion from school or job, and no education beyond elementary school), and indices of social functioning in adulthood, we determined the outcome for each of the contrasting developmental trajectories of psychopathology.
Method
Assessment of Psychopathology
The Youth Self-Report (8, 9) is a self-report questionnaire for subjects 11–18 years of age that was modeled on the Child Behavior Checklist (5), a parent questionnaire for the assessment of psychopathology in children and adolescents. The Youth Self-Report contains 103 items covering a broad range of emotional and behavioral problems during the previous 6 months. Examples of problem items are “I cry a lot,” “I feel lonely,” and “I physically attack people.” Responses are rated on a 3-point scale in which 0=not true, 1=somewhat or sometimes true, and 2=very true or often true. Summing the scores for each problem item derives a total problem score. The good reliability and validity of the Youth Self-Report (8) were supported for the Dutch version (9, 10).
The Young Adult Self-Report (6) is a questionnaire for 19–30-year-olds. It contains 110 problem items that are scored in the same way as the Youth Self-Report items. Good reliability and validity for this measure have been reported by Achenbach (6) and were supported by the Dutch version (11, 12). The 18-day test-retest reliability for the total problem score in a general population sample was high (r=0.89) (11).
To determine the course of psychopathology across time, Youth Self-Report and Young Adult Self-Report total problem scores were computed, omitting items that were not included in both instruments (12 from the Youth Self-Report and 24 from the Young Adult Self-Report).
The computerized version of the Composite International Diagnostic Interview (13) and three sections of the National Institute of Mental Health Diagnostic Interview Schedule (DIS) (14) were used to obtain lifetime and 12-month histories of DSM-IV mental disorders. The Composite International Diagnostic Interview and DIS are fully structured interviews that allow administration by lay interviewers. Good reliability and validity have been reported for the Composite International Diagnostic Interview (15). Test-retest studies showed agreement for diagnoses to be acceptable to excellent, with all but simple phobia having kappa values greater than 0.40. Because the Composite International Diagnostic Interview lacks diagnoses of disruptive disorders (oppositional defiant disorder, antisocial personality disorder, and attention deficit hyperactivity disorder), three sections of the DIS covering these disorders were translated into Dutch and administered. Each Composite International Diagnostic Interview and DIS was conducted by a lay interviewer trained by the Dutch World Health Organization training center.
Because of small cell sizes for the individual disorders, we combined disorders into the following groupings: 1) any DSM-IV disorder, 2) anxiety disorders, 3) mood disorders, 4) substance abuse/dependence, and 5) disruptive disorders.
Assessment of Functioning and Outcome
The Global Assessment Scale (16) was used to evaluate the overall functioning of a subject during the 6 months preceding the interview. The Global Assessment Scale was scored by the interviewer after the interview. The scale values range from 1 to 100 and are divided into 10 equal intervals. The highest interval (91–100) indicates good functioning, while lower scores indicate problems in functioning. The test-retest intraclass correlation coefficients of the Global Assessment Scale in subjects from different populations ranged from 0.69 to 0.91 (16).
For each subject, we determined whether the following poor outcome events occurred in the preceding 12 months: referral to mental health services, suicide attempt, police arrest, and expulsion from school or termination from a job, as well as whether the subject had only finished elementary school.
Current social functioning was assessed with the Groningen Questionnaire About Social Behavior (17, 18). Three subscales covering the following areas were assessed: 1) social activities (activities with friends); 2) daily activities (schoolwork, employment, and housekeeping); and 3) spare-time activities. The first two scales pertain to problems with interpersonal contacts; five of the six items of the last subscale refer to the subject’s satisfaction with the quality of his or her spare-time activities. Subjects were asked to rate situations applicable to their situation in the preceding 4 weeks. Each item had four response options, ranging from never to always. By summing the scores, a total score for each subscale was derived. Applying a cutoff dichotomized these scores. Subjects who scored above the cutoff were considered to be impaired on that specific subscale (18). The test-retest reliability correlations of the scales ranged from good to excellent (r=0.69–0.97). About 4% of the subjects (N=26 of 705) did not have any close friends and therefore could not answer questions regarding social activities (the scale was limited to activities with friends). Thus, a separate sign of impaired social functioning was added: having no close friend.
Study Group and Procedure
The original sample consisted of children aged 4 to 16 years, drawn in 1983 from the Dutch province of Zuid-Holland. Municipal registers that listed all residents were used from which a random sample of 100 children of each sex and age with Dutch nationality was drawn (N=2,600). Of the parents of the 2,447 target children who were reached, 2,076 (84.8%) completed a Child Behavior Checklist (5) on their child. The first assessment was designated time 1 (see Verhulst et al. [19, 20] for details on the initial data collection).
The present study, which used self-report information, stemmed from follow-ups of the original sample. As can be seen in Figure 1, subjects were asked to complete self-reports in 1987 (time 3), 1989 (time 4), 1991 (time 5), and 1997 (time 6). The Youth Self-Report was used for subjects 11–18 years of age; the Young Adult Self-Report was used for those 19–28 years of age. Subjects whose parents declined to participate at time 1 were not contacted. Because the Youth Self-Report was constructed for ages 11–18 years, only subjects within this age range were asked to complete the Youth Self-Report at time 3 (1987). These subjects were between the ages of 7 and 14 years at initial assessment (time 1, 1983) and form the original sample of the present study.
In 1997 (time 6), DSM-IV diagnoses and information on social functioning and on general signs of maladjustment were obtained as well. In total, complete information (e.g., self-reports at times 3, 4, 5, and 6) was obtained for 705 subjects—310 male and 395 female subjects.
In 1983 (time 1), 1,285 parents of 7–14-year-olds completed Child Behavior Checklists. The 705 individuals on whom we obtained complete information at times 3, 4, 5, and 6 represented 54.9% of the original (1983) sample on whom we obtained Child Behavior Checklists from parents.
To assess selective attrition, we compared dropouts (subjects who were in the right age range to complete self-reports in 1987 [time 3] but on whom we did not obtain complete information at all four assessments, N=580) and remainers (N=705) with respect to age, sex, Child Behavior Checklist total problem score at time 1, and parent socioeconomic status at time 1. Socioeconomic status was assessed by using a 6-point scale of parental occupation (21), with 1=lowest socioeconomic status. There was a significant difference in age between the dropouts (mean=10.8 years, SD=2.3) and the remainers (mean=10.2 years, SD=2.3) at time 1 (t=4.4, df=1283, p<0.001). Also, the response rate was higher in female subjects (60.4%, N=395 of 654) than in male subjects (49.1%, N=310 of 631) (χ2=16.47, df=1, p<0.001). The Child Behavior Checklist total problem score at time 1 for dropouts (mean=22.3, SD=18.7) was slightly but significantly higher than that for remainers (mean=20.0, SD=15.0) (t=2.4, df=1099, p<0.02). The mean socioeconomic status for dropouts (mean=3.4, SD=1.6) was slightly but significantly lower than that for remainers (mean=3.8, SD=1.6) (t=–4.38, df=1280, p<0.001).
Subjects were considered deviant when they scored above the cutoff for the total problem score on the Youth Self-Report or Young Adult Self-Report. The cutoff was set at the 82nd percentile as recommended by Achenbach (5, 6). This cutoff has been shown to discriminate between referred and nonreferred subjects and can therefore be considered to be of clinical relevance. The 50th percentile of the cumulative frequency distribution of the total problem score was chosen as the arbitrary level below which individuals were considered to function well. The use of the 50th percentile enabled us to identify individuals whose functioning improved or worsened considerably across time. Cutoffs were determined for each sex separately. When selecting subjects for the four contrasting developmental pathways of psychopathology, we only used those on whom we obtained complete information at all four assessments (N=705).
Subjects who scored in the deviant range at all four assessments were considered to have persistent psychopathology (N=22). Subjects who scored below the 50th percentile at the initial assessment (time 3, 1987) and in the deviant range at time 6 (1997) were considered to have increasing psychopathology (N=20). Subjects who scored in the deviant range at time 3 but below the 50th percentile at time 6 were considered to have decreasing psychopathology (N=32). Finally, to compare the other three groups with subjects that could be regarded as normal, we selected a fourth category consisting of subjects who scored under the 50th percentile at all four assessments (N=145).
Each assessment phase of this study was approved by the Committee for Medical Ethics, University Hospital/Erasmus University Rotterdam. Informed consent was obtained from all subjects after the procedure had been fully explained.
Statistical Analyses
We used t tests to test differences between mean scores at time 6. To test the significance of differences in the prevalence of DSM-IV diagnoses and signs of maladjustment among the four contrasting developmental pathways, chi-square tests with Yates’s correction were used. Fisher’s exact test significance levels were used for tables with cell sizes <5. Analyses of variance were used to test differences in Young Adult Self-Report total problem scores and Global Assessment Scale scores among the four contrasting developmental pathways at time 6, followed by the least significant difference post hoc test to test pairwise comparisons. Bonferroni correction was used to adjust the significance level for testing multiple variables. Only significant results were further analyzed, with the significance level set at 0.05 again. This procedure equals the least significant difference post hoc test used in the analyses of variance. All statistical tests were computed with SPSS 9.0 for Windows (SPSS, Chicago).
Results
Figure 2 shows the mean total problem scores on the self-report measures at each of the four assessment points over the 10-year period for the subjects grouped by their developmental pathways of psychopathology. The total problem score at time 6 for subjects with persistent psychopathology (mean=52.14, SD=19.07) was significantly higher than that of subjects with increasing psychopathology (mean=43.70, SD=9.25), and the total problem score for subjects with decreasing psychopathology (mean=13.44, SD=3.78) was significantly higher than that of the subjects who were consistently normal (mean=8.90, SD=4.67).
DSM-IV Diagnoses
Table 1 shows the 12-month and lifetime histories of DSM-IV diagnoses for subjects in each developmental pathway. Significant differences in the rate of anxiety disorders, mood disorders, disruptive disorders, and any DSM-IV disorder were seen in the preceding 12 months and over the lifetime for the four groups. No subject in the increasing psychopathology group met criteria for a disruptive disorder in the preceding 12 months.
When looking at the contrasting pathways of psychopathology in more detail, post hoc pairwise comparisons found that subjects with persistent psychopathology had a higher lifetime prevalence of DSM-IV disorders (p=0.04) and higher rates of disruptive disorders in the preceding 12 months (p=0.02) and over the lifetime (p=0.02) than did those with increasing psychopathology. Subjects with decreasing psychopathology and those who were consistently normal did not differ significantly in the prevalence of any category of DSM-IV disorder category.
Comorbidity
Comorbidity rates of DSM-IV disorders in the preceding 12 months differed significantly (p<0.001, Fisher’s exact test) among the four groups (persistent psychopathology: 18.2% [N=4]; increasing psychopathology: 10.0% [N=2]; decreasing psychopathology: 6.3% [N=2]; consistently normal: 0.7% [N=1]).
Signs of Maladjustment
We compared subjects from each of the developmental pathway groups in terms of maladjustment at time 6. The results are shown in Table 2. A one-way analysis of variance with the four pathway groups as a factor and Global Assessment Scale scores at time 6 as the dependent variable revealed a significant effect. Post hoc pairwise comparisons revealed that subjects with persistent psychopathology had significantly lower mean Global Assessment Scale scores (p=0.001) than did those with increasing psychopathology, whereas subjects with decreasing psychopathology did not significantly differ from those subjects with consistently normal scores.
Significant differences in the occurrence of events indicating poor outcome were found among the four groups for suicide attempts and experiencing any poor outcome event (i.e., one or more outcome events). Post hoc pairwise comparisons revealed that subjects with persistent psychopathology differed significantly from those with increasing psychopathology in terms of experiencing any poor outcome event (p=0.04), whereas subjects with decreasing psychopathology did not significantly differ from those subjects with consistently normal scores.
When looking at signs of impaired social functioning, we found significant differences among subjects from the four developmental groups in terms of experiencing impairment in daily activities and spare-time activities as well as any sign of impaired social functioning. Subjects with persistent and increasing psychopathology did not differ significantly on the overall scale (i.e., any sign of impaired social functioning) (χ2=0.00, df=1, p=1.00), nor did subjects with decreasing psychopathology and those whose scores were consistently normal (χ2=1.78, df=1, p=0.18).
Discussion
The main limitation of the current study concerns the generalizability of the findings. The sample used in the present study consisted of subjects who showed somewhat less behavioral and emotional problems in childhood than subjects who, at some point in time, refused to cooperate. Moreover, because we made use of information that had to be complete across four assessment points covering a period of 10 years, the sample was substantially smaller than the original sample. This may have weakened our findings. Furthermore, because we did not include standardized information on treatment and help-seeking during the years of follow-up, we do not know the effect of possible interventions on the course of problem behaviors from adolescence into adulthood. Moreover, the current study used information given by the adolescents or adults themselves. Sole reliance on self-reports may have influenced our results.
Despite these limitations, we could determine the differential impact on adult outcome of four contrasting 10-year developmental pathways of psychopathology in a general population sample. The four groups were selected on the basis of their a priori-defined developmental trajectories (deviant at each of the four assessments, high levels of problems in adolescence but normal in adulthood, high problem levels in adulthood but not in adolescence, and consistently normal from adolescence into adulthood). These trajectories were chosen because of their contrasting character and clinical relevance.
We wanted to know whether individuals with similar levels of psychopathology at outcome as measured with the Young Adult Self-Report, but who differed in developmental trajectories, also differed in their level of adaptation in adulthood. Not only did subjects with persistent psychopathology differ significantly in their mean total problem scores on the Young Adult Self-Report from those whose problem levels increased, but both groups differed on a number of other outcome variables in adulthood. Not only was the prevalence of any lifetime DSM-IV diagnosis more prevalent among subjects with persistent psychopathology than among those for whom problem levels increased, the persistent psychopathology subjects also had a greater probability of experiencing, in general, poor outcome events. In addition, subjects with persistent psychopathology had significantly lower overall functioning than did those with increasing psychopathology. In contrast, social functioning in adulthood seemed to be influenced to a lesser extent by earlier psychopathology. However, when we determined the co-occurrence of impaired social functioning signs in more detail, it was found that of those in whom psychopathology increased over time (N=20), only one individual had two signs of impaired social functioning and one individual had three. Subjects with persistent psychopathology (N=22), on the other hand, showed a higher rate of co-occurrence of impaired social functioning signs (three individuals had two signs of impaired social functioning and five individuals had three). Thus, impaired social functioning was more pervasive and affected a broader range of everyday life activities in those with persistent psychopathology than among subjects whose psychopathology increased over time.
These findings converge to the conclusion that individuals who persistently show high levels of psychopathology from adolescence into adulthood seem to be captured in a trajectory toward a broad range of maladaptation signs in adulthood and seem to be more vulnerable than those whose psychopathology emerged de novo during a later phase in their development. This fits well in the theoretical framework indicating that the more persistent a deviant pathway is followed, the more difficult it is to reclaim a normal developmental trajectory (1, 22).
We also wanted to know whether individuals who reported normal levels of functioning at outcome as measured with the Young Adult Self-Report, but who differed in the developmental trajectories before the follow-up assessment, differed in their level of adaptation. Although subjects whose psychopathology decreased over time showed slightly but significantly elevated Young Adult Self-Report total problem scores at follow-up compared with those who scored consistently normal at all four assessments, we could not detect significant differences between the two groups on any of the measures of maladaptation. It thus seems that the experience of having had high levels of problems in adolescence did not leave these individuals with greater vulnerability once their developmental pathway returned to normal in adulthood.
The overriding conclusion we can draw from this study’s findings is that when psychopathology is chronic from adolescence into adulthood, the consequences for overall adaptive functioning reach beyond the level of psychopathology, extending to a broad range of maladaptive functioning. Because our study was not intended to elucidate the underlying etiological mechanisms of the continuities and discontinuities of psychopathology, we do not know to what extent genetic or environmental factors are responsible for the chronicity of psychopathology and for its negative effects on so many domains of functioning.
Our findings emphasize the importance of obtaining the developmental history of an adult individual’s emotional and behavioral problems in the assessment of psychopathology. Moreover, we found that people with high levels of problems in early adolescence whose problems decrease during later adolescence seem to be almost as healthy in adulthood as those people who never attained a serious level of psychopathology, which is a powerful argument for early intervention to try to prevent an ongoing devious pathway into adulthood.
 |
 |
Received July 24, 2000; revisions received March 16 and July 30, 2001; accepted Aug. 15, 2001. From the Department of Child and Adolescent Psychiatry, University Hospital Rotterdam-Sophia/Erasmus University. Address reprint requests to Dr. Verhulst, Department of Child and Adolescent Psychiatry, University Hospital Rotterdam-Sophia, Dr. Molewaterplein 60, 3015 GJ Rotterdam, the Netherlands; [email protected] (e-mail). Supported by a grant from the Health Research and Development Council, Zorgonderzoek, Nederland (002827230).
Figure 1. Formation of the Subject Groups Representing Four Developmental Pathways of Psychopathology Taken From a General Population Sample Followed From Adolescence to Adulthood
aNo self-reports were available.
Figure 2. Developmental Pathways of Psychopathology as Determined by Scores on the Youth Self-Report and Young Adult Self-Reporta for Subjects From a General Population Sample Followed From Adolescence to Adulthood
aSubjects 11–18 years of age completed the Youth Self-Report; subjects 19–28 years of age completed the Young Adult Self-Report.
bSignificantly lower than score of subjects in the persistent group (p=0.001, least significant difference post hoc test).
cSignificantly lower than score of subjects in the decreasing group (p=0.003, least significant difference post hoc test).
1. Sroufe LA: Pathways to adaptation and maladaptation: psychopathology as developmental deviation, in The Emergence of a Discipline: Rochester Symposium on Developmental Psychopathology, vol 1. Edited by Cicchetti D. Hillsdale, NJ, Lawrence Erlbaum Associates, 1990, pp 13-40Google Scholar
2. Andrews G, Page AC, Neilson M: Sending your teenagers away: controlled stress decreases neurotic vulnerability. Arch Gen Psychiatry 1993; 50:585-589Crossref, Medline, Google Scholar
3. Rutter M: Pathways from childhood to adult life. J Child Psychol Psychiatry 1989; 30:23-51Crossref, Medline, Google Scholar
4. Achenbach TM, Howell CT, McConaughy SH, Stanger C: Six-year predictors of problems in a national sample, III: transition to young adult syndromes. J Am Acad Child Adolesc Psychiatry 1995; 34:658-669Crossref, Medline, Google Scholar
5. Achenbach TM: Manual for the Child Behavior Checklist/4-18 and 1991 Profile. Burlington, University of Vermont, Department of Psychiatry, 1991Google Scholar
6. Achenbach TM: Manual for the Young Adult Self-Report and Young Adult Behavior Checklist. Burlington, University of Vermont, Department of Psychiatry, 1997Google Scholar
7. Ferdinand RF, Verhulst FC: Psychopathology from adolescence into young adulthood: an 8-year follow-up study. Am J Psychiatry 1995; 152:1586-1594Link, Google Scholar
8. Achenbach TM: Manual for the Youth Self-Report and 1991 Profile. Burlington, University of Vermont, Department of Psychiatry, 1991Google Scholar
9. Verhulst FC, Van der Ende J, Koot HM: [Manual for the Youth Self-Report (YSR)]. Rotterdam, the Netherlands, Erasmus University, Department of Child and Adolescent Psychiatry, Sophia Children’s Hospital, 1997 (Dutch)Google Scholar
10. De Groot A, Koot HM, Verhulst FC: Cross-cultural generalizability of the Youth Self-Report and Teacher’s Report Form cross-informant syndromes. J Abnorm Child Psychol 1996; 24:651-664Crossref, Medline, Google Scholar
11. Ferdinand RF, Verhulst FC, Wiznitzer M: Continuity and change of self-reported problem behaviors from adolescence into young adulthood. J Am Acad Child Adolesc Psychiatry 1995; 34:680-690Crossref, Medline, Google Scholar
12. Wiznitzer M, Verhulst FC, Van den Brink W, Koeter M, van der Ende J, Giel R, Koot HM: Detecting psychopathology in young adults: the Young Adult Self Report, the General Health Questionnaire and the Symptom Checklist as screening instruments. Acta Psychiatr Scand 1992; 86:32-37Crossref, Medline, Google Scholar
13. World Health Organization: Composite International Diagnostic Interview (CIDI), version 2.1. Geneva, Switzerland, WHO, 1992Google Scholar
14. Robins LN, Cottler L, Bucholtz K, Compton W: National Institute of Mental Health Diagnostic Interview Schedule, version IV (DIS-IV). St Louis, Washington University, Department of Psychiatry, 1997Google Scholar
15. Andrews G, Peters L: The psychometric properties of the Composite International Diagnostic Interview. Soc Psychiatry Psychiatr Epidemiol 1998; 33:80-88Crossref, Medline, Google Scholar
16. Endicott J, Spitzer RL, Fleiss JL, Cohen J: The Global Assessment Scale: a procedure for measuring overall severity of psychiatric disturbance. Arch Gen Psychiatry 1976; 33:766-771Crossref, Medline, Google Scholar
17. Van der Lubbe PM: [Manual for the Groningen Questionnaire About Social Behavior]. Groningen, the Netherlands, Rijksuniversiteit Groningen, 1995 (Dutch)Google Scholar
18. De Jong A, Van der Lubbe PM: [The Development of the Groningen Questionnaire About Social Behavior]. Groningen, the Netherlands, Rijksuniversiteit Groningen, Department of Social Psychiatry, 1994 (Dutch)Google Scholar
19. Verhulst FC, Akkerhuis GW, Althaus M: Mental health in Dutch children, I: a cross-cultural comparison. Acta Psychiatr Scand Suppl 1985; 323:1-108Medline, Google Scholar
20. Verhulst FC, Berden GFMG, Sanders-Woudstra JAR: Mental health in Dutch children, II: the prevalence of psychiatric disorder and relationship between measures. Acta Psychiatr Scand Suppl 1985; 324:1-45Medline, Google Scholar
21. Van Westerlaak JH, Kropman JA, Collaris JWM: [Manual for Occupational Level]. Nijmegen, the Netherlands, Institute for Sociology, 1975 (Dutch)Google Scholar
22. Cicchetti D, Cohen DJ: Perspectives on developmental psychopathology, in Developmental Psychopathology, vol 1: Theory and Methods. Edited by Cicchetti D, Cohen DJ. New York, John Wiley & Sons, 1995, pp 3-20Google Scholar
