Cognitive impairment is a persistent and functionally relevant feature of schizophrenia. Deficits span multiple ability domains and are more closely linked to outcomes than overall clinical symptoms, making them a treatment priority (1). Cognitive remediation treatments for schizophrenia have flourished over the past decade after a long period of refinement. Early approaches, which were borrowed from restorative techniques for traumatic brain injury (2), produced equivocal results. Positive studies were criticized for generating small effects of questionable durability and modest transfer to everyday functioning (3). Substantial recent refinements in cognitive remediation have produced greater success. Two recent meta-analyses suggest that cognitive remediation produces durable improvements in cognition with moderate effect sizes (4, 5).
Although contemporary cognitive remediation strategies result in statistically significant effects, the true measure of effectiveness is improved everyday functioning. The likelihood that changes in cognition alone would result in behavioral changes is questionable, considering that schizophrenia is a neurodevelopmental condition with cognitive and functional impairments that predate illness onset. Moreover, these early difficulties likely reduce the individual's achievement of functional milestones (e.g., work and residential independence) prior to onset. Furthermore, the relationship of cognitive performance and real-world functional outcome may be indirect, mediated by functional competence (6–9). Thus, whether cognitive improvements result in improved everyday functioning likely requires supplemental approaches to support the acquisition of living skills. Skills training treatments have a long history in schizophrenia but only moderate success in terms of magnitude of effects and durability.
The meta-analyses supporting the efficacy of cognitive remediation reveal that it has greater transfer to everyday functioning when it is delivered in conjunction with other evidence-based programs, such as work therapy (10), job placement (11), and intensive social cognitive training (12). However, no published studies to date have directly compared the benefits of cognitive remediation to functional competence and real-world functioning in a randomized trial of cognitive remediation with and without supplemental skills training. In this study, we examined the effects, in individuals with schizophrenia living in community settings, of cognitive remediation on functional competence and real-world functional behaviors when the treatment was delivered alone or in conjunction with supplemental skills training; a third group received the skills training alone. We hypothesized that cognitive remediation and functional skills training alone would produce domain-specific effects (i.e., changes in cognition or functional skills, respectively). We further hypothesized that combining cognitive remediation with functional skills training would result in larger and more durable effects on both functional competence and real-world functioning.
Clinicians at outpatient centers affiliated with three sites—Mount Sinai School of Medicine, New York; Bronx VA Medical Center, New York; and Queen's University, Kingston, Ontario—referred patients who met study entry criteria. Inclusion criteria were a diagnosis of schizophrenia or schizoaffective disorder, current enrollment in outpatient psychiatric treatment, a reading level at or above grade 6, and age between 18 and 65 years. Exclusion criteria were psychiatric hospitalization of more than 24 hours within 1 month before baseline and comorbid psychiatric or neurological conditions. There were no medication exclusions. All participants provided written informed consent approved by the local ethics boards.
Cognitive performance was assessed with the Brief Assessment of Cognition in Schizophrenia (13). Six subtests are used to assess the domains of reasoning and problem solving (Tower of London), processing speed (verbal fluency, token motor task, symbol coding), verbal memory (list learning), and working memory (digit sequencing). Raw scores are transformed to z-scores (mean=0, SD=1) based on performance relative to the age- and gender-matched healthy comparison subjects and converted to a neurocognitive composite score in which all domains are given equal weight (14). The battery has been used in multiple treatment studies and has been shown to have practice effects in the range of 0.1 standard deviations per retest assessment.
Severity of clinical symptoms was examined with the Positive and Negative Syndrome Scale (PANSS) (15). After chart review and a structured interview with the patient and an informant, seven positive symptoms, seven negative symptoms, and 16 general aspects of psychopathology were rated on a 7-point scale. Interrater reliability for the PANSS for our raters ranged from 0.77 to 0.87. For the present analyses, we report the mean item score for the positive and negative symptom subscales derived from the five-factor model (16).
Social competence was examined using the Social Skills Performance Assessment (17). In this brief interactive assessment, participants engage in role playing tasks that simulate two everyday social situations: greeting a new neighbor and attempting to persuade a recalcitrant landlord to repair a leak. Sessions were audiorecorded and scored by a trained rater who was blind to diagnostic, cognitive, and treatment status. Dimensions of social competence scored include interest, affect, fluency, clarity, focus, negotiation ability, persistence, and social appropriateness. The rater was trained to the gold standard ratings proposed by the instrument developers (intraclass correlation coefficient=0.84). We report total scores for the two scenes; higher scores indicate better performance.
Functional competence refers to skills that are important for independent functioning (18). We included three measures of everyday functional skills and computed a composite score with equal weights across the three measures. The composite score had adequate internal consistency (Cronbach's alpha=0.78) and was converted to a standard score ranging from 0 to 100; higher scores indicate better performance.
In the University of California San Diego Performance-Based Skills Assessment Battery (UPSA) (19), participants are examined with a series of role playing tasks for their comprehension and planning of recreational activities, financial skills in handling money and writing checks, use of a telephone for instrumental communication, and ability to use information from bus schedules and maps to use public transportation effectively. We developed location-specific measures for the transportation/mobility and comprehension/planning items, as the initial version of the UPSA had items based on San Diego. We excluded the household chores subtest because it was not portable enough to be used in this study, since we collected data at various outpatient centers. This modified version was used in our previous studies with the UPSA (6) and demonstrated substantial correlations with neuropsychological testing performance that was consistent with the UPSA (20).
In the advanced finances test (21), participants perform higher-level financial management skills, including depositing a check, paying several bills, planning to leave money in the account, and balancing a checkbook. Raw scores range from 0 to 11.
The Medication Management Ability Assessment (22) is a role playing task similar in complexity to what someone receiving multiple medications is likely to encounter. Correct responses include taking the correct medication at the appropriate time, with or without food as prescribed, the correct number of doses per day, and the correct number of pills per dose.
+
Real-world functional behavior.
Real-world functional behavior was rated using the Specific Levels of Functioning Scale (23). This observer-rated scale indexes the patient's behavior and functioning. Ratings are made by an observer on a 5-point Likert scale for each item based on frequency of the behavior or the amount of assistance required for the subject to perform real-world skills. Informants in this study were case workers who indicated that they knew the patient at least “well” on a 5-point Likert scale. This scale has excellent interrater reliability, factorial validity, and internal consistency (23). Consistent with validating measures of real-world outcome in schizophrenia study recommendations (9, 20), we used only three of the original five domains of the Specific Levels of Functioning Scale: interpersonal relationships (e.g., initiating, accepting), activities (e.g., shopping, using the telephone, paying bills, use of leisure time, use of public transportation), and work skills (e.g., employable skills, level of supervision required to complete tasks, ability to stay on task, punctuality), and we did not collect self-reports of functioning. We report percent of total score, with higher scores representing better functioning.
Assessments for all of these domains were completed at baseline, at end of treatment (immediately after completion of the active treatments), and 12 weeks after end of treatment (durability assessment). Testing technicians completed the assessments of cognition and functional competence, and more experienced examiners conducted the symptom interviews. The patients' clinicians rated real-world functioning on the Specific Levels of Functioning Scale and were not involved in any of the other assessments. All parties were unaware of the other two sources of data as well as group assignment, study hypotheses, and study design.
Participants were randomly assigned, in a predetermined sequence, to one of three treatment conditions: 12 weeks of cognitive remediation therapy followed by 12 weeks of treatment as usual; 12 weeks of functional adaptation skills training (24) followed by 12 weeks of treatment as usual; or 12 weeks of cognitive remediation followed by 12 weeks of functional skills training. Participants received $20 per evaluation, and they received transportation funds but not compensation for attending treatment sessions. See Figure 1 for a description of the treatments. Treatment as usual consisted of a mean of 1.6 (SD=1.1) visits per month for case management services. These services did not include cognitive-enhancing or functional-adaptive strategies, and they did not differ significantly by group in number of visits.
Cognitive remediation was based on a previously validated treatment, the Thinking Skills for Work Program (25). It used computer-based exercises (Cogpack, Version 5.1 [Marker Software, Ladenburg, Germany], PSSCogRehab [Psychological Software Services, Inc., Indianapolis], and Scientific Brain Training PRO [HAPPYneuron, Inc., Mountain View, Calif.]) and included a large role for therapist involvement in stimulating the forming and testing of alternative strategies and discussing how the cognitive skills are used in everyday life. Bridging strategies to help transfer cognitive skills to everyday activities were informed by the Thinking Skills for Work Program (25) and the Neuropsychological Educational Approach to Remediation programs (26). Functional adaptation skills training (24) is a standardized psychosocial intervention developed for outpatients with schizophrenia that uses props and role playing to increase competence in social skills and various aspects of independent living. Systematic therapist training was provided by the developers of the treatments (all of them investigators in this study: S.R.M., B.M., T.L.P.), using treatment manuals, didactic instruction, and role playing. These investigators remained available for periodic teleconferences for follow-up training and to address specific issues as needed. The first author was responsible for ensuring completion of training, adherence to protocol, and review of session notes on a weekly basis. Two therapists—doctoral-level clinical psychologists or doctoral clinical psychology students—were used at each site.
Univariate analyses of variance were used to examine differences between groups on demographic and illness variables at baseline. Mixed models for repeated measures were used to analyze changes in the dependent variables by treatment group, including all subjects in an intent-to-treat analysis. A diagonal covariance structure was used, with treatment site and treatment dose (number of hours of treatment actually received) as random effects with intercept and slope modeled and variance components selected as the covariance type. The restricted maximum likelihood method was selected for estimation. Pairwise comparisons for main effects were made with the Bonferroni adjustment. Interaction effects were examined with a Bonferroni adjustment for the six primary analyses set to an alpha of 0.008 (0.05/6). Bonferroni-corrected significant interactions were followed by pairwise comparisons. We calculated effect sizes for the change scores in two ways. Cohen's d (27) was calculated for all treatment completers for the baseline assessment to the end-of-treatment assessment and the baseline assessment to the 12-week durability assessment. We also calculated the number-needed-to-treat effect size using the formula described by Furukawa and Leucht (28). In these analyses, the number needed to treat refers to the number of patients who would need to receive the combined treatment in order to show improvement at the various levels of change (10% through 60%) compared with a patient who received only the functional adaptation skills training.
Participants (cognitive remediation, N=36; functional skills training, N=35; combined treatment, N=36) who completed baseline assessment were considered in the intent-to-treat analyses (Figure 2). The proportion of the intended treatment dose (120 minutes per week) that was completed did not differ significantly between the cognitive remediation (mean=0.87, SD=0.13), functional skills training (mean=0.94, SD=0.09), and combined treatment (mean=0.93, SD=0.16) groups.
There were no significant differences between groups at baseline on demographic or clinical variables (Table 1) or on antipsychotic medication type at baseline (Table 2).
+
Intent-to-Treat Analysis
Symptom changes were not part of the primary analyses; exploratory analyses revealed no significant main or interaction effects on negative or positive symptoms. Estimated marginal means derived from linear mixed-model analyses with the intent-to-treat sample are presented in Table 3. Improvement in fit was identified for each of the subsequent models when random effects were entered. Degrees of freedom using linear mixed models produce noninteger variables because they are estimates based on nonexact F distributions and thus they do not reflect sample sizes. We rounded to integers in the results below for clarity. The group-by-time interaction for neurocognition was significant (F=28.9, df=4, 72, p<0.001). The cognitive remediation and combined treatment groups improved significantly from baseline to end of treatment, and these effects were maintained at the 12-week durability assessment. The functional skills training group did not significantly improve at either time point. Both the cognitive remediation (F=42.7, df=2, 48, p<0.001) and combined (F=44.8, df=2, 51, p<0.001) treatment groups showed greater improvement in the neurocognitive composite score compared with the functional skills training group, but they did not differ from each other.
The group-by-time interaction for social competence was significant (F=9.1, df=4, 85, p<0.001). The functional skills training and combined treatment groups improved from baseline to end of treatment, and both groups maintained these treatment effects at the 12-week durability assessment. The cognitive remediation group did not significantly improve on social competence. Both the functional skills training (F=3.7, df=2, 53, p=0.03) and combined treatment (F=12.1, df=2, 53, p<0.001) groups had greater improvement in social competence compared with the cognitive remediation group, and their treatment effects did not differ significantly. The group-by-time interaction for functional competence was significant (F=19.5, df=4, 68, <0.001). The combined treatment group improved to a greater extent than the cognitive remediation group (F=40.7, df=2, 43, p<0.001) and the functional skills training group (F=19.7, df=2, 46, p<0.001). The functional skills training group demonstrated significant improvement from baseline to end of treatment, but these effects were not statistically significant at the 12-week durability assessment relative to baseline.
Main effects for time but not group were significant for social functioning. The group-by-time interaction was not significant. The group-by-time interaction for real-world community activities was significant (F=3.9, df=4, 70, p=0.007). For the combined treatment group, improvement was significantly greater than the functional skills training group (F=9.1, df=2, 50, p<0.001); the combined treatment group also had greater improvement than the cognitive remediation group, but this difference fell short of significance (F=3.2, df=2, 44, p=0.051). The functional skills training and cognitive remediation groups did not differ statistically. The group-by-time interaction for real-world work skills was significant (F=12.8, df=4, 72, p<0.001). The functional skills training and combined treatment groups improved from baseline to end of treatment, and both groups maintained these improvements at the 12-week durability assessment. However, the combined treatment group had significantly larger treatment effects than the functional skills training group (F=3.5, df=2, 43, p=0.037) and the cognitive remediation group (F=32.5, df=2, 37, p<0.001).
Number-needed-to-treat analyses were conducted with the functioning variables for which we observed a significantly greater improvement in the combined treatment group compared with the functional skills training group. For functional competence, the number needed to treat with both treatments compared with functional skills training alone was 10.2 for a treatment response of 10%; 3.3 for a treatment response of 20%; 3.0 for a treatment response of 30%; 7.2 for a treatment response of 40%; 9.0 for a treatment response of 50%; and 18.0 for a treatment response of 60%. For clinician-rated real-world work skills, the number needed to treat was 5.4 for a treatment response of 10%; 3.3 for a treatment response of 20%; 9.4 for a treatment response of 30%; 19.6 for a treatment response of 40%; and 42.0 for a treatment response of 50%. For clinician-rated real-world community activities, the number needed to treat was 18.5 for a treatment response of 10%; 18.0 for a treatment response of 20%; and 36.0 for a treatment response of 30%.
This study is, to our knowledge, the first to directly compare the independent and additive effects of cognitive remediation therapy and a psychosocial intervention designed to improve real-world behavior in schizophrenia. The major findings are that cognitive remediation produces robust improvements in cognition, has limited transfer to everyday behavior as a standalone treatment, and increases the effects and durability of treatments that directly target functional disability.
The major implication of these findings is that cognitive remediation can potentiate other psychological treatments, making them more substantial and more durable. The additive effects were pronounced in the performance-based measure of adaptive skills, in real-world community or household responsibilities, and in real-world work skills. All groups showed improvement in the real-world ratings of social behavior, suggesting an effect neither specific to treatment type nor facilitated by their combination. Similarly, preceding functional skills training with cognitive remediation did not improve the effects of the former treatment on social skills or behavior. Previous studies have found cognition to have smaller relationships with social, compared to adaptive, skills (7). Social skills are also influenced more by individual characteristics such as social cognition, negative symptoms, and disordered speech than cognition or social competence (7, 29), as well as by external factors such as living circumstances and financial strain (30, 31).
Cognitive remediation without a supplemental skills intervention produced less robust effects on measures of functional competence and on clinician ratings of everyday functional behaviors. This restricted transfer of skills was observed in spite of robust improvements in neurocognition. The relationship between cognition and everyday functioning is well replicated, and thus it is important to consider why a treatment that improves these deficits does not manifest in measures of functioning. Skill acquisition is disrupted early in the illness process, when cognitive impairments are present but psychosis has not emerged. Limited opportunities to learn and utilize skills (30) might explain why cognitive improvements need to be supplemented by habilitative functional opportunities. In contrast to this idea, data from one study suggested that performance on measures of competence were minimally related to differences in environmental enrichment, while real-world outcomes (independence in residence) were notably different across environments (8). Another study, following 1 year of similarly nonsocial cognitive remediation, found significant improvements in UPSA scores (32). However, a study of 15 weeks of nonsocial cognitive remediation treatment failed to find transfer of cognitive gains to UPSA results (33). The ideal dose or duration of cognitive remediation for transfer to functional competence is thus still in question.
Transfer of cognitive gains to everyday behaviors will likely require time and opportunities to practice new skill sets and for others to adapt to these changes. In cognitive remediation studies with longer follow-up assessment intervals and a psychosocial treatment component (11, 12), transfer of behavior to real-world indices such as work outcomes has been greater than that observed in this study. Other intrinsic and extrinsic factors are likely to limit the extent to which cognitive abilities transfer to everyday performance. Mood symptoms (6), defeatist beliefs (34), social cognition (35), and reduced motivation (36, 37) have been identified as factors more closely associated with behavioral performance than competence and should also be considered as treatment targets or potential mediators.
There are potential clinical and health care policy implications to these findings. In a mental disorder characterized by severe and persistent impairments in everyday functioning, we were able to see meaningful changes in patients' ability to acquire community living skills and use those skills when cognitive remediation was combined with functional skills training. The extra cost of combining these interventions must be considered in the context of the impressive number-needed-to-treat statistics—as few as three patients for the combined treatments to improve functioning by 20% to 30%, an effect size that is relative to an active treatment comparison group. These results may be even more impressive when one considers the short-term nature and financial and resource burden of the treatments relative to pharmacologic treatments.
Our study had several limitations. The duration and intensity of cognitive remediation have been variable across previous studies; treatment duration in this study was short. However, a weekly 1-hour treatment for 3 months might reflect clinical reality in many health care models, and we nevertheless observed a robust effect on cognition. Functional adaptation skills training was modified from its original form, which may have changed its effectiveness as a standalone intervention. Milestone changes in everyday behavior, such as work status, romantic relationships, and independence in living, were not examined; however, these measures might not be expected to undergo change in short-term trials. Our sample consisted primarily of male patients who were well connected with outpatient treatment centers and had low symptom severity, so we do not know whether our findings generalize to samples with other characteristics. However, the degree of cognitive impairment in our sample is consistent with that in other studies. Finally, the combined treatment group received more total treatment sessions than the other two groups. Thus, the effects, despite statistical control for time-related dosing, cannot be conclusively proven not to derive from more contact. A design with inactive cognitive remediation and inactive functional skills training-like treatment would be required to address this point and would be particularly valuable for informing time- and resource-limited clinics about the necessary type and amount of treatment needed to reduce functional disability, although the number-needed-to-treat statistics in this study provide some insight into these issues.
The authors thank Hannah Anderson, Winnie Leung, Amelia Bowman, Shannon Xavier, Stephanie Taillefer, Emma Bassett, Jeremy Stewart, Allisha Patterson, Talia Troister, and Sylvia Magrys for assistance with data collection and treatment delivery.