Determinants of Real-World Functional Performance in Schizophrenia Subjects: Correlations With Cognition, Functional Capacity, and Symptoms

Despite 50 years of pharmacological and psychosocial intervention, schizophrenia remains one of the top causes of disability in the world (1). Schizophrenia is associated with lifelong impairment across a number of domains (2). This is true especially among young (3) and middle-aged (4) adults and even among patients who respond to antipsychotic treatment and have only residual psychotic symptoms (5). These functional impairments involve social, occupational, and independent living activities, and for the first 30 years of the antipsychotic treatment era of the 1950s through the 1980s, functional outcomes remained essentially unchanged (6). Schizophrenia outpatients evidence widespread functional impairments regardless of level of residential independence (7). It has been argued that one of the primary reasons for the historical lack of improvement in functional outcome is a general lack of success in treating the aspects of schizophrenia that have the strongest associations with functional outcome, such as cognitive impairment and negative symptoms (8).

On the basis of findings from other neuropsychiatric conditions (9), recent studies have examined the relationship between neuropsychological performance and functional status. Meta-analytic reviews of these early studies have clearly demonstrated a cross-sectional correlational relationship between neuropsychological impairment and several domains of functional outcome (10, 11); these findings have been replicated in longitudinal studies as well (12). The discovery of these consistent relationships has led to the hypothesis that cognitive-enhancing treatments (13) may result in improved functional outcomes. However, a potential limitation of this hypothesis is the fact that correlations between individual cognitive ability areas and functional outcomes are generally moderate, and even composite measures of neuropsychological performance rarely account for more than 25%–50% of the variance in real-world functional outcomes (14).

With the recognition that cognitive and functional domains are potential treatment targets, valid assessment of these ability areas is a critical step. Further, the entire breadth of the relationship between neuropsychological performance and functional outcome requires study in that functional skills can be conceptualized in terms of capacity (performance under optimal conditions—what the person can do) versus performance (real-world outcomes—what the person does do). In contrast to functional capacity, functional performance in everyday life may be influenced not only by cognitive and other abilities but also by a variety of motivational and environmental factors (e.g., opportunity and social support for performing activities such as financial and medication management, seeking employment). Assessment strategies have been developed to address all of these domains. The NIMH MATRICS initiative (15–18) was created to assist in the refinement of neuropsychological and functional assessment measures for studies of pharmacological enhancement of cognition. This initiative led to a “state of the art” cognitive assessment for use in treatment studies of cognition in schizophrenia. Important issues that remain include how functional capacity and functional performance should be conceptualized and measured and how competence and performance domains are related to neuropsychological performance.

Assessments of functional outcomes can be derived from reports by the patient or caregiver/informants or direct observation of the patient, while performance-based measures of skills competence are collected in analogue settings. Examples of the latter include performance of social, personal care, and financial management in laboratory settings. Patients with schizophrenia have cognitive impairments that may invalidate their self-reporting, and many have no caregivers to provide information. Both types of direct assessment (direct observation versus analogue assessment) have advantages and limitations. Real-world observations are necessarily individualized and nonstandardized as well as costly and potentially reactive (presence of an observer may alter the environment and resulting behaviors). Performance-based measures of functional capacity are valid to the extent that they measure the relevant skills accurately, but other factors may influence real-world outcomes. Again, factors such as financial resources, motivation, and possibly other symptoms of the illness may limit the extent to which skills that are present in the behavioral repertoire are actually performed in real-world settings. Cognitive deficits are associated with impairments in laboratory-based functional skills (19, 20), and previous data have indicated that performance-based measures of functional skills are related to several aspects of real-world outcome in patients with schizophrenia (20). It is interesting that in one recent study (21), two patient self-report measures of outcome, a subjective quality-of-life scale and a disability assessment, were well correlated with each other but essentially uncorrelated with either neuropsychological performance or performance-based measures of functional skills, both of which were also well correlated with each other. These data suggest that multiple measures may be required to capture the complexity of functional outcome in schizophrenia.

This article reports the results of a study of the relationship between neuropsychological performance, functional capacity, and real-world functional skills performance in outpatients with schizophrenia. The patterns of direct and indirect influence between neuropsychological performance, functional capacity, and various domains of real-world adaptive skills performance were examined with confirmatory path analyses. In these models, several other variables that could influence these relationships were also examined, including the current severity of positive and negative symptoms and symptoms of depression. The contributions of these other variables were examined in terms of whether they primarily influenced the “competence domain,” that is neuropsychological performance and measures of functional capacity, or whether the influence was “postcompetence” in nature, affecting real-world outcomes but not functional capacity or neuropsychological performance. This distinction is important, since previous research with HIV-infected patients (22) has indicated that this relationship is complex, with neuropsychological performance affecting functional capacity and real-world performance, whereas depression affects real-world performance but is not related to functional capacity or neuropsychological performance. Thus, our analyses were aimed at determining if 1) neuropsychological performance would be associated with real-world outcome other than through its relationship with functional capacity, and 2) whether other factors such as depression or negative symptoms would affect outcome and, if so, were these influences at the level of affecting functional capacity and neuropsychological performance or real-world performance.

Method

Subjects

Subjects in this study were older, ambulatory schizophrenia patients enrolled in a longitudinal study of the course of cognitive and functional status. The long-term goal of this project was to examine the course and correlates of change in functional status among older, ambulatory schizophrenia patients. Exclusion criteria for this study were a primary DSM-IV axis I diagnosis other than schizophrenia or schizoaffective disorder, Mini-Mental Status Examination (MMSE) score below 18, or any medical illnesses that might interfere with the assessment of cognitive functioning. All subjects were receiving outpatient treatment at the time of recruitment at a Department of Veterans Affairs, New York State, or academic research site. Outpatient status was defined as living outside of any institutional setting, including a nursing home. Study subjects were also required to have evidence of continued illness at the time of recruitment, as evidenced by meeting at least one of three criteria: 1) an inpatient admission for psychosis in the past 2 years; 2) an emergency room visit for psychosis in the past 2 years; or 3) a score of 4 (moderate) or more at the baseline assessment on the positive symptom items delusions, hallucinations, or conceptual disorganization. All subjects signed a written informed consent form approved by the institutional review board at each research site after the testing procedures were fully explained.

All subjects met DSM-IV criteria for schizophrenia or schizoaffective disorder. The Comprehensive Assessment of Symptoms and History (23) was completed by a trained research assistant, and diagnosis was confirmed by a senior clinician. Trained raters at each site performed all testing. Patients were only enrolled in this part of the study if they were receiving case management services and were actively involved in psychiatric rehabilitation services. The case managers were used as the informants for the real-world functional status ratings, leading to the exclusion of 64 patients who completed other aspects of this assessment. All patients were receiving treatment with second-generation antipsychotic medications.

Measures

All subjects completed the test battery in a fixed order. Screening measures were the MMSE (24) to assess global cognition and the recognition reading subtest of the Wide-Range Achievement Test 3 (25) to estimate premorbid functioning. These measures were followed by functional skills assessment, a cognitive test battery, and a symptom interview. Comprehensive reviews of chart notes and interviews with case managers were performed to provide data for rating scales. All raters received extensive training in performing all of the assessments, and every 3 months their performance was reevaluated through rerating of training tapes, dual ratings of the functional status measures with the first author, and quality assurance assessments of all testing. These raters were trained to adequate reliability on symptom ratings with 2 full days of training, four standardized videotapes, and in-person interviews that yielded intraclass correlation coefficients from 0.857 to 0.917.

Performance-Based Measure of Functional Capacity

The UCSD Performance-Based Skills Assessment (19) is designed to directly assess functional skills competence among the severely mentally ill. This test is designed for older outpatients and measures performance in a number of domains of everyday functioning through the use of props and standardized skills performance situations. The comprehension/planning domain measures the patient’s ability to comprehend written material describing recreational outings and then plan the activities and list appropriate items necessary to bring to the outings. In the financial skills domain, the patient must count out given amounts from real currency, make change, and fill out a check to pay a utility bill. The communication domain involves a series of role play situations that require the patient to make emergency calls, call directory assistance to request a telephone number, call the number, and then reschedule a medical appointment. In the transportation/mobility domain, patients use information from bus schedules and maps to determine appropriate fare, state telephone numbers to answer relevant questions, decide which map to use to get to a certain location, and determine the appropriate route and transfers to reach a destination. For the purposes of these analyses, the UCSD Performance-Based Skills Assessment total score was used as the dependent variable to reduce the number of total correlations.

Cognitive Assessment

Patients completed a comprehensive assessment of the cognitive ability areas previously shown to be most consistently correlated with functional skills (10, 11, 26), including attention, motor skills, verbal learning and memory, verbal fluency, and several aspects of executive functioning. For the analyses in the present study, a composite cognitive score was calculated by standardizing scores from the following measures into z scores: animal naming, perseverative errors and categories attained on the Wisconsin Card Sorting Test (64-card computerized version [27]), Trail-Making Test (28) parts A and B, learning trials 1–5 and short-delay free recall from the Rey Auditory Verbal Learning Test (29), the Consortium to Establish a Registry for Alzheimer’s Disease versions of the Boston Naming Test and constructional praxis assessment (30), WAIS-III digit span and letter-number sequencing subtests (31), and the Stroop Color and Word Test interference condition (32).

Symptom Assessment

Severity of schizophrenia symptoms was assessed with the Positive and Negative Syndrome Scale (PANSS) (33). This is a 30-item scale with seven items measuring positive symptoms, seven items measuring negative symptoms, and 16 items measuring general aspects of psychopathology. The PANSS was completed after a structured interview. We used the positive and negative domains from the empirically derived five-factor model of the PANSS that was developed by White et al. (34). We obtained self-reports of depression from the patients by using the second edition of the Beck Depression Inventory (35).

Real-World Functional Performance

In order to examine real-world functional performance, the Specific Level of Function Scale (36) was employed. This scale is a 43-item caretaker report of a patient’s behavior and functioning across the following domains: physical functioning (e.g., vision, hearing), personal care skills (e.g., eating, grooming), interpersonal skills (e.g., initiating, accepting, and maintaining social contacts; effectively communicating), social acceptability (e.g., absence of verbal and physical abuse, absence of repetitive behaviors), community activities (e.g., shopping, using telephone, paying bills, use of leisure time, use of public transportation), and work skills (e.g., employable skills, level of supervision, punctuality). This scale is rated on the basis of 1) the amount of assistance that the patient requires to perform real-world skills (personal care, community activities), 2) the effect on functioning (physical functioning), or 3) the frequency of the behavior (interpersonal skills, social acceptability, work skills). In order to obtain these ratings, a corroborated informant report method was used. With the patient’s permission, a caseworker for the patient was directly interviewed in order to obtain information on real-world performance. All informants indicated that they knew the patient “very well” on the Specific Level of Function Scale’s 5-point Likert scale. The scale has excellent interrater reliability, factorial validity, and internal consistency (36).

Statistical Analyses

Confirmatory path analyses were conducted to determine which of the variables predicted or mediated the relationship with the functional outcome domains from the Specific Level of Function Scale. The goodness of fit of these models was tested statistically and compared with several comparison models. This procedure has the advantage of identifying the best-fitting model that optimally relates to the outcome variables, in this case the Specific Level of Function Scale domains. In contrast to a nonconfirmatory evaluation in which the significance level of correlations is used as the index of whether or not they are important, this procedure determines whether elimination of certain paths in the model either improves, or at least does not degrade, the fit of the overall model. Such a procedure enhances parsimony in that statistically significant but uninformative paths are eliminated while identifying the set of correlations that explains optimum variance in the preselected outcome variable. In these analyses, we used the Specific Level of Function Scale domains as the outcome variables, since real-world outcome is a putative “gold standard” and the informants making the ratings would not be influenced by neuropsychological performance or functional capacity, of which they were unaware. We examined score on the UCSD Performance-Based Skills Assessment as a mediating variable, with the hypothesis that functional capacity would influence the effect of neuropsychological status on real-world outcomes.

These models included an independent model and an optimized (final) model. The independent model posited that all variables were uncorrelated with each other. The final model was developed through an iterative procedure in which nonsignificant paths were eliminated from a saturated model (in which all variables are correlated, providing no degrees of freedom, which precludes comparisons to other models) until the best-fitting model, defined by several different goodness-of-fit statistics, was identified. Goodness-of-fit statistics are reported for the final model and the independent comparison models. Statistical tests included absolute and relative goodness-of-fit statistics. In addition, chi-square change scores and incremental fit indices are presented as the difference from the independent models to the final model. All analyses were performed by using SPSS, version 13 (37) and its affiliated software, AMOS.

Results

Seventy-eight patients completed all of the baseline assessments without any missing data. Demographic, clinical, performance, and functioning data are presented in Table 1. Bivariate Pearson correlations are presented in Table 2. Parametric comparisons of fit for the independent and final path models are shown in Table 3. Path coefficients are presented in terms of standardized regression coefficients originating from the final models.

The physical functioning, personal care skills, and social acceptability domains of the Specific Level of Function Scale had little variance and were close to ceiling because of the ambulatory, generally stable nature of the patients. These domains were not examined further. The interpersonal skills domain was correlated on a zero-order basis with neuropsychological performance, negative symptoms, depressive symptoms, and functional capacity. The outcome domain of community activities was correlated with neuropsychological performance and functional capacity. Finally, work skills were correlated with neuropsychological scores, functional capacity, and depressive symptoms.

For the Specific Level of Function Scale outcome domains of interpersonal skills, community activities, and work skills, in all cases the independent models were very poor fits to the data, with large and significant chi-square statistics and goodness-of-fit indices close to 0. For each of these, a final model that fit the data was extracted from the saturated model.

For interpersonal skills, the best-fitting path model is shown in Figure 1. This model represents a significant improvement over the independent model (χ² change=82.08, df=6, p=0.001). In this model, neuropsychological performance had no significant direct effect on interpersonal skills but was mediated by the effects of functional capacity. Further, severity of the negative symptoms and depressive symptoms had significant and negative effects on interpersonal skills that were independent of functional capacity.

The final model predicting community activities is shown in Figure 2. This model represents a significant improvement over the independent model (χ² change=93.8, df=6, p=0.001). Neuropsychological scores had a direct influence on community activities as well as an effect mediated by functional capacity. Negative symptoms did not contribute to the prediction of community activities. While depression was not a significant independent predictor of activities, it was retained in the model because of its overall impact on goodness of fit.

In predicting work skills, the best-fitting path model is shown in Figure 3, and represents a significant improvement over the independent model (χ² change=86.6, df=10, p=0.001). Neuropsychological scores did not independently predict work skills but were mediated by functional capacity. Depression had an independent effect on real-world occupational functioning, whereas negative symptoms did not enter the final model.

Discussion

The purpose of the present study was to examine the complex relationships of neuropsychological performance, symptom severity, and functional capacity in predicting real-world functional performance of schizophrenia patients. The results support previous work that found neuropsychological performance to be highly correlated with functional outcome. However, when considering functional capacity, as measured by performance in an analogue setting, neuropsychological scores only occasionally provide further predictive power in determining the level of performance in a number of real-world outcome domains. In most domains, the relationship between neuropsychological performance and real-world functional performance was mediated by the subject’s functional capacity, which is in large part correlated with neuropsychological performance. Moreover, while functional capacity is the most consistent predictor of functional performance, additional variance is accounted for by symptoms such as affective and negative symptoms, which do not appear to have an impact at the “competence” level. Thus, the patients’ deployment of functional skills appears to be affected by other apparent limiting factors such as negative and affective symptoms.

This study has several limitations, many shared with other similar studies. The ratings of functional capacity came from a psychometrically sound measure (19) that—like neuropsychological tests—was performance-based and required cognitive skills, which raises the potential for measurement artifact in predicting performance. It is impossible to determine if the current real-world functional performance rating measure is the optimal one, and it is equally impossible to determine if the caregiver ratings contain some bias. Unfortunately, the most reliably rated functional outcome milestones (marriage, independent living, and full-time competitive employment) are also rarely attained in patients with schizophrenia. Also, the study group size was not extremely large, and while we were able to identify a good-fitting model for each real-world outcome domain, which improved on various comparison models, some of the nonsignificant predictors might be influenced by the study group size. Obviating this concern is the fact that this group of subjects was quite well suited for these analyses in that they were community dwelling and involved in rehabilitation programs that aim to enhance their functional skills. As a result, they did not constitute a group of patients who were either not motivated to engage in functional activities or were in some way restricted from these activities (e.g., being in an institution). All patients were clinically stable and ambulatory, so the results might be different for refractory populations or inpatients, and we do not know how representative their performance scores would be relative to younger patients. Perhaps most importantly, this study did not attempt to measure all potential factors that could interfere with real-world functional outcomes. As noted, different systems of societal, environmental, personal, and familial incentives and disincentives may influence the deployment of skills in the repertoire when in community settings.

It is not surprising that the patients in this study, who were relatively treatment responsive with a less severe course of illness than some with schizophrenia, had intact physical functioning and personal care skills as well as minimal evidence of disruptive behavior, making a search for predictors of these daily living skills and characteristics of illness difficult due to the truncated range. While the patients had mild to moderate levels of symptom severity, these scores were not too low to preclude prediction of the various outcome domains, and negative symptoms and depression symptoms were found to be related to some other aspects of functioning in this patient group. Despite this relatively mild symptom profiles, the patients manifested deficits in a number of outcome domains that measured more complex instrumental functional skills.

In these analyses, we found that negative symptoms interfered with interpersonal relationships independent of neuropsychological ability and competence to perform specific everyday tasks. Depression further appeared to limit interpersonal functioning independently of other competence domains. Thus, according to the results of these analyses, a patient with neuropsychological deficits is likely to have impairments in measures of functional capacity, which in turn are associated with impairments in the ability to form and maintain relationships in the community. Regardless of functional capacity, severity of negative and depressive symptoms appears related to schizophrenia patients’ interpersonal functioning in community settings. A person’s performance of community activities (e.g., handling finances, using transportation, and engagement in recreational activities) is strongly predicted by their functional capacity in the laboratory. Neuropsychological functioning contributed to performance of community activities both independently and as mediated by functional capacity. Severity of depressive symptoms tended to be associated with further impairment of community activities, which while nonsignificant still contributed to the overall fit of the model. Neuropsychological scores did not contribute to prediction of work skills after considering the mediating effects of functional capacity, while depression had an independent effect on work skills.

It is possible that other factors not included in this study (e.g., motivation and reward; facilitation from caretakers, friends, or family; socioeconomic status) further predict involvement and success in community activities. As in other domains, the prediction of work skills outcome domain is likely to be further influenced by factors not measured in this study, such as access to employment opportunities. Employability might also be influenced by external factors such as stigma, financial need, and access to job training and coaching. These results suggest, however, that treatment-related changes in neuropsychological performance or in functional capacity should not be expected to be reflected on a one-to-one basis by changes in real-world outcomes. Such outcomes appear multiply determined and might require additional time to respond above and beyond that associated with changes in more performance-based measures. Thus, lack of changes in real-world outcomes would not necessarily impugn the usefulness of treatments aimed at neuropsychological performance or functional capacity. Cognitive enhancement may be important in improving outcomes, but even highly effective interventions may not lead to deployment of skills in the real world. Further work will be needed to determine how postcompetence and independent inhibiters such as negative and affective symptoms can best be addressed in improving functional outcome. Additional functional outcome assessment measures, such as direct observation, may provide more detailed information regarding specific functional domains and further elucidate relationships. An advantage of the present study is that ratings of real-world performance with the Specific Level of Function Scale were completed by a third-party rater who was completely blind to the patient’s performance on all other tests and ratings. More precise definition and measurement of functional outcomes should be a goal for the field as we move closer toward improving outcomes.

TABLE 1

Enlarge table

TABLE 2

Enlarge table

TABLE 3

Enlarge table