Generalized Cognitive Impairments, Ability to Perform Everyday Tasks, and Level of Independence in Community Living Situations of Older Patients With Psychosis

Cognitive deficits have been related to impaired everyday functioning in patients with schizophrenia and have been shown to explain more variance in daily functioning than have psychiatric symptoms (1–3). A review of studies of neurocognitive deficits and functional outcome in schizophrenia concluded that vigilance, working memory, verbal memory, and executive functioning were strongly correlated with functional outcomes, including community functioning, social skills, and skill acquisition (4). However, some studies included in the review assessed everyday functioning with self-report or collateral-report measures that may have suboptimal validity.

Objective, performance-based assessment of everyday functioning involves standardized evaluation during role playing of tasks and situations encountered in “real-world” community living. Performance-based testing of everyday functioning combines reliability and validity with practicality and has considerable advantages compared with other assessment modalities. For example, observation in real-life settings can provide valuable information about patient functioning, but it is time-consuming and subject to uncontrolled environmental/situational effects. Self-report and collateral-report measures of patient functioning may be subject to bias because of social desirability and respondents’ lack of insight, as well as situational influences; clinicians’ ratings may be based on such reports or on very limited behavioral observations. Our group previously reported on the development and validation of the University of California, San Diego, Performance-Based Skills Assessment (UPSA) (5), a measure of functional capacity of psychiatric patients. The UPSA was adapted from the Direct Assessment of Functional Status (6), a performance-based assessment of functioning that had been used with Alzheimer’s disease patients. The adaptation was necessary because some subtests of the Direct Assessment of Functional Status (e.g., time orientation and eating) did not differentiate schizophrenia patients from normal subjects and because tests of higher-order domains of functioning were needed for psychiatric patients. In the UPSA normative study of 50 patients, performance on the UPSA was positively associated with performance on the Mini-Mental State Examination (MMSE) (7), replicating a pattern of association between the MMSE and the Direct Assessment of Functional Status (8). In addition, worse performance on the UPSA has been correlated with increasing severity of negative symptoms (5).

The current study was designed to examine further 1) how specific cognitive abilities, measured with neuropsychological tests that provide more sensitive and detailed information than the MMSE, are associated with performance on the UPSA; 2) how cognitive abilities and clinical factors predict UPSA performance; and 3) how the UPSA reflects everyday functioning in the patient’s usual living situation. Furthermore, the study included a larger (N=111) and diagnostically more heterogeneous group of subjects than the initial UPSA normative study. Study participants were middle-aged and elderly outpatients with schizophrenia, schizoaffective disorder, mood disorder with psychosis, or psychosis not otherwise specified. We hypothesized that cognitive performance (particularly in the domains of attention, learning, memory, and executive functioning, as suggested by an earlier literature review [5]) would be associated with UPSA scores and that UPSA scores would in turn be associated with patients’ level of independence in their community living situations.

Method

Participants

The 111 study participants (N=58 men, 52%) ranged in age from 41 to 87 years (mean=54.6 years, SD=9.6) and had DSM-IV diagnoses of schizophrenia (N=67, 60%), schizoaffective disorder (N=29, 26%), mood disorder with psychotic features (N=14, 13%), or psychosis not otherwise specified (N=1, 1%). Diagnoses were confirmed with the Structured Clinical Interview for DSM-III-R or DSM-IV (9, 10). Most participants were Caucasian (N=83, 75%) and had completed high school (mean=12.5 years of education). The mean duration of psychotic illness was 27 years, and 91% of the patients (N=100) were taking antipsychotic medication at the time of the study. Exclusion criteria were physical or mental illness severe enough to require current hospitalization or to preclude completion of the assessments, history of head injury with >30 minute loss of consciousness, history of seizure disorder, current diagnosis of dementia, and current diagnosis of substance abuse or dependence. The participants were psychiatrically, medically, and pharmacologically stable. Thirty-nine of the 111 patients had also participated in the UPSA normative study (5), and 23 patients had also participated in a study examining use of the Direct Assessment of Functional Status (11). Characteristics of the participants are summarized in Table 1.

Procedure

The study subjects were participants in the University of California, San Diego, Intervention Research Center for Psychosis in Older People. They were recruited from San Diego County Adult Mental Health Services; the University of California, San Diego, Medical Center; the Department of Veterans Affairs San Diego Healthcare System; and the San Diego community. Recruitment procedures and clinical assessments have been described previously (12). Many subjects in the current study contributed functional and cognitive data to previous reports from the research center (e.g., reference 13). This study, however, is the center’s first investigation of the relation between UPSA performance and specific dimensions of cognitive functioning.

All cognitive measures were administered by trained examiners within a few months of the administration of the UPSA (median number of months between assessments=1.9). We previously reported that cognitive performance in the center’s patient population was quite stable over several years (14), so we believe that comparison of cognitive and noncognitive measures administered within a few months of each other was appropriate. Other study assessments were completed at the time the UPSA was administered.

The examiners who administered the assessments used in this study were trained to a high level of interrater reliability on all measures (intraclass correlation coefficient [ICC] >0.80 [15]). Interrater reliability on the UPSA was high (ICC=0.91 [5]).

After a complete description of the study to the subjects, written informed consent was obtained. The University of California, San Diego, Institutional Review Board approved this study.

Measures

The UPSA was designed to evaluate the abilities of individuals to perform everyday tasks that are considered necessary for independent functioning in the community. The UPSA uses role-playing situations to evaluate skills in five areas: household chores (e.g., preparing a shopping list for a specific cooking task and shopping for items in a mock grocery store), communication (e.g., emergency and nonemergency telephone use), finance (e.g., counting change and paying a bill by personal check), transportation (e.g., planning the use of a public bus system), and planning recreational activities (e.g., preparing for an outing to the beach or zoo). Subscale scores range from 0 to 20 points, and total scores range from 0 to 100 points; higher scores reflect better performance. Administration of the UPSA requires about 30 minutes. In a group of 49 patients with schizophrenia or schizoaffective disorder, 1-week test-retest reliability was 0.94 (5).

The Mattis Dementia Rating Scale (16) is a 20-minute screening assessment for dementia that includes subscales measuring attention, construction (e.g., figure drawing/copying), initiation/perseveration (e.g., verbal fluency), conceptualization (e.g., verbal abstraction), and memory. Possible scores range from 0 to 144 points, with higher scores reflecting better performance.

A comprehensive neuropsychological test battery (14) was administered to a subgroup of participants (N=67). Mean scaled scores (in which a mean of 10 and a standard deviation of 3 apply to normal adults, and higher scaled scores reflect better performance) were computed in the domains of verbal ability, attention/working memory, psychomotor ability, motor ability, learning, memory, and abstraction/cognitive flexibility (i.e., executive functioning). Scaled scores without demographic corrections were used to reflect absolute levels of functioning (as is the case with the UPSA and the Mattis Dementia Rating Scale). The specific tests used to assess each domain are reported by Heaton et al. (14).

Severity of psychiatric symptoms was assessed with the Scale for the Assessment of Positive Symptoms (SAPS) (17), the Scale for the Assessment of Negative Symptoms (SANS) (18), and the 17-item Hamilton Depression Rating Scale (19). Higher scores reflect greater severity of symptoms on all three scales.

We rated each patient’s community residential setting on a scale from 1 to 5, with 5 representing the highest independence in living. Thus, skilled nursing facilities were rated 1; locked board-and-care facilities were rated 2; open board-and-care facilities were rated 3; situations in which the patient lived with a spouse, partner, or family member were rated 4; and those in which the patient lived alone in a house or apartment were rated 5. These ordinal rankings were used in Spearman rho correlations. Patients were classified as living in a nonindependent setting (1, 2, and 3) or a relatively more independent setting (4 and 5) for a logistic regression analysis in which the level of independence in the community living situation was the dependent variable.

Statistical Analyses

Data were checked for normality of distribution. A square root transformation was performed for age. Some independent variables could not be normalized with appropriate transformations, so nontransformed variables were used in the analyses. Most correlations were computed by using Pearson’s correlation coefficient; however, Spearman rho correlations were used to examine associations between UPSA scores and level of independence in the community living situation, which was measured with an ordinal scale. All correlations were two-tailed, and Bonferroni corrections were used, if appropriate, to control for multiple comparisons.

We used stepwise regression analysis (p to enter <0.05, p to remove >0.10) to identify predictors of total UPSA score. Demographic, illness burden, and cognitive performance variables were entered into the analysis. Although the stepwise regression procedure is atheoretical, it was appropriate for use in this study because the goal was to identify the strongest predictors of performance on the UPSA. In addition, we used logistic regression analysis to assess the ability of the total UPSA score, compared to the other variables, to predict the level of independence in the community living situation. Scores for the neuropsychological abilities were not used in these regression analyses because of the small size of the subgroup that completed the neuropsychological assessment and the lack of significant demographic and clinical differences between this subgroup and the full study group.

Results

UPSA subscale scores and total scores did not differ among the diagnostic groups (all F<1.65, df=3, 107, all p>0.19). Hence, data for all diagnostic groups were aggregated for the analyses.

UPSA Performance and Cognition

Table 2 presents the correlations of UPSA scores with Mattis Dementia Rating Scale scores and scaled scores for the neuropsychological ability domains. Because 14 correlations were computed for each UPSA score, we used a Bonferroni-corrected alpha level of 0.003 (0.05/14) to determine significance. In the current study, total scores on the UPSA were correlated with total Mattis Dementia Rating Scale scores (r=0.61, df=109, p<0.001) as well as with scores on all five subscales of the Mattis Dementia Rating Scale (0.42≤r≤0.60, df=109, all p values<0.001); correspondingly, total Mattis Dementia Rating Scale scores were significantly correlated with scores on all five domains measured by the UPSA (0.39≤r≤0.60, df=109, all p values<0.001). The highest subscale correlations existed between the UPSA total score and the score on the memory subscale of the Mattis Dementia Rating Scale (r=0.60, df=109, p<0.001) and between the total Mattis Dementia Rating Scale score and the UPSA finances subscale score (r=0.60, df=109, p<0.001).

Correlations between the UPSA scores and the neuropsychological ability domains were also computed. The results of t tests analyzing differences between the participants who received neuropsychological testing and those who did not suggested that those who received neuropsychological testing were older (mean age=56.6 years versus 51.7 years), had higher education levels (mean=13.0 years versus 11.7 years), had less severe negative symptoms (mean SANS total score=7.1 versus 10.0), and had better performances on the Mattis Dementia Rating Scale conceptualization (mean score=35.5 versus 32.6 points) and memory (mean score=21.3 versus 19.8 points) subscales (all differences significant at the p<0.05 level, but not significant when a Bonferroni correction was applied). The two groups of patients did not differ significantly, however, in scores on the other Mattis Dementia Rating Scale subscales or in the total scale score, the Hamilton Depression Rating Scale or SAPS scores, or antipsychotic medication doses. In addition, the groups did not differ in gender or living situation. Finally, the two groups did not differ significantly on any of the UPSA scores (e.g., mean UPSA total score=69.1, SD=22.8, versus 62.1, SD=23.1) (t=1.6, df=109, p=0.12). UPSA total scores also were positively correlated with all seven neuropsychological domain scores (0.49≤r≤0.68, df=30–65, all p values<0.001), with the highest correlation between the UPSA total score and the verbal ability score (r=0.68, df=33, p<0.001). The mean scaled score on the aggregated neuropsychological tests was positively correlated with each of the UPSA subscale scores (0.43≤r≤0.57, df=32, all p values<0.05), with the highest correlation with the UPSA communication subscale score (r=0.57, df=32, p<0.001).

UPSA Performance and Living Situation

The total score on the UPSA was positively associated with the degree of independence in the community living situation (r_s=0.48, df=109, p<0.001, N=111). The UPSA subscales measuring financial (r_s=0.45, df=109, p<0.001) and communication (r_s=0.43, df=109, p<0.001) abilities had the strongest correlations with independence.

For comparative purposes, we computed bivariate Spearman rho correlations between independence in the community living situation and the two indicators of global cognitive performance (Mattis Dementia Rating Scale total score and the mean scaled score on the aggregated neuropsychological tests). The correlation between neuropsychological performance and independence (r_s=0.54, df=32, p<0.001, N=34) was nonsignificantly higher than the correlation between the Mattis Dementia Rating Scale total score and independence (r_s=0.30, df=109, p<0.001, N=111).

Other Correlates of UPSA Performance

Bivariate correlations indicated no significant association of UPSA performance with either age (r=–0.11, df=109, p=0.25) or education (r=0.07, df=108, p=0.45). Total UPSA scores were not associated with Hamilton Depression Rating Scale scores (r=–0.01, df=108, p=0.95) or SAPS scores (r=–0.17, df=76, p=0.15), but were inversely related to SANS scores (r=–0.52, df=75, p<0.01).

Stepwise and Logistic Regression Analyses

Stepwise regression analysis was used to identify the strongest predictors of total UPSA score. The following variables were entered in the analysis: demographic variables (age, education, and gender), illness burden variables (SANS, SAPS, and Hamilton Depression Rating Scale scores; duration of illness; and daily antipsychotic dose), and cognitive performance variables (Mattis Dementia Rating Scale subscale scores). (It should be noted that antipsychotic dose may reflect not only illness burden but also physician behavior.) The Mattis Dementia Rating Scale memory subscale score explained 41.1% of the variance in total UPSA scores, and the SANS score accounted for an additional 10.1% of the variance, bringing the total explained variance to 51.2% (Table 3).

To determine whether the association between the Mattis Dementia Rating Scale memory subscale score and the UPSA total score was in fact related to memory performance or was related to more general cognitive performance, we repeated the stepwise regression without the memory subscale score. The Mattis Dementia Rating Scale conceptualization subscale score then explained 36% of the variance in UPSA scores. When both the memory and the conceptualization subscale scores were removed, the Mattis Dementia Rating Scale initiation/perseveration subscale score explained 33% of the variance in UPSA scores.

To assess whether UPSA score was uniquely associated with the level of independence in the community living situation, we conducted a logistic regression analysis of variables predicting whether the living situation was independent or nonindependent. The variables included in the analysis were the demographic and illness burden variables that had been entered in the stepwise regression analysis, the Mattis Dementia Rating Scale total score, and the UPSA total score. Lower levels of education, shorter duration of illness, less severe negative symptoms, and better performance on the UPSA were significant predictors of an independent living situation, accounting for 50% of the variance and correctly classifying 75.7% of the patients (Table 4).

Discussion

We expected that performance on the UPSA would be associated with discrete cognitive domains, as suggested by Green et al. (4). Correlations of UPSA score with measures of attention (the Mattis Dementia Rating Scale attention subscale, the attention/working memory domain of the neuropsychological assessment), learning (learning domain of the neuropsychological assessment), memory (Mattis Dementia Rating Scale memory subscale, memory domain), and executive functioning (Mattis Dementia Rating Scale initiation/perseveration and conceptualization subscales, abstraction/cognitive flexibility domain) were indeed strong. However, our findings suggest that additional cognitive abilities are also associated with UPSA performance. In fact, all of the cognitive abilities we measured were positively correlated with UPSA total scores at a stringent, Bonferroni-corrected level of significance. A similar pattern emerged when we separately examined each UPSA subscale: most cognitive variables were associated with performance on each UPSA subscale. Thus, our findings are similar to those of Evans et al. (11), who found significant associations between most neuropsychological domains and the subscales of the Direct Assessment of Functional Status that are similar to those of the UPSA. The lack of specific relationships between cognitive abilities and UPSA scores probably reflects the multifactorial nature of the tasks included in the UPSA and, indeed, in most aspects of everyday functioning. As an example, successful completion of the UPSA finance tasks would require attention to the task instructions, working memory to keep the relevant task data in mind, mathematical ability, sequential planning, and monitoring performance along the way. To some degree, these results also probably reflect the challenge of separating the effects of generalized brain integrity or impairment from the effects of specific cognitive abilities or deficits.

UPSA total and subscale scores did not differ among patients with different psychotic disorders, suggesting that community functioning skills may be about equally impaired among patients with schizophrenia, schizoaffective disorder, and mood disorder with psychotic features. We found that total scores on the UPSA were related to ultimate functioning in the real world, as measured by level of independence in the community living situation. Correlations with independence were strongest for aggregated neuropsychological test scores and second strongest for the UPSA total score. The UPSA total score remained significantly associated with independence in the living situation when demographic and illness burden variables and Mattis Dementia Rating Scale total score were accounted for, supporting the utility of the UPSA in assessing real-world functioning.

Confirming the results of Patterson et al. (5), we found that everyday functioning abilities, as measured by the UPSA, were associated with less severe cognitive impairment (particularly as assessed with the Mattis Dementia Rating Scale memory subscale, but also with measures of generalized cognitive impairment) and with less severe negative symptoms. The correlations between cognitive performance and performance on everyday functioning tasks were similar to those previously reported for more global indicators of cognitive status (e.g., the MMSE [8]) and other measures of functioning (e.g., the Direct Assessment of Functional Status [unpublished 2001 manuscript of Evans et al.] and medication management ability [20]).

The relationship between depressive symptoms and everyday functioning in psychosis remains unclear. In the current study, depressive symptoms were not associated independently with UPSA scores or with independence in the community living situation. Similarly, Palmer et al. (1) and Evans et al. (11) found that severity of depressive symptoms was not associated with any of the functional domains measured objectively (independence in the community living situation, driving ability, employment history, and performance on the Direct Assessment of Functional Status). Auslander et al. (13), too, found that psychotic patients in assisted care and independent living settings did not differ in terms of severity of depressive symptoms. In contrast, previous research at our center has suggested that severity of depressive symptoms is correlated with worse functioning, when functioning is measured subjectively (21, 22).

The cross-sectional nature of the present study precludes causal inferences about the association between functional capacity (UPSA score) and functional status (independence in the living situation). Nonetheless, it seems likely that basic living skills are at least one of the prerequisites for independent living in the community. Which skills, and at what ability level, may be needed for specific types of living environments are questions needing further research. Of course, it is also possible that patients who have lived predominantly in structured settings have not had the opportunity to develop, practice, and/or maintain such skills. In other words, the relation between functional capacity and functional status may be bidirectional. Thus, impairments in functional capacity should not be viewed as restrictive determinants of living situation, but rather as points of potential intervention. For example, can patients who currently lack the ability to manage public transportation systems be taught to navigate them? Can information about patients’ specific cognitive strengths and weaknesses provide clues for appropriate intervention strategies? Further research in the areas of schizophrenia, neuropsychology, and rehabilitation should address such questions to assist clinicians in helping their patients maximize their independence.

The current findings have several other limitations. A conceptual limitation is that the UPSA measures capacity to perform certain skills in a contrived situation; it does not measure the actual performance of those skills in daily life, where initiation or motivation deficits may interfere with successful role functioning. Our study focused on older outpatients with primary psychotic disorders; therefore, the results may not generalize to younger patients or those who have been chronically hospitalized. Another methodological concern is that the smaller size of the subgroup for which we had additional neuropsychological data precluded including these data in all analyses. However, the correlations of these data with the UPSA scores were similar to those between the Mattis Dementia Rating Scale and UPSA subscale scores, suggesting a generalized pattern of moderate to strong correlations between most cognitive tests and most UPSA subscales.

Like neuropsychological tests, performance-based assessments such as the UPSA may underestimate impairment in real-world functioning, partly because they assess only a sample of the abilities needed for everyday functioning and partly because of differences between quiet testing laboratories and distracting real-world situations. Yet, performance-based assessment of functioning is an important advance in the estimation of real-world functioning. Further research using additional indicators of community functioning (e.g., frequency of actual behavior, such as socializing or using transportation) would provide useful information about the gaps between capacity and initiation of behaviors needed for independent living. Such research should optimally measure additional skill domains (e.g., self-care and medication management, which are not measured by the UPSA). The current investigation, like the cumulative body of research clarifying the links between cognitive abilities, psychiatric symptoms, and community functioning, suggests that a focus on reducing cognitive deficits and negative symptoms may result in improved everyday functioning.

TABLE 1

Enlarge table

TABLE 2

Enlarge table

TABLE 3

Enlarge table

TABLE 4

Enlarge table