Negative symptoms are resistant to treatment and impede functional recovery in schizophrenia. Recognizing the clinical importance of negative symptoms, the top recommendation of the Consensus Development Conference on Negative Symptoms (convened by the National Institute of Mental Health [NIMH] and the Measurement and Treatment Research to Improve Cognition in Schizophrenia [MATRICS] initiative) for stimulating novel treatment development was to develop a new negative symptom measure for treatment trials and research on negative symptoms (1). The Collaboration to Advance Negative Symptom Assessment of Schizophrenia was established to develop and validate this “next-generation” scale, applying a data-driven, iterative, and transparent process (2, 3). In this final report, we describe the measure resulting from the development, validation, and psychometric evaluation of the Clinical Assessment Interview for Negative Symptoms (CAINS).
We developed the CAINS to address conceptual and psychometric limitations of existing instruments (2–4) by systematically assessing the domain of negative symptoms with an eye toward capturing underlying processes that go awry and contribute to these symptoms. Thus, items in the CAINS tap constructs covering approach motivation, pleasure, social engagement, and affective expression, which are also part of the NIMH Research Domain Criteria (5, 6). The CAINS ratings combine assessments of behavioral engagement in relevant activities and reported experiences of motivation and emotion, enabling comprehensive assessment of negative symptoms (2).
We conducted a two-study scale development project at four sites with nearly 500 people with schizophrenia or schizoaffective disorder. The CAINS-beta included 23 items that oversampled the consensus domains of negative symptoms (asociality, avolition, anhedonia, blunted affect, and alogia [1]), recognizing that our systematic and rigorous data-analytic approach to scale development would result in a shorter yet psychometrically sound instrument.
In our first study (3), we evaluated the CAINS-beta in nearly 300 outpatients with schizophrenia using state-of-the-art analytic techniques derived from complementary classical test theory and item response theory (7, 8). Results indicated that the CAINS comprised two moderately correlated factors, one reflecting motivation and pleasure for and engagement in social, vocational, and recreational activities, the other reflecting emotion expression and speech. Item-level analyses revealed good distributional properties, good interrater agreement, discriminating anchor points, and preliminary convergent and discriminant validity.
Our multistep data-analytic approach provided a rational means for item deletion, retention, and modification, and we thus discarded several items that did not meet stringent empirical criteria for inclusion in the revised CAINS (3). Specifically, items that were redundant, had poor psychometric properties, or did not load cleanly on one of the scale factors were eliminated. The remaining items were revised to increase their discriminating power and to capture more clearly the underlying construct (motivation, pleasure expression).
The present study provides final validation of the CAINS in another large and diverse sample of outpatients with schizophrenia or schizoaffective disorder. We examined the scale’s structure through complementary structural analyses, evaluating whether the two-dimensional structure in our first study was replicated with a different sample and a shortened revised CAINS. We evaluated the interrater agreement of the CAINS by examining whether raters at different sites demonstrated good agreement. We also evaluated the test-retest reliability, an important metric for clinical trials, as well as the convergent and discriminant validity of the CAINS. Finally, a key objective was to develop a measure that was not simply a reflection of functional outcome but was nonetheless meaningfully associated with functioning (2). Thus, we examined the relationship of the CAINS to functional capacity and real-world functioning. The overarching goal was to produce a validated, user-friendly, and practical yet comprehensive CAINS that could be used across research and clinical contexts.
Participants were 162 patients with schizophrenia (N=139) or schizoaffective disorder (N=23), recruited from outpatient clinics at four sites. Diagnoses were determined using the Structured Clinical Interview for DSM-IV Axis I Disorders (9). Exclusion criteria were mood episode within the past month; substance dependence in the past 6 months; substance abuse in the past month; IQ <70; history of head injury or neurological disorder; and insufficient English fluency. The sample was a diverse, moderately ill outpatient group with a high level of disability (Table 1). All participants provided written informed consent.
All raters attended a training workshop to review administration and scoring for the CAINS and all other measures. Extensive training included manual review, didactics, and ratings of videotaped interviews (see reference 3 for details). Raters were credentialed for all instruments with videotaped and in-person interviews. Procedures were identical at all sites and were approved by each site’s institutional review board.
Participants attended two sessions approximately 2 weeks apart (mean=15.4 days, SD=3.5). Measures (listed below) were administered in a fixed order. The CAINS and the Brief Psychiatric Rating Scale (BPRS) were readministered during the second session to assess the CAINS’s test-retest reliability in the context of current symptoms. We selected the test-retest interval as an initial test of stability, opting for a time frame that is within the range reported for other negative symptom instruments (1 day to 6 months) and that would lessen the likelihood of clinical changes biasing stability estimates. CAINS interviews were videotaped, and 10 videos from each site were randomly selected to be rated independently by another rater at each of the sites to evaluate between-site agreement on a common set of 40 interviews. Thus, 40 CAINS videos were rated by four raters, one from each site, to assess between-site agreement. To assess the convergent validity of the CAINS expression items, these same 40 CAINS videos were coded by a different set of raters (blind to symptom ratings) using the Facial Expression Coding System (FACES) for the frequency of positive and negative facial expressions. FACES provides comprehensive ratings of outward expressions and has been well validated in healthy and schizophrenia populations (10).
After our first study (3), the CAINS-beta was revised to include 16 items covering motivation and pleasure across social (five items), vocational (three items), and recreational life (three items) domains, as well as emotion expression and speech (five items). Motivation and pleasure items were assessed on the basis of patients’ reports of experienced motivation, interest, and emotion, as well as reports of actual engagement in relevant social, vocational, and recreational activities. The social domain included family, romantic, and friend relationships. Vocational domain activities included work, school, and volunteer activities. Recreational domain activities included hobbies and free-time activities. Given evidence that anticipatory pleasure may be a prominent deficit in schizophrenia (3, 4, 11), we assessed pleasure in two ways: past-week pleasure and expected upcoming-week pleasure. All items were rated on a scale of 0–4, with higher scores reflecting greater impairment. The time period covered by the interview was the past 7 days except for expected pleasure, which covered the next 7 days. One important feature of the CAINS that sets it apart from most other negative symptom interviews is the standardized interview probes built into the measure as well as comprehensive and descriptive anchor points. This standardized interview ensures that different sites will base ratings on the same information.
Additional clinical characteristics were assessed with the 24-item BPRS (12), assessing positive, negative, depression-anxiety, and agitation domains (13); the Calgary Depression Scale for Schizophrenia (14); the modified Simpson-Angus Rating Scale for medication side effects (15); and the Scale for the Assessment of Negative Symptoms (SANS) (16). The CAINS and the SANS were rated by different raters in order to assess the linkage between the measures independent from shared rater variance.
To assess cognitive functioning, we administered the Brief Cognitive Assessment Tool for Clinicians (17), which includes the digit symbol test, the Trail-Making Test, part B, and the category fluency test as well as the Wechsler Test of Adult Reading (18) to estimate full-scale IQ.
To assess functional capacity and real-world functioning (19), we administered the brief version of the UCSD Performance-Based Skills Assessment (UPSA-Brief) (20) and the Role Functioning Scale (21), assessing work, self-care, family, and social functioning. Self-report measures included the Temporal Experience of Pleasure Scale (22), assessing anticipatory and consummatory pleasure; the Behavioral Inhibition/Behavioral Activation Scales (23), assessing sensitivity of approach and avoidance motivation systems; the Social Anhedonia Scale (24), assessing decreased social pleasure; and the Social Closeness Scale (25), assessing social engagement and desire for close relationships.
As in our first study (3), we inspected scree plots to explore the number of common factors in CAINS item responses, followed by exploratory factor analysis using principal axis extraction with promax rotation to assess scale structure. Hierarchical cluster analysis was used as a complementary approach for exploring meaningful content groupings of items (26). After we confirmed the CAINS structure, additional analyses assessed between-site interrater agreement with intraclass correlation (ICC) (27); test-retest reliability; convergent validity (i.e., whether the CAINS significantly correlated with negative symptoms assessed with the BPRS, the SANS, FACES coded facial expressions, and self-report measures); discriminant validity (i.e., whether the CAINS was not strongly correlated with positive symptoms, depression, agitation, medication side effects, and cognitive functioning); and the relationship between the CAINS and functioning. For comparison, we also present correlations between the SANS, BPRS, and other study measures. Correlations with SANS subscales are presented in the data supplement that accompanies the online edition of this article.
We excluded two items for psychometric reasons. First, the vocational past-week pleasure item was missing for nearly half of the sample (47%) because many participants were not in a relevant role (work, school, or volunteer) and thus could not report on experienced pleasure. This item was strongly correlated with vocational expected pleasure (r=0.74, p<0.001, N=160), indicating that the expected pleasure item could capture vocational pleasure. Second, the two vocal expression items were redundant (r=0.86, p<0.001, N=160); we opted to retain the vocal prosody item given its ease of use by raters and its item-level psychometric properties. Descriptive statistics for individual CAINS items are presented in supplementary tables 1 and 2 in the online data supplement.
Results from structural analyses replicated the two-dimensional structure reported in our first study (3). The scree plot suggested two major dimensions, and subsequent results from principal-axis factor analysis (Table 2) and hierarchical cluster analysis yielded two clearly interpretable and relatively independent factors: expression (four items reflecting diminished outward expression and speech) and motivation/pleasure (nine items reflecting diminished motivation, pleasure, and social engagement). Because the item assessing romantic relationships did not load clearly on either factor, we deleted it as a separate item from the final CAINS. The final CAINS has 13 items (see the online data supplement).
Results of factor and cluster analyses did not suggest a single factor, and the internal consistency of the overall CAINS scale (Cronbach’s alpha=0.76) was no better than the internal consistency of the two scales: expression (alpha=0.88) and motivation/pleasure (alpha=0.74), suggesting that the negative symptom domain is not unidimensional. The correlation between the two scales was also modest (r=0.24, N=160). Thus, the CAINS is optimally implemented with two scales (expression and motivation/pleasure). However, a single composite of the two subscales can also be computed.
+
Rater Agreement and Test-Retest Reliability
Average ICCs between the four sites for the motivation/pleasure and expression scales were 0.93 and 0.77, respectively, indicating good rater agreement by raters from different sites. ICCs ranging from 0.31 to 0.70 have been reported for the individual SANS items (28); ICCs for CAINS items range from 0.67 to 0.94 (see supplementary table 1 in the online data supplement). Correlations between the two testing occasions for the motivation/pleasure and expression scales were 0.69 and 0.69, respectively, indicating adequate test-retest reliability. In our study, test-retest reliability of the BPRS negative symptom subscore and total score were 0.47 and 0.60, respectively. Test-retest reliability in other studies ranges from 0.37 to 0.54 for the SANS (28) and 0.68 for the negative symptom subscale of the Positive and Negative Syndrome Scale (PANSS) (29). Cronbach’s alpha was 0.82 for the SANS and 0.77 for the BPRS.
Men and women did not differ on either CAINS scale. There were no racial or ethnic differences in either scale, with one exception: European Americans (mean=5.47, SD=3.70) were rated higher on the expression scale than African Americans (mean=4.26, SD=3.22) (t=2.09, df=142, p=0.038).
Convergent validity (Table 3) was assessed by examining correlations between the CAINS scales and 1) other negative symptom measures, 2) self-report measures, and 3) observer-coded facial expressions. Both CAINS scales were correlated with the BPRS negative symptom subscale and the SANS subscales. The correlations between the CAINS and the SANS are noteworthy because they are not biased by shared rater variance. As expected, the CAINS motivation/pleasure scale was modestly correlated with self-report scales assessing emotion, motivation, and social engagement, but significantly more so than the CAINS expression scale and the BPRS negative symptom subscale. Specifically, the CAINS motivation/pleasure scale was related to anticipatory and consummatory pleasure and the Social Anhedonia Scale. In addition, the CAINS motivation/pleasure scale was inversely related to the desire for close relationships and social engagement as assessed with the Social Closeness Scale, and this correlation was greater for the CAINS than for the SANS. Sensitivity to approach and avoidance motivational systems was not related to the motivation/pleasure scale, but it was related to the expression scale. Finally, the CAINS expression scale was significantly and more strongly correlated with independently coded positive and negative facial expressions than either the CAINS motivation/pleasure scale or the BPRS negative symptom subscale.
Discriminant validity (Table 4) was assessed by examining the correlations between the CAINS and 1) other symptoms (positive symptoms, agitation, and depression), 2) medication side effects, and 3) cognitive functioning. The CAINS scales were not related to depression as assessed with the BPRS or the Calgary Depression Scale or with medication side effects as assessed with the Simpson-Angus Rating Scale. In addition, the CAINS scales, the SANS, and the BPRS negative symptom subscale were not related to cognition, a finding consistent with other studies (30, 31). The CAINS motivation/pleasure scale was modestly related to positive symptoms and agitation as assessed by the BPRS, although the SANS and the BPRS negative symptom subscale were significantly more strongly related to these other symptoms than the CAINS expression scale was.
+
Relationship With Functional Outcome
The CAINS scales were not related to functional capacity on the performance-based skills test (nor were the SANS or the BPRS), but the motivation/pleasure scale was related to social, family, independent living, and vocational functioning, and more strongly so than the BPRS negative symptom scale, but not the SANS (Table 5). The expression scale was related to independent living and family functioning. Critically, the correlations were not so large as to suggest that the CAINS scales are redundant with measures of functioning. Indeed, that the CAINS minimizes overlapping item content with functioning (e.g., self-care) is a strength of the measure.
Following the recommendation of the NIMH-MATRICS Consensus Development Conference on Negative Symptoms (1), the CAINS is a clinical rating scale for negative symptoms with potent and clear treatment targets for the next generation of pharmacological and psychosocial treatments. The CAINS development process was unique in that it included the recommended sample types for negative symptom treatment trials (32, 33); included input from multiple stakeholders (industry, government, academia); followed a rigorous empirical approach; elicited and incorporated feedback at each stage, keeping results available and transparent; and developed training materials for dissemination.
Converging structural analyses replicated the two-factor structure we reported in our first study (3). Further structural and item-level analyses indicated that three of the 16 items should be dropped, and the final CAINS contains 13 items. Given the modest association between scales and their differential validity correlations, the CAINS is optimally implemented as a two-scale measure, one tapping expression (four items) and the other assessing motivation/pleasure (nine items). These scales are consistent with the domains identified in reviews of the negative symptom on older negative measures, such as the SANS and the PANSS (34, 35). However, should certain indications require the use of a single score (e.g., clinical trial designs), a composite of the two subscales can be computed in much the same way that a composite is computed from the separable cognitive domains in the MATRICS Consensus Cognitive Battery. Notably, the CAINS can be administered in a timely manner while also comprehensively covering the domains.
Analyses indicated that the CAINS is stable across a 2- to 3-week period, an outcome that portends well for studies assessing change in treatment trials. As in our first study (3), we adopted a rigorous approach to assessing interrater agreement, comparing raters from different sites, and our findings indicate high agreement for both the expression and motivation/pleasure scales. The CAINS includes interview questions built into the measure, explicit anchor descriptions, and a detailed manual, which will help preserve rater agreement at other sites. CAINS scores will be easier to compare across sites and studies given that the ratings will be made from the same standardized interview probes.
The CAINS exhibited strong convergent validity, as evidenced by linkages to other negative symptom measures; self-report measures of constructs covering social engagement, pleasure experience, and motivation; and independently assessed emotion expressions. That the correlations between the CAINS motivation/pleasure scale and SANS were not stronger reflects the fact that the CAINS taps underlying domains of motivation and pleasure and distinguishes experience and behavioral engagement. The CAINS also exhibited strong discriminant validity by its nonsignificant correlations with depression, medication side effects, IQ, and cognition. Although other studies have found modest correlations between negative symptoms and cognition (36), these correlations may not hold longitudinally, may partly reflect inclusion of cognition in older negative symptom scales, and are perhaps accounted for by overlap between older negative symptom scales and functioning measures (31).
The CAINS also demonstrated linkages to functional outcome. The CAINS motivation/pleasure scale was related to all aspects of functioning, and the expression scale was related to independent living and family functioning. Improving the lives of people with schizophrenia involves not only decreasing symptoms but also improving functioning. Evidence from recent longitudinal studies indicates that improvements in negative symptoms predict improvements in functioning, particularly among people early in the course of illness (e.g., 37). Interestingly and unexpectedly, neither of the CAINS scales was related to functional capacity as assessed by the UPSA-Brief. Thus, with respect to functioning, the CAINS is not directly related to what the individual can do but instead is related to what he or she actually does, and researchers interested in functioning and negative symptoms should keep this distinction in mind. Given the CAINS’s psychometric properties, it is well positioned to be included in future studies designed to assess symptom and functional recovery in schizophrenia.
Important next steps for the CAINS include assessing it for use across cultures and populations outside of academic settings (e.g., adolescents at risk for psychosis, patients early in the course of illness, inpatients, patients with more severe symptoms, patients in community clinics), as well as comparisons with a broader range of neurocognitive tests and other negative symptom scales (e.g., the Negative Symptom Assessment [38]). Of particular importance is inclusion of the CAINS in treatment trials to ascertain its sensitivity to change. Research that seeks to delineate the neurobiological and behavioral processes that undergird these symptoms, processes that are also a current focus of the Research Domain Criteria (5, 6, 39), will further help to focus the development of treatments for the motivation/pleasure and expression domains. Indeed, to move the field forward, progress must be made on complementary fronts: isolating mechanisms and processes underlying these domains and developing treatments to improve them. The CAINS provides a means for assessing these domains comprehensively, reliably, and validly such that progress on these fronts may be propelled forward more quickly and in a more focused manner.
Why might researchers and clinicians opt to use the CAINS instead of other negative symptom measures? The CAINS was developed to address conceptual and psychometric limitations of existing instruments, and our approach included an emphasis on constructs with a strong cognitive and affective neuroscience research base (approach motivation, pleasure, social engagement, and affective expression), large sample sizes, and an iterative data-analytic approach. Other strengths of the CAINS include the standardized interview prompts built into the measure; the clear, comprehensive, and discriminating anchor points for items; the instrument’s combination of brevity and comprehensiveness; and the availability of a training manual and videos (http://www.med.upenn.edu/bbl/downloads/CAINSVideos.shtml). The CAINS’s inclusion of prospections of pleasurable experiences is another strength that has been at the center of recent views on negative symptoms (40). Our results demonstrate that interrater agreement for the CAINS is higher than that reported for the SANS (28), and the test-retest reliability is comparable to those of the SANS and the PANSS (28, 29). The CAINS demonstrates greater convergent validity than the BPRS negative symptom scale, and its convergent validity is comparable to or greater than that of the SANS. The CAINS scales are less strongly related to positive symptoms, agitation, and depression than other measures, and the CAINS demonstrates linkages to functioning comparable to the SANS.
Although clinical trials may opt to use a single composite CAINS score, it is important to emphasize that the CAINS also distinguishes two clear treatment targets with substantial grounding in neuroscience: expression and motivation/pleasure. For example, translational efforts to understand processes related to expression are pursued in human studies (41) as well as informative mouse models that examine underlying regulatory mechanisms (42), and contemporary models of negative symptoms emphasize linkages between motivation and pleasure (11, 40). Indeed, an important issue for the field to address is whether there are common or distinct mechanisms underpinning the two dimensions of negative symptoms. Current models and studies of negative symptoms suggest both distinct mechanisms (40, 43) as well as different types of patients who may have more severe deficits in one but not both dimensions (44). Practically speaking, clinicians and researchers can gauge improvement in negative symptoms across these two dimensions, rather than five overlapping symptoms or a total composite that does not distinguish the two core domains of negative symptoms.
Across two studies with nearly 500 diverse outpatients with schizophrenia or schizoaffective disorder, and using state-of-the-art scale development methods, we have demonstrated that the CAINS is a reliable and valid clinical rating scale for negative symptoms that is grounded in basic science on the processes putatively undergirding these symptoms. The CAINS is ready to be disseminated and tested in clinical trials as well as in other studies of negative symptoms. Clinicians will be able to assess the two dimensions of negative symptoms across three life domains, measures that will be informative when considering the ways in which changes in these two dimensions may differentially affect real-world functioning.
The authors thank Janelle Caponigro, Erin Moran, Timothy Campellone, Ori Elis, Mary March, Elizabeth Hanson, Melanie Bennett, Mara Hart, Alison Petrow, Amanda Bender, Crystal Gibson, Heidi Kuppinger, and Christen Waldon for assistance with data collection and scale development.