The American Psychiatric Association (APA) has updated its Privacy Policy and Terms of Use, including with new information specifically addressed to individuals in the European Economic Area. As described in the Privacy Policy and Terms of Use, this website utilizes cookies, including for the purpose of offering an optimal online experience and services tailored to your preferences.

Please read the entire Privacy Policy and Terms of Use. By closing this message, browsing this website, continuing the navigation, or otherwise continuing to use the APA's websites, you confirm that you understand and accept the terms of the Privacy Policy and Terms of Use, including the utilization of cookies.

×

Abstract

Objective:

Little is known about factors influencing the rate of progression of Alzheimer’s dementia. Using data from the Cache County Dementia Progression Study, the authors examined the link between clinically significant neuropsychiatric symptoms in mild Alzheimer’s dementia and progression to severe dementia or death.

Method:

The Cache County Dementia Progression Study is a longitudinal study of dementia progression in incident cases of this condition. Survival analyses included unadjusted Kaplan-Meier plots and multivariate Cox proportional hazard models. Hazard ratio estimates controlled for age at dementia onset, dementia duration at baseline, gender, education level, General Medical Health Rating, and apolipoprotein E epsilon 4 genotype.

Results:

Three hundred thirty-five patients with incident Alzheimer’s dementia were studied. Sixty-eight (20%) developed severe dementia over the follow-up period. Psychosis (hazard ratio=2.007), agitation/aggression (hazard ratio=2.946), and any one clinically significant neuropsychiatric symptom (domain score ≥4, hazard ratio=2.682) were associated with more rapid progression to severe dementia. Psychosis (hazard ratio=1.537), affective symptoms (hazard ratio=1.510), agitation/aggression (hazard ratio=1.942), mildly symptomatic neuropsychiatric symptoms (domain score of 1–3, hazard ratio=1.448), and clinically significant neuropsychiatric symptoms (hazard ratio=1.951) were associated with earlier death.

Conclusions:

Specific neuropsychiatric symptoms are associated with shorter survival time from mild Alzheimer’s dementia to severe dementia and/or death. The treatment of specific neuropsychiatric symptoms in mild Alzheimer’s dementia should be examined for its potential to delay time to severe dementia or death.

The increasing number of people diagnosed with dementia is a well-known phenomenon driven by an aging population and increased public recognition of its signs and symptoms (1). The United States annual costs for health care, long-term care, and hospice care of people with dementia are expected to increase from $200 billion in 2012 to $1.1 trillion in 2050 (2). Many of these costs are related to the long-term care required for those with severe dementia. Delaying progression to late-stage dementia has the potential of increasing meaningful time spent with those afflicted. Several studies have examined predictors of progression from onset of dementia to severe dementia (35). Factors shown to accelerate progression include younger age at onset, higher level of education, greater severity of cognitive impairment (defined as lower baseline modified Mini-Mental State Examination scores or higher clinical dementia rating scores), greater severity of behavioral disturbance, and presence of psychosis or other neuropsychiatric symptom (3, 4). Storandt et al. (6) showed that rate of decline on psychometric testing accelerates as dementia severity worsens, but in their study no individual test was predictive of dementia progression (i.e., nursing home placement).

Using the same population-based study utilized in the present study, the Cache County Dementia Progression Study, Rabins et al. (5) found that female gender, less than high school education, and at least one clinically significant neuropsychiatric symptom at baseline were predictive of shorter time to severe Alzheimer’s dementia. Age at onset of dementia was predictive in that persons in the youngest (68–80 years old) and the oldest (87–104 years old) tertiles of age progressed to severe Alzheimer’s dementia faster than those in the middle tertile of age (81–86 years old). In addition, subjects with mild neuropsychiatric symptoms or at least one clinically significant neuropsychiatric symptom and subjects with worse health were more likely to progress to severe dementia or death. The present article aims to expand on this work.

The Cache County Dementia Progression Study (7, 8) is a longitudinal study with regular reassessment of cognition and detailed collection of neuropsychiatric symptom data. Although it is known that neuropsychiatric symptoms are associated with a worse prognosis in dementia (9), the relationship between individual neuropsychiatric symptoms or clusters of neuropsychiatric symptoms and progression to severe dementia or death is not fully understood. In the present analysis, we examine the association between clinically significant neuropsychiatric symptoms, including psychotic and affective clusters of symptoms, and progression to severe dementia and/or death. We hypothesized that the presence of psychotic symptoms, and the individual symptom of agitation/aggression, would predict shorter time to severe dementia.

Method

Methods of the Cache County Study and the Dementia Progression Study have been described in detail elsewhere (7, 8). Briefly, all permanent residents of Cache County, Utah, who were age ≥65 in January 1995 (N=5,677) were invited into the study. We enrolled 5,092 (90%) in wave 1 of the Cache County Study, all of whom were screened for dementia in a multistage assessment protocol. Rate of dementia was 9.6% in the prevalence wave, similar to many epidemiological samples. Individuals were reassessed at 3- to 5-year intervals (mean=3.53 years [SD=0.6]) in three incidence waves. A consensus panel made diagnoses of dementia and dementia type. Diagnosis of Alzheimer’s dementia followed the criteria of the National Institute of Neurological and Communicative Disorders and Stroke and the Alzheimer’s Disease and Related Disorders Association (10). The Dementia Progression Study (11) limited analyses to those individuals from the Cache County Study who converted from no dementia to Alzheimer’s dementia with follow-up rates, excluding mortality, exceeding 90%. After complete description of the study to the subjects, written informed consent was obtained.

Severe Alzheimer’s dementia was defined as a Mini-Mental State Examination (12) score ≤10 or a Clinical Dementia Rating Scale (13) score of 3 (severe). If only the Mini-Mental State Examination criteria were met, inclusion required a Clinical Dementia Rating Scale score ≥2 (moderate); and if only Clinical Dementia Rating Scale criteria were met, inclusion required a Mini-Mental State Examination score <16.

To identify potentially predictive neuropsychiatric symptoms, the 10-item Neuropsychiatric Inventory (14) was utilized. The Neuropsychiatric Inventory is a fully structured informant-based interview that provides a systematic assessment of the following 10 neuropsychiatric symptom domains: delusions, hallucinations, agitation/aggression, depression/dysphoria, anxiety, elation/euphoria, apathy/indifference, disinhibition, irritability/lability, and aberrant motor behavior. The presence of symptoms in each domain during the past 30 days was queried, and if endorsed, specific follow-up questions were asked to clarify the nature of the symptoms, including the degree of change from premorbid and treatment. Once the disturbances relevant to each domain were defined, the informant was asked about the frequency of these on a scale from 1 (occasionally) to 4 (very frequently, more than once a day). The informant was also asked to rate the severity of the behavior on a 3-point severity scale (1=mild, 2=moderate, or 3=severe). Neuropsychiatric Inventory ratings were obtained at the time of diagnosis of Alzheimer’s dementia and scored as follows: 1=presence of at least one of the psychosis neuropsychiatric symptom domains (delusions and hallucinations); 2=presence of at least one of the affective neuropsychiatric symptom domains (depression, anxiety, and irritability); 3=presence of the individual neuropsychiatric symptom of apathy/indifference or agitation/aggression; and 4=frequency-by-severity Neuropsychiatric Inventory score across all domains trichotomized as no symptoms, at least one neuropsychiatric symptom domain score of 1 to 3 (mild), or at least one neuropsychiatric symptom domain score ≥4 (clinically significant).

To identify individual factors associated with time to develop severe Alzheimer’s dementia, we constructed unadjusted Kaplan-Meier plots for each of the Neuropsychiatric Inventory groups described above. Additionally, we ran bivariate and multivariate Cox proportional hazard models. Based on the results from Rabins et al. (5), the hazard models controlled for age at dementia onset, gender, education level, and General Medical Health Rating score (15). General Medical Health Rating scores were obtained at the time of diagnosis of Alzheimer’s dementia and were coded as excellent, good, or fair/poor. Furthermore, given results of previous studies (16), we also controlled for apolipoprotein E epsilon 4 (APOE-ε4) genotype status, with positive designation coded if at least one ε4 allele was present. Lastly, we controlled for time between dementia onset and diagnosis (dementia duration at baseline). The same analyses were run for association with time to death. All analyses met the proportional hazards assumption and were conducted with SPSS version 21 (IBM, Armonk, New York).

Results

Three hundred thirty-five incident cases of possible or probable Alzheimer’s dementia were identified. The mean age at onset was 84.3 years (SD=6.4), and the mean time between dementia onset and diagnosis was 1.7 years (SD=1.3). Sixty-eight individuals (20% of the incident sample) developed severe Alzheimer’s dementia over the course of the study (1995–2009). After extended follow-up through October 21, 2010, 273 individuals were deceased. The median time to severe Alzheimer’s dementia for the sample was 8.4 years (95% confidence interval [CI]=7.6–9.2) and to death was 5.742 years (95% CI=5.423–6.061). Age at onset showed a nonlinear relationship for time to severe Alzheimer’s dementia and a linear association with time to death. Global Medical Health Rating was not associated with time to severe Alzheimer’s dementia but was associated with time to death, and those with poor/fair scores had 1.6 times the death hazard compared with individuals with good/excellent scores (neuropsychiatric symptom results were not affected significantly by this).

Neuropsychiatric symptoms were common, with 50.9% of the sample having at least one neuropsychiatric symptom. The baseline percentages of the neuropsychiatric symptom clusters were as follows: psychosis cluster, 18.1%; affective cluster, 38.8%. The individual neuropsychiatric symptom domain of apathy/indifference was seen in 16.9% of individuals at baseline, and the individual domain of agitation/aggression was seen in 10%. At baseline, 25.9% of individuals had at least one mild neuropsychiatric symptom, and 25.0% of individuals had at least one clinically significant neuropsychiatric symptom.

The results of the bivariate and multivariable Cox regression models (controlled for age at dementia onset, dementia duration at baseline, gender, education level, Global Medical Health Rating, and APOE-ε4 status) are summarized in Table 1 for hazard of severe dementia and in Table 2 for hazard of death. The psychosis cluster (hazard ratio=2.007, p=0.03), agitation/aggression (hazard ratio=2.946, p=0.004), and at least one clinically significant neuropsychiatric symptom (hazard ratio=2.682, p=0.001) were predictive of progression to severe dementia. The psychosis cluster (hazard ratio=1.537, p=0.01), affective cluster (hazard ratio=1.510, p=0.003), agitation/aggression (hazard ratio=1.942, p=0.004), at least one mild neuropsychiatric symptom (hazard ratio=1.448, p=0.02), and at least one clinically significant neuropsychiatric symptom (hazard ratio=1.951, p≤0.001) were predictive of progression to death. Unadjusted Kaplan-Meier plots for the outcome of severe dementia and death as predicted by agitation/aggression are presented in Figure 1 and meant to serve as a proxy illustration for other predictors as well.

TABLE 1. Multivariate Cox Regression Models for Time to Severe Dementiaa

VariablePsychosis ClusterAffective ClusterAgitation/AggressionApathy/IndifferenceNeuropsychiatric Inventory Significanceb
Hazard RatiopHazard RatiopHazard RatiopHazard RatiopHazard Ratiop
Unadjusted, bivariate valuec2.0240.091.3870.192.3210.0091.1760.60d0.03d
(1.214/2.129)(0.560/0.008)
Adjusted model
Alzheimer’s dementia age at onset0.290<0.0010.313<0.0010.3510.0010.279<0.0010.295<0.001
Alzheimer’s dementia age at onset squared1.008<0.0011.007<0.0011.0070.0011.008<0.0011.008<0.001
Female1.9490.031.8880.041.8850.041.9980.031.852<0.05
Educatione1.7910.801.910<0.051.9340.041.8880.560.7560.10
Apolipoprotein E-e4 carrier0.9400.821.0760.771.1090.711.0700.811.1060.71
Global Medical Health Rating1.5540.141.5850.131.5110.171.5270.151.7370.07
Dementia duration at baseline0.8110.030.8270.040.7530.0060.8210.040.7630.005
Psychosis cluster2.0070.03XXXXXXXX
Affective clusterXX1.5120.1XXXXXX
Agitation/aggressionXXXX2.9460.004XXXX
Apathy/indifferenceXXXXXX1.5520.17XX
Neuropsychiatric Inventory significancebXXXXXXXXXX
 TrichotomizedXXXXXXXX0.002
 Mild symptom(s)XXXXXXXX1.0770.83
 Clinically significant symptoms(s)XXXXXXXX2.6820.001

aCox regression models were controlled for age at dementia onset, dementia duration at baseline, gender, education level, Global Medical Health Rating, and apolipoprotein E-e4 status.

bNeuropsychiatric Inventory score was examined as a trichotomous variable (none versus mild versus clinically significant).

cData represent the bivariate unadjusted model for each Neuropsychiatric Inventory symptom or cluster.

dData are shown as trichotomized comparison followed (in parentheses) by mild symptoms versus none/clinically significant symptoms versus none.

eReference category is a high school education or greater.

TABLE 1. Multivariate Cox Regression Models for Time to Severe Dementiaa

Enlarge table

TABLE 2. Multivariate Cox Regression Models for Time to Deatha

VariablePsychosis ClusterAffective ClusterAgitation/AggressionApathy/IndifferenceNeuropsychiatric Inventory Significanceb
Hazard RatiopHazard RatiopHazard RatiopHazard RatiopHazard Ratiop
Unadjusted, bivariate valuec1.5670.0061.1920.201.2760.251.070.68d0.07d
(1.425/1.267)(0.029/0.147)
Adjusted model
Alzheimer’s dementia age at onset1.092<0.0011.100<0.0011.096<0.0011.095<0.0011.102<0.001
Female0.7060.020.7250.030.7010.020.7330.030.6940.01
Educatione1.2260.231.2680.161.2780.151.2690.171.2210.24
Apolipoprotein E-e4 carrier1.0670.661.1140.451.1680.281.1160.441.1340.39
Global Medical Health Rating1.5930.0021.5600.0041.5580.0041.6220.0021.5770.003
Dementia duration at baseline0.781<0.0010.780<0.0010.7610.0040.781<0.0010.752<0.001
Psychosis cluster1.5370.01XXXXXXXX
Affective clusterXX1.5100.003XXXXXX
Agitation/aggressionXXXX1.9420.004XXXX
Apathy/indifferenceXXXXXX1.2610.21XX
Neuropsychiatric Inventory significancebXXXXXXXXXX
 TrichotomizedXXXXXXXX<0.001
 Mild symptom(s)XXXXXXXX1.4480.02
 Clinically significant symptoms(s)XXXXXXXX1.951<0.001

aCox regression models were controlled for age at dementia onset, dementia duration at baseline, gender, education level, Global Medical Health Rating, and apolipoprotein E-e4 status.

bNeuropsychiatric Inventory sore was examined as a trichotomous variable (none versus mild versus clinically significant).

cData represent the bivariate unadjusted model for each Neuropsychiatric Inventory symptom or cluster.

dData are shown as trichotomized comparison followed (in parentheses) by mild symptoms versus none/clinically significant symptoms versus none.

eReference category is a high school education or greater.

TABLE 2. Multivariate Cox Regression Models for Time to Deatha

Enlarge table
FIGURE 1.

FIGURE 1. Unadjusted Kaplan-Meier Plots for Agitation and Severe Dementia (left) and for Agitation and Death (right).

Additional models were constructed controlling for psychotropic medication use. Analyses were run for any antidepressant use (with separate analysis for selective serotonin reuptake inhibitor use), any antipsychotic use (with separate analyses for first- and second-generation antipsychotic use), and benzodiazepine use. Medication use was consistently not significant, and there was no appreciable change in the results (i.e., the same predictors were predictive).

Discussion

In this population-based study of individuals with incident Alzheimer’s dementia, psychosis, agitation/aggression, and clinically significant neuropsychiatric symptoms were predictive of earlier progression to severe dementia and death. Affective neuropsychiatric symptoms and mild neuropsychiatric symptoms were associated with earlier death but not earlier progression to severe dementia. These results expand on the analyses performed by Rabins et al. (5), in which women, participants with less than high school education, participants with at least one clinically significant Neuropsychiatric Inventory domain, and the youngest or oldest age-at-onset cohorts progressed more rapidly to severe dementia. Age at onset showed a nonlinear relationship for time to severe Alzheimer’s dementia and a linear association with time to death. Global Medical Health Rating was not associated with time to severe Alzheimer’s dementia but was associated with time to death, and individuals with poor/fair scores had 1.6 times the death hazard compared with those with good/excellent scores.

In other samples, psychosis was shown to be predictive of progression to nursing home care, but not mortality, in Alzheimer’s dementia patients (4). Additionally, behavioral disturbance has been associated with faster cognitive decline over 24 weeks among untreated patients (3). In this study, early agitation/aggression was a robust predictor of both accelerated progression and mortality. Other studies have commented on an apathy syndrome in Alzheimer’s disease as predictive of increased mortality (17). In the present study, apathy was not predictive of accelerated mortality or progression to severe Alzheimer’s dementia. The treatment of specific neuropsychiatric symptoms in early dementia should be examined for its potential to delay time to severe dementia or death.

Although the causal nature of these predictive associations is not known, several possibilities exist. First, it is possible that a confounder that was not being measured in this study exists. Or perhaps localized pathology of brain regions associated with agitation/aggression or psychosis occurs in more aggressive forms of Alzheimer’s dementia. Alternatively, the presence of these neuropsychiatric symptoms may influence the care environment in some way that in turn affects progression. For example, one may speculate that the presence of psychosis or agitation/aggression may lead to behaviors, situations, and relationships that are more conducive to worsening of disease. Affective symptoms could lead to similar modification. Although not evident in this study, it is also possible that the treatment of these symptoms with antipsychotic medication increased mortality, since these drugs are associated with a 1.5- to 1.7-fold mortality increase in randomized trials and in large-scale cohort studies (18). The present study did not show a modification in progression based on medication use of any kind.

Limitations of this study include the lack of incident case subjects <65 years old, the small number of cases with severe Alzheimer’s dementia, and the homogeneity of the population (low rates of alcohol and illicit substance abuse and low representation of nonwhite persons). In general, the fact that this is a one-time look at the presence of neuropsychiatric symptoms based on frequency scores on the Neuropsychiatric Inventory without including severity scores or longitudinal measures of behavioral burden over the course from diagnosis of Alzheimer’s dementia to study endpoint is a limitation. Lastly, there was not a delirium screen included in the assessment because there is a very high risk of delirium in advancing dementia, and psychosis/agitation is a known manifestation of delirium (19). A formal delirium assessment method, such as the confusion assessment method (20), might identify undetected delirium and therefore subgroups at risk.

Strengths of the study include its epidemiologic sampling frame, a high participation rate, a prospective, longitudinal data collection, and use of state-of-the-art clinical diagnostic assessments of Alzheimer’s dementia.

From the Department of Psychiatry, Johns Hopkins University, Baltimore; and the Department of Psychiatry, Utah State University, Logan, Utah.
Address correspondence to Dr. Lyketsos ().

Dr. Lyketsos receives grant support (research or CME) from the Associated Jewish Federation of Baltimore, AstraZeneca, Bristol-Myers Squibb, Eisai, Elan, Forest, Functional Neuromodulation, GlaxoSmithKline, Lilly, the National Football League, NIA, NIMH, Novartis, Ortho-McNeil, Pfizer, and the Weinberg Foundation; he serves as a consultant/advisor to Abvie, Adlyfe, AstraZeneca, Avanir, Bristol-Myers Squibb, Genentech, GlaxoSmithKline, Eisai, Elan, Forest, Janssen, Lilly, Lundbeck, Merz, NFL Benefits Office, NFL Players Association, Novartis, Orion, Pfizer, Supernus, Takeda, Wyeth, and Zinfandel; and he has received honorarium or travel support from Forest, GlaxoSmithKline, Health Monitor, and Pfizer. Dr. Rabins has provided legal testimony for Janssen Pharmaceutica. All other authors report no financial relationships with commercial interests.

Supported by the Cache County Memory Study, the Dementia Progression Study, and the Joseph and Kathleen Bryan Alzheimer's Disease Research Center (National Institute on Aging grants R01AG11380, R01AG21136, and R01AG18712).

Previously presented in part at the Alzheimer’s Association International Conference, Vancouver, British Columbia, July 14–19, 2012; and the American Association of Geriatric Psychiatry Conference, Orlando, Fla., March 14–17, 2014.

References

1 Rabins P, Lyketsos C, Steele C: Practical Dementia Care, 2nd ed. New York, Oxford University Press, 2006CrossrefGoogle Scholar

2 Alzheimer’s Association: 2012 Alzheimer's Disease Facts and Figures. Chicago, Alzheimer’s Association, 2012Google Scholar

3 Lopez OL, Schwam E, Cummings J, et al.: Predicting cognitive decline in Alzheimer’s disease: an integrated analysis. Alzheimers Dement 2010; 6:431–439Crossref, MedlineGoogle Scholar

4 Stern Y, Tang MX, Albert MS, et al.: Predicting time to nursing home care and death in individuals with Alzheimer disease. JAMA 1997; 277:806–812Crossref, MedlineGoogle Scholar

5 Rabins PV, Schwartz S, Black BS, et al.: Predictors of progression to severe Alzheimer’s disease in an incidence sample. Alzheimers Dement 2013; 9:204–207Crossref, MedlineGoogle Scholar

6 Storandt M, Grant EA, Miller JP, et al.: Rates of progression in mild cognitive impairment and early Alzheimer’s disease. Neurology 2002; 59:1034–1041Crossref, MedlineGoogle Scholar

7 Breitner JC, Wyse BW, Anthony JC, et al.: APOE-epsilon4 count predicts age when prevalence of AD increases, then declines: the Cache County Study. Neurology 1999; 53:321–331Crossref, MedlineGoogle Scholar

8 Lyketsos CG, Steinberg M, Tschanz JT, et al.: Mental and behavioral disturbances in dementia: findings from the Cache County Study on Memory in Aging. Am J Psychiatry 2000; 157:708–714LinkGoogle Scholar

9 Lyketsos CG, Colenda CC, Beck C, et al.: Position statement of the American Association for Geriatric Psychiatry regarding principles of care for patients with dementia resulting from Alzheimer disease. Am J Geriatr Psychiatry 2006; 14:561–572Crossref, MedlineGoogle Scholar

10 McKhann G, Drachman D, Folstein M, et al.: Clinical diagnosis of Alzheimer’s disease: report of the NINCDS-ADRDA Work Group under the auspices of Department of Health and Human Services Task Force on Alzheimer’s Disease. Neurology 1984; 34:939–944Crossref, MedlineGoogle Scholar

11 Tschanz JT, Corcoran CD, Schwartz S, et al.: Progression of cognitive, functional, and neuropsychiatric symptom domains in a population cohort with Alzheimer dementia: the Cache County Dementia Progression Study. Am J Geriatr Psychiatry 2011; 19:532–542Crossref, MedlineGoogle Scholar

12 Folstein MF, Folstein SE, McHugh PR: “Mini-mental state”: a practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 1975; 12:189–198Crossref, MedlineGoogle Scholar

13 Hughes CP, Berg L, Danziger WL, et al.: A new clinical scale for the staging of dementia. Br J Psychiatry 1982; 140:566–572Crossref, MedlineGoogle Scholar

14 Cummings JL: The Neuropsychiatric Inventory: assessing psychopathology in dementia patients. Neurology 1997; 48(suppl 6):S10–S16Crossref, MedlineGoogle Scholar

15 Lyketsos CG, Galik E, Steele C, et al.: The General Medical Health Rating: a bedside global rating of medical comorbidity in patients with dementia. J Am Geriatr Soc 1999; 47:487–491Crossref, MedlineGoogle Scholar

16 Peters ME, Rosenberg PB, Steinberg M, et al.: Neuropsychiatric symptoms as risk factors for progression from CIND to dementia: the Cache County Study. Am J Geriatr Psychiatry 2013; 21:1116–1124Crossref, MedlineGoogle Scholar

17 Vilalta-Franch J, Calvó-Perxas L, Garre-Olmo J, et al.: Apathy syndrome in Alzheimer’s disease epidemiology: prevalence, incidence, persistence, and risk and mortality factors. J Alzheimers Dis 2013; 33:535–543Crossref, MedlineGoogle Scholar

18 Steinberg M, Lyketsos CG: Atypical antipsychotic use in patients with dementia: managing safety concerns. Am J Psychiatry 2012; 169:900–906LinkGoogle Scholar

19 Witlox J, Eurelings LS, de Jonghe JF, et al.: Delirium in elderly patients and the risk of postdischarge mortality, institutionalization, and dementia: a meta-analysis. JAMA 2010; 304:443–451Crossref, MedlineGoogle Scholar

20 Inouye SK, van Dyck CH, Alessi CA, et al.: Clarifying confusion: the confusion assessment method: a new method for detection of delirium. Ann Intern Med 1990; 113:941–948Crossref, MedlineGoogle Scholar