Suicide Prediction With Machine Learning

Case Vignette

“Mr. A” is a 35-year-old Caucasian veteran who just completed a tour of Iraq and returned stateside. He presents to the emergency department at the Veterans Affairs (VA) Hospital complaining of persistent nightmares, inability to go out at all, and inability to be normal around his children. He says he wants to die. He has no family history of suicide, and his wife has given away his guns. He is deemed to be a safety risk to himself and is admitted to the inpatient service. His symptoms are controlled on an optimal medication regimen, but a week after he leaves the hospital he discontinues his medications because they make him feel dull. At his outpatient follow-up appointment 2 weeks later, he reports feeling okay but endorses transient thoughts of dying, which become increasingly severe over the next few weeks. He expresses being at the end of the rope and wanting to die. He spends most of his time thinking about ways to kill himself, and one day calls the crisis line when his wife is away at work.

For anyone who has worked at the Veterans Affairs (VA), this situation described in the above case vignette may seem familiar. Despite all the psychotropic medications we have at our disposal, and despite our best efforts, as any mental health provider who has lost a patient to suicide knows all too well, there is no way to accurately predict what this veteran will do next.

Suicide is the 10th leading cause of death in the United States. A total of 41,149 people died by suicide in 2015. Suicide costs the health care industry $51 billion annually (1). Firearms are responsible for more than 50% of suicides, and middle-aged white men have the highest rates (1). Given these statistics combined with the numerous stressors associated with deployment and reintegration, it is perhaps unsurprising that suicide prevention is a top priority for the U.S. military.

Multiple studies have investigated factors correlated with completed suicide; a well-known strong predictor of completed suicide is a previous suicide attempt. A 2016 meta-analysis of longitudinal studies pointed out discrepancies between studies that have examined the influence of previous suicidal attempts on suicidal behavior (2). Individual studies report risk ranging from nonsignificant (3) to 40-fold (4) to 70-fold (5).

Clinical prediction rules are increasingly used to facilitate evidence-based decision making regarding diagnosis and treatment; in essence, a clinical prediction tool helps a clinician weigh the odds and arrive at an average predicted risk (6). Mental health prediction rules have been slower to develop than clinical prediction rules, such as the Wells Criteria (7) (to help assess risk of pulmonary embolism) or the CHADS2 score (8) (to help assess risk of stroke with atrial fibrillation). Efforts to validate specific scales to predict suicide risk have been undertaken, with studies evaluating the Columbia Suicide Severity Rating Scale, the Suicide Trigger Scale, and the Barwon Health Suicide Risk Assessment (9–11). Of these, the Columbia Suicide Severity Rating Scale is standardized for use in different populations, ranging from children to adults and veterans, and has reasonably good data for validity and reliability (12, 13).

Compounding the complexity of predicting suicide risk is the fact that 60% of deaths from suicide come about from the first suicide attempt, and a complex relationship exists between previous suicide attempts, current suicidal ideation, and lifetime suicide risk (2, 13, 14). Adding the high degree of variability of type of illness associated with suicidal ideation (major depressive disorder, posttraumatic stress disorder [PTSD], borderline personality), mental health clinicians currently have no evidence-based or systematic way of arriving at a composite risk score for each patient from all of the individual risk factors (14). To date, the identification of a biomarker or biomarkers to predict suicide risk has remained elusive, and there is no blood test to predict suicide risk (15). In summary, the ability of a mental health provider to predict an individual patient's suicide risk with any certainty is limited by lack of clinical prediction rules, a problem that is compounded and highlighted by individual diagnostic, psychosocial, and medical comorbidity.

It is at this point where machine learning can enter the landscape of psychiatry care. For the last 10 years, machine learning has made its foray into medical practice and biomedical applications and has facilitated the development of well-accepted clinical prediction rules (6). A machine-learning algorithm is a statistical technique that utilizes complex calculations to look at large data sets to predict factors or variables that can influence outcomes. Using variables that have been identified as significantly predictive, a software interface can be designed so that a provider in a hospital setting can ascertain the variables relevant to the patient being examined, input the variables into the system, and receive output in the form of a risk calculated through the available data. When set up properly, the information entered by providers can be harnessed to add to the data set and improve the accuracy of the clinical prediction rule.

Machine learning was initially used to build faster search engines like Google, for signal detection and for many other engineering feats. A recent article in JAMA highlights how machine learning is instrumental for health care in the 21st century (16). Studies have already shown the use of machine learning in risk stratification and outcome prediction in multiple medical and surgical specialties. It follows that machine learning may be a useful adjunct to the clinical assessment of suicide risk (14).

There are a few studies that have already used machine learning to predict suicidal risk in clinical and nonclinical settings. Querying PubMed with the MeSH terms “machine learning” and “suicide” and selecting studies that used clinical populations for assessment of suicidal risk, we chose two studies. Both studies applied machine learning to retrospective data sets encompassing clinical and demographic details of patients. The first study applied machine learning to predict suicide risk in a sample of outpatients with mood disorders and determined a sensitivity of 70% and specificity of 70% while finding previous hospitalization for major depressive disorder (using DSM-IV), a history of psychosis, cocaine dependence, and comorbid PTSD to be the strongest predictors of completed suicide risk (17).

The second study comes from the STARRS [Study to Assess Risk and Resilience in Service Members] project. This study utilized machine learning to predict suicide risk among 53,769 previously deployed soldiers and veterans after discharge from inpatient hospitalization from 2004 to 2009 (18). Variables considered included demographics, diagnoses (as distinguished by appropriate ICD-9 codes), assessment tool results, pharmacotherapy, psychotherapy (if any), and information about hospital course. The STARRS model was able to predict suicide risk with sensitivity and specificity each approaching 70% and identified male sex, late age at enlistment, criminal offenses, and presence of previous suicidal ideation as the strongest predictors of completed suicide.

Generation of a composite risk score for an individual patient using machine learning relies on a computational process based on patterns seen in previously analyzed data sets. Both the score and the clinical prediction rule used for generating the score need adjunctive clinical interpretation before assigning relevance to the score. A trained mental health provider knows that presence of comorbid substance use disorders and psychotic symptoms (elucidated as predictive of completed suicide in a study by Passos et al. [17]) increase suicidal risk in patients with major depressive disorder or PTSD. The benefit of a machine-learning approach permits the validation and strengthening of clinical prediction rules as numbers of inputs rise while, at the same time, facilitating more accurate triage of patients and more reliable assessment of suicide risk in cases in which the clinical situation seems ambiguous, as in the case of the patient in the above clinical vignette. As with any computer application, the technology is only as good as the information and programming that goes in to it, and misclassification or wrongful assignment of risk is possible. It is for this reason that adjunctive clinical assessment and ongoing modifications are necessary to optimize the utility of the strategy.

Another avenue for strengthening risk prediction is applying machine learning to biomarker data in conjunction with clinical assessment data. Numerous candidate biomarkers have been postulated for suicide (15, 19). Some of them include neurotransmitter systems (dopamine, norepinephrine, serotonin, GABA), cytokine levels, imaging biomarkers (e.g., PET, diffusion tensor imaging), and cortisol/HPA systems. An optimal biomarker should be unique to suicide and have good validity and reliability. It is possible that lack of an optimal biomarker speaks to the complex neurobiology of suicide. An ideal research goal would be to apply machine learning to databases comprising clinical data, as well as candidate biomarker data. This would result in being able to choose both clinical and biomarker variables with the highest capability of suicide risk prediction to generate a composite score for patients seen in the emergency department or inpatient/outpatient settings, irrespective of diagnoses. These endeavors highlight a future direction in psychiatry that will help reduce our margin of error in suicide prediction.

Key Points/Clinical Pearls