Introduction

Suicide is a major public health issue and is the second-most leading cause of death among adolescents and young adults in the United States [1]. According to the National Center for Health Statistics, the national suicide rate has increased year-over-year since 2007 although there has been a slight decline in 2019 and 2020 [2-4]. From 1999 to 2017, the age-adjusted suicide rate increased 33% from 10.5 to 14.0 per 100,000 [3]. Suicide rates among the younger age group have continued to increase or stayed the same even during 2019-2020 [2]. The rate of suicide attempt has increased at an alarming rate over the last decade due to several factors such as lack of an effective national strategy and slow movement from research to action [4]. The historical trend documenting increase in the suicide rates, particularly among adolescents and younger adults, calls for an urgent and dire need to respond and address the needs of individuals who are at risk [5].

 As one of the most preventable causes of death, suicide is a very complex problem affecting individuals, families, and our communities at large. To target and support the individuals who are most at risk, it is important to understand the specific factors that increase an individual’s risk for suicide [5,6].

While we cannot point to a single factor for the increased rate of suicide over the last decade, the cause of suicide can be attributed to several factors ranging from lack of access to affordable mental health services to lack of a national strategy that addresses the multiple risk factors involved in increasing an individual’s risk for suicide [5]. According to National Institute of Mental Health and the National Action Alliance for Suicide Prevention, the key areas where we need improvement are risk identification, intervention, and comprehensive follow-up care to address any unmet needs of the individuals [5,6].

Literature review has shown that other studies using predictive modeling approach have used varied sources of data such as electronic health records, self-reported data, or social media data to identify high-risk suicidal behavior [12-14]. We incorporated a broad range of independent variables, specifically social drivers of health variables, in our model to create more meaningful insights and get a better picture of an individual’s overall health, which cannot be captured, solely through medical/pharmacy claims.

Methods

Analytics

We pulled claims data for our commercial members aged 12-26 years. Members with mental health diagnoses or those who have had any substance abuse event and/or members who had a prescription for antidepressants, antipsychotics, and/or mood stabilizers were included in the model.

The look-back period for most severe types of events such as prior suicide attempts, and overdoses was 24 months. For events such as prescription utilization patterns, inpatient admissions, ED utilization, and other diagnosis and utilization categories, the look-back period was 12 months. Outcome metrics were looked at 30 days to 395 days post-identification

Predictor variables

Our robust model included over 300 independent variables in several categories such as demographics, physical/behavioral health diagnoses, social drivers of health, abnormal labs, drug fill count, health care utilization, and dosage.

For each of the categories above, the specific variables included in each of the categories are listed as follows:

a) Demographics: Age, gender

b) Prior Physical Health Diagnoses: Diabetes, Asthma, Headache, Pain, Liver disease, Migraines, Obesity, Renal disease, pulmonary disease, and Osteomyelitis

c) Prior Mental Health Diagnoses: Mood, Bipolar, Anxiety, Psychosis, Personality Disorder, and substance use disorders. As well as Suicidal Ideation and prior Suicide Attempts.

d) Abnormal labs: Anemia, Diabetes, Infection, Kidney, Liver. Member testing positive for any of the controlled substances such as Opioids, Stimulants, or Benzodiazepines.

e) Drug fill count (by class and ingredient): Antidepressants, Antipsychotics, Mood Stabilizers, Muscle Relaxers, Migraine medications, Stimulants, Opioids, Sedatives, and Hypnotics.

f) Social Drivers of Health (SDOH): Known prior physical abuse, sexual abuse, exploitation, neglect, and/or maltreatment. Known work problems, social problems, familial instability, involvement with child welfare, economic problems, educational problems, homelessness, and food insecurity.

g) Center for Disease Control’s (CDC) Social Vulnerability Index (SVI) variables: Based on the member’s address that was then mapped to a census tract, we tested all CDC Social SVI metrics for potential inclusion in the model.

h) Utilization: ER Admissions by severity level, Inpatient admits (acute and subacute), Ambulatory visits (Mental Health, Substance Abuse, Other), PCP visits, Prior Psychotherapy, Psychiatric Assessment

i) Dose: Avg. Morphine Milligram Equivalents (MME) per day, Avg. Diazepam Milligram Equivalents (DME) per day, and Avg. Amphetamine Milligram Equivalents (AME) per day.

A member’s opiates prescription gets converted to Morphine Milligram Equivalents (MME) and the total MME is divided by the days’ supply gives the average opioid use per day. Similarly, a member’s Diazepam milligram equivalent total divided by the days’ supply gives the Average Benzo Usage per day. The same method is used for stimulants as well to convert to AME.

In addition to the SDOH data elements such as the Z-codes on claims data, we also incorporated CDC’s SVI that uses census level data to make a number of social driver variable estimates for every US census tract [11]. The CDC’s SVI is a publicly available online tool, and its purpose is to help identify and map communities that will require support in preparing and responding to disasters.

The SVI indicates the relative vulnerability of every U.S. Census tract or subdivisions of counties for which the Census collects statistical data. SVI ranks the tracts on 15 social factors, including unemployment, minority status, and disability, and further groups them into four related themes. Thus, each tract receives a ranking for each census variable and for each of the four themes, as well as an overall ranking. We assigned members to census tracts in order to derive their social vulnerability index variables.

We tested both the base variables and interaction variables for their significance. Lasso regression was used to test for potential interaction between the co-variates for two and three-way interaction terms.

A suicidal event was defined as a member with any of the following: member having a first/repeat suicide attempt, an ER Level 4 or 5 visit with a diagnosis of suicidal ideation, or an Inpatient Acute admission with a diagnosis of suicidal ideation. In an early pass at the modeling, it was observed that many of the members who received high risk scores but did not appear to have actual suicide attempts in the subsequent prediction window did often have inpatient admissions with a suicidal ideation diagnosis or a high intensity ER visit with a suicidal ideation diagnosis. Subsequent consultation with clinical partners led to the decision that these sorts of events should be treated as true positives rather than false positives as the planned intervention would aim to not only reduce actual suicide attempts but also high intensity injurious ideation that would lead to admissions and high-severity ER visits with that diagnosis. Therefore, rather than focusing the model on the prediction of suicide attempts alone, the target event-class was expanded to include inpatient admissions and high-intensity ER visits with the same diagnosis. All subsequent modeling was then focused on this broader group of events which we labeled “suicidal events.” Referral to intervention was then based on a member’s risk of suicidal events in the subsequent 12 months of 10% or greater.

It is also important to note that the date window for target event prediction was 1 month to 13 months after the date of the risk score calculation. We aimed to predict events in this window so that our clinicians would have time to both receive the risk model referrals, outreach, and engage the identified individuals hopefully before they entered the time period in which the suicidal event, they were predicted to have a 10% (or greater) risk of was predicted to occur.

In our dataset, there were a total of 6,188,641 observations with a unique member count of 1,516,023. Since members were, therefore, in the observation set multiple times due to calculations of prior predictors and subsequent outcomes on four separate dates, care was taken to ensure that members were either wholly in the training or wholly in the validation datasets.

Statistical methods

We used SAS Enterprise Guide 7.1 for our statistical analyses. A two-step regression was conducted to identify significant risk factors for suicidal events and to develop a model to predict the likelihood of a member having a suicidal event in the next 12 months. The dataset was split into training and validation sets with 60% of members in training while the remaining 40% were in the validation set.

In the first step, a multivariate regression was performed on the continuous dependent variable and the set of predictor variables. A custom intensity metric dependent variable was built that increases with the paid claim amount (as a proxy for the intensity of an event) associated with suicide attempts, inpatient acute claims with suicidal ideation, and ER Level 4 or 5 with suicidal ideation; and increases as the subsequent suicidal event occurred more nearly in the future. With more intense and nearer in the future events having higher outcome metrics in this regression step, the model resulting from this first phase was then considering more intense outcomes more heavily.

In the second step, a logistic regression with a binary outcome variable was conducted based on the resulting risk score from the first phase regression as well as the underlying predictors to the first phase regression. The regression coefficients were used to calculate a risk score for each member and categorize the members into three distinct categories of low-moderate risk, high risk, and critical risk based on the rate of suicidal event. We generated scatterplots to examine the relationship between risk scores and outcome metrics such as the subsequent rate of suicidal event.

We built two separate models for first attempt vs. repeat-attempt members as those with a prior attempt are inherently at a higher risk for subsequent suicidal event compared to those who have never had a previous suicide attempt. In order to evaluate the predictive power of the model, we used AUC (area under the curve) in the Receiver Operating Curve (ROC) plot to measure the predictive power of the model. It is a plot of the true positive rate (sensitivity) against the false positive rate (1-specificity) for the different possible thresholds. The closer AUC is to 1.0, the better [10].

Intervention

With our predictive modeling approach, we were better able to understand which specific risk factors increase an individual’s likelihood for attempting suicide and aimed to target those individuals through a multi-dimensional approach involving various members of the health care team.

A centralized team providing telephonic case management focusing on safety planning, risk reduction, and family support was formed. Telephonic peer support worked to create connection to community resources and supports and provide afterhours access to Behavioral Health Resource Center.

Case managers had the opportunity to bring risky or complicated patient scenarios to clinical rounds, which included the Suicide Prevention Team and a psychiatrist to collaborate regarding resources and other interventions that might help the member.

The goal was to offset the adolescent’s suicidal impulsivity, create safety, and provide 24/7 crisis support. We got parental consent and youth consent for participation for minors and provided a telephonic care manager and/or peer support partner to the parent(s) if applicable to help to decrease parental anxiety and stress and to provide coaching on threat assessment, means reduction, appropriate boundaries, and provide related psychosocial support and skills training.

Using the model results, we identified and targeted approximately 10,000 members within our commercial fully insured population who had at least 10% risk of a suicidal event and those who were not in any existing case management program. Members who agreed to participate in our program had weekly touchpoints and were followed for the next 6 months. Participants were given the option to opt out of the program at any point in time. To measure the efficacy of the program, we compared the total medical spend and healthcare utilization of the members in the engaged vs. the matched control group 12 months pre and post intervention.

Results

Model building

The top predictor variables for both the first attempt and repeat-attempt model are summarized in Table 1 based on our regression output.

Table 1: Top Risk Factors for the suicide predictive model^*.

Model Validation

The AUC for the training and the validation dataset for the first-attempt model was 0.79 while that for the repeat-attempt model was 0.76 (Figures 1 and 2).

 

Figure 1: Receiver Operating curve (ROC): First-attempt.

 

Figure 2: Receiver Operating curve (ROC): Repeat –attempt.

An AUC between 0.7 and 0.8 is considered moderately accurate and with both the training and validation, datasets having an AUC that is close further suggests that our model works well on scoring new datasets. To test the positive predictive value of our model, we plotted a scatterplot to show the relationship between risk score and rates of suicidal events (Figure 3). As seen in the graph below, as the risk score increases, the rate of suicidal events comprising of actual suicide attempts, high-intensity ER visits, inpatient acute admissions with suicidal ideation diagnosis also goes up substantially.

 

Figure 3: Risk score vs. Average Suicide Event Rate.

We also found that higher risk scores correlated to members having suicidal events sooner with an average time of five months to first suicidal event. Table 2 summarizes the demographic and risk profile of members who fall in the critical risk category.

Table 2: Sample characteristics of critical risk members.

Program evaluation

To evaluate the effectiveness of the case management program, we compared the twelve-month pre and post outcome metrics related to the total cost of care and the rate of suicidal event between the control group and the intervention group. The control group was comprised of members who were identified by the predictive model but either had inaccurate contact information on file or had benefit packages that meant they were not yet eligible for the program. The intervention group consisted of members who actively engaged in the program for at least 7 days and had at least twelve months of enrollment before and after engagement. Both the control and intervention groups were matched on risk score, demographics, clinical profile, and spend patterns to ensure accuracy of the study results.

There were 1,386 unique members in the enrollment file. Of those, certain members were excluded for reasons as follows: a) members outside the age range of 12-26 years b) engagement date outside the outcomes time frame c) members with extreme medical spend patterns d) members who had less than six months of enrollment post-engagement. After applying the above exclusions, 894 members were included in the final analysis.

In the no prior attempt group, there were 554 members in the control group while 562 members in the engaged group. Among those in the matched control, the rate of suicidal events from the twelve months prior to potential referral to the twelve months post potential referral fell by 25.81% (95% Margin of Error (MOE): 3.64%); for those engaged in the program 7 or more days, their rate change fell by 42.53% (95% MOE: 4.09%); indicating a net intervention effect of 16.71%. Similarly, the rates of high-intensity ER visits and inpatient acute admissions with a diagnosis of suicidal ideation had a net decrease of 10.11% and 9.55%, respectively in the engaged group.

In the prior attempt group, there were 340 members in the control group and 332 members in the engaged group. The 12-month pre/post net decrease in suicidal events was 27.06% (95% MOE: 4.72%) in the control group while 54.82% (95% MOE: 5.32%) in the engaged group with a net intervention effect of 27.76%. The biggest improvement was seen in the actual suicide attempts going from a prior 12-month rate of nearly 80% to a post 12-month rate of 14%, a nearly 66% drop (95% MOE: 5.00%) in the engaged group. Those in the matched control group went from 63% to 30%, a drop of 33% (95% MOE: 5.11%) with a net effect of nearly 33%. At 95% CI, our results show statistically significant differences in the suicidal events, including suicide attempts, between the engaged and the matched control group.

Discussion

Our study results make a strong contribution to help us better understand the specific risk factors for first time and repeated suicidal events. The variables included in the model not only look at claims/pharmacy data but also look at social factors giving a more holistic perspective on an individual’s overall mental health. Our intervention tailored to address specific needs of high-risk individuals has shown to be very effective in reducing suicide attempts and healthcare utilization, especially inpatient admissions, and emergency room visits. The results of our study will help the research community and healthcare practitioners transform the disturbing trend of increased suicidal rates in the youth population through early identification, education, crisis support and encouragement to acknowledge the problem and seek help from various resources.

Prior studies in hospitals and community-based settings focusing on implementing strategies for suicide prevention have shown promising results with reduction in suicide attempts in the intervention group [7-9]. Our results corroborate with prior findings and further strengthen the consensus that individuals with high risk for suicide need proper care coordination with strong support from families and mental health professionals to address their needs and overall well-being.

Since our study included only members who have prior diagnosis of mental health conditions or those with a prescription for drugs known to treat specific mental health disorders, the results of our study may not be generalizable to the population at large. Additionally, we cannot establish a cause-and-effect relationship as the members in our study were not randomly assigned to a control or intervention group.

Conclusion

Despite the limitations, our study forms a strong basis for implementing comprehensive case management programs aiming to not only reduce suicide attempts but also decrease healthcare utilization and costs. Our study shows that those individuals at high risk could significantly benefit from regular engagement with professionals who can work with them closely and connect them with resources to improve their mental health and further alleviate their risk for future negative health outcomes. Furthermore, our cutting-edge algorithm is industry leading and offers intervention based on risk score.

Acknowledgements

This project was supported in its entirety by Elevance Health, Inc. as part of Elevance Health’s usual business, which includes quality improvement and cost of care initiatives.

Authors' Contributions

BF was the lead analyst who built the main framework for the predictive model and performed the program evaluation. SK, along with BF, built the predictive model and collaborated with clinical/business leaders to implement the project. SK was a major contributor in writing the manuscript. RB designed the main SQL server database used to build the predictive model. JC was the owner and clinical lead for the described intervention. She oversaw the project in its entirety from ideation through implementation and post-evaluation. All the authors read and provided feedback on the manuscript before finalizing and approving it.

Ethical Guidelines

Although we did not conduct a randomized clinical trial, we confirm that all methods were carried out in accordance with relevant guidelines and regulations. We obtained informed consent from all participants. Additionally, we got parental consent and youth consent for participation for minors. Members who agreed to participate in our program were enrolled in the program and followed for the next 6 months. Participants were also given the option to opt out of the program at any point in time. This publication has been reviewed and approved by the internal review committee at Elevance Health.

Availability of Data and Materials

The data that support the findings of this study are available from the Elevance Health Sandbox, but restrictions apply to the availability of those data and so are not publicly available. Data is available from the corresponding author upon reasonable request and with permission of Elevance Health, Inc.

Conflicts of Interest

The authors declare that they have no conflicts of interests (i.e., financial, and non-financial).

References

1. National Center for Health Statistics (2021) About multiple cause of death, 1999-2020. CDC WONDER online database.
2. Ehlman DC, Yard E, Stone DM, Jones CM, Mack KA (2022) Changes in Suicide Rates — United States, 2019 and 2020. MMWR Morb Mortal Wkly Rep 71: 306-312.
3. Hedegaard H, Curtin SC, Warner M (2018) Suicide mortality in the United States, 1999-2017. NCHS Data Brief, no 330: 1-8.
4. Hedegaard H, Curtin SC, Warner M (2021) Suicide mortality in the United States, 1999-2019. NCHS Data Brief no 398: 1-8.
5. Weir K (2019) Worrying trends in U.S. suicide rates. American Psychological Association 50: 24.
6. The Surgeon General's Call to Action on Suicide Prevention. A Report of the U.S. Surgeon General and of the National Action Alliance for Suicide Prevention.
7. Chen WJ, Chen CC, Ho CK, Lee MB, Lin GG, et al. (2012) Community-based case management for the prevention of suicide reattempts in Kaohsiung, Taiwan. Community Ment Health J 48: 786-791.
8. Kim MH, Lee J, Noh H, Hong JP, Kim H, et al. (2020) Effectiveness of a Flexible and Continuous Case Management Program for Suicide Attempters. Int J Environ Res Public Health 17: 2599.
9. Norimoto K, Ikeshita K, Kishimoto T, Okuchi K, Yonemoto N, et al. (2020) Effect of assertive case management intervention on suicide attempters with comorbid Axis I and II psychiatric diagnoses: secondary analysis of a randomised controlled trial. BMC Psychiatry 20: 311.
10. Hanley JA, McNeil BJ (1982) The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143: 29-36.
11. Centers for Disease Control and Prevention/Agency for Toxic Substances and Disease Registry/Geospatial Research, Analysis, and Services Program. CDC/ATSDR Social Vulnerability Index 2018 Database US   Accessed on Sep 9^th 2021.
12. Nock MK, Millner AJ, Ross EL, Kennedy CJ, Al-Suwaidi M, et al. (2022) Prediction of Suicide Attempts Using Clinician Assessment, Patient Self-report, and Electronic Health Records. JAMA Netw Open 5: e2144373.
13. Castillo-Sánchez G, Marques G, Dorronzoro E, Rivera-Romero O, Franco-Martin M, et al. (2020) Suicide Risk Assessment Using Machine Learning and Social Networks: a Scoping Review. J Med Syst 44: 205.
14. Su C, Aseltine R, Doshi R, Chen K, Rogers SC, et al. (2020) Machine learning for suicide risk prediction in children and adolescents with electronic health records. Transl Psychiatry 10: 413.