THE EFFECTS OF SUICIDE PREVENTION GATEKEEPER TRAINING ON BEHAVIORAL INTENTION, AND INTERVENTION BEHAVIOR: A SYSTEMATIC REVIEW AND META-ANALYSIS by SARAH G. SPAFFORD A DISSERTATION Presented to the Department of Counseling Psychology and Human Services and the Division of Graduate Studies of the University of Oregon in partial fulfillment of the requirements for the degree of Doctor of Philosophy March 2023 DISSERTATION APPROVAL PAGE Student: Sarah G. Spafford Title: The Effects of Suicide Prevention Gatekeeper Training on Behavioral Intention and Intervention Behavior: A Systematic Review and Meta-Analysis This dissertation has been accepted and approved in partial fulfillment of the requirements for the Doctor of Philosophy degree in the Counseling Psychology and Human Services by: John R. Seeley, PhD Chairperson Emily E. Tanner-Smith, PhD Core Member James R. Muruthi, PhD Core Member Geovanna Rodriguez, PhD Institutional Representative and Krista Chronister Vice Provost for Graduate Studies Original approval signatures are on file with the University of Oregon Division of Graduate Studies. Degree awarded March 2023 ii © 2023 Sarah G. Spafford This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs (United States) License iii DISSERTATION ABSTRACT Sarah G. Spafford Doctor of Philosophy Department of Counseling Psychology and Human Services March 2023 Title: The Effects of Suicide Prevention Gatekeeper Training on Behavioral Intention and Intervention Behavior: A Systematic Review and Meta-Analysis Suicide is a major public health concern worldwide and the tenth leading cause of death for all ages in the United States. As suicidal ideation and suicide behaviors have continued to grow in the U.S., this has encouraged national and statewide efforts to implement comprehensive suicide prevention plans. A critical component of a comprehensive plan is ensuring individuals experiencing suicidal ideation receive mental health treatment. Suicide prevention gatekeeper trainings, which aim to improve knowledge regarding suicide risk as well as increase gatekeepers’ intentions to ask about suicidality and make referrals to appropriate mental health treatment, are a frequently implemented program to increase mental health help-seeking behaviors, despite the lack of evidence around effective implementation. To understand the current state of research regarding suicide prevention gatekeeper training, a systematic review and meta-analysis were conducted that synthesized evidence on the effects of suicide prevention gatekeeper training on behavioral intention to intervene and suicide intervention behaviors and whether these changes are maintained over time. A secondary aim of this meta-analysis was to examine implementation setting, training modality, and training level as moderators for the effectiveness of suicide prevention gatekeeper training on behavioral intention, and suicide intervention behaviors. Results from 43 studies revealed that iv suicide prevention gatekeeper training had an overall positive effect on behavioral intention and intervention behaviors. Furthermore, subgroup analyses for training level and training modality were unable to be conducted due to the lack of studies examining advanced or online trainings. Results from the implementation setting subgroup analysis revealed no significant differences in behavioral intention or intervention behavior based on the implementation setting. However, these results must be considered with caution as there were significant methodological concerns of the included studies and limited studies that conducted long-term follow-up. Although this meta-analysis reveals an overall positive effect for suicide prevention gatekeeper training on behavioral outcomes, the low methodological quality of the current available evidence limits the ability to draw conclusions from the synthesis. To inform policymakers and interventionists on best practices for suicide prevention gatekeeper training, additional rigorous research is needed. v CURRICULUM VITAE NAME OF AUTHOR: Sarah G. Spafford GRADUATE AND UNDERGRADUATE SCHOOLS ATTENDED: University of Oregon, Eugene Griffith University, Brisbane, Queensland, Australia Azusa Pacific University, Azusa, California DEGREES AWARDED: Doctor of Philosophy, Prevention Science, 2023, University of Oregon Master of Suicidology, Suicidology, 2018, Griffith University Bachelor of Arts, Psychology, 2013, Azusa Pacific University AREAS OF SPECIAL INTEREST: Suicide Prevention Mental Health and Wellbeing Older Adult Mental Health Mental Health Workforce PROFESSIONAL EXPERIENCE: Independent Evaluator, Department of Counseling Psychology and Human Services, 2020-Present Graduate Research Assistant, Department of Counseling Psychology and Human Services, 2019-Present Research Consultant, Australian Institute for Suicide Research and Prevention, 2018-Present Course Reviewer, Australian Institute for Suicide Research and Prevention, 2020-2021 Intentional Injury Prevention Coordinator, Wyoming Department of Health, 2017- 2019 Graduate Research Assistant, Australian Institute for Suicide Research and Prevention, 2017-2018 vi GRANTS, AWARDS, AND HONORS: Society for Prevention Research Early Career Travel Award, 2022 SOPHE/CDC Student Fellowship in Injury Prevention, Mental Wellbeing & Resilience in Suicide Prevention Crisis Line Volunteers, Society for Public Health Education, 2020 Raymund Fellowship, University of Oregon, 2019 Senior Research Award, Azusa Pacific University, 2013 PUBLICATIONS: Muruthi, J., Spafford, S., & Osa, M. L., DeLouize, A., Biritwum, R., Naidoo, N., Kowal, P., & Snodgrass, J. J. (2021). Suicidal ideation in older Ghanaian women: The roles of bodily pain, functional limitation, and chronic health conditions. International Journal of Social Psychiatry. Advanced online publication. https://doi.org/10.1177/0020764021991179 Metcalfe, R. E., Spafford, S. G. & Ziring, A. (2021). Calling in for care: Best practices for telehealth administrations of adolescent suicide assessments. The Oregon Psychologist. Advanced Online Publication. https://doi.org/10.13140/RG.2.2.32812.80002 Hawgood, J., Spafford, S. G., Evans, A., Webster, A., & Koo, Y. W. (2022). Suicide risk assessment fears, attitudes and behaviours of lifeline crisis supporters. Health & Social Care in the Community, 00, 1– 12. https://doi.org/10.1111/hsc.13923 vii ACKNOWLEDGMENTS They say it takes a village to raise a child–I’d say this also applies to finishing a PhD. First and foremost, I would like to express my gratitude to my advisor, Dr. John Seeley, as he has pushed me to deepen my understanding of prevention science and provided endless opportunities for me to grow personally and professionally. I would not have made it through this program or dissertation without your mentorship and continued support, especially within the past year. Additionally, I would like to thank Dr. Emily Tanner-Smith for her input and support throughout this dissertation work. I have learned so much from you, not only how to excel in research, but also in how to create a balance between the valuable work being done in academia and at home. I am deeply grateful for your encouragement and detailed feedback. Thank you, also, to Dr. James Muruthi for your friendship and mentorship over my graduate school career. I have appreciated your perspective, encouragement, and support, especially in serving on my committee. Finally, thank you to Dr. Geovanna Rodriguez for your support and encouragement in serving on my committee. I appreciate the time and effort each of you have put into this project. Also, Marielena, a special shoutout to you for the countless hours that you put into this project. You dove in headfirst as my second coder, not knowing what you were getting into, yet were always encouraging and excited to continue working. This project truly would not have been possible without your love, support, and dedication. I would not have survived these years without the continued love and support of my grad school friends. A special thank you to Mavis, for always being there for me and to remind me that while this is hard, we will make it; Lauren, for the many times you took my panicked call about meta-analysis, our gymnastics chats, and all the Panera runs; viii Aubrey, for your continued encouragement and positivity in the most trying times; Jean, for the many times you fed me and made sure I had the support I needed to make it through, and Maggie, for always being there when I needed a reminder that I wasn’t allowed to quit and that we were all in this together. Also, a huge thank you to all my friends and family (if only I had room to name each of you individually) that have understood my lengthy disappearances in the past few years but have still stood by me. I would not have been able to survive this without you. And Juliana, thank you, especially, for commiserating with me and taking the time read this lengthy document to make sure no wild grammar mistakes were hanging on. Lastly are my two biggest supporters in life. Mom, you have been an incredible support to me throughout my educational career and have shown me that nothing is impossible if you set your mind to it. Your inquisitive nature always encouraged me to ask big questions and seek big answers. Your kindhearted spirit and servant heart has inspired me to engage in work that finds ways to better the lives of people in our communities near and far. I hope to follow the legacy you have been writing of loving others well in all you do. My dearest and deepest love, Kevin, you have stood by me through so many ups and downs of this program with nothing but love, patience, and support. Although these have been some of the most challenging years so far, we have only grown closer and deeper in love. This degree would not have been possible without your commitment and sacrifice to my success. Thank you for being my sounding board, my safe space, and my biggest cheerleader even when I wanted to quit. There are no words to describe my gratitude for you. ix To all my friends, family, colleagues, and loved ones who made this journey possible; those still with us and those deeply missed. x TABLE OF CONTENTS Chapter Page I. INTRODUCTION .................................................................................................... 1 Suicide Prevention Gatekeeper Training ............................................................... 2 Types of Gatekeeper Training ............................................................................... 3 Theoretical Foundations of Gatekeeper Training .................................................. 4 Gatekeeper Training Outcomes ............................................................................. 8 Knowledge About Suicide ............................................................................... 8 Beliefs and Attitudes About Suicide Prevention ............................................. 9 Self-Efficacy to Intervene ................................................................................ 9 Behavioral Intention ........................................................................................ 10 Suicide Intervention Behaviors ....................................................................... 10 Implementation of Gatekeeper Training ............................................................... 11 Rationale for a Meta-Analysis ............................................................................... 12 Objectives .............................................................................................................. 14 Research Questions ............................................................................................... 15 II. METHODS ............................................................................................................. 16 Eligibility Criteria .................................................................................................. 16 Types of Studies .............................................................................................. 16 Types of Participants ....................................................................................... 16 Types of Interventions ..................................................................................... 17 Types of Outcome Measures ........................................................................... 17 Search Strategy ...................................................................................................... 18 xi Chapter Page Electronic Searches ......................................................................................... 18 Searching Other Resources .............................................................................. 19 Data Collection and Analysis ................................................................................ 19 Selection of Studies ......................................................................................... 19 Data Collection Process ................................................................................... 19 Assessment of Risk of Bias and Study Quality in Included Studies ............... 20 Effect Size Measures ....................................................................................... 20 Assessment of Heterogeneity .......................................................................... 22 Assessment of Publication/Small Study Bias .................................................. 22 Data Synthesis ................................................................................................. 22 Subgroup Analysis and Investigation of Heterogeneity .................................. 22 Registration Information ................................................................................ 23 III. RESULTS .............................................................................................................. 24 Search Results ....................................................................................................... 24 Results of Meta-Analyses ..................................................................................... 27 Behavioral Intention ........................................................................................ 27 Pre-Training to Post-Training .................................................................... 28 Pre-Training to Short-Term Follow-Up .................................................... 30 Pre-Training to Long-Term Follow-Up .................................................... 32 Intervention Behavior ...................................................................................... 33 Pre-Training to Short-Term Follow-Up .................................................... 34 Pre-Training to Long-Term Follow-Up .................................................... 35 xii Chapter Page Sensitivity Analyses ........................................................................................ 37 Subgroup Analyses .......................................................................................... 38 Training Level ........................................................................................... 38 Training Modality ..................................................................................... 39 Implementation Setting ............................................................................ 39 Behavioral Intention, Pre- to Post-Training ....................................... 39 Behavioral Intention, Pre-Training to Short-Term Follow-Up ........... 40 Intervention Behavior, Pre-Training to Short-Term Follow-Up ......... 41 IV. DISCUSSION ....................................................................................................... 43 Summary of Evidence ........................................................................................... 43 Limitations ............................................................................................................ 45 Implications ........................................................................................................... 49 Practice and Policy Implications .................................................................... 49 Research Implications .................................................................................... 51 Conclusion ............................................................................................................. 52 APPENDICES ............................................................................................................. 53 A. CODING MANUAL ........................................................................................ 53 B. RISK OF BIAS CODING CRITERIA ............................................................. 63 C. TABLE OF SUPPLEMENTARY REPORTS .................................................. 64 D. TABLE OF INCLUDED STUDIES ................................................................ 65 E. RISK OF BIAS TABLE ................................................................................... 76 F. ADDITIONAL PUBLICATION/SMALL STUDY BIAS FIGURES ............. 78 xiii Chapter Page G. SENSITIVITY ANALYSES ............................................................................ 81 REFERENCES CITED ............................................................................................... 83 xiv LIST OF FIGURES Figure Page 1. An integrated theory of suicide prevention gatekeeper training ............................ 6 2. PRISMA flow diagram adapted from Page et al. (2021) ...................................... 25 3. Forest plot for pre-post training behavioral intention meta-analysis ..................... 29 4. Funnel plot for pre-post training behavioral intention meta-analysis ................... 30 5. Forest plot for pre- to short-term follow-up behavioral intention analysis ........... 31 6. Funnel plot for pre- to short-term follow-up behavioral intention analysis .......... 33 7. Forest plot for pre- to short-term follow-up intervention behavior analysis ......... 35 8. Funnel plot for pre- to short-term follow-up intervention behavior analysis ........ 36 9. Forest plot for pre- to long-term follow-up intervention behavior analysis .......... 37 10. Forest plot for implementation setting subgroup analysis for pre- to post- Training behavioral intention outcome ................................................................. 40 11. Forest plot for implementation setting subgroup analysis for pre-training to short-term follow-up behavioral intention outcome .............................................. 41 12. Forest plot for implementation setting subgroup analysis for pre-training to short-term follow-up intervention behavior outcome ............................................ 42 xv LIST OF TABLES Table Page 1. Features of included studies (k = 43) .................................................................... 26 xvi CHAPTER I INTRODUCTION Suicide is a global public health concern causing nearly 800,000 deaths each year (World Health Organization, 2019). In the United States, suicide rates have increased 35% between the years 1999 and 2018, making it the tenth leading cause of death for all ages (Hedegaard et al., 2020). While the rising rates of suicide alone cause alarm for public and mental health professionals alike, these numbers fail to account for the increasing number who suffer from suicidal ideation or attempt suicide each year. Recent data for the United States estimates that nearly 5% of adults over the age of 18 have experienced serious suicidal ideation within the past year (Substance Abuse and Mental Health Services Administration [SAMHSA], 2021), a rate that has been increasing for over a decade (SAMHSA, 2019). The concerning growth of suicide and the prevalence of suicidal ideation in the United States has encouraged national and statewide efforts to implement comprehensive suicide prevention plans. According to the Suicide Prevention Resource Center (SPRC; 2020a), a leading organization for technical assistance in the field of suicide prevention, an effective and comprehensive suicide prevention plan targets nine elements: (a) identify and assist persons at risk, (b) increase help-seeking, (c) ensure access to effective mental health and suicide care treatment, (d) support safe care transitions and create organizational linkages, (e) respond effectively to individuals in crisis, (f) provide for immediate and long-term postvention, (g) reduce access to means of suicide, (h) enhance life skills, and (i) promote social connectedness and support. Within this guidance, a critical component of suicide prevention for individuals experiencing suicidal ideation is 1 to seek appropriate mental health treatment. However, research indicates that a significant barrier to pursuing professional help is the stigma associated with seeking mental health treatment (Clement et al., 2015; Corrigan, 2004; Henderson et al., 2013). One effective strategy for increasing the likelihood of mental health service utilization is through referrals to treatment from family and friends (Han et al., 2018). As individuals in crisis often seek help or advice from their social networks prior to seeking professional help (Wong et al., 2014), community members play a critical role in supporting those experiencing suicidal thoughts and encouraging help-seeking behavior. Suicide Prevention Gatekeeper Training A common approach to increasing mental health help-seeking and suicide awareness is to provide suicide prevention gatekeeper trainings, which aim to improve knowledge regarding suicide risk as well as increase gatekeepers’ intentions to ask about suicidality and make referrals to appropriate mental health treatment (Wyman et al., 2008). In the context of suicide prevention, those who make referrals for individuals experiencing a mental health concern are often called “gatekeepers.” A “gatekeeper” refers to an individual who has contact with a large number of community members, including those who may be experiencing suicidal thoughts, and is able to identify suicide risk, refer to mental health treatment, develop a suicide safety plan, or conduct brief suicide intervention (Isaac et al., 2009; U.S. Department of Health and Human Services (HHS) Office of the Surgeon General & National Action Alliance for Suicide Prevention, 2012). Individuals may be gatekeepers in their personal or professional lives and can be conceptualized into two categories: “designated” or “emergent” gatekeepers (Isaac et al., 2009; Ramsay et al., 1990). Designated gatekeepers are those trained and educated in a 2 professional role that provides care and support to suicidal individuals, such as mental health professionals or physicians. On the other hand, emergent gatekeepers are community members who have not been specifically trained to provide support to those experiencing suicidal ideation but emerge as direct supports for individuals with suicidal thoughts, such as clergy members, teachers, or coaches (Isaac et al., 2009). Types of Gatekeeper Training Although suicide prevention gatekeeper training is often discussed as a homogenous intervention, gatekeeper trainings vary widely in their target audience, length, and depth of content related to suicide prevention and intervention. Gatekeeper trainings also vary by training modality, with some trainings utilizing online platforms (e.g., Youth SAVE) and others offering in-person trainings (e.g., ASIST, SafeTalk). Osteen and colleagues (2014) highlight the differing training needs for designated and emergent gatekeepers suggesting that designated gatekeepers need a higher level of training that focuses on screening, brief intervention, and risk management practices. In comparison, emergent gatekeepers need more basic skills such as knowledge of suicide warning signs and community resources, communication skills with a person experiencing suicidal thoughts, and self-efficacy to put the skills into practice. In line with Osteen and colleagues’ (2014) argument, gatekeeper trainings can be conceptualized into two categories targeting basic or advanced gatekeeping skills. Basic gatekeeper training skills typically focus on increasing an individual’s knowledge of recognizing warning signs of suicide, how to ask a person if they are experiencing thoughts of suicide, and how to refer an individual with thoughts of suicide to a mental health professional (Osteen et al., 2014). These trainings may be more appropriate for 3 emergent gatekeepers without professional experience in working with individuals with suicidal thoughts. A common example of training focused on basic gatekeeping skills is Question, Persuade, Refer (QPR; Quinnett, n.d.), which is a 1½ to 2-hour training that addresses myths around suicide, asking an individual whether they are having thoughts of suicide, and referring an individual with thoughts of suicide to mental health treatment. Additional trainings in this category may include LivingWorks Start (LivingWorks Education Inc., n.d.), and LivingWorks SafeTALK (Turley, 2018). Advanced gatekeeper trainings build on basic gatekeeper training skills to focus on equipping individuals to provide brief suicide interventions and conduct safety planning with individuals with suicidal thoughts (Osteen et al., 2014). These trainings may be more commonly implemented in settings with higher levels of designated gatekeepers (e.g., healthcare settings, mental health clinics) but remain accessible to emergent gatekeepers. An advanced training example is the Applied Suicide Intervention Skills Training (ASIST; Rodgers, 2010), which is a two-day, 15-hour workshop that addresses attitudes toward suicide, suicide intervention skills, and safety planning with a person with thoughts of suicide. An additional example of advanced gatekeeper training includes Youth Suicide Assessment in Virtual Environments (Youth SAVE; Ziring et al., 2020), an Oregon-based virtual training focused on improving assessment and intervention skills in practitioners working with youth with thoughts of suicide in online spaces. Theoretical Foundations of Gatekeeper Training Overall, the goal of suicide prevention gatekeeper training is to educate gatekeepers on the warning signs of suicide and provide them with the skills to either 4 conduct a brief suicide intervention with safety planning or refer the suicidal individual to the appropriate mental health resource (Hawgood et al., 2021; Isaac et al., 2009). Two theories relevant to suicide prevention gatekeeper training are the theory of planned behavior (TPB; Ajzen, 1991) and Burnette and colleagues (2015) model for gatekeeper training. TPB posits that the intention to perform a behavior, such as a suicide intervention learned in gatekeeper training, is correlated with an individual’s perceived behavioral control (self-efficacy), attitude toward the behavior, and subjective norms (Ajzen, 1991; Hawgood et al., 2021). This theory highlights the importance of addressing attitudes toward suicide and norms around suicide intervention behavior during gatekeeper training to improve behavioral change outcomes. Additionally, Burnette and colleagues (2015) developed a theoretical model for gatekeeper training based on available literature that identifies four primary factors impacting a gatekeeper’s decision to intervene when they encounter an individual at risk for suicide: (a) knowledge about suicide, (b) beliefs and attitudes about suicide prevention, (c) reluctance to intervene, and (d) self-efficacy to intervene. This model, which is rooted in Bandura’s (2001) social cognitive theory, describes gatekeeper training competencies and the pathways that lead to changes in suicide intervention behavior (Burnette et al., 2015). Based on these two theories, I have developed an integrated theory for suicide prevention gatekeeper training, which is presented in Figure 1. This integrated model posits that gatekeeper training, broadly defined as training that addresses suicide prevention topics or specifically teaches suicide intervention and gatekeeping skills, influences gatekeeper intervention behavior by increasing knowledge about suicide, acknowledging and improving attitudes around suicide, and developing 5 Figure 1 An integrated theory of suicide prevention gatekeeper training self-efficacy for intervention behavior. Although subjective norms are a critical component to this model, they have not been widely measured in previous research regarding gatekeeper training and will not be discussed here. Behavioral intention, or the likelihood in which an individual expects they will engage in a suicide intervention when warranted, acts as a mediator between the proximal outcomes of knowledge, attitudes, and self-efficacy and the distal outcome of suicide intervention behavior. This theory posits that an increase in behavioral intention to conduct a suicide intervention will lead to an increase in true intervention behaviors, which include actions such as asking about suicidality, referring to a mental health professional, or conducting a safety plan with the individual experiencing suicidal thoughts. These intervention behaviors are considered to be a primary mechanism towards the reduction of suicidal behavior within a community 6 as more individuals experiencing suicidal thoughts are recognized and encouraged to seek treatment (Burnette et al., 2015). Additionally, this model has been contextualized within individual characteristics (e.g., demographics, gender, sex, occupation) and the larger social context in which a person is expected to act as a gatekeeper. Both individual and social contexts may have a significant influence on the type of training the gatekeeper has received as well as the gatekeeper’s ability to act on the skills they have learned (Burnette et al., 2015). As the goal of suicide prevention gatekeeper training is to increase the network of individuals available to intervene with a person experiencing thoughts of suicide (Isaac et al., 2009), it is critical to examine whether gatekeeper trainees are translating the skills they have learned into actual intervention behavior. Through targeting the domains of knowledge, attitudes, and self-efficacy, gatekeeper training aims to increase an individual’s intention to intervene and activate intervention behavior with those who appear to be experiencing suicidal thoughts by conducting a safety plan or referring them to a mental health professional (Burnette et al., 2015). Previous research has indicated that changes in beliefs and attitudes do not always correspond with changes in behaviors (Webb & Sheeran, 2006), therefore, to understand whether these trainings are having an impact, it is essential to examine behavioral outcomes. Overall, studies have indicated those who attend a gatekeeper training show increases in intentions to intervene with individuals experiencing thoughts of suicide, although minimal research exists on whether these effects are maintained over the long-term, with mixed results over varying periods (Holmes et al., 2021a; Isaac et al., 2009). Furthermore, there is mixed evidence for the effectiveness of gatekeeper training on intervention behaviors across previous 7 systematic reviews (Holmes et al., 2021a; Isaac et al., 2009; Mo et al., 2018; Morton et al., 2021). Following the integrated theory, evidence for each of these outcomes is explored further below. Gatekeeper Training Outcomes Knowledge About Suicide Increasing knowledge regarding suicide is a core component of gatekeeper training (Hawgood et al., 2021). This suicide-related knowledge typically focuses on statistics around suicide, risk and protective factors related to suicide, and identification of warning signs in an individual at risk for suicide (Hawgood et al., 2021). Knowledge may be declarative, in which an individual demonstrates an ability to recall information, or perceived, which refers to an individual’s feelings about their level of knowledge regarding a subject (Burnette et al., 2015). Gatekeeper training has been shown to increase both declarative and perceived knowledge about suicide immediately after training (Burnette et al., 2015; Holmes et al., 2021a; Isaac et al., 2009; Mo et al., 2018; Pistone et al., 2019; Wolitzky-Taylor et al., 2020) to equip individuals with the ability to identify those who may be experiencing suicidal thoughts and intervene appropriately. However, a recent systematic review demonstrated mixed evidence for whether these gains in suicide prevention knowledge are sustained in the long-term, with 11 of 18 studies showing decreases in knowledge at follow-up that remain above baseline levels, three studies showing maintenance of gains at follow-up, and two studies indicating a significant drop in knowledge at follow-up. These studies measured follow-up ranging from one month to two years after training (Holmes et al., 2021a). 8 Beliefs and Attitudes About Suicide Prevention Attitudes toward suicide prevention most commonly addressed in gatekeeper training include whether suicide is preventable, the appropriateness of intervening with an individual at risk, and perspectives regarding help-seeking for mental health concerns (Burnette et al., 2015). In several studies, gatekeeper trainings have demonstrated positive impacts on attitudes toward suicide prevention at immediate post-test with improvements maintained for up to six months (Cross et al., 2010; Jacobson et al., 2012; Keller et al., 2009). However, a recent systematic review that examined the long-term impact of gatekeeper training on attitudes toward suicide prevention found weak evidence for maintaining increases seen post-training (Holmes et al., 2021a). Of seven studies included in this review that measured long-term change in attitudes, three saw increases maintained above baseline levels, while four studies saw attitudes decrease to baseline levels at three- to six-months post-training or no significant change across time at all. Self-Efficacy to Intervene Self-efficacy refers to a gatekeeper’s comfortability and confidence to conduct a suicide intervention with an individual experiencing suicidal thoughts (Burnette et al., 2015). Gatekeeper training has been shown to increase self-efficacy around suicide intervention behaviors (Mo et al., 2018; Wolitzky-Taylor et al., 2020; Yonemoto et al., 2019) and previous research indicates that self-efficacy may be the most long-lasting outcome of gatekeeper training (Holmes et al., 2021a). A recent systematic review indicated that all 16 studies reporting on self-efficacy saw an increase at post-training 9 measurement with most seeing self-efficacy maintained above baseline levels at follow- up timepoints from one month up to two years (Holmes et al., 2021a). Behavioral Intention Individuals may often feel uncomfortable with discussing the topic of suicide with another person due to the stigma associated with mental illness, fear of increasing distress levels for an individual in crisis, or being unsure of how to approach the topic of mental health and suicide. Gatekeeper training programs aim to decrease this reluctance to intervene with an individual in crisis and increase behavioral intention to conduct a suicide intervention when warranted. Research has shown evidence of the effectiveness of gatekeeper training in reducing reluctance to intervene with a suicidal individual from pre- to post-training (Holmes et al., 2021b; Jacobson et al., 2012; Osteen, 2018). Although some studies have shown these effects decrease significantly over time, levels remain significantly above baseline even with this degradation (Holmes et al., 2021b). Suicide Intervention Behaviors Gatekeeper training ultimately aims to increase suicide intervention behavior, which is broadly defined as any action taken by a gatekeeper to identify and ask an individual about suicidal thoughts and plans, develop a safety plan with the individual in crisis, or refer them directly to a mental health professional (Burnette et al., 2015). Intervention behaviors may be categorized as identifying, intervening (asking about suicide), safety planning, and referring. Research has shown little evidence for the impact of gatekeeper training on intervention behaviors. A systematic review from Holmes and colleagues (2021a) indicated some support for referral behavior but far weaker support for identification and intervention behaviors. Additionally, Mo and colleagues (2018) 10 showed negative effects for asking about suicide and null effects for identification of suicidal individuals and referral behavior, while Pistone and colleagues (2019) showed no increases in suicide prevention skills for the gatekeeper intervention group over the control group. However, Wolitzky-Taylor and colleagues (2020) did show moderate effect sizes for improving suicide prevention skills from pre- to post-training, although these measures were more focused on skill acquisition than skill application as in previously mentioned studies. Overall, the evidence for suicide prevention gatekeeper training’s ability to improve intervention behavior remains unclear. Implementation of Gatekeeper Training Suicide prevention gatekeeper training has grown in popularity nationally and internationally, as public and mental health officials are seeking feasible and widely available strategies for suicide prevention. Additionally, SPRC (2020a) recommends gatekeeper training as a part of an effective, comprehensive approach to suicide prevention. However, there is minimal guidance regarding how to effectively implement gatekeeper training as a widespread initiative. Though the SPRC provides a brief worksheet with questions and factors to consider when selecting and implementing a gatekeeper training (Suicide Prevention Resource Center, 2020b), the author is not aware of any studies that have examined implementation factors as moderators for the effectiveness of gatekeeper training on behavioral outcomes. Therefore, in an effort to expand the field’s understanding of how to effectively implement suicide prevention gatekeeper training within communities, moderators of implementation setting (e.g., healthcare, schools), training modality (online vs. in-person), and training level (basic vs. advanced) have been proposed for exploration. 11 Rationale for a Meta-Analysis Despite the increasing suicide rates and growing popularity of suicide prevention gatekeeper training as states are seeking evidence-based prevention efforts, limited research has taken a broad approach to summarize its longitudinal and overall effects. This meta-analysis is critical to providing evidence as to whether suicide prevention gatekeeper trainings are effective in increasing behavioral intention, and suicide intervention behaviors of those interacting with suicidal individuals. The focus of this review is on behavioral intention and intervention behavior as previous reviews have examined knowledge, attitudes, and self-efficacy (Holmes et al., 2021a; Mo et al., 2018; Pistone et al., 2019; Wolitzky-Taylor et al., 2020; Yonemoto et al., 2019) and there is minimal understanding to whether gatekeeper training creates change in behavior that may lead to a reduction in suicide rates. Additionally, this review will provide policymakers and statewide suicide prevention offices with information on whether implementing widespread gatekeeper trainings should continue, or whether these trainings should be targeted to specific professions or workplaces. Few meta-analyses have been conducted on the topic of suicide prevention gatekeeper training (Pistone et al., 2019; Wolitzky-Taylor et al., 2020). Pistone and colleagues (2019) examined randomized controlled trials (RCT) and quasi-experimental design studies that evaluated educational interventions on outcomes of suicidal behavior or intermediate outcomes of knowledge, attitudes, and skills for suicide intervention. Results included 41 studies and found no evidence for an increase in gatekeeping skills and no significant decrease in suicidal ideation or suicide attempts in the group the gatekeepers were intended to serve. However, these findings were limited by the high 12 risk of bias of the included studies due to low sample sizes, quality of evidence, and short follow-up times (Pistone et al., 2019). The focus of Wolitzky-Taylor and colleagues (2020) was on suicide prevention programs for college campuses but included a separate meta-analysis for gatekeeper trainings. The meta-analysis included 11 studies that examined some type of gatekeeper intervention and findings for gatekeeper trainings indicated effectiveness for increasing suicide knowledge, intervention skills, and self- efficacy to address suicide risk among college students. Though these meta-analyses provide important information, they are limited in scope to the educational sector. Other narrative systematic reviews have been conducted on suicide prevention gatekeeper trainings (Holmes et al., 2021a; Mo et al., 2018; Yonemoto et al., 2019). For example, Holmes and colleagues (2021a) examined the long-term efficacy of gatekeeper training programs on knowledge, attitudes, self-efficacy, behavioral intention, and behaviors, but did not conduct a meta-analytic review. Their findings indicated maintenance of knowledge and self-efficacy gains, but weak evidence for sustained effects on attitudes, behavioral intention, and intervention behaviors. Additionally, Yonemoto and colleagues (2019) conducted a review of the effectiveness of gatekeeper training programs from randomized controlled trials and intervention studies but were unable to conduct a meta-analysis due to heterogeneity in outcome measures. Findings from this review concluded that the effects of gatekeeper training were unclear for knowledge, appraisals, and self-efficacy. Other narrative reviews have examined gatekeeper training on their own or as a comprehensive review of suicide prevention strategies (Clifford et al., 2013; Katz et al., 2013; Mo et al., 2018; Morton et al., 2021; Nasir et al., 2016; Torok et al., 2019; Zalsman et al., 2016). However, to my knowledge, 13 none of these reviews have focused on understanding the effects of gatekeeper training on behavioral intention and intervention behaviors or for different implementation settings, training levels, or training modalities. Since increasing help-seeking and responding effectively to individuals in crisis are critical components to the SPRC’s comprehensive approach that are frequently addressed through gatekeeper trainings, findings from this review will provide information on the effects of these trainings on behavioral outcomes to better understand how to effectively implement these programs. Specifically, this review will (a) longitudinally examine the overall effects of suicide prevention gatekeeper trainings on behavioral intention and suicide intervention behaviors (b) provide information on whether gatekeeper trainings are more effective within specific populations or implementation settings, (c) whether gatekeeper trainings are more effective in-person or online, and (d) whether basic or advanced trainings are more effective at improving behavioral outcomes. These findings have the potential to inform state- and national-level policy guidance on the effective implementation of suicide prevention gatekeeper training programs. Objectives The primary objective of this review is to synthesize the evidence on whether suicide prevention gatekeeper training improves behavioral intention for suicide interventions and applied suicide intervention behaviors as well as whether these changes are maintained over time. For the purposes of this review, suicide prevention gatekeeper training is defined as a training focused on suicide-specific interventions for individuals who may come face-to-face with individuals at risk of suicide. A secondary aim of this 14 meta-analysis is to examine whether the effect of suicide prevention gatekeeper training on behavioral intention and suicide intervention behaviors of training participants differ based on implementation setting, training modality, or training level. Research Questions This meta-analysis aims to answer the following research questions: 1. What is the impact of suicide prevention gatekeeper training on behavioral intention and suicide intervention behavior over time? 2. Does the effect of suicide prevention gatekeeper training on behavioral intention and suicide intervention behavior differ based on implementation setting (e.g., healthcare, school, community-based)? 3. Does the effect of suicide prevention gatekeeper training on behavioral intention and suicide intervention behavior differ based on the modality (in-person vs. online) of the training implemented? 4. Does the effect of suicide prevention gatekeeper training on behavioral intention and suicide intervention behavior differ based on the level (basic vs. advanced) of the training implemented? 15 CHAPTER II METHODS Eligibility Criteria The purpose of this study was to synthesize the current evidence on the effectiveness of suicide prevention gatekeeper training to improve behavioral intention, and suicide intervention behaviors. The studies included in this review met the following eligibility criteria. Types of Studies Given the limited number of randomized controlled trials, eligible studies included the gatekeeper training treatment arms from any pre-experimental, quasi- experimental, or experimental longitudinal designs that included pre-test and post-test assessments. Eligible studies measured change in at least one outcome prior to and after the implementation of the intervention. A comparison group was not required. Qualitative studies and review papers were not considered for this review. Types of Participants Participants of the training were 18 years of age or older as the focus of this meta- analysis was on behavioral intention and suicide intervention behaviors of adult populations who traditionally play the role of a gatekeeper, as opposed to adolescent populations who typically play a peer support role. Studies could contain a few 16- or 17- year-olds (e.g., university students) if the sample majority was 18 and over. To include studies with a majority of individuals under the age of 18, data to calculate an effect size had to be provided for participants aged 18 and over. For example, if a study included 16 most training participants aged 16 and 17 but did not disaggregate results for adults over the age of 18, it was excluded. Types of Interventions Eligible interventions were suicide prevention gatekeeper trainings, defined as trainings that focused on understanding suicide and providing information on how to help an individual experiencing a suicidal crisis. Gatekeeper trainings included already established programs (e.g., ASIST, QPR) or researcher-developed training programs designed to improve suicide prevention and intervention skills. General mental health programs (e.g., Mental Health First Aid) or programs focused on psychological distress (e.g., Kognito) without an explicit focus on suicide prevention and intervention skills were excluded from the study as the focus of the review is on suicide prevention and intervention-related outcomes. Additionally, suicide prevention trainings that focused on the clinical treatment of suicidality (e.g., Collaborative Assessment and Management of Suicidality) or lethal means trainings (e.g., Counseling on Access to Lethal Means) were excluded as this was beyond the scope of this review. Types of Outcome Measures To be eligible for this review, studies measured at least one outcome variable of interest for gatekeeper training participants: (a) behavioral intention regarding suicide intervention, and (b) suicide intervention behaviors. Behavioral intention is operationally defined as any measure of behavioral intention that includes the likelihood to intervene (e.g., ask about suicide, refer to mental health services) with a person considered at risk for suicide. This could include multi-item measures covering multiple facets of intentions to intervene (e.g., asking, referring) or a single-item measure regarding intentions to take 17 actions to intervene (e.g., contacting a helpline). Intervention behavior refers to any measure of suicide intervention behavior that may include identifying, asking, intervening, safety planning, or referring an individual at risk for suicide. Separate meta- analyses were conducted for each outcome. Study outcomes were measured at a minimum of two timepoints, with the first measurement time point occurring prior to the intervention implementation (pre-training) and the second timepoint occurring after the training has occurred (post-training). Studies with follow-up timepoints beyond pre- and post-training were included, but post-only studies were excluded as the question of interest examines change over time. Search Strategy Electronic Searches Multiple search methods were used to locate all the relevant literature on suicide prevention gatekeeper training. The electronic databases PubMed, PsycINFO via APA PsycNet, ProQuest Dissertation and Theses, Education Resources Information Center (ERIC) via ProQuest, Social Science Database (SSD) via ProQuest, and Applied Social Science Index and Abstracts (ASSIA) via ProQuest were searched for published and unpublished studies. To capture a broad evidence base for gatekeeper trainings, no limitations were placed on the date or location of studies included. However, only studies published in English were included as the researcher was unable to employ translation services for this review. The following search terms and strategy were used to search the PubMed database and were modified appropriately for other databases: ((gatekeep*[Title/Abstract]) AND (suicid*[Title/Abstract])) OR ("suicide prevention training"[Title/Abstract]). The initial 18 search took place on August 29, 2022, and the final search was conducted on October 31, 2022. Searching Other Resources To find other relevant literature, three additional search methods were employed. Grey literature sources searched for program evaluation reports and unpublished literature included the RAND Corporation (https://www.rand.org/), LivingWorks (https://www.livingworks.net/), and QPR Institute (https://qprinstitute.com/). Additionally, reference lists from relevant systematic reviews found during the database searches were hand searched for relevant studies. Finally, forward citation searching was implemented using Google Scholar to search papers that cited the included studies. Data Collection and Analysis Selection of Studies Two researchers independently reviewed all titles and abstracts of the studies found through electronic searches. These two researchers then independently screened the full texts of all abstract-eligible studies to determine final eligibility. Any discrepancies were resolved through discussion and consensus. Data Collection Process Two researchers independently extracted data from the included studies. Study level data included study details, study methods, program information, study population, trainer characteristics, and quality assessment. Additionally, statistical findings data were collected to code effect sizes for the outcomes of behavioral intention and intervention behavior for each study. The full codebook detailing extracted data and operational definitions can be found in Appendix A. If data to code an effect size were unavailable in 19 the published study, the primary study author was contacted to provide the information. Upon initial contact, each study author was provided with a table of the missing information being requested and a deadline of 2-3 weeks to return this information. If attempted contact was unsuccessful and data were unable to be obtained, the study was not included in the meta-analysis but retained in the systematic review. In the case that multiple reports for a single study contained conflicting information for coding an effect size, the information from the peer-reviewed publication was used. Studies with multiple reports utilized information from all reports for study-level coding to ensure the most complete and accurate information. Any discrepancies in coding were resolved through discussion and consensus. Assessment of Risk of Bias and Study Quality in Included Studies To measure the risk of bias and study quality of the included studies, two researchers extracted data on attrition at the final timepoint, trainer adherence rating, training evidence rating, trainer qualifications, study design, data collection method, and the relationship between the funding agency and the training developer. Each domain of study bias was then categorized as low, high, or unclear risk. See Appendix B for the full risk of bias coding procedures. Any discrepancies in coding were resolved through discussion and consensus. Effect Size Measures The effect size metric employed for analysis was the standardized mean gain as this allows for quantifying change over time and standardization of heterogeneous outcome measures. The standardized mean gain, as defined by Borenstein et al. (2009), 20 utilizes the mean gain of each study and divides it by the study’s standard deviation to create a comparable index across studies, such that 𝑑 = ! "!"#!"!# $$%&&⁄%&((#)) and " 𝑆𝐸, = & &((#)) + , . - &- Only two studies reported the pre-post correlations necessary for the standardized mean gain calculation, ranging from 0.09 (Rallis, 2016) to 0.66 (Orovecz, 2020). Therefore, a pre-post correlation of .5 was assumed for the remainder of studies as a more conservative estimation, and sensitivity analyses were conducted assuming a .1 and .9 correlation to determine whether any changes in results are detected. All effect size metrics were coded such that larger effect sizes indicate positive changes over time (e.g., increased behavioral intention, increase in intervention behaviors). Studies with multiple effect sizes for one outcome (e.g., three items for behavioral intention analyzed individually) were combined to create a summary effect size for this study using the following formula as illustrated by Borenstein et al. (2009), 𝑌) = ( *∑.   𝑌 - . / / where, & & 𝑉 = / (0 var *∑.   𝑌 - = / (0 /∑.0" /1( 2 /1(   𝑉2 +∑/34   *𝑟/45𝑉/5𝑉4-0. . . Where the correlation between items was not reported, a correlation of .5 was assumed. 21 Assessment of Heterogeneity To assess heterogeneity, the Q, I2, and τ2 statistics were examined. Prediction intervals were also estimated to assess the dispersion of effects and variance across studies. Additionally, heterogeneity was inspected using a forest plot. Assessment of Publication/Small Study Bias Publication bias was assessed by examining funnel plots, conducting a trim and fill analysis (Duval & Tweedie, 2000), and using Egger’s regression test (Egger et al., 1997). Additionally, a cumulative meta-analysis was conducted to determine whether the mean effect size shifted by including studies with smaller sample sizes. Data Synthesis Meta-analyses were conducted using the random-effects models with inverse variance weights and reported 95% confidence intervals with mean effect sizes. Separate meta-analyses were conducted for behavioral intention and intervention behavior and grouped by timepoint. Timepoints analyzed included pre-training to post-training, pre- training to short-term follow-up (6-months after training or less), and pre-training to long-term follow-up (greater than 6-months). If there was more than one effect size for the follow-up category (e.g., 3- and 6-month follow-up from one study) the effect size for the longer follow-up period was used. All meta-analyses were also displayed with forest plots. Analyses were performed using the metafor package (version 3.9.20) (Viechtbauer, 2010) in R (version 4.0.2) (R Core Team, 2020). Subgroup Analysis and Investigation of Heterogeneity Mixed-effect meta-regression analyses were conducted to detect potential moderators. The researcher examined the following moderators: (a) implementation 22 setting (the setting or environment in which the training was administered), (b) training modality (online or in-person) and (c) training level (basic or advanced). Registration Information The protocol for this systematic review and meta-analysis was registered with the international prospective register of systematic reviews (PROSPERO) in accordance with PRISMA-P guidelines and can be found under registration number CRD42022368607. 23 CHAPTER III RESULTS Search Results The initial search retrieved 1,104 reports. Of 169 reports retrieved through the electronic searches that were considered eligible for a full-text review, 123 were excluded due to (a) not measuring the outcomes of interest (n = 59), (b) the training not fitting the definition of gatekeeper training (n= 35), (c) outcomes of interest only measured after the training took place (n = 21), (d) the age of the sample was under 18 years old or not reported (n =4), (e) the study was not a primary study (n = 3), or (f) the study was qualitative (n = 1). See Figure 2 for the PRISMA diagram and further information. The remaining 46 reports eligible to be included in this review reported on 43 unique samples of participants. Appendix C provides a list of main reports that have corresponding supplementary reports. These 46 reports included 36 journal articles, 8 dissertations and theses, 1 technical report, and 1 book chapter. Of these studies, 84.7% took place in the United States with other studies conducted in Australia, Germany, the Netherlands, Canada, or multiple locations. Table 1 provides descriptive statistics from the 43 studies eligible to be included in the review. The majority of studies (74.4%) were pre-experimental in design. Training participants were predominantly female (M = 73.8%, SD = 9.4%) and White (M = 64.1%, SD = 23.0%). Over half of the studies (65.2%) examined standardized gatekeeper trainings as opposed to researcher-developed trainings. The average training length was 2.9 hours and few studies (8.7%) included trainings with culturally responsive content. Only two studies (4.3%) evaluated advanced trainings and only three studies (6.5%) 24 Figure 2 PRISMA flow diagram adapted from Page et al. (2021) 25 Table 1 Features of the included studies (k = 43) k Frequency Mean (SD) Range (%) Study design Pre-experimental design 32 (74.4) Quasi-experimental design 8 (18.6) Randomized controlled trial 3 (7.0) Gatekeeper training arm sample size 43 510.5 (723.5) 16-3,958 Participant characteristics Average age (years) 28 32.7 (8.6) 20-45 Percent male 21 26.2% (12.5%) 9.0-64.0% Percent female 34 73.8% (9.4%) 56.0-95.2% Percent non-binary 2 2.01% (1.4%) 1.0-3.0% Percent White 29 64.1% (23.0%) 0.0-93.0% Percent Black 21 14.0% (12.4%) 1.0-43.0% Percent Hispanic 20 15.8% (16.1%) 1.0-63.9% Percent Native/Aboriginal 13 13.6% (30.2%) 0.0-100.0% Percent Asian 19 13.9% (17.3%) 0.1-60.0% Previous suicide prevention training 8 38.6% (29.3%) 0.0-75.0% Training characteristics a Standardized training 30 (65.2%) Advanced level training 2 (4.3%) Online training 3 (6.5%) School/university setting 28 (60.9%) Community setting 13 (28.3%) Government agency setting 3 (6.5%) Social services/Non-profit setting 2 (4.3%) Average length of training (hours) 36 2.9 (3.2) 0.75-16 Culturally responsive content 4 (8.7%) Outcome characteristics Behavioral intention measured 33 (76.7%) Intervention behavior measured 17 (39.5%) Notes. k refers to the number of studies providing data for rows with means and standard deviations. a Frequencies calculated from 46 samples as two studies included multiple types of trainings. 26 evaluated online trainings. Studies most commonly took place in a school or university setting (60.9%). Overall, 33 studies included measures of behavioral intention and 17 studies included measures of intervention behavior. Appendix D provides detailed information regarding individual study results. Risk of bias in included studies was measured in seven domains: (a) attrition at the final timepoint, (b) trainer adherence ratings, (c) training evidence rating, (d) trainer qualifications, (e) study design, (f) data collection method, and (g) relationship between the sponsor and the training developer. Overall, studies included in this review had several areas with potential risk of bias to consider. The areas with the greatest percentage of studies at high risk were the data collection method (100%), study design (74.4%), and attrition at the final timepoint (53.5%). Additionally, trainer adherence ratings were only reported in two studies, indicating an unclear risk of bias in this category for most studies. However, only one study had a high risk of bias due to a relationship between the funding agency and the training developer. The results of the following meta-analysis should be evaluated within the context of these risk of bias considerations. A complete risk of bias table is displayed in Appendix E. Results of Meta-Analyses Behavioral Intention Of 33 studies that measured behavioral intention, 25 studies were able to be included in the meta-analyses. Eight studies were excluded due to not measuring an outcome in a manner that could be compared across studies or an inability to retrieve necessary data to code an effect size from the report or from contacting the study authors. 27 Pre-Training to Post-Training. A total of k = 24 studies were included in this meta-analysis with a total of n = 26 effect sizes. Lancaster et al. (2014) contributed two effect sizes as they analyzed separate samples for online training and in-person training. Additionally, Wood et al. (2022) contributed two effect sizes as they analyzed separate participant samples for educational and religious settings. The observed standardized mean gain for this analysis ranged from -0.22 to 2.28, with the majority of estimates indicating a positive change (96%). The estimated average standardized mean gain based on the random effects model was 1.03, demonstrating that on average, behavioral intention improved by 1.03 standard deviations from pre- to post-training. The confidence interval for the average effect size [0.80, 1.25], which indicates that the mean effect size in the universe of comparable studies could fall within this range, does not include any effect size of zero, suggesting the mean effect size is not equal to zero. Additionally, the null hypothesis testing indicates that the average effect size differed significantly from zero (z = 8.95, p < 0.0001). Based on these findings, the null hypothesis that behavioral intention is the same before and after attending gatekeeper training can be rejected, suggesting that behavioral intention is higher after attending gatekeeper training. A forest plot showing the observed effect sizes and mean effect size estimate based on the random-effects model is shown in Figure 3. The Q-statistic provides a test of the null hypothesis that all studies in the analysis share a common effect size. In the case that all studies shared the same effect size, the value of Q would be equal to the degrees of freedom (the number of studies minus 1). According to the Q-test, the true effect sizes appear to be heterogeneous (Q(25) = 1666.59, p < 0.0001) and the null hypothesis that the true effect size is identical in all 28 Figure 3 Forest plot for pre-post training behavioral intention meta-analysis studies may be rejected. The I2 statistic indicates that 98.94% of the variance in the observed effect sizes reflects variance in true effects rather than sampling error. The variance of true effects (τ2) is 0.33, and the standard deviation of true effects (τ) is 0.57. The 95% prediction interval for the true effect sizes is -0.12 to +2.17, suggesting that in the universe of populations represented by these studies, the true effect size will fall within this range in 95% of cases. Given the wide range of the prediction interval, t some studies may yield a negative, null, or positive effect. However, most values within the prediction interval are positive, a new study will likely indicate beneficial effects though the effects could vary widely regarding magnitude of effect (small, medium, or large). 29 A funnel plot of the estimates is shown in Figure 4. Upon visual inspection, there appeared to be no indications of funnel plot asymmetry and Egger’s regression test supports this conclusion (p = 0.52). However, the trim and fill analysis estimated five missing studies on the right side. Finally, the cumulative meta-analysis shows a slight shift towards smaller effect sizes with the addition of smaller sample size studies. See Based on the funnel plot, trim and fill, and cumulative meta-analysis, the bias appears to be minimal overall and can likely be trusted. Appendix F for the trim and fill and cumulative meta-analysis plots. Figure 4 Funnel plot for pre-post training behavioral intention analysis Note. SE = standard error. Pre-Training to Short-Term Follow-Up. A total of k = 11 studies were included in this analysis with a total of n = 12 effect sizes (two effect sizes for Lancaster et al., 30 2014). The observed standardized mean gain for this analysis ranged from 0.34 to 1.73, with 100% of estimates indicating a positive change. The estimated average standardized mean gain based on the random effects model was 0.78, showing that on average, behavioral intention improved by 0.78 standard deviations from pre-training to up to 6- months after training. The 95% confidence interval for the average effect size [0.59, 0.97] suggests the mean effect size is not equal to zero. This is supported by the null hypothesis testing (z = 8.06, p < 0.0001). Based on these findings, the null hypothesis that behavioral intention is the same before and up to 6 months after attending a gatekeeper training can be rejected, demonstrating that behavioral intention is higher up to 6 months after attending a gatekeeper training. A forest plot showing the observed effect sizes and the estimate based on the random-effects model is shown in Figure 5. Figure 5 Forest plot for pre-training to short-term follow-up behavioral intention analysis 31 According to the Q-test, the true effect sizes appear to be heterogeneous (Q(11) = 147.81, p < 0.0001) and the null hypothesis that the true effect size is identical in all studies may be rejected. The I2 statistic indicates that 93.40% of the variance in the observed effect sizes reflects variance in true effects rather than sampling error. The variance of true effects (τ2) is 0.09, and the standard deviation of true effects (τ) is 0.30. The 95% prediction interval for the true effect sizes is +0.15 to +1.40. This prediction interval indicates that most studies will yield a positive effect, but the magnitude of the effect could vary greatly. A funnel plot of the estimates is shown in Figure 6. Upon visual inspection, there appeared to be missing studies in the bottom left quadrant and Egger’s regression test suggests there may be asymmetry (p = 0.04). However, the trim and fill analysis estimated no missing studies. Finally, the cumulative meta-analysis shows a minimal shift towards larger effect sizes with the addition of smaller sample size studies. See Based on the funnel plot, trim and fill, and cumulative meta-analysis, the bias appears to be minimal overall and can likely be trusted. Appendix F for the trim and fill and cumulative meta-analysis plots. Pre-Training to Long-Term Follow-Up. Only one study (Adams et al., 2018) provided data to code an effect size for long-term follow-up beyond 6-months for behavioral intention. Results from this study revealed positive effects for behavioral intention from pre-training to 9-month follow-up (d = 0.49, 95% CI [0.09, 0.90]. However, the author was unable to conduct a meta-analysis for this timepoint as only one study was eligible so these results should be taken caution. 32 Figure 6 Funnel plot for pre-training to short-term follow-up behavioral intention analysis Note. SE = standard error. Intervention Behavior Of 17 studies that measured intervention behavior, nine studies were able to be included in the two meta-analyses. Eight studies were excluded from the meta-analyses due to not measuring an outcome in a manner that could be compared across studies or an inability to retrieve necessary data to code an effect size from the report or from contacting the study authors. No studies provided data to code an effect size for a pre-to- post evaluation of intervention behaviors as training participants would not have had a chance to utilize the skills learned during the training, therefore, the author did not conduct a meta-analysis for this timepoint. 33 Pre-Training to Short-Term Follow-Up. A total of k = 9 studies were included in this analysis with a total of n = 10 effect sizes (two effect sizes for Lancaster et al., 2014). The observed standardized mean gain for this analysis ranged from 0.12 to 0.84, with 100% of estimates indicating a positive change. The estimated average standardized mean gain based on the random effects model was 0.33, showing that on average, intervention behavior improved by 0.33 standard deviations from pre-training to up to 6 months after training. The 95% confidence interval for the average effect size [0.21, 0.46] suggests the mean effect size is not equal to zero. This is supported by the null hypothesis testing (z = 5.22, p < 0.0001). Based on these findings, the null hypothesis that intervention behavior is the same before and up to 6-months after attending a gatekeeper training can be rejected, demonstrating that intervention behavior is higher up to 6 months after attending a gatekeeper training. A forest plot showing the observed effect sizes and the estimate based on the random-effects model is shown in Figure 7. According to the Q-test, the true effect sizes appear to be heterogeneous (Q(9) = 26.37, p =0.0018) and the null hypothesis that the true effect size is identical in all studies may be rejected. The I2 statistic indicates that 62.65% of the variance in the observed effect sizes reflects variance in true effects rather than sampling error. The variance of true effects (τ2) is 0.02, and the standard deviation of true effects (τ) is 0.15. The 95% prediction interval for the true effect sizes is +0.15 to +0.65. This prediction interval indicates that most studies will yield a positive effect and will have a small to medium magnitude of effect. 34 Figure 7 Forest plot for pre-training to short-term follow-up intervention behavior analysis A funnel plot of the estimates is shown in Figure 8. Upon visual inspection, there appeared to be no indications of funnel plot asymmetry and Egger’s regression test supports this conclusion (p = 0.10). Additionally, the trim and fill analysis estimated no missing studies. Finally, the cumulative meta-analysis shows a minimal shift towards larger effect sizes with the addition of smaller sample size studies. Based on the funnel plot, trim and fill, and cumulative meta-analysis, the bias appears to be minimal overall and can likely be trusted. See Appendix F for the trim and fill and cumulative meta- analysis plots. Pre-Training to Long-Term Follow-Up. A total of k = 2 studies were included in this analysis with a total of n = 2 effect sizes. The observed standardized mean gain for 35 Figure 8 Funnel plot for pre-training to short-term follow-up intervention behavior analysis Note. SE = standard error. this analysis ranged from 0.20 to 0.30, with 100% of estimates indicating a positive change. The estimated average standardized mean gain based on the random effects model was 0.22, showing that on average, intervention behavior improved by 0.22 standard deviations from pre- to more than 6-months after training. The 95% confidence interval for the average effect size [0.14, 0.30] suggests the mean effect size is not equal to zero. This is supported by the null hypothesis testing (z = 5.49, p < 0.0001). Based on these findings, the null hypothesis that intervention behavior is the same before and more than 6 months after a gatekeeper training may be rejected, demonstrating that intervention behavior is higher more than 6 months after attending a gatekeeper training. 36 However, these results must be considered with caution as only two studies were included. A forest plot showing the observed effect sizes and the estimate based on the random-effects model is shown in Figure 9. Figure 9 Forest plot for pre-training to long-term follow-up intervention behavior analysis The 95% prediction interval for the true effect sizes is +0.14 to +0.30. This prediction interval indicates that most studies will yield a positive effect and are likely to have a small magnitude of effect. The Q, I2, and τ2 statistics for heterogeneity, funnel plots, Egger’s regression test, and trim and fill analyses were unable to be conducted as there were less than ten studies included in the analysis. Sensitivity Analyses Sensitivity analyses were conducted for the pre-post correlation, assuming a .1 and a .9 correlation to determine whether this changed the results of the meta-analyses 37 since most studies did not report the pre-post correlation (and a .5 correlation was assumed in the main analyses). These sensitivity analyses were conducted for the pre- to post-training behavioral intention meta-analyses and demonstrated a minimal shift in the summary effect size. Results revealed a standardized mean gain of 0.86 for a .9 correlation and a 1.06 for a .1 correlation, indicating the results of the meta-analyses with a .5 correlation are robust and have the potential to be an underestimate of effects. Appendix G shows the forest plots for the sensitivity analyses. Subgroup Analyses Training Level. Only two studies (Coleman & Del Quest, 2015; Ewell Foster et al., 2017) examined an advanced training. While both studies measured the outcome of intervention behavior, Coleman and Del Quest (2015) included only a short-term follow- up, while Ewell Foster and colleagues (2017) included a long-term follow-up. Therefore, no formal statistical comparisons could be conducted for this moderator. A narrative comparison of effect sizes for intervention behavior from pre-training to short-term follow-up revealed a larger magnitude of effect for advanced trainings when compared to basic training. The summary effect size for basic training across seven studies with nine effect sizes was 0.30, 95% CI [0.18, 0.42], while the effect size for advanced training was 0.86, 95% CI [0.39, 1.28]. However, effect sizes for basic and advanced trainings at long- term follow up remain similar. Ewell Foster and colleagues (2017) reported an effect size of 0.20, 95% CI [0.11, 0.29] for advanced training, while Wyman and colleagues (2008) reported an effect size of 0.30, 95% CI [0.13, 0.47] for basic training. Overall, there is not enough evidence to make substantial conclusions regarding whether there were significant differences in effect sizes for basic in comparison to advanced training. 38 Training Modality. Only one study (Lancaster et al., 2014) examined an online training, therefore no formal statistical comparisons were conducted for this moderator. Lancaster and colleagues (2014) measured behavioral intention at post-training and short- term follow-up, as well as intervention behavior at short-term follow-up. For behavioral intention at post-training, the summary effect size for in-person training across 24 studies with 25 effect sizes was 1.04, 95% CI [0.81, 1.27], while the effect size for online training was 0.72, 95% CI [0.51, 0.93], indicating a slightly larger magnitude of effect for in-person training. For behavioral intention at short-term follow-up, the summary effect size for in-person training across 11 studies was 0.78, 95% CI [0.58, 0.99], which was similar to the online training, which was 0.75, 95% CI [0.44, 1.06]. Additionally, for intervention behavior at short-term follow-up, the summary effect size for in-person training across seven studies and nine effect sizes was 0.37, 95% CI [0.25, 0.49], while the online training had a slightly smaller effect size of 0.20, 95% CI [-0.01, 0.41]. Though there is not enough evidence to draw firm conclusions from these findings, in- person training and online training seem to result in comparable effect sizes for both behavioral intention and intervention behavior outcomes. Implementation Setting. Mixed effect meta-regression models with the Knapp- Hartung correction were conducted to explore whether the implementation setting contributed to a difference in the magnitude of effect sizes between studies for behavioral intention and intervention behavior. Behavioral Intention, Pre- to Post-Training. Studies from the meta-analysis examining pre- to post-training changes in behavioral intention were conducted in either a community setting or a school/university setting. Results from the meta-regression 39 model indicated there were no significant differences in the magnitude of effect sizes for the community setting (d = 0.81, 95% CI [0.42, 1.20]) in comparison to the school/university setting (d = 1.14, 95% CI [0.86, 1.43]). A subgroup forest plot is displayed in Figure 10. Figure 10 Forest plot for implementation setting subgroup analysis for pre- to post-training behavioral intention outcome Behavioral Intention, Pre-Training to Short-Term Follow-Up. Studies from the meta-analysis examining pre-training to short-term follow-up changes in behavioral intention were conducted in either a community setting or a school/university setting. Results from the meta-regression model indicated there were no significant differences in 40 the magnitude of effect sizes for the community setting (d = 0.65, 95% CI [0.20, 1.11]) in comparison to the school/university setting (d = 0.83, 95% CI [0.55, 1.10]). A subgroup forest plot is displayed in Figure 11. Figure 11 Forest plot for implementation setting subgroup analysis for pre-training to short-term follow-up behavioral intention outcome Intervention Behavior, Pre-Training to Short-Term Follow-Up. Studies from this meta-analysis examining pre-training to short-term follow-up changes in intervention behavior were conducted in either a community, school/university, or government agency setting. Results from the meta-regression model indicated there were no significant differences in the magnitude of effect sizes for community settings (d = 0.23, 95% CI 41 [0.04, 0.48]), government agency settings (d = 0.53, 95% CI [0.17, 0.88], or school/university settings (d = 0.31, 95% CI [0.09, 0.53]). A subgroup forest plot is displayed in Figure 12. Figure 12 Forest plot for implementation setting subgroup analysis for pre-training to short-term follow-up intervention behavior outcome 42 CHAPTER IV DISCUSSION The following section will summarize the findings from this systematic review and meta-analysis as well as provide a discussion around the limitations of the current evidence available for suicide prevention gatekeeper training and the methodology for this meta-analysis. Finally, practice, policy and research implications will be discussed. Summary of Evidence This systematic review summarized findings from 43 studies examining the effects of suicide prevention gatekeeper training on behavioral intention to intervene and actual intervention behavior with a person with thoughts of suicide. The meta-analyses quantitatively synthesized findings from 25 studies, which indicated that individuals who participated in a suicide prevention gatekeeper training saw increases in behavioral intention to intervene and intervention behavior. Specifically, large summary effect sizes at post-training (d = 1.03) and short-term follow-up (d = 0.78) were found for behavioral intention, while small summary effect sizes for short-term follow-up (d = 0.33) and long- term follow-up (d = 0.22) were found for intervention behavior. The results of the current meta-analysis are in line with a previous meta-analysis that found a moderate effect of suicide prevention gatekeeper training on intervention behavior (labeled as suicide prevention skills) from pre- to post-training (Wolitzky-Taylor et al., 2020). However, Wolitzky-Taylor and colleagues’ (2020) review was limited in scope to university and college campuses. Overall, this meta-analysis expands current research by giving a comprehensive examination of the suicide prevention gatekeeper training literature for adult populations without limiting the scope to a particular setting. Furthermore, these 43 results are in line with the hypotheses and proposed theory which posits suicide prevention gatekeeper training increases behavioral intention and intervention behaviors. Subgroup analyses were limited and unable to be completed as planned for training modality and training level due to the lack of available evidence for online or advanced trainings. Though the summary effect size for in-person training was slightly larger than the single study effect size for online training for both behavioral intention and intervention behavior outcomes, it is not appropriate to consider this evidence conclusive. Similarly, advanced trainings showed a slightly larger magnitude of effect on intervention behavior in comparison to basic trainings, but this was also based on very limited evidence for advanced trainings, so this is not considered conclusive. Statistical comparisons were made to examine whether effect sizes differed based on the implementation setting. No significant differences were found for behavioral intention or intervention behavior, indicating that suicide prevention gatekeeper trainings may have a similar effect in a variety of settings and populations. Overall, the results from this meta- analysis indicate that, regardless of implementation setting, suicide prevention gatekeeper training appears to have a clinically meaningful positive effect on behavioral intention and intervention behavior for trainees attempting to intervene with a person with thoughts of suicide. Of note when considering the results of this meta-analysis is the wide range of effect sizes from the individual studies for each outcome. For example, the effect sizes for pre- to post-training changes in behavioral intention ranged from -0.22 (Capp et al., 2001) to 2.28 (Wood et al., 2022). This wide range of effect sizes then leads to wider prediction intervals for what may be expected for future studies’ effect sizes. This 44 variability of effect sizes that spans between negative and positive impacts lowers confidence in the ability to conclude that suicide prevention gatekeeper trainings have an overall positive effect. However, upon further examination, the studies that had negative or null effects had specific features to account for. First, the Capp and colleagues (2001) study resulting in negative effects for behavioral intention was conducted with a sample of Aboriginal participants from Australia, which posed unique cultural considerations (e.g., mistrust of mental health professionals, attitudes that suicide is an individual right) when understanding intentions to help or refer to a mental health professional. Behavioral intention was also high at pre-training so changes in scores may have been impacted by ceiling effects at later measurement timepoints. Second, Burket (2017) showed null effects for behavioral intention to contact a helpline on behalf of a peer, which may have been too specific of a measure to understand how participants would help an individual with thoughts of suicide. Capp and colleagues (2001) and Burket (2017) highlight the need for additional research in culturally tailored interventions and standardized measures for gatekeeper trainings. Limitations At first glance, this meta-analysis supports federal and statewide spending on gatekeeper training as a suicide prevention initiative due to its effectiveness, however, there are key aspects of these analyses that should be considered. First, this meta-analysis is not only limited by the small quantity of studies that were included, but also the methodological rigor of these studies. Although there were a greater number of studies included in the behavioral intention analyses (k = 24 for pre- to post-training, k = 11 for pre-training to short-term follow-up), there were very limited studies available for the 45 intervention behavior analyses (k = 9 for pre-training to short-term follow-up, k = 2 for pre-training to long-term follow-up). Moreover, there was significant evidence for risk of bias in several categories for the included studies, specifically related to study design and attrition, which causes concern for the validity of the results of these meta-analyses. With high attrition rates and lower methodological quality of the included studies, there remains a potential for these results to be biased. On the other hand, there was minimal evidence of publication bias, suggesting that the included studies are likely representative of the overall population of studies. To the author’s knowledge, there have only been three randomized controlled trials measuring changes in behavioral intention and intervention behavior before and after participating in a suicide prevention gatekeeper training (Burket, 2017; Jacobson et al., 2012; Wyman et al., 2008). The majority (74.4%) of studies included in this meta- analysis synthesized effect sizes from pre-experimental research studies, meaning there was no comparison or control group to compare the results to. This introduces risk of bias in understanding the true effectiveness of suicide prevention gatekeeper training on behavioral intention and intervention behavior as other variables may contribute more significantly to the increases seen over time. This is an overall limitation in the research and reveals a major gap in the field, especially given the rising popularity of suicide prevention gatekeeper trainings for policymakers and interventionists. Additional methodological concerns for the studies included in the meta-analysis are that all studies relied on self-report measures for their data collection. Self-report measures are susceptible to bias associated with social desirability, or the desire to present oneself favorably, and recall bias, in which participants may or may not be able to 46 accurately recall past events (Althubaiti, 2016). The potential risk of bias is increased as most of the included studies use researcher-developed measures that have not been previously validated. Additionally, over half of the studies had a high rate of attrition (>20%) with many not addressing issues of missing data, which poses a concern as there may be systematic differences in those who choose to continue participation in a study in comparison to those who dropped out or are lost at follow-up (Nunan et al., 2018). For example, a study could see inflated effect sizes if those who utilized their suicide intervention skills were more likely to report at follow-up than those who did not use their intervention skills. Furthermore, only two studies included measures of training fidelity, which introduces concern regarding whether study results are influenced by trainer variations or implementation differences. Without knowledge of whether these suicide prevention gatekeeper trainings were conducted to fidelity, there remain questions regarding whether other training variables could be influencing the outcomes. Overall, these risks of bias may lead to an overestimation of effect sizes, inferring that suicide prevention gatekeeper training has a more positive effect than the true effect. Furthermore, the inability to conduct formal statistical comparisons for training modality and training level as moderators due to the lack of research available for online programs and advanced training programs. Only one study (Lancaster et al., 2014) in this review examined behavioral intention and intervention behavior for an online training program. It is critical to understand how in-person and online trainings differ, if at all, as the use of online trainings increases due to accessibility issues and the necessity of moving trainings to a virtual space during the COVID-19 pandemic. The field may see an increase in research regarding online training programs in the coming years as a result of 47 the pandemic. This meta-analysis not only highlights the need for future research to examine how suicide prevention gatekeeper trainings are functioning in a virtual space and whether training modality has any influence on the effect size outcomes, but also should be updated in the future to capture the potential studies regarding online training programs in the coming years. Regarding training level, one study (Coleman & Del Quest, 2015) measured intervention behavior at short-term follow-up and one study (Ewell Foster et al., 2017) measured intervention behavior at long-term follow-up. Overall, the majority of research included in this review focused on basic gatekeeper trainings (93.0%) and more specifically, QPR (51.2%). Not only did this limit the ability to answer the question of whether basic or advanced trainings have a different influence on behavioral intention and intervention behavior, but it also demonstrates a lack of research on more advanced trainings, despite their popularity. In addition to the limitations within the available literature, there are also limitations to the meta-analytic procedures to consider. First, although sensitivity analyses indicated that the analytic choices made produced robust results, there is still inherent risk in making assumptions (e.g., pre-post correlations) when the primary studies do not report the necessary information to code an effect size. Second, the choice to include pre-experimental and quasi-experimental research in the meta-analyses leads to additional threats to validity such as selection bias, that may impact the results based on who chose to participate in the research. Furthermore, while the inclusion of a variety of measures of behavioral intention and intervention behavior provides an overall understanding of how gatekeeper training impacts these constructs broadly, there may be additional nuances that this analysis does not capture. For example, there may be 48 differences in the effect sizes of gatekeeper training’s impact on participants’ intention to ask about suicide in comparison to their intention to refer a suicidal person to a mental health professional. Future research would benefit from a more nuanced examination of these constructs. Finally, though the author attempted to capture the currently available literature, there are limitations to the search procedures in which databases were chosen and where grey literature was retrieved. It is possible that some studies were not located during the data collection, especially as there is not a standardized nomenclature for suicide prevention gatekeeper trainings. Implications Practice and Policy Implications Based on the information from this meta-analysis, policymakers and interventionists may cautiously conclude that the majority of suicide prevention gatekeeper trainings show positive effects on behavioral intention and intervention behavior in the short-term. Overall, larger effect sizes were seen for behavioral intention than intervention behavior, although the goal of gatekeeper training is to train individuals to take action to intervene with persons with thoughts of suicide. Though a weak correlation between changes in behavioral intention and intervention behavior for bystander intervention programs is often common (Webb & Sheeran, 2006), this could also be due to the shorter follow-up periods of the studies on intervention behavior where training participants have not yet had the opportunity to use their intervention skills. This highlights a need for additional research on gatekeeper trainings with longer follow-up time periods to help inform policymakers and interventionists on the effectiveness of 49 gatekeeper training programs and implementation strategies that help training participants maintain their skills based on how frequently they utilize them. Given the lack of randomized controlled trials or other rigorous methodologies (e.g., randomized roll-out, stepped wedge) and the focus on program evaluation and pre- experimental research, these results must be taken with caution. Policymakers and interventionists should advocate for additional research in effectiveness and implementation trials to inform their efforts and determine the validity of utilizing gatekeeper training as a main strategy for suicide prevention. Rigorous trials focused on implementation outcomes such as reach, penetration, sustainment, cost, and feasibility (Proctor et al., 2011) may provide leaders with guidance on the effective implementation of gatekeeper training to create a wider network of gatekeepers for suicidal individuals. This is especially critical as there have been an increasing number of states implementing legislation around suicide prevention training for behavioral health and medical professionals, such as Oregon House Bill 2315 (2021) which requires suicide assessment and treatment training for behavioral health workers. Until the research is available that provides insight as to whether suicide prevention gatekeeper training creates behavior change, practitioners and policymakers are unable to make informed decisions or draw conclusions regarding whether these trainings are effective at accomplishing their intended goal. With additional implementation research, policymakers will be better informed on how to develop effective and sustainable policies to ensure a suicide-safer workforce and practitioners will be able to strategically implement gatekeeper trainings for workforces and communities who need it most. 50 When practitioners are considering the results of this meta-analysis, is important to note that most research was predominantly conducted on a majority of White and female samples. This indicates that these findings are particularly relevant to a more homogenous set of participants and may not be generalizable to more diverse populations. A lack of generalizability is further highlighted by the finding that over 60% of studies were conducted in the school or university setting. However, the subgroup analyses did not indicate a difference in magnitude of effect based on setting, so it is possible that similar results would be seen for a variety of settings. Overall, practitioners should be cautioned to interpret that suicide prevention gatekeeper trainings are generalizable to diverse populations and settings. Research Implications This systematic review and meta-analysis reveal significant gaps in current literature regarding suicide prevention gatekeeper training and highlights the need for more robust and rigorous examinations of the effectiveness of these trainings. Most notably, current evidence for gatekeeper training has focused on skill acquisition rather than skill application, as evidenced by the limited number of studies that examined intervention behavior as an outcome of interest and the limited follow-up timeframes. The goal of suicide prevention gatekeeper training is to expand the network of individuals available to perform these interventions. However, the current body of evidence has failed to measure whether these trainings are effective at producing intervention behavior in gatekeepers, especially in the long term. Without this research, we are unable to determine whether gatekeeper training is making an impact on creating suicide-safer communities and workforces. 51 Cultural responsivity is a critically important aspect of gatekeeper training, especially given the high rates of suicide in marginalized populations (Centers for Disease Control and Prevention, 2023; Troya et al., 2022). However, there is very minimal research on culturally responsive suicide prevention gatekeeper trainings and their effectiveness on behavioral outcomes. With less than ten percent of studies included in this review containing a culturally responsive component, this is a call to the field to prioritize and respond to the needs of those most at risk for suicide. Future research must address, first, whether gatekeeper training is acceptable to marginalized groups, and secondly, how these trainings or other interventions can be designed to meet their needs. Conclusion Despite the limited research available and the low methodological quality of the studies currently available on behavioral outcomes, this meta-analysis was conducted using rigorous evaluation methods. These results may inform policymakers and interventionists that suicide prevention gatekeeper trainings are showing overall positive effects but lack robust research to determine the true effects of this training. Additionally, this meta-analysis provides a strong call for future research to utilize rigorous methodologies (e.g., randomized controlled trials) with a focus on intervention behavior to determine whether suicide prevention gatekeeper training is having the desired effect of expanding the network of individuals available to help a person with suicidal thoughts receive the support they need. 52 APPENDIX A CODING MANUAL Suicide Prevention Gatekeeper Training Meta-Analysis Coding Manual Last updated: December 14, 2022 Table of Contents Eligibility Criteria .................................................................................................................... 54 Full-Text Coding ...................................................................................................................... 54 Study Level ........................................................................................................................................... 55 Outcomes .............................................................................................................................................. 60 Effect Sizes ........................................................................................................................................... 61 53 Eligibility Criteria Intervention 1. The intervention must be a suicide prevention gatekeeper training. Trainings may include already established programs or new training programs designed to improve suicide prevention and intervention skills. The training program must focus on suicide and provide information on how to help an individual experiencing a suicidal crisis. General mental health programs, lethal means trainings, and trainings focused on clinical treatment of suicidality will be excluded. Participants 2. The study sample on which the intervention outcomes are measured should consist of adults, defined as persons aged 18 or over. Studies may include persons aged 16 and 17 years as long as the majority of participants are aged 18 and over (e.g., university student samples with some participants under the age of 18). If the study sample includes a majority of persons under the age of 18 years, it must include separate data to calculate effect sizes for those aged 18 and over. Research Design 3. Eligible studies include the gatekeeper training treatment arms from any pre- experimental, quasi-experimental, or experimental longitudinal designs that include at least pre-test and post-test assessments. A comparison group is not required. Outcome Measures 4. The study reports at least one of the following outcomes: behavioral intention or intervention behavior. Outcomes can be measured using researcher-developed measures or standardized measures. Outcome definitions are listed below. a. Behavioral intention: any measure of behavioral intention, which may include the likelihood or reluctance to intervene with a person with thoughts of suicide. b. Intervention behavior: any measure of suicide intervention behavior that may include identifying, intervening, safety planning, or referring. Publication Date, Type, and Source 5. Studies are limited to those published in English. 6. There are not restrictions on the date, type, or form of publication. Any relevant article, chapter, technical report, conference paper, etc. is eligible as long as it meets the other eligibility criteria. 54 Full-Text Coding Study Level The unit coded here is at the study level. Note that a single study may be reported in multiple manuscripts (e.g., journal articles). In this case, the coding should be done from the full set of relevant reports, using whichever report is best for each item to be coded. Be sure to have the full set of relevant reports prior to coding. Sometimes a single report describes more than one study. In this case, each study should be coded separately as if each has been described in a separate report. Each study has its own study identification number (StudyID, e.g., 619). Each report also has an identification number (ReportID, e.g., 619.01). The ReportID has two parts; the part before the decimal is the StudyID, and the part after the decimal is used to distinguish the reports within a study. When coding, use the studyID (e.g., 619) to refer to the study as a whole, and use the appropriate reportID (e.g., 619.01) when referring to an individual report. While reading reports for coding, be alert to any references to other gatekeeper training studies that may be appropriate to include in this meta-analysis. If you find appropriate- looking references that are not currently entered into the Bibliography, the references may need to be entered by hand and retrieved in full-text format. Study Identifiers [studyid] Study ID. Enter the three-digit study ID number here. [studyname] Study Name. Enter a brief descriptive name of the study. If the study has a name (e.g., Project QUIT), use that. If not, use First Author and Year (e.g., Richmond 1995). [coder] Coder Name. Coder’s initials [cdate] Date Coded. Date initially coded (mm/dd/yy) [mdate] Date Modified. Date coding was modified (mm/dd/yy) Study Details [author] Enter author’s last name. If multiple authors, follow APA format using et al. [pubyear] Year of publication. [pubtype] Publication type: 1. Journal article 2. Thesis-dissertation 3. Technical report 4. Conference report 5. Book 6. Book chapter 55 7. Other (Specify):__________ [country] Country. Country in which the study was implemented. List with a semi-colon between names of countries. [size] Sample size. Enter the initial number of participants recruited into the study. [gktsize] Gatekeeper training arm initial sample size. Enter the number of participants in the gatekeeper training arm of the study. Study Methods [design] Research design. 1. Pre-experimental (no control group) 2. Quasi-experimental (control group, no randomization) 3. Randomized controlled trial (control group, randomized to condition) [timepoint] Timepoints assessed. Check all that apply for the timepoints that were assessed during the study. 1. Baseline/pre-test 2. Immediate post-test (within 1 week of training) 3. Follow-up(s) [funum] Number of follow-up assessments. Enter # of follow-ups conducted. [behavint] Behavioral intention measure. Indicate how behavioral intention was measured. Enter nr if not measured. [intervbeh] Intervention behavior measure. Indicate how intervention behaviors were measured. Enter nr if not measured. 56 Program and Participants Program Information [program] Name of program implemented. Enter the name of the gatekeeper training program implemented. [desc] Program description. Enter a brief program description if it is not a standardized program (e.g., ASIST, QPR). As much as possible, quote or give a close paraphrase of the relevant descriptive text in the study report; include page numbers to the report when appropriate. It is acceptable to copy and paste directly from the article as long as you include the information in quotations and provide a page number for the quotation. Enter nr if it is a standardized program. [developer] Training developer. Enter the name of the training developer. If researcher-developed, indicate as such. Enter nr if not reported. [modality] Modality of program. Enter nr if not reported. 0. In-person 1. Online 2. Hybrid [level] Level of program implemented. Basic programs present information on warning signs, asking about suicide, and referring an individual to a mental health professional. Advanced programs may include basic skills but will also teach brief suicide intervention skills and safety planning. Enter nr if not reported. 0. Basic 1. Advanced [length] Length of training. Enter the length of training in number of hours. Enter nr if not reported. [setting] Setting. This refers to the setting or environment (e.g., healthcare, community) in which the gatekeeper training program was administered. If there is no specific setting implicated in the study, indicate Community, assumed. 1. Healthcare 2. School/University 3. Mental Health Care 4. Government Agency 5. Social Services/Non-Profit 6. Community, indicated. 7. Community, assumed. 8. Other (Specify):__________ [culresp] Cultural responsiveness. Did this program employ any culturally responsive practices? Cultural responsiveness in gatekeeper training will be defined as a training designed or adapted “in response to local cultural meanings and practices,” taking into account “an individual’s 57 historical context, social network, and community resources.” (Wexler & Gone, 2012, p. 804). Enter nr if not reported. [culrespdesc] Description of cultural responsiveness. Enter a brief description of how the program employed culturally responsive practices. As much as possible, quote or give a close paraphrase of the relevant descriptive text in the study report; include page numbers to the report when appropriate. It is acceptable to copy and paste directly from the article as long as you include the information in quotations and provide a page number for the quotation. Participant Characteristics [permale] Enter the percent of males in this group. Use nr for not reported. [perfemale] Enter the percent of females in this group. Use nr for not reported. [pernonbin] Enter the percent of non-binary participants in this group. Use nr for not reported. [perwhite] Enter the percent of White participants in this group. Use nr for not reported. [pernonwhite] Enter the percent of Non-White participants in this group. Use nr for not reported. [perblack] Enter the percent of Black participants in this group. Use nr for not reported. [perhisp] Enter the percent of Hispanic/Latinx participants in this group. Use nr for not reported. [perasian] Enter the percent of Asian participants in this group. Use nr for not reported. [pernative] Enter the percent of Native participants in this group. Use nr for not reported. [age] Enter the average age of the sample in this group using number of years. Use nr for not reported. [agerange] Enter the age range of the sample in this group using number of years. Use nr for not reported. [occupation] Enter the occupation(s) of the participants in this group. Use nr for not reported. [yearsexp] Enter the mean years of suicide prevention experience of the participants in this group. Use nr for not reported. [prevtrain] Enter the percent of participants who have received previous suicide prevention training in this group. Use nr for not reported. Trainer Characteristics [trainexp] Enter the mean years of training experience of the trainer. Enter nr if not reported. [trainmodel] Enter whether the trainers were a part of the development team or have been trained in a train the trainer model. Use nr for not reported. 1. Development Team 58 2. Train the trainer [trainocc] Enter the trainer(s) occupation. Enter nr if not reported. [trainqual] Describe any qualifications trainers are required to have to provide the training. If none reported, enter nr. 59 Quality Assessment [collection] Enter the method of data collection. 1. Self-report surveys 2. Assessment (e.g., multiple choice test) 3. Observations 4. Other [attrition] Enter percent of attrition for each timepoint. 1. Post: (att_post) 2. Follow-up 1: (att_fu1) 3. Follow-up 2: (att_fu2) [evidence] Enter the amount of evidence behind the training. 1. Standardized training program 2. Researcher-developed program [adhere] Enter training adherence rating and describe how this was measured. If none reported, enter nr. [sponsorrel] Sponsor Relationship to Program. Does this sponsor have a potential conflict of interest with the program implemented (e.g. funded by LivingWorks to implement ASIST)? 0. No 1. Yes 2. Unknown 60 Effect Sizes Although this is the final section of coding, it is a good idea to identify at least one codable effect size before you start coding a study, because studies that appear eligible frequently end up presenting data that cannot be coded into an effect size. [reported] Report ID for this effect size. Indicate the report number (e.g., 2098.01) for the report in which you found the information for this effect size. This is important so that we can find the source information for the effect sizes later on, if necessary, and is especially important for studies with multiple reports. [page] Page number for this effect size. Indicate the page number of the report identified above on which you found the effect size data. If you used data from two different pages, you can type in both, but use a comma or dash between the page numbers. [estype] Type of effect size you are coding. 1. Pre to Post 2. Pre to Follow-Up (indicate timing) 3. Post to Follow-Up (indicate timing) [esfu] If effect size for follow-up, indicate timing of follow-up. [esout] Outcome being measured for this effect size. 1. Behavioral intention 2. Intervention behavior [escorr] Correlation between Time 1 and Time 2 measures. If this is not reported, assume a .5 correlation. [esmeant1] Mean score of outcome at time 1. [esmeant2] Mean score of outcome at time 2. [essdt1] Standard deviation of outcome at time 1. [essdt2] Standard deviation of outcome at time 2. [essddiff] Standard deviation of the difference scores. Note. If not reported, calculate using, 61 & & 2 (𝑠5)6 + 𝑠5789) [esn] Number of participants included in analysis. [essmg] Effect size value – standardized mean gain. 𝑋) − 𝑋) 𝑑 = ;& ;(∆ 𝑠,2<<⁄52(1 − 𝑟) 61 Note. You must make sure that the sign of the effect size matches the way we think about direction, meaning that a positive effect size indicates that the posttest had better scores than the pretest and a negative effect size indicates that a pretest had better scores. Effect sizes can range anywhere from around –3 to +3. However, you will most commonly see effect sizes in the –1 to +1 range. [esse] Standard error of the effect size. 2(1 − 𝑟) 𝑑& 𝑆𝐸,∆ = 6 𝑛 + 2𝑛 [esprob] Any problems coding this effect size? [notes] Any additional notes regarding the effect size. 62 APPENDIX B RISK OF BIAS CODING CRITERIA Variable Criteria Attrition at final timepoint Low: 0-19% High: >20% Unclear: not reported Trainer adherence rating Low: >90% of trainings to fidelity High: <89% of trainings to fidelity Unclear: not reported Training evidence rating Low: standardized training High: researcher-developed training Unclear: not reported Trainer qualifications Low: Experienced/certified High: No training requirements Unclear: not reported Study design Low: RCT, QED High: Pre-experimental Unclear: not reported Data collection method Low: Assessment, observations High: Self-report surveys Sponsor relationship Low: No relationship High: Relationship Unclear: not reported Note. Attrition cutoff points based on rationale from Schulz and Grimes (2002). 63 APPENDIX C TABLE OF SUPPLEMENTARY REPORTS Main report Supplementary report Jacobson et al., 2012 Osteen et al., 2018 Lancaster et al., 2014 Quinnett & Baker, 2009 Rallis, 2016 Rallis et al., 2018 64 APPENDIX D TABLE OF INCLUDED STUDIES Author(s) (year), Study design Name of intervention Outcome measure country Longest Length of intervention Effect size (Standardized mean gain), [95% CI] follow-up Implementation setting Gatekeeper Cultural responsiveness Behavioral Intention Intervention Behavior training arm Pre-Post Pre-Short Pre-Short Pre-Long sample size (n) Follow-Up Follow-up Follow-Up Adams et al., (2018), Pre- QPR 0.96, [0.79, 0.49, [0.25, USA experimental Not reported 1.14] 0.74] 9-months School/university 182 No cultural responsiveness Aldrich et al., Pre- QPR 1.40, [1.16, (2018), USA experimental 1 hour 1.64] Post-training School/university 108 No cultural responsiveness Armstrong et al., QED Talking about suicide 0.95, [0.78, 0.51, [0.29, (2020), Australia 4-months 5 hours 1.12] 0.73] 192 Community setting Designed to be culturally appropriate for Aboriginal and Torres Strait Islander communities and non- Indigenous individuals working with these communities 65 Author(s) (year), Study design Name of intervention Outcome measure country Longest Length of intervention Effect size (Standardized mean gain), [95% CI] follow-up Implementation setting Gatekeeper Cultural responsiveness Behavioral Intention Intervention Behavior training arm Pre-Post Pre-Short Pre-Short Pre-Long sample size (n) Follow-Up Follow-up Follow-Up Bell, (2015), USA Pre- QPR 1.04, [0.92, 0.77, [0.66, experimental 1.5 hours 1.17] 0.88] 3-months School/university 413 No cultural responsiveness Bockhoff et al., QED No name 1.29, [0.63, 1.18, [0.54, (2022), Germany 3-months 12 hours 1 .95] 1.81] 45 School/university No cultural responsiveness Burket, (2017), USA RCT We are the Safety Net 0.03, [-0.34, Post-training 1.5 hours 0.42] 27 School/university No cultural responsiveness Capp et al., (2001), Pre- No name 0.22, [-0.51, Australia experimental 8 hours 0.08] Post-training Community setting 44 Developed by and for the Aboriginal community; workshops tailored to the needs of each workforce 66 Author(s) (year), Study design Name of intervention Outcome measure country Longest Length of intervention Effect size (Standardized mean gain), [95% CI] follow-up Implementation setting Gatekeeper Cultural responsiveness Behavioral Intention Intervention Behavior training arm Pre-Post Pre-Short Pre-Short Pre-Long sample size (n) Follow-Up Follow-up Follow-Up Cerel et al., (2012), Pre- QPR 0.91, [0.87, USA experimental 0.5-3+ hours 0.94] Post-training Community setting 3,958 No cultural responsiveness Clark et al., (2010), Pre- Samaritans of NY Training 0.31, [0.21, USA experimental programme 0.42] Post-training 3 hours 365 Community setting No cultural responsiveness Coleman & Del QED QPR / ASIST / QPR: Quest, (2015), USA 6-months RESPONSE 0.36, [0.01, 126 Not reported 0.72] Community setting, School/university RESPONSE: No cultural responsiveness 0.27, [0.06, 0.49] ASIST: 0.84, 0.39, 1.28] 67 Author(s) (year), Study design Name of intervention Outcome measure country Longest Length of intervention Effect size (Standardized mean gain), [95% CI] follow-up Implementation setting Gatekeeper Cultural responsiveness Behavioral Intention Intervention Behavior training arm Pre-Post Pre-Short Pre-Short Pre-Long sample size (n) Follow-Up Follow-up Follow-Up Davidson & Range, Pre- No name N/A (1999), USA experimental 1 hour Post-training School/university 75 No cultural responsiveness Diamond et al., QED More Than Sad 0.64, [0.57, 0.52, [0.45, (2021), USA 2-months 2 hours 0.70] 0.58] 1,030 School/university No cultural responsiveness Duong-Killer, Pre- QPR 1.15, [1.04, (2015), USA experimental 1-1.5 hours 1.27] Post-training School/university 734 No cultural responsiveness Ewell Foster et al., Pre- ASIST 0.20, [0.11, (2017), USA experimental 16 hours 0.29] 6-9 months Community setting 435 No cultural responsiveness Goldstein, (2017), Pre- QPR N/A N/A USA experimental 2 hours 6-weeks School/university 16 No cultural responsiveness 68 Author(s) (year), Study design Name of intervention Outcome measure country Longest Length of intervention Effect size (Standardized mean gain), [95% CI] follow-up Implementation setting Gatekeeper Cultural responsiveness Behavioral Intention Intervention Behavior training arm Pre-Post Pre-Short Pre-Short Pre-Long sample size (n) Follow-Up Follow-up Follow-Up Hangartner et al., Pre- QPR N/A (2019), USA experimental 2 hours 3-months Social services/Non-profit 2,389 No cultural responsiveness Hjelvik et al., Pre- No name N/A (2022), USA experimental 1.75 hours Post-training School/university 273 Curriculum was modified to be more relevant and applicable to a medical student audience Indelicato et al., Pre- QPR N/A N/A (2011), USA experimental Not reported 3-months School/university 917 No cultural responsiveness Jacobson et al., RCT QPR 0.45, [0.09, (2012) / Osteen, 6-months 1.5 hours 0.80] (2018), USA 34 School/university No cultural responsiveness 69 Author(s) (year), Study design Name of intervention Outcome measure country Longest Length of intervention Effect size (Standardized mean gain), [95% CI] follow-up Implementation setting Gatekeeper Cultural responsiveness Behavioral Intention Intervention Behavior training arm Pre-Post Pre-Short Pre-Short Pre-Long sample size (n) Follow-Up Follow-up Follow-Up Kahsay et al., Pre- SafeTalk 0.60, [0.40, (2020), USA experimental 4 hours 0.79] 6-months Government agency 248 No cultural responsiveness Kaplan Research Pre- SafeTalk N/A Associates, (2018), experimental 4 hours Canada Post-training School/university 638 No cultural responsive ness Lancaster et al., QED QPR Online: Online: Online: (2014) / Quinnett & 6-months Not reported 0.72, [0.51, 0.75, [0.44, 0.20, [-0.01, Baker, (2009), 1,056 Social services/Non-profit, 0.93] 1.06] 0.41] Australia & USA Community setting No cultural responsiveness Face-to-Face: Face-to-Face: Face-to-Face: 0.76, [0.69, 0.70, [0.55, 0.12, [0.01, 0.84] 0.86] 0.23] Lapidos et al., Pre- No name 0.87, [0.50, (2021), USA experimental 1 hour 1.24] Post-training Community setting 40 No cultural responsiveness 70 Author(s) (year), Study design Name of intervention Outcome measure country Longest Length of intervention Effect size (Standardized mean gain), [95% CI] follow-up Implementation setting Gatekeeper Cultural responsiveness Behavioral Intention Intervention Behavior training arm Pre-Post Pre-Short Pre-Short Pre-Long sample size (n) Follow-Up Follow-up Follow-Up Mitchell et al., Pre- QPR N/A N/A (2013), USA experimental 1.5 hours 3-6 months School/university 911 No cultural responsiveness Muehlenkamp & Pre- LifeSavers Program 1.71, [1.62, 1.16, [1.03, Quinn-Lee, (2021), experimental 0.75 hours 1 .79] 1.29] USA 3-months School/university 1,345 No cultural responsiveness Orovecz, (2020), Pre- UW-Madison suicide 0.42, [0.31, 0.34, [0.41, USA experimental prevention trainings 0.53] 0.98] 2-4 months 1-3 hours 517 School/university No cultural responsiveness Osteen et al., (2018), Pre- YDS: Let's Talk 0.23, [-0.32, USA experimental 3-4 hours 0.78] 6-months Government agency 42 No cultural responsiveness Osteen et al., (2021), Pre- QPR for Law Enforcement N/A USA experimental 5.5-6.5 hours 3-months Government agency 95 No cultural responsiveness 71 Author(s) (year), Study design Name of intervention Outcome measure country Longest Length of intervention Effect size (Standardized mean gain), [95% CI] follow-up Implementation setting Gatekeeper Cultural responsiveness Behavioral Intention Intervention Behavior training arm Pre-Post Pre-Short Pre-Short Pre-Long sample size (n) Follow-Up Follow-up Follow-Up Pullen et al., (2016), Pre- QPR N/A USA experimental 1-1.5 hours Post-training School/university 147 No cultural responsiveness Pullen et al., (2017), Pre- QPR N/A USA experimental 1.5 hours Post-training Community setting 894 No cultural responsiveness Rallis, (2016) / Pre- No name 0.24, [0.05, Rallis et al., (2018), experimental 1 hour 0.43] USA 3-months School/university 231 No cultural responsiveness Ramsey-Wilson, Pre- QPR / SafeTalk 1.00, [0.69, (2021), USA experimental Not reported 1.31] Post-training School/university 60 No cultural responsiveness Rice, (2016), USA Pre- QPR 1.67, [1.54, experimental 1 hour 1.79] Post-training School/university 568 No cultural responsiveness 72 Author(s) (year), Study design Name of intervention Outcome measure country Longest Length of intervention Effect size (Standardized mean gain), [95% CI] follow-up Implementation setting Gatekeeper Cultural responsiveness Behavioral Intention Intervention Behavior training arm Pre-Post Pre-Short Pre-Short Pre-Long sample size (n) Follow-Up Follow-up Follow-Up Samuolis et al., Pre- QPR 1.24, [1.03, (2020), USA experimental Not reported 1.44] Post-training School/university 161 N o cultural responsiveness Teo et al., (2016), QED No name 0.69, [0.42, USA Post-training 2 hours 0.96] 67 Community setting Training targeted Japanese American individuals; provided materials in English and Japanese Terpstra et al., Pre- No name N/A (2018), Netherlands experimental 4 hours 6-weeks Community setting 502 No cultural responsiveness Tompkins & Witt, QED QPR 0.70, [0.41, 0.36, [0.09, (2009), USA 5-months 1 hour 0.98] 0.62] 122 School/university No cultural responsiveness 73 Author(s) (year), Study design Name of intervention Outcome measure country Longest Length of intervention Effect size (Standardized mean gain), [95% CI] follow-up Implementation setting Gatekeeper Cultural responsiveness Behavioral Intention Intervention Behavior training arm Pre-Post Pre-Short Pre-Short Pre-Long sample size (n) Follow-Up Follow-up Follow-Up Tompkins et al., QED QPR 1.53, [1.20, 1.72, [1.00, (2009), USA 3-months 2 hours 1.86] 2.45] 78 School/university No cultural responsiveness Tsong et al., (2019), Pre- QPR 1.09, [0.98, USA experimental 1.5 hours 1.21] Post-training School/university 477 No cultural responsiveness Walsh et al., (2013), Pre- No name 0.64, [0.48, USA experimental 1.5 hours 0.80] Post-training School/university 287 No cultural responsiveness Wood et al., (2022), Pre- QPR Educational USA experimental Not reported setting: Post-training Community setting 2.28, [2.15, 1,445 No cultural responsiveness 2.42] Religious setting: 2.23, [2.09, 2.37] 74 Author(s) (year), Study design Name of intervention Outcome measure country Longest Length of intervention Effect size (Standardized mean gain), [95% CI] follow-up Implementation setting Gatekeeper Cultural responsiveness Behavioral Intention Intervention Behavior training arm Pre-Post Pre-Short Pre-Short Pre-Long sample size (n) Follow-Up Follow-up Follow-Up Wyman et al., RCT QPR 0.30, [0.13, (2008), USA 1-year 1.5 hours 0.47] 166 School/university No cultural responsiveness Zinzow et al., Pre- No name 1.31, [1.21, 1.21, [1.01, (2020), USA experimental 1.5 hours 1 .40] 1.42] 3-months School/university 542 No cultural responsiveness Notes. RCT = randomized controlled trial. QED = Quasi-experimental design. N/A indicates the study either could not provide appropriate data to code an effect size or the variable was not measured in a manner that was comparable across studies. Cells left blank indicate that variable was not measured. 75 APPENDIX E RISK OF BIAS TABLE Study Adams et al., 2018 H U L U H H U Aldrich et al., 2018 H U L U H H L Armstrong et al., 2020 H U H L L H L Bell, 2015 L U L L H H L Bockhoff et al., 2022 U U H L L H L Burket, 2017 H U H U L H L Capp et al., 2001 U U H U H H L Cerel et al., 2012 U U L L H H L Clark et al., 2010 L U H U H H L Coleman & Del Quest, 2015 U U L U L H U Davidson & Range, 1999 U U U U H H U Diamond et al., 2021 H U L U L H H Duong-Killer, 2015 U U L U H H U Ewell Foster et al., 2017 H U L L H H L Goldstein, 2017 H U L U H H U Hangartner et al., 2019 H L L L H H L Hjelvik et al., 2022 L U H U H H U Indelicato et al., 2011 H U L L H H U Jacobson et al., 2012 L U L L L H L Kahsay et al., 2020 H U L L H H U Kaplan, G., 2018 H U L L H H U Lancaster et al., 2014 H U L U L H L Lapidos et al., 2021 L U H U H H L Mitchell et al., 2013 H U L L H H L Muehlenkamp & Quinn-Lee, 2021 H U H L H H L Orovecz, 2020 H U H U H H U Osteen et al., 2018 H U H U H H U Osteen et al., 2021 H U L U H H U Pullen et al., 2016 U U L U H H L 76 Attrition at final timepoint Trainer adherence rating Training evidence rating Trainer qualifications Study design Data collection method Sponsor relationship Study Pullen et al., 2017 L U L U H H U Rallis, 2017 H U H L H H L Ramsey-Wilson, 2021 U U L U H H U Rice, 2016 U U L U H H U Samuolis et al., 2020 U U L L H H L Teo et al., 2016 U U H U L H L Terpstra et al., 2018 H U H L H H U Tompkins & Witt, 2009 H U L L L H L Tompkins et al., 2010 H U L L L H U Tsong et al., 2019 L U L L H H L Walsh et al., 2013 L U H U H H L Wood et al., 2022 U U L U H H L Wyman et al., 2008 H H L L L H L Zinzow et al., 2020 H U H L H H L Notes. H = High risk, L = Low risk, U = Unclear risk 77 Attrition at final timepoint Trainer adherence rating Training evidence rating Trainer qualifications Study design Data collection method Sponsor relationship APPENDIX F ADDITIONAL PUBLICATION/SMALL STUDY BIAS FIGURES Figure F1 Trim and fill funnel plot for pre-post training behavioral intention analysis Figure F2 Cumulative meta-analysis for pre-post training behavioral intention analysis 78 Figure F3 Trim and fill funnel plot for pre-training to short-term follow-up behavioral intention analysis Figure F4 Cumulative meta-analysis for pre-training to short-term follow-up behavioral intention analysis 79 Figure F5 Trim and fill funnel plot for pre-training to short-term follow-up intervention behavior analysis Figure F6 Cumulative meta-analysis for pre-training to short-term follow-up intervention behavior analysis 80 APPENDIX G SENSITIVITY ANALYSES Table G1 Forest plot for pre-post training behavioral intention analysis with .1 pre-post correlation 81 Table G2 Forest plot for pre-post training behavioral intention analysis with .9 pre-post correlation 82 REFERENCES CITED References marked with an asterisk indicate studies included in the systematic review and meta-analysis. *Adams, L. M., Nguyen, T., Morgan, K. B., & Gumbleton, C. (2018). RU OK: Evaluating the effectiveness of a gatekeeper training program. Journal of College Student Development, 59(5), 614–617. https://doi.org/10.1353/csd.2018.0056 Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179–211. https://doi.org/10.1016/0749- 5978(91)90020-T *Aldrich, R. S., Wilde, J., & Miller, E. (2018). The effectiveness of QPR suicide prevention training. Health Education Journal, 77(8), 964–977. https://doi.org/10.1177/0017896918786009 Althubaiti, A. (2016). Information bias in health research: Definition, pitfalls, and adjustment methods. Journal of Multidisciplinary Healthcare, 9, 211–217. https://doi.org/10.2147/JMDH.S104807 *Armstrong, G., Sutherland, G., Pross, E., Mackinnon, A., Reavley, N., & Jorm, A. F. (2020). Talking about suicide: An uncontrolled trial of the effects of an Aboriginal and Torres Strait Islander mental health first aid program on knowledge, attitudes and intended and actual assisting actions. PLOS ONE, 15(12), e0244091. https://doi.org/10.1371/journal.pone.0244091 Bandura, A. (2001). Social cognitive theory: An agentic perspective. Annual Review of Psychology, 52(1), 1–26. https://doi.org/10.1146/annurev.psych.52.1.1 83 *Bell, K. (2015). Examining the effectiveness of gatekeeper training in suicide prevention [Doctoral dissertation]. Texas Woman’s University. *Bockhoff, K., Ellermeier, W., & Bruder, S. (2022). Evaluation of a suicide prevention program encompassing both student and teacher training components. Crisis, 1–9. https://doi.org/10.1027/0227-5910/a000862 Borenstein, M., Hedges, L. V., Higgins, P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. John Wiley & Sons, Ltd. *Burket, R. (2017). We are the safety net: Skills for suicide prevention evaluating a training to increase recognition and response to signs of suicide among at-risk peers [Doctoral dissertation]. University of Alaska Fairbanks and Anchorage. Burnette, C., Ramchand, R., & Ayer, L. (2015). Gatekeeper training for suicide prevention (p. 19). RAND Corporation. https://www.rand.org/pubs/research_reports/RR1002.html *Capp, K., Deane, F. P., & Lambert, G. (2001). Suicide prevention in Aboriginal communities: Application of community gatekeeper training. Australian and New Zealand Journal of Public Health, 25(4), 315–321. https://doi.org/10.1111/j.1467- 842X.2001.tb00586.x Centers for Disease Control and Prevention. (2023, January 12). Disparities in suicide. https://www.cdc.gov/suicide/facts/disparities-in-suicide.html *Cerel, J., Padgett, J. H., Robbins, V., & Kaminer, B. (2012). A state’s approach to suicide prevention awareness: Gatekeeper training in Kentucky. Journal of Evidence-Based Social Work, 9(3), 283–292. https://doi.org/10.1080/15433714.2010.498672 84 *Clark, T. R., Matthieu, M. M., Ross, A., & Knox, K. L. (2010). Training outcomes from the Samaritans of New York Suicide Awareness and Prevention Programme among community- and school-based staff. British Journal of Social Work, 40(7), 2223–2238. https://doi.org/10.1093/bjsw/bcq016 Clement, S., Schauman, O., Graham, T., Maggioni, F., Evans-Lacko, S., Bezborodovs, N., Morgan, C., Rüsch, N., Brown, J. S. L., & Thornicroft, G. (2015). What is the impact of mental health-related stigma on help-seeking? A systematic review of quantitative and qualitative studies. Psychological Medicine, 45(1), 11–27. https://doi.org/10.1017/S0033291714000129 Clifford, A. C., Doran, C. M., & Tsey, K. (2013). A systematic review of suicide prevention interventions targeting indigenous peoples in Australia, United States, Canada and New Zealand. BMC Public Health, 13, 463. https://doi.org/10.1186/1471-2458-13-463 *Coleman, D., & Del Quest, A. (2015). Science from evaluation: Testing hypotheses about differential effects of three youth-focused suicide prevention trainings. Social Work in Public Health, 30(2), 117–128. https://doi.org/10.1080/19371918.2014.938397 Corrigan, P. (2004). How stigma interferes with mental health care. American Psychologist, 59(7), 614–625. https://doi.org/10.1037/0003-066X.59.7.614 Cross, W., Matthieu, M. M., Lezine, D., & Knox, K. L. (2010). Does a brief suicide prevention gatekeeper training program enhance observed skills? Crisis: The Journal of Crisis Intervention and Suicide Prevention, 31(3), 149–159. https://doi.org/10.1027/0227-5910/a000014 85 *Davidson, M. W., & Range, L. M. (1999). Are teachers of children and young adolescents responsive to suicide prevention training modules? Yes. Death Studies, 23(1), 61–71. https://doi.org/10.1080/074811899201190 *Diamond, G., Ogunkua, L., Atte, T., Rosen, P., Wintersteen, M., & Gallop, R. (2021). The effectiveness of the More Than Sad school-based gatekeeper training program. School Mental Health, 13(4), 655–666. https://doi.org/10.1007/s12310- 021-09444-8 *Duong-Killer, J. (2015). Suicide prevention training: Its impact on college students of color [Master’s thesis]. California State University, Long Beach. Duval, S., & Tweedie, R. (2000). Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56(2), 455–463. https://doi.org/10.1111/j.0006-341x.2000.00455.x Egger, M., Davey Smith, G., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ (Clinical Research Ed.), 315, 629–634. https://doi.org/10.1136/bmj.315.7109.629 *Ewell Foster, C. J., Burnside, A. N., Smith, P. K., Kramer, A. C., Wills, A., & A. King, C. (2017). Identification, response, and referral of suicidal youth following Applied Suicide Intervention Skills Training. Suicide and Life-Threatening Behavior, 47(3), 297–308. https://doi.org/10.1111/sltb.12272 *Goldstein, J. (2017). Increasing perceived self-efficacy of accelerated BSN students in suicide assessment and management: Is online training a viable option? [Doctoral dissertation]. Fairleigh Dickinson University. 86 Han, J., Batterham, P. J., Calear, A. L., & Randall, R. (2018). Factors influencing professional help-seeking for suicidality: A systematic review. Crisis, 39(3), 175– 196. https://doi.org/10.1027/0227-5910/a000485 *Hangartner, R. B., Totura, C. M. W., Labouliere, C. D., Gryglewicz, K., & Karver, M. S. (2019). Benchmarking the “Question, Persuade, Refer” program against evaluations of established suicide prevention gatekeeper trainings. Suicide and Life-Threatening Behavior, 49(2), 353–370. https://doi.org/10.1111/sltb.12430 Hawgood, J., Woodward, A., Quinnett, P., & De Leo, D. (2021). Gatekeeper training and minimum standards of competency: Essentials for the suicide prevention workforce. Crisis, 1–7. https://doi.org/10.1027/0227-5910/a000794 Hedegaard, H., Curtin, S. C., & Warner, M. (2020). Increase in suicide mortality in the United States, 1999–2018 (NCHS Data Brief, no 362; p. 8). National Center for Health Statistics. Henderson, C., Evans-Lacko, S., & Thornicroft, G. (2013). Mental illness stigma, help seeking, and public health programs. American Journal of Public Health, 103(5), 777–780. https://doi.org/10.2105/AJPH.2012.301056 *Hjelvik, A., Eldridge, A., Furnari, M., Hoeflich, H., Chen, J. I., Roth, B., & Black, W. (2022). A peer-to-peer suicide prevention workshop for medical students. MedEdPORTAL, 18(11241), 1–7. https://doi.org/10.15766/mep_2374-8265.11241 Holmes, G., Clacy, A., Hermens, D. F., & Lagopoulos, J. (2021a). The long-term efficacy of suicide prevention gatekeeper training: A systematic review. Archives of Suicide Research, 25(2), 177–207. https://doi.org/10.1080/13811118.2019.1690608 87 Holmes, G., Clacy, A., Hermens, D. F., & Lagopoulos, J. (2021b). Evaluating the longitudinal efficacy of safeTALK suicide prevention gatekeeper training in a general community sample. Suicide and Life-Threatening Behavior, 51(5), 844– 853. https://doi.org/10.1111/sltb.12741 House Bill 2315, Ore. Stat. § 675 (2021). https://olis.oregonlegislature.gov/liz/2021R1/Downloads/MeasureDocument/HB2 315 *Indelicato, N. A., Mirsu-Paun, A., & Griffin, W. D. (2011). Outcomes of a suicide prevention gatekeeper training on a university campus. Journal of College Student Development, 52(3), 350–361. https://doi.org/10.1353/csd.2011.0036 Isaac, M., Elias, B., Katz, L. Y., Belik, S.-L., Deane, F. P., Enns, M. W., & Sareen, J. (2009). Gatekeeper training as a preventative intervention for suicide: A systematic review. The Canadian Journal of Psychiatry, 54(4), 260–268. https://doi.org/10.1177/070674370905400407 *Jacobson, J. M., Osteen, P. J., Sharpe, T. L., & Pastoor, J. B. (2012). Randomized trial of suicide gatekeeper training for social work students. Research on Social Work Practice, 22(3), 270–281. https://doi.org/10.1177/1049731511436015 *Kahsay, E., Magness, C. S., Persky, S., Smith, P. K., & Foster, C. E. (2020). Suicide prevention training in the child welfare workforce: Knowledge, attitudes, and practice patterns prior to and following safeTALK training. Child Welfare, 98(1), 95–114. *Kaplan Research Associates. (2018). A formative evaluation of the safeTALK program in Manitoba. https://legacy.livingworks.net/dmsdocument/830 88 Katz, C., Bolton, S., Katz, L. Y., Isaak, C., Tilston‐Jones, T., & Sareen, J. (2013). A systematic review of school-based suicide prevention programs. Depression & Anxiety (1091-4269), 30(10), 1030–1045. https://doi.org/10.1002/da.22114 Keller, D. P., Schut, L. J. A., Puddy, R. W., Williams, L., Stephens, R. L., McKeon, R., & Lubell, K. (2009). Tennessee Lives Count: Statewide gatekeeper training for youth suicide prevention. Professional Psychology : Research and Practice, 40(2), 126. https://doi.org/10.1037/a0014889 *Lancaster, P. G., Moore, J. T., Putter, S. E., Chen, P. Y., Cigularov, K. P., Baker, A., & Quinnett, P. (2014). Feasibility of a web-based gatekeeper training: Implications for suicide prevention. Suicide and Life-Threatening Behavior, 44(5), 510–523. https://doi.org/10.1111/sltb.12086 *Lapidos, A., Spinner, J., Garlick, J., & Kees, M. (2021). Immediate impact of a brief suicide prevention training for veteran volunteers. Pedagogy in Health Promotion, 1–7. https://doi.org/10.1177/2373379921989383 LivingWorks Education Inc. (n.d.). LivingWorks Start. LivingWorks. Retrieved July 1, 2022, from https://www.livingworks.net/start *Mitchell, S. L., Kader, M., Darrow, S. A., Haggerty, M. Z., & Keating, N. L. (2013). Evaluating Question, Persuade, Refer (QPR) suicide prevention training in a college setting. Journal of College Student Psychotherapy, 27(2), 138–148. https://doi.org/10.1080/87568225.2013.766109 89 Mo, P. K. H., Ko, T. T., & Xin, M. Q. (2018). School-based gatekeeper training programmes in enhancing gatekeepers’ cognitions and behaviours for adolescent suicide prevention: A systematic review. Child and Adolescent Psychiatry and Mental Health, 12(1), 29. https://doi.org/10.1186/s13034-018-0233-4 Morton, M., Wang, S., Tse, K., Chung, C., Bergmans, Y., Ceniti, A., Flam, S., Johannes, R., Schade, K., Terah, F., & Rizvi, S. (2021). Gatekeeper training for friends and family of individuals at risk of suicide: A systematic review. Journal of Community Psychology, 49(6), 1838–1871. https://doi.org/10.1002/jcop.22624 *Muehlenkamp, J. J., & Quinn-Lee, L. (2021). Effectiveness of a peer-led gatekeeper program: A longitudinal mixed-method analysis. Journal of American College Health, 1–10. https://doi.org/10.1080/07448481.2021.1891080 Nasir, B. F., Hides, L., Kisely, S., Ranmuthugala, G., Nicholson, G. C., Black, E., Gill, N., Kondalsamy-Chennakesavan, S., & Toombs, M. (2016). The need for a culturally-tailored gatekeeper training intervention program in preventing suicide among Indigenous peoples: A systematic review. BMC Psychiatry, 16(1), 357. https://doi.org/10.1186/s12888-016-1059-3 Nunan, D., Aronson, J., & Bankhead, C. (2018). Catalogue of bias: Attrition bias. BMJ Evidence-Based Medicine, 23(1), 21–22. https://doi.org/10.1136/ebmed-2017- 110883 *Orovecz, J. J. (2020). Evaluating the effects of suicide prevention gatekeeper trainings in a university setting [Doctoral dissertation]. University of Wisconson - Madison. 90 *Osteen, P. J. (2018). Suicide intervention gatekeeper training: Modeling mediated effects on development and use of gatekeeper behaviors. Research on Social Work Practice, 28(7), 848–856. https://doi.org/10.1177/1049731516680298 Osteen, P. J., Frey, J. J., & Ko, J. (2014). Advancing training to identify, intervene, and follow up with individuals at risk for suicide through research. American Journal of Preventive Medicine, 47(3, Suppl 2), S216–S221. https://doi.org/10.1016/j.amepre.2014.05.033 *Osteen, P. J., Lacasse, J. R., Woods, M. N., Greene, R., Frey, J. J., & Lane Forsman, R. (2018). Training youth services staff to identify, assess, and intervene when working with youth at high risk for suicide. Children and Youth Services Review, 86, 308–315. https://doi.org/10.1016/j.childyouth.2018.02.008 *Osteen, P. J., Ohme, K., Morris, R. C., Arciniegas, J., Frey, J. J., Woods, M., & Forsman, R. L. (2021). Suicide intervention training with law enforcement officers. Suicide and Life-Threatening Behavior, 51(4), 785–794. https://doi.org/10.1111/sltb.12763 Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo- Wilson, E., McDonald, S., … Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372, n71. https://doi.org/10.1136/bmj.n71 91 Pistone, I., Beckman, U., Eriksson, E., Lagerlöf, H., & Sager, M. (2019). The effects of educational interventions on suicide: A systematic review and meta-analysis. The International Journal of Social Psychiatry, 65(5), 399–412. https://doi.org/10.1177/0020764019852655 Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011). Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda. Administration and Policy in Mental Health, 38(2), 65–76. https://doi.org/10.1007/s10488-010-0319- 7 *Pullen, J. M., Gilje, F., & Tesar, E. (2016). A descriptive study of baccalaureate nursing students’ responses to suicide prevention education. Nurse Education in Practice, 16(1), 104–110. https://doi.org/10.1016/j.nepr.2015.09.007 *Pullen, J. M., Loftsgaarden, E., Stahley, N., & Gilje, F. (2017). A community’s response to gatekeeper training. Journal of Community & Public Health Nursing, 03(04). https://doi.org/10.4172/2471-9846.1000195 Quinnett, P. (n.d.). QPR: Ask a question, save a life. The QPR Institute. https://whs.avondale.edu.au/public/Mental%20Health%20Resources/QPR%20Sui cide%20Prevention/QPR%20Review%20Booklet.pdf *Quinnett, P., & Baker, A. (2009). Web-based suicide prevention education: Innovations in research, training, and practice. In L. Sher & A. Vilens (Eds.), Internet and Suicide (p. 22). Nova Science Publishers, Inc. R Core Team. (2020). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/ 92 *Rallis, B. A. (2016). A brief peer gatekeeper suicide prevention training: Preliminary examination and individual factors that influence outcomes [Doctoral dissertation]. George Mason University. *Rallis, B. A., Esposito-Smythers, C., Disabato, D. J., Mehlenbeck, R. S., Kaplan, S., Geer, L., Adams, R., & Meehan, B. (2018). A brief peer gatekeeper suicide prevention training: Results of an open pilot trial. Journal of Clinical Psychology, 74(7), 1106–1116. https://doi.org/10.1002/jclp.22590 Ramsay, R. F., Cooke, M. A., & Lang, W. A. (1990). Alberta’s suicide prevention training programs: A retrospective comparison with Rothman’s developmental research model. Suicide and Life-Threatening Behavior, 20(4), 335–351. *Ramsey-Wilson, G. M. (2021). Campus gatekeeper trainings: An analysis of Question, Persuade, Refer (QPR) and SafeTALK [Master’s thesis, Old Dominion University]. https://digitalcommons.odu.edu/psychology_etds/373/ *Rice, J. (2016). The effectiveness of QPR suicide prevention training [Master’s thesis]. California State University, Sacramento. Rodgers, P. (2010). Review of the Applied Suicide Intervention Skills Training Program (ASIST): Rationale, evaluation results, and directions for future research. LivingWorks Education Inc. https://vivatselvmordsforebygging.net/wp- content/uploads/2015/04/Review-of-the-Applied-Suicide-Intervention-Skills- Training-Program-ASIST.pdf *Samuolis, J., Harrison, A. J., & Flanagan, K. (2020). Evaluation of a peer-led implementation of a suicide prevention gatekeeper training program for college students. Crisis, 41(5), 331–336. https://doi.org/10.1027/0227-5910/a000638 93 Schulz, K. F., & Grimes, D. A. (2002). Sample size slippages in randomised trials: Exclusions and the lost and wayward. The Lancet, 359(9308), 781–785. https://doi.org/10.1016/S0140-6736(02)07882-0 Substance Abuse and Mental Health Services Administration [SAMHSA]. (2019). Key substance use and mental health indicators in the United States: Results from the 2019 National Survey on Drug Use and Health (HHS Publication No. PEP20-07- 01-001, NSDUH Series H-55; p. 114). Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration. https://www.samhsa.gov/data/ Substance Abuse and Mental Health Services Administration [SAMHSA]. (2021). Key substance use and mental health indicators in the United States: Results from the 2020 National Survey on Drug Use and Health (HHS Publication No. PEP21-07- 01-003, NSDUH Series H-56; p. 156). Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration. https://www.samhsa.gov/data/ Suicide Prevention Resource Center. (2020a). A comprehensive approach to suicide prevention. https://www.sprc.org/effective-prevention/comprehensive-approach Suicide Prevention Resource Center. (2020b). Selecting and Implementing a Gatekeeper Training. Education Development Center. https://www.sprc.org/sites/default/files/Selecting%20and%20Implementing%20a %20Gatekeeper%20Training.pdf 94 *Teo, A. R., Andrea, S. B., Sakakibara, R., Motohara, S., Matthieu, M. M., & Fetters, M. D. (2016). Brief gatekeeper training for suicide prevention in an ethnic minority population: A controlled intervention. BMC Psychiatry, 16(1), 211. https://doi.org/10.1186/s12888-016-0924-4 *Terpstra, S., Beekman, A., Abbing, J., Jaken, S., Steendam, M., & Gilissen, R. (2018). Suicide prevention gatekeeper training in the Netherlands improves gatekeepers’ knowledge of suicide prevention and their confidence to discuss suicidality, an observational study. BMC Public Health, 18(1), 637. https://doi.org/10.1186/s12889-018-5512-8 *Tompkins, T. L., & Witt, J. (2009). The short-term effectiveness of a suicide prevention gatekeeper training program in a college setting with residence life advisers. The Journal of Primary Prevention, 30(2), 131–149. https://doi.org/10.1007/s10935- 009-0171-2 *Tompkins, T. L., Witt, J., & Abraibesh, N. (2009). Does a gatekeeper suicide prevention program work in a school setting? Evaluating training outcome and moderators of effectiveness. Suicide and Life-Threatening Behavior, 39(6), 671–681. https://doi.org/10.1521/suli.2009.39.6.671 Torok, M., Calear, A. L., Smart, A., Nicolopoulos, A., & Wong, Q. (2019). Preventing adolescent suicide: A systematic review of the effectiveness and change mechanisms of suicide prevention gatekeeping training programs for teachers and parents. Journal of Adolescence, 73, 100–112. https://doi.org/10.1016/j.adolescence.2019.04.005 95 Troya, M. I., Spittal, M. J., Pendrous, R., Crowley, G., Gorton, H. C., Russell, K., Byrne, S., Musgrove, R., Hannam-Swain, S., Kapur, N., & Knipe, D. (2022). Suicide rates amongst individuals from ethnic minority backgrounds: A systematic review and meta-analysis. EClinicalMedicine, 47, 1–23. https://doi.org/10.1016/j.eclinm.2022.101399 *Tsong, Y., Young, J. T., Killer, J. D., Takemoto, M. A., & Compliment, B. (2019). Suicide prevention program on a diverse college campus: Examining the effectiveness of a peer-to-peer model. Journal of College Student Psychotherapy, 33(2), 131–144. https://doi.org/10.1080/87568225.2018.1434716 Turley, B. (2018). safeTALK literature review: An overview of its rationale, conceptual framework, and research foundations. LivingWorks Education Inc. https://legacy.livingworks.net/assets/Public/5508-safeTALK-Literature- Review.pdf U.S. Department of Health and Human Services (HHS) Office of the Surgeon General, & National Action Alliance for Suicide Prevention. (2012). 2012 National Strategy for Suicide Prevention: Goals and Objectives for Action. Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1–48. https://doi.org/10.18637/jss.v036.i03 *Walsh, E., Hooven, C., & Kronick, B. (2013). School-wide staff and faculty training in suicide risk awareness: Successes and challenges. Journal of Child and Adolescent Psychiatric Nursing, 26(1), 53–61. https://doi.org/10.1111/jcap.12011 96 Webb, T. L., & Sheeran, P. (2006). Does changing behavioral intentions engender behavior change? A meta-analysis of the experimental evidence. Psychological Bulletin, 132(2), 249. https://doi.org/10.1037/0033-2909.132.2.249 Wolitzky-Taylor, K., LeBeau, R. T., Perez, M., Gong-Guy, E., & Fong, T. (2020). Suicide prevention on college campuses: What works and what are the existing gaps? A systematic review and meta-analysis. Journal of American College Health, 68(4), 419–429. https://doi.org/10.1080/07448481.2019.1577861 Wong, J., Brownson, C., Rutkowski, L., Nguyen, C. P., & Becker, M. S. (2014). A mediation model of professional psychological help seeking for suicide ideation among Asian American and White American college students. Archives of Suicide Research, 18(3), 259–273. https://doi.org/10.1080/13811118.2013.824831 *Wood, D. S., Ohri, F., Hudnall, G., & Cahoon, L. (2022). Suicide gatekeeper training outcomes in educational and religious settings. Journal of Human Behavior in the Social Environment, 1–7. https://doi.org/10.1080/10911359.2022.2049414 World Health Organization. (2019). Suicide in the world; Global health estimates. *Wyman, P. A., Brown, C. H., Inman, J., Cross, W., Schmeelk-Cone, K., Guo, J., & Pena, J. B. (2008). Randomized trial of a gatekeeper program for suicide prevention: 1-year impact on secondary school staff. Journal of Consulting and Clinical Psychology, 76(1), 104–115. https://doi.org/10.1037/0022-006X.76.1.104 Yonemoto, N., Kawashima, Y., Endo, K., & Yamada, M. (2019). Gatekeeper training for suicidal behaviors: A systematic review. Journal of Affective Disorders, 246, 506–514. https://doi.org/10.1016/j.jad.2018.12.052 97 Zalsman, G., Hawton, K., Wasserman, D., van Heeringen, K., Arensman, E., Sarchiapone, M., Carli, V., Höschl, C., Barzilay, R., Balazs, J., Purebl, G., Kahn, J. P., Sáiz, P. A., Lipsicas, C. B., Bobes, J., Cozman, D., Hegerl, U., & Zohar, J. (2016). Suicide prevention strategies revisited: 10-year systematic review. The Lancet Psychiatry, 3(7), 646–659. https://doi.org/10.1016/S2215-0366(16)30030- X *Zinzow, H. M., Thompson, M. P., Fulmer, C. B., Goree, J., & Evinger, L. (2020). Evaluation of a brief suicide prevention training program for college campuses. Archives of Suicide Research, 24(1), 82–95. https://doi.org/10.1080/13811118.2018.1509749 Ziring, A., Frutiger-Parmley, M., Alvarez, A., May, E., Allen, C., Parson, G., & Winkle, J. (2020). Youth Suicide Assessment in Virtual Environments (Youth SAVE). Oregon Pediatric Society. 98