- Study protocol
- Open Access
Predictive models for suicidal thoughts and behaviors among Spanish University students: rationale and methods of the UNIVERSAL (University & mental health) project
BMC Psychiatry volume 16, Article number: 122 (2016)
Suicide is a leading cause of death among young people. While suicide prevention is considered a research and intervention priority, longitudinal data is needed to identify risk and protective factors associate with suicidal thoughts and behaviors. Here we describe the UNIVERSAL (University and Mental Health) project which aims are to: (1) test prevalence and 36-month incidence of suicidal thoughts and behaviors; and (2) identify relevant risk and protective factors associated with the incidence of suicidal thoughts and behaviors among university students in Spain.
An ongoing multicenter, observational, prospective cohort study of first year university students in 5 Spanish universities. Students will be assessed annually during a 36 month follow-up. The surveys will be administered through an online, secure web-based platform. A clinical reappraisal will be completed among a subsample of respondents. Suicidal thoughts and behaviors will be assess with the Self-Injurious Thoughts and Behaviors Interview (SITBI) and the Columbia-Suicide Severity Rating Scale (C-SSRS). Risk and protective factors will include: mental disorders, measured with the Composite International Diagnostic Interview version 3.0 (CIDI 3.0) and Screening Scales (CIDI-SC), and the Epi-Q Screening Survey (EPI-Q-SS), socio-demographic variables, self-perceived health status, health behaviors, well-being, substance use disorders, service use and treatment. The UNIVERSAL project is part of the International College Surveys initiative, which is a core project within the World Mental Health consortium. Lifetime and the 12-month prevalence will be calculated for suicide ideation, plans and attempts. Cumulative incidence of suicidal thoughts and behaviors, and mental disorders will be measured using the actuarial method. Risk and protective factors of suicidal thoughts and behaviors will be analyzed by Cox proportional hazard models.
The study will provide valid, innovative and useful data for developing prevention programs for youth suicide and for improving early identification for high-risk students. The longitudinal design of this study will improve causal interpretation of analyzed associations, needed for generating and validating predictive models. It will represent the first results about suicidal thoughts and behaviors in the Spanish university population. The World Mental Health Survey collaboration will permit accurate cross-national comparisons.
Among individuals aged 15-29 years, suicide is a leading cause of death in many European countries . And, for every person who completes suicide, it is believed that 25 people attempt [1, 2]. Given the enormity of the socioeconomic and emotional burden [3, 4], the European Commission and the World Health Organization (WHO) have identified suicide prevention as a core public health target – with a particular focus on adolescents and young adults .
Adolescence and early adulthood is a key developmental period . On the one hand, it is time when youth learn to more effectively navigate stress, hone effective coping strategies, and develop the competencies, attitudes, values, and social network necessary to make a successful transition into adulthood [7, 8]. Conversely, it also is a peak period of mental disorder onset, as it is estimated that 75 % of mental disorders have an age of onset . Moreover, earlier disorder onset is associated with worse prognosis and greater suicide risk .
For some, the transition to university may be stressful and increase risk for mental disorder onset [11, 12]. Perhaps one of the most challenging aspects is managing increased psychosocial stress and academic pressures in a new, unfamiliar environment. Presently, in the United States (US), suicide is the second leading cause of death for university students [13, 14] and between 4 and 10 % of college students report having serious suicidal thoughts in the previous 12 months . In Spain, mental health research in university students is limited [16–18], and thus, there is a dire need to better understand suicidality within this important population segment.
Research synthesized by the Centers for Disease Control and Prevention (CDC) suggests that risk and protective factors vary as a function of age, gender, and ethnicity . Although previous research has explored these factors in other geographical contexts (e.g., United States) [20, 21] it is not clear if these findings would extend to Spanish university students. There are some differences in the transition from school to university in other countries. For example, in Spain, in contrast to US, a majority of the students reside at home or close to their family and they are financially supported by their family during the university period. Thus the stress associated with this transition might be weaker than in other countries. On the other hand, in Spain, although universities have their own psychological or counselling services, the amount of services offered is typically lower than in other countries, such as the US. This might imply a lower access to specific services.
To address these critical gaps in our knowledge, we have initiated the UNIVERSAL project. Here we present the overall study background, objectives, design, and analysis plan. The project is part of the International College Surveys initiative in the context of the World Mental Health (WMH) surveys consortium (http://www.hcp.med.harvard.edu/wmh/), an international initiative created to improve the scientific knowledge about suicidal behavior among university students worldwide. The general objective of the UNIVERSAL project is to assess the prevalence and the incidence of suicidal thoughts and behaviors in Spanish university students. Mental disorders and academic performance are evaluated both as associated factors as well as relevant health and social outcomes. Further, cross-national analyses will be completed to test differences across WMH countries.
The study will address the following issues: i) test prevalence of suicidal thoughts and behaviors among first-year university students in Spain; ii) test incidence of suicidal thoughts and behaviors up to 36 months; iii) identify relevant risk associated with a higher incidence of suicidal thoughts and behaviors, and iv) delineate protective factors associated with less incidence of suicidal thoughts and behaviors.
As a part of the World Mental Health International College Student (WMH-ICS) project (see http://www.hcp.med.harvard.edu/wmh/college_student_survey.php), the UNIVERSAL project is an ongoing multicenter, observational, and prospective cohort study of first-year university students followed over 36 months. Surveys will be conducted via an online, secure web-based platform. Additionally, a clinical reappraisal interview will be completed in a subsample of respondents to ensure validity of assessment methods of mental disorders. Predictive models of suicidal thoughts and behaviors will be estimated and validated. At the time of the manuscript submission the study is in the data collection stage.
Five public universities from different regions of Spain participate: Cadiz University (UCA); Balearic Islands University (UIB); Basque Country University (UPV-EHU); Pompeu Fabra University (UPF); and Miguel Hernández University (UMH). These universities represent about 8 % of the undergraduate enrollment capacity annually offered in Spain.
Eligibility and recruitment
Inclusion criteria: a) ages18 to 24 years; and b) enrolled in the first year of university for the first time. Exclusion criteria: a) poor knowledge of Spanish language; and b) not accept the informed consent to the study. Students under 18 years old are eligible when they turn 18 and satisfy other inclusion criteria. Based on eligibility criteria we estimate that about 18,000 students will be eligible to participate.
All first year undergraduates from participating universities will be invited to participate. Invitation methods across different college campuses, including advertising campaigns at the campus (e.g., stands with information, information in the classrooms, university web) and up to 4 personal e-mail invitation letters from the university authorities. To increase participation, a raffle of academic material (€ 40) is held at the end of the second year among all respondents who complete the surveys.
Data collection on-line platform
Registration and verification
To participate in the study, eligible students are invited to complete the study registration form first through the UNIVERSAL website (see https://encuesta.estudio-universal.net/) and must agree with the informed consent by checking the “I Agree” checkbox. In the registration form, students are asked to provide personal contact information, so that they can be re-contacted to enter the survey. A verification e-mail is then automatically sent to the student’s University e-mail address, which includes a personal access code to the baseline survey (web link and personal password) and a copy of the informed consent form.
Timing of assessments
Respondents will be complete five assessments throughout the study. The baseline survey (T0) is completed after registration, during the 1st year of the university degree (from October to July). The administration time of T0 is about 40 min. Students who complete at least 5 % of the T0 survey will receive an invitation e-mail at the end of the first year (from July to September) with an electronic link to complete a brief online survey (T1). The T1 survey will include questions pertaining to students’ first year experiences at the university. The information gathered at T1 supplements the baseline data (e.g., academic performance) and the expected administration time is 8 min. Then, 12, 24 and 36 months after completion of the baseline survey (T0), respondents will be invited to complete follow-up questionnaires (T2-T3-T4, respectively), with an estimated administration time of 30 min. The study timeline is depicted in Fig. 1.
Data collection platform (DCP)
The DCP is specially designed and developed for the UNIVERSAL project, follows international recommendations and guidelines for computerized assessment (International Test Commission–ITC, 2005). This ensures proper use, technical handling, quality control, and security of the data. The system allows for automatic e-mail reminders to complete the survey, storage of responses as well as output reports of participation and data consistency checks throughout the interview, ensuring for instance that the responses provided are within acceptable ranges and checking for consistency across ages reported throughout the interview (e.g., age of onset younger or equal the age at interview). The link to the survey that is sent to each student is personal, associated to a single code, and it is active for a limited period of time. If the participant does not access the DCP within a week, an automatic email reminder is sent. For security reasons, the platform renews the links and sends new access links and passwords to all the respondents who have never accessed the questionnaire or who have not completed the survey.
Based on previous development form vulnerability-stress models [22–25] that distinguish between vulnerability or distal factors, and stressors or proximal factors, our project gathers self-reported data about suicide thoughts and behaviors (ideation, plans, and attempts) and candidate risk and protective factors: sociodemographic, self-perceived health status, health behaviors, mental wellbeing, mental disorders (attention-deficit hyperactivity disorder (ADHD), major depressive episode (MDE), bipolar disorder (BP), generalized anxiety disorder (GAD), panic disorder (PD), posttraumatic stress disorder (PTSD), eating disorders, and psychosis), alcohol and substance use disorders, history of suicidal thoughts and behaviors, and non-suicidal self-injuries (NSSI), services use and treatment, personal history, stressful live events, psychological factors (personality, hopelessness, stress management, impulsivity and anxiety coping), sexuality, religiosity/spirituality, university expectations and experiences, and academic performance.
The questionnaire content is summarized in Table 1 and is described in more detail below.
Suicidal related behaviors
Suicidal thoughts and behaviors items are taken mostly from the Self-Injurious Thoughts and Behaviors Interview (SITBI)  and Columbia-Suicide Severity Rating Scale (C-SSRS) . Items probe death wishes (“wish you were dead or would go to sleep and never wake up?”), suicidal ideation (“have thoughts of killing yourself?”), suicide plans (“think about how you might kill yourself [e.g. taking pills, shooting yourself] or work out a plan of how to kill yourself?”), and suicide attempts (“make a suicide attempt [i.e., purposefully hurt yourself with at least some intent to die]?”). Several questions about Non-suicidal self-injury (NSSI) query (hurting yourself on purpose, without wanting to die? [for example, cutting yourself, hitting yourself or burning yourself]?). All items will be assessed about the past 12 months and lifetime.
Attention deficit hyperactivity disorder (ADHD), major depressive episode (MDE), bipolar disorder (BP), generalized anxiety disorder (GAD), panic disorder (PD), posttraumatic stress disorder (PTSD), eating disorders, and psychosis will be assessed. Item prompts are drawn from versions of the Composite International Diagnosis Interview version 3.0 (CIDI 3.0) , and Screening Scales (CIDI-SC) , and Epi-Q Screening Survey (EPI-Q-SS) . Adult ADHD Self-Report Scale (ASRS)  will be used to assess attention deficit hyperactivity disorder (ADHD). Evaluation of depressive episode (MDE), bipolar disorder (BD), generalized anxiety disorder (GAD), and panic disorder (PD) include items about onset of the disorder and presence during the last 12 month. Posttraumatic stress disorder (PTSD), eating disorders and psychosis will be assessed with adapted screening items from CIDI-SC.
Age, gender, nationality, race and descriptive information about their status at the university (e.g. grade, part time/full time) will be collected.
Self-perceived health status
Perceptions of physical and mental health “overall physical/mental health” will be evaluated with items adapted from CIDI Screening Scales (CIDI-SC). Additionally, disability and medical conditions (e.g., chronic health problems, head injury) will be evaluated. Disability will be measured with selected items adapted from Sheehan Disability Scale (SDS) .
Health behaviors (e.g., exercise, sleep) diet will be evaluated with items from the Youth Risk Behavior Survey (YRBS)  and CIDI 3.0.
The short version of Warwick-Edinburgh Mental Well-Being Scale instrument (SWEMWBS) , which had been validated for the Spanish general population  and university students , will be utilized.
Substance Use (alcohol and substances)
The Alcohol Use Disorders Identification Test, 10-item version (AUDIT-10) , the Alcohol Substance Involvement Screening Test (ASSIST)  and selected items from the CIDI 3.0. will be used to screen substance use problems and disorders.
Services use and treatment
A selection of items from the Land Combat Study (LCS) , Joint Mental Health Advisory Team 7 (J-MHAT 7) , Stages of Change Readiness and Treatment Eagerness Scale SOCRATES , Katrina Survey , and CIDI 3.0 will be used to assess: (a) current and past use of psychological counseling and medication as well as (b) barriers to and motivation for seeking treatment.
Psychiatric family history, adverse childhood experiences, and family support will be assessed with parts of the CIDI 3.0. Adverse Childhood Experiences Scale (CES) , and Childhood Trauma Questionnaire (CTQ) . Bullying experiences and other interpersonal violence are collected with the Bully Survey (BS) and J-MHAT 7 . Additional childhood information includes questions about childhood experience, perceived social support and social network at school-age.
Stressful life events
Stressful experiences in the past 12 month will be collected with set of items from Live Events Questionnaire (LEQ) , Deployment Risk and Resilience inventory  and others developed for department of defense survey of health related behaviors among active duty military personnel  and for Army STARRS . Items include: (a) injury and/or death of friends or family members, (b) romantic break-ups, (c) life-threatening events, (d) financial stressors, (e) health stressors, (f) social stressors.
We include: personality, hopelessness, impulsivity and anxiety coping/stress management. Personality traits will be evaluated with Ten Item Personality Inventory (TIPI) , which probes extraversion, agreeableness, conscientiousness, emotional stability and openness to experience. Hopelessness will be assessed with selected items from Beck Hopelessness Scale (BHS) . Four items from UPPS Impulsive Behavior scale (UPPS)  will be used for evaluate impulsive behavior.
Sexuality items were developed based on recommendations from best practices . Sexuality items will probe sexual orientation, sexual behavior, and discrimination related to sexual orientation. Multiple Discrimination Scale (MDS)  and Internalized Homophobia scale (IHP)  will be administrated for students who answer non-heterosexual orientation.
Preferences and level of involvement in religious/spirituality events reflect items from the CIDI 3.0.
University expectations and experiences
Expectations about alcohol or drug consumption at university will be collected with items from the Seattle Social Development Project Survey . Other items in this section probe: (a) student experience and expectations in university , (b) reasons for attending university, (c) where they are living. In the follow-up surveys, a set of questions about involvement in activities during the university year, satisfaction with university services, and intentions to leave universities studies are probed.
Final grades will be tracked among students in the follow- up surveys and objectively calculated by information from Universities administration offices.
Adaptation of the questionnaires into Spanish
Many instruments used in this study already have a Spanish version available and have been used in several previous studies. Examples include the SITBI , the CIDI 3.0  and AUDIT . Adaptations of the remaining instruments and items into Spanish were performed by the project study group (approximately 15 % of the questionnaire content). A forward translation by two independent teams was carried out. Each independent team included at least one linguist professional and one mental health professional. An expert panel took part in the adaptation, identifying, and resolving discrepancies between the forward translations. Further adaptations into Basque and Catalan languages (using the Spanish version as the source) were performed in the same way for universities with more than one official language (e.g., Basque for UPV-EHU and Catalan for UPF, UIB, and UMH).
Study quality control procedures
Data quality control procedures will be implemented and the results will be reported on a weekly basis with regard to the following aspects: survey participation and duration, and individual and item level quality data.
A table with the number of students registered to the survey and summary of the participation, and completion rate. The table includes a summary of the validation status of registered individuals as provided by the universities. Approximately once a month the corresponding universities will validate new registrations and will report the causes of exclusion. For each participant, there will be three possible validation status: valid (i.e., inclusion criteria are fulfilled), excluded (i.e., inclusion criteria are not fulfilled), or pending validation.
A table will report descriptive statistics of the duration time for each University and the overall sample. Additionally, the number of individuals below and over a pre-specified threshold will be reported in order to identify possible outliers. We have set as values to flag attention about response quality: a total time of less than 15 min. and of more than 60 min (as they correspond to the 25th and 75th percentiles of the response distribution).
Quality checks at the individual level
To identify possible unreliable respondents, the respondents with a higher proportion of empty responses and a higher proportion of “I don’t want to answer” responses will be listed and the corresponding proportions will be displayed.
Quality checks at the item level
In order to identify problematic items or possible skip errors of the interview, those items with a higher proportion of empty responses and those not willing to answer are listed. The 95th percentile cutoff will be used to identify candidate problematic items.
More details on the data quality control procedures are provided in Table 2.
Confidentiality and ethical issues
Information collected will be treated as strictly confidential, in compliance with the provisions of the Spanish Law 41/2002, 14th November, and the 1999 Spanish Data Protection Act (LOPD). The UNIVERSAL project was approved by Parc de Salut MAR-Clinical Research Ethics Committee. Reference number 2013/5252/I. All the information will be protected in compliance with data gathering procedures following the Code of Ethics and the Helsinki Declaration (Seoul 2008 revision).
An online informed consent is obtained for every respondent. The respondent must agree with the informed consent by checking the “I Agree” checkbox in https://encuesta.estudio-universal.net/es/inscripcion.
Although inquiring about suicidal behavior does not increase the risk of suicidal behavior, it is important to minimize any possible risk [59, 60]. Hence, at the end of the survey, all respondents will receive a general notification which provides information on how to access specialized mental health services. For the participants responding “yes” to any of the screening items for suicidal thoughts and behavior (i.e., ideation, plan or intent) or NSSI in the last 12 month, a specific alert will provide information for consulting with a health professional.
The instrument used in the UNIVERSAL project for assessing mental disorders is an adapted version of the CIDI 3.0 and CIDI-SC for which a good concordance with clinical diagnoses has been reported in both instruments [61, 62, 63]. However given that the instrument includes additional items from other scales, as well as the fact that it has not been tested among university students so far, it was decided to perform a clinical reappraisal.
The Mini International Neuropsychiatric Interview 5.0.0 (MINI)  will be administrated to a subsample of respondents shortly after the completion of the online survey, to assess diagnostic concordance between. Administration of the MINI will be completed by trained mental health professionals by telephone. The following MINI sections are administered; Major Depression Episode (MDE); Hypomania/Mania (BP); Panic disorder (PD); Generalized Anxiety Disorder (GAD); Substance Use Disorders (SUD). Also, the Suicidal Behavior is reappraised. For consistency with the survey data, the MINI has been adapted as necessary to account for lifetime and 12-month recall periods, with the exception of SUD, for which only 12 months are assessed. Interviewers will be blinded to participants’ mental health status. Eligible respondents will be selected after completion of T0 and after completion of T2.
Statistical power calculations
The statistical power needed for the main result has been computed for a universe estimated of 18,000 eligible subjects (undergraduate students enrolled for the first time). The absolute precision that would be achieved for the prevalence estimate of suicidal thoughts and behaviors, assuming a 12-month prevalence of 10 % (conservative estimate) and a significance level alpha of 0.05 has been evaluated for three different assumptions of baseline response rate: a) for a response rate of 60 % (n = 10,800) precision would be ±0.40 %; b) for a response rate of 30 % (n = 5,400) precision would be equal to ±0.57 %; and c) for a response rate of 15 % (n = 2,700) precision would be ±0.81 %.
With regard to the relative risk (RR), different scenarios are presented graphically (see Fig. 2), showing how power increases with sample size (i.e., final participation rate at follow-up), for significance level alpha of 0.05, using a unilateral contrast and assuming a fixed prevalence of exposure to a given risk factor of 20 %, two different levels of relative risk to be detected (with Ho: RR < =1), RR = 2 (in gray) or RR = 2.5 (in black), and three different levels of the event probability in the non-exposed group (p0) : p0 = 0.01 (diamond), p0 = 0.02 (square), p0 = 0.03 (triangle). In a restrictive scenario where we want to detect a RR = 2 assuming p0 = 0.01 (i.e., overall low incidence rate of 1.2 %) (see point A in the figure), a final sample size of n = 5,500 students (equivalent to a final participation of 30 %) would be needed to achieve a statistical power over 80 %. In the case scenario of a final sample size of n = 1,750 (i.e. final participation rate of almost 10 %) we would reach a power = 0.81 for a RR = 2 and p0 = 0.03, representing an overall incidence rate of 3.6 % (see point B in the figure). For RR = 2.5, a power of 0.88 would be reached if p0 = 0.02 (point C) and a power of 0.96 if p0 = 0.03 (point D). The test statistic used was the pooled variance Z-test.
For the clinical reappraisal study, a random sample of 100 subjects with a mental disorder and 100 subjects without a mental disorder will be selected. A sample of 100 from the positive group and 100 from the negative group would achieve 80 % power to detect a difference of 0.10 between the area under the ROC curve (AUC) under the null hypothesis of 0.80 and an AUC under the alternative hypothesis of 0.90, using a one-sided z-test at a significance level alpha of 0.05. The data are discrete (rating scale) responses. The AUC is computed between false positive rates of 0.10 and 0.15
Survey response and participation bias assessment
Study representativeness depends on the extent to which participation is high and respondents are representative of the target population. Participation rates are continuously monitored the calculation of response rate) using the American Association for Public Opinion Research (AAPOR) definitions . In case of a low response rate (i.e., below 50 %), a two-phase sampling for non-response , denominated “end-game strategy”, will be carried out. The end-game strategy consists of selecting a random subsample of eligible students that have not been enrolled in the study so far from each university and offer them an economic incentive (range € 5 to 25) to complete the interview. End-game respondents could be considered an approximation to non-responders and will be compared to initial respondents in terms of a selection of variables of interest. Weights will be applied in the analysis and response rate calculations to restore the probabilities of selection for the end-game strategy .
Participation bias assessment
A key issue in survey research is whether respondents differ from non-respondents in some way that is likely to impact systematically on the estimated outcome values. To evaluate possible bias, the distribution of the sample in terms of age, sex, and university degree categories, among others, will be compared with aggregated information provided by the participating universities. If differences are found, post-stratification weights will be used to restore distributions population distributions in each University in terms of these variables.
Prevalence, incidence and risk and protective factors: modeling
At T0, lifetime prevalence and prevalence in the last 12-months will be estimated for suicidal ideation, plans, and attempts. Additionally, the prevalence of the mental disorders assessed will be calculated. Cumulative incidence of suicidal thoughts and behavior and mental disorders will be measured at follow-up using the actuarial method.
The effect of risk and protective factors on suicidal thoughts and behaviors will be evaluated through Cox proportional hazards models. Candidate variables to be included in the models are the variables indicated in the Table 1. Different models will be estimated for suicidal ideation, plans and attempts. Cox proportional hazards models will be used to derive separate risk equations of developing suicidal thoughts and suicidal behavior within a real time in the future, among first year university students. Discrimination ability (the ability of the model to separate individuals who develop the event from those who do not) will be assessed with the C statistic. Model calibration, to assess how accurately model predictions match overall observed event rates, will be evaluated with a version of the goodness of fit of Hosmer and Lemeshow developed by D’Agostino and Nam . Internal model validation will be carried out with k-fold cross-validation methods. The external validity of the models will be assessed using data from similar studies conducted in other countries (i.e., Belgium, United States, among others) within the WMH surveys initiative.
In all future publications we will follow the STROBE criteria (Strengthening the Reporting of Observational Studies in Epidemiology) 
Diagnostic accuracy of the scales of mental disorder included in the questionnaire will be carried out with ROC curves analysis, including the area under the ROC curve (AUC) and the estimation of diagnostic performance indices for different cut off points (i.e., sensitivity, specificity, positive predictive value, negative predictive value, positive and negative likelihood ratios and diagnostic odds ratio).
Presently, there is limited information on protective factors of suicidal behaviors among university students, and the current study sought to provide valid, innovative and useful data for developing prevention programs for youth suicide and for improving early identification for high-risk students.
The longitudinal design of this study will enable us to develop predictive models, and importantly, this project will provide the first data on suicidal thoughts and behaviors among Spanish university students -- with data from different regions in the country. It is important to emphasize that cross-national data will permit the comparison with other countries of the WMH Surveys initiative. This will increase the magnitude of observations and will offer relevant information about differences between countries. Finally, the online platform provides an efficient data collection methodology, which is especially useful in young populations. There is evidence that it conveys more reliable information about suicidal behavior .
Although the validity of the instruments used to derive the final content of the survey questionnaire is well-established, additional information will be necessary to establish the validity: i) of the modifications introduced in the questionnaire; and ii) in this particular population group. It is for this reason that we will perform the clinical reappraisal study. Its results will be used to calibrate prevalence data as well as to conduct sensitivity analyses of the predictive models of risk factors.
A major challenge for this study is the potential low participation and/or the loss of subjects during the follow-up. Low response/retention rates would reduce the statistical power and can generate a selection bias which affect at the generalization of the results. The potential low participation will be minimized with successive invitation to participate reminders and with an end-game strategy. In this strategy, non-responders will be randomly selected for a specific. An important effort for reducing the loss during the follow-up it will be also necessary.
Finally it should be noted that although there is geographical variation in the universities participating in the UNIVERSAL project, its external validity will be still limited, as they are part of a convenience rather than random sampling strategy. Some results of this study will not be extrapolated to students from other universities or to non-university students. However, incidence and predictive models will not be affected by this bias.
The UNIVERSAL project had the potential to provide valid, new, and useful knowledge about important risk and protective factors. The project will emphasize the development and validation of predictive information based on these factors. To the extent that factors are modifiable, the results of this project have the potential to inform and improve future intervention strategies.
Interventions are more effective when there is concurrent focus on both risk and protective factors . The UNIVERSAL project has the potential to provide useful predictive models based on both types of factors, and thus, will inform early identification, prevention, and treatment.
Finally, as the study will be replicated in several countries around the world, as part of the WMH International College Surveys initiative, international comparisons will be made possible. They will inform about universal factors as well as cultural and context factors that are associated with suicidal behaviors among university students.
Ethics approval and consent to participate
The UNIVERSAL project was approved by the Parc de Salut MAR-Clinical Research Ethics Committee (Reference number 2013/5252/I). All the information will be protected in compliance with data gathering procedures following the Code of Ethics and the Helsinki Declaration (Seoul 2008 revision).
An online informed consent is obtained for every respondent. The respondent must agree with the informed consent by checking the “I Agree” checkbox in https://encuesta.estudio-universal.net/es/inscripcion. The authorization to check with the universities if they are first year students from the participating universities is obtained too through the consent informed page. The informed consent include: i) Information regarding the research project; ii) Details about confidentiality, and data management and storage; iii) project contact for more information]. The respondent testify: i) I have read the information regarding the research project; ii) I have received enough information about the study; iii) I was able to request additional information about the study; iv) I understand that my participation is voluntary and can withdraw from the study; v) I hereby agree to participate in the study; and vi) I have read the information concerning measures to ensure confidentiality and protection of my data. A copy of the informed form is sent to the respondent by mail.
Consent to publish
Availability of data and materials
Data: Non applicable.
Materials: Can be obtained through the correspondent author (email address: firstname.lastname@example.org)
public opinion research
attention deficit hyperactivity disorder
adult ADHD self-report scale
alcohol substance involvement screening test
area under the ROC curve
alcohol use disorders identification test
beck hopelessness scale
centers for disease control and prevention
adverse childhood experiences scale
- CIDI 3.0:
composite international diagnosis interview
composite international diagnostic interview screening scales
Columbia-suicide severity rating scale
childhood trauma questionnaire
survey data collection platform
epi-Q screening survey
generalized anxiety disorder
internalized homophobia scale
- J-MHAT 7:
joint mental health advisory team 7
land combat study
live events questionnaire
major depressive episode
multiple discrimination scale
mini international neuropsychiatric interview
posttraumatic stress disorder
Sheehan disability scale
self-injurious thoughts and behaviors interview
stages of change readiness and treatment eagerness scale
strengthening the reporting of observational studies in epidemiology
substance use disorders
Short Warwick-Edinburgh Mental Well-being Scale
ten item personality inventory
University of Cadiz
University of Balearic Islands
Miguel Hernandez University
Pompeu Fabra University
UPPS impulsive behavior scale
University of the Basque Country
world health organization
world mental health
youth risk behavior survey
Control C for D. WISQARS Leading Causes of Death Reports, National and Regional, 1999--2010. 2012.
Drapeau CW, McIntosh JL. (for the American Association of Suicidology). U.S.A. Suicide 2012: Official final data. Washington, DC: American Association of Suicidology; 2014. downloaded from http://www.suicidology.org.
Chang B, Gitlin D, Patel R. The depressed patient and suicidal patient in the emergency department: evidence-based management and treatment strategies. Emerg Med Pract. 2011;13:1–23. quiz 23–4.
Murray CJ L, Vos T, Lozano R, Naghavi M, Flaxman AD, Michaud C, Ezzati M, Shibuya K, Salomon JA, Abdalla S, Aboyans V, Abraham J, Ackerman I, Aggarwal R, Ahn SY, Ali MK, AlMazroa MA, Alvarado M, Ross Anderson H, Anderson LM, Andrews KG, Atkinson C, Baddour LM, Bahalim AN, Barker-Collo S, Barrero LH, Bartels DH, Basáñez M-G, Baxter A, Bell ML, et al. Disability-adjusted life years (DALYs) for 291 diseases and injuries in 21 regions, 1990–2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012;380:2197–223.
World Health Organization and others. Preventing Suicide A Global Imperative. World Health Organization; 2014. http://www.who.int/mental_health/suicide-prevention/world_report_2014/en/.
Arnett JJ. Emerging adulthood. A theory of development from the late teens through the twenties. Am Psychol. 2000;55:469–80.
Zarrett N, Eccles J. The passage to adulthood: Challenges of late adolescence. New Dir Youth Dev. 2006;111:13–28.
Jessor R, Donovan JE, Costa FM. Beyond Adolescence: Problem Behaviour and Young Adult Development. 1994
Kessler RC, Demler O, Frank RG, Olfson M, Pincus HA, Walters EE, Wang P, Wells KB, Zaslavsky AM. Prevalence and treatment of mental disorders, 1990 to 2003. N Engl J Med. 2005;352:2515–23.
de Girolamo G, Dagani J, Purcell R, Cocchi A, McGorry P: Age of onset of mental disorders and use of mental health services: needs, opportunities and obstacles. Epidemiol Psychiatr Sci 2012, 21:47–57.
Gollust SE, Eisenberg D, Golberstein E. Prevalence and correlates of self-injury among university students. J Am Coll Health. 2008;56:491–8.
Stallman HM. Psychological distress in university students: A comparison with general population data. Aust Psychol. 2010;45:249–57.
Schwartz AJ. College student suicide in the United States: 1990-1991 through 2003-2004. J Am Coll Heal. 2006;54:341–52.
Eisenberg D, Hunt J, Speer N, Zivin K. Mental Health Service Utilization Among College Students in the United States. J Nerv Ment Dis. 2011;199:301–8.
Drum DJ, Denmark AB. Campus suicide prevention: bridging paradigms and forging partnerships. Harv Rev Psychiatry. 2012;20:209–21.
Galindo SB, Morales I, Guerrero J. Prevalence of Anxiety and Depressive Disorders in University Students : Associated Academic and Socio-Family Factors Prevalencia de Ansiedad y Depresión en una Población de Estudiantes Universitarios : Factores Académicos y Sociofamiliares Asociados. Clínica y Salúd. 2009;20:177–87.
Vázquez FL, Blanco V. Prevalence of DSM-IV major depression among Spanish university students. J Am Coll Heal. 2008;57:165–72.
Vázquez FL, Torres Á, Otero P, Díaz O. Prevalence, comorbidity, and correlates of DSM-IV axis I mental disorders among female university students. J Nerv Ment Dis. 2011;199:379–83.
Prevention C for DC and. Suicide: Risk and Protective Factors. 2010.
Eisenberg D, Gollust SE, Golberstein E, Hefner JL. Prevalence and correlates of depression, anxiety, and suicidality among university students. Am J Orthopsychiatry. 2007;77:534–42.
Wilcox HC, Arria AM, Caldeira KM, Vincent KB, Pinchevsky GM, O’Grady KE. Prevalence and predictors of persistent suicide ideation, plans, and attempts during college. J Affect Disord. 2010;127:287–94.
Mann JJ. Neurobiology of suicidal behaviour. Nat Rev Neurosci. 2003;4:819–24.
Glenn CR, Nock MK. Improving the short-term prediction of suicidal behavior. Am J Prev Med 2014;47(3 suppl. 2):176–80.
Mann JJ, Waternaux C, Haas GL, Malone KM. Toward a clinical model of suicidal behavior in psychiatric patients. Am J Psychiatry. 1999;156:181–9.
Ingram RE, Luxton DD. Vulnerability-Stress Models. In: Hankin BL, Abela JRZ, editors. Development of Psychopathology. A vulnerability-stress perspective. Thousand Oaks California: Sage Publications; 2005:32–46.
Nock M, Holmberg E, Photos V, Michel. Self-Injurious Thoughts and Behaviors Interview: development, reliability, and validity in an adolescent sample. Psychol Assess. 2007;19:300–17.
Posner K, Oquendo MA, Gould M, Barbara Stanley M, Davies M. Columbia Classification Algorithm of Suicide Assessment (C-CASA): Classification of Suicidal Events in the FDA’s Pediatric Suicidal Risk Analysis of Antidepressants. Am J Psychiatry. 2007;164:1035–43.
Kessler RC, Ustun TB. The world mental health (WMH) survey initiative version of the world health organization (WHO) composite international diagnostic interview (CIDI). Int J Methods Psychiatr Res. 2004;13:93–121.
Ursano RJ. Modifiable risk and protective factors for suicidal behaviors in the US Army. Bethesda; 2012. (available at https://projectreporter.nih.gov/project_info_description.cfm?projectnumber=3U01MH087981-01S1).
Kessler R, Farley P, Gruber M, Harshaw Q, Jewell M, Sampson N, Shillington A. Concordance of Computerized Self-Report Measures of DSM-IV-TR Mood and Anxiety Disorders Compared To Gold Standard Clinical Assessments in Primay Care, ISPOR 15th Annual International Meeting. 2010.
Kessler RC, Adler LA, Gruber MJ, Sarawate CA, Spencer T, Van Brunt DL. Validity of the World Health Organization Adult ADHD Self-Report Scale (ASRS) Screener in a representative sample of health plan members. Int J Methods Psychiatr Res Int J Methods Psychiatr Res. 2007;16:52–65.
Sheehan DV, Harnett-Sheehan K, Raj BA. The measurement of disability. Int Clin Psychopharmacol. 1996;11 Suppl 3:89–95.
Brener ND, Collins JL, Kann L, Warren CW, Williams BI. Reliability of the Youth Risk Behavior Survey Questionnaire. Am J Epidemiol. 1995;141:575–80.
Stewart-Brown SL, Platt S, Tennant A, Maheswaran H, Parkinson J, Weich S, Tennant R, Taggart F, Clarke A. The Warwick-Edinburgh Mental Well-being Scale (WEMWBS): a valid and reliable tool for measuring mental well-being in diverse populations and projects. J Epidemiol Community Health. 2011;65:A38–9.
Castellví P, García-Forero C, Codony M, Vilagut G, Brugulat P, Medina A, Gabilondo A, Mompart A, Colom J, Tresserras R, Ferrer M, Stewart-Brown S, Alonso J. The Spanish version of the Warwick-Edinburgh Mental Well-Being Scale (WEMWBS) is valid for use in the general population. Qual Life Res. 2014;23:857–68.
López MA, Gabilondo A, Codony M, García-Forero C, Vilagut G, Castellví P, Ferrer M, Alonso J. Adaptation into Spanish of the Warwick--Edinburgh Mental Well-being Scale (WEMWBS) and preliminary validation in a student sample. Qual Life Res. 2013;22:1099–104.
Saunders JB, Aasland OG, Babor TF, De JR, Fuente L, Grant M. Development of the Alcohol Use Disorders Identification Test (AUDIT): WHO Collaborative Project on Early Detection of Persons with Harmful Alcohol Consumption—II. Addiction. 1993;88:791–804.
Group WAW. The Alcohol, Smoking and Substance Involvement Screening Test (ASSIST): development, reliability and feasibility. Addiction. 2002;97:1183–94.
Hoge CW, Castro CA, Messer SC, McGurk D, Cotting DI, Koffman RL, Abstract M. Combat Duty in Iraq and Afghanistan, Mental Health Problems, and Barriers to Care. N Engl J Med. 2004;351:13–22.
Mhat V. Joint mental health advisory team 7 (J-MHAT 7) Operation Enduring Freedom 2010, Washington, DC Off Surg Gen United States Army Med Command Off Command Surg HQ, USCENTCOM, Off Command Surg US Forces Afghanistan. 2011.
Miller WR, Tonigan JS. Assessing drinkers’ motivation for change: the Stages of Change Readiness and Treatment Eagerness Scale (SOCRATES). Psychol Addict Behav. 1996;10:81–9.
Kessler R, Galea S, Gruber M, Sampson N, Ursano R, Wessely S. Trends in mental illness and suicidality after Hurricane Katrina. Mol Psychiatry. 2008;13:374–84.
Felitti VJ, Anda RF, Nordenberg D, Williamson DF, Spitz AM, Edwards V, Koss MP, Marks JS. Relationship of Childhood Abuse and Household Dysfunction to Many of the Leading Causes of Death in Adults. Am J Prev Med. 1998;14:245–58.
Bernstein DP, Ahluvalia T, Pogge D, Handelsman L. Validity of the Childhood Trauma Questionnaire in an adolescent psychiatric population. J Am Acad Child Adolesc Psychiatry. 1997;36:340–8.
Swearer SM, Cary PT. Perceptions and attitudes toward bullying in middle school youth: A developmental examination across the bully/victim continuum. J Appl Sch Psychol. 2003;19:63–79.
Vogt DS, Proctor SP, King DW, King LA, Vasterling JJ. Validation of scales from the Deployment Risk and Resilience Inventory in a sample of Operation Iraqi Freedom veterans. Assessment. 2008;15:391–403.
Bray RM, Pemberton MR, Hourani LL, Witt M, Olmsted KLR, Brown JM, Weimer B, Lane ME, Marsden ME, Scheffler S, Vandermaas-Peeler R, Aspinwall KR, Anderson E, Spagnola K, Close K, Gratton JL, Calvin S, Bradshaw M. Department of Defense Survey of Health Related Behaviors Among Active Duty Military Personnel. North Carolina. 2008;2009(September).
Gosling SD, Rentfrow PJ, Swann WB. A very brief measure of the Big-Five personality domains. J Res Pers. 2003;37:504–28.
Beck AT, Weissman A, Lester D, Trexler L. The measurement of pessimism: The Hopelessness Scale. J Consult Clin Psychol. 1974;42:861–5.
Whiteside SP, Lynam DR. The Five Factor Model and impulsivity: using a structural model of personality to understand impulsivity. Five factor Model Impuls Using a Struct Model Personal to understand Impuls. 2001;30:669–89.
Lee MV. Title: Best Practices for Asking Questions about Sexual Orientation on Surveys. 2009.
Bogart LM, Landrine H, Galvan FH, Wagner GJ, Klein DJ. Perceived discrimination and physical health among HIV-positive Black and Latino men who have sex with men. AIDS Behav. 2013;17:1431–41.
Meyer I. Minority stress and mental health in gay men. J Health Soc Behav. 1995;36:38–56.
Hawkins JD, Catalano RF, Morrison DM, O’Donnell J, Abbott RD, Day LE. The Seattle Social Development Project: Effects of the first four years on protective factors and problem behaviors. In: McCord J, Tremblay RE, editors. Preventing antisocial behavior: Interventions from birth through adolescence. New York: Guilford Press; 1992. p. 139–61.
Round A. A Survey of Student Attitudes, Experiences and Expectations on Selected Vocational Courses at the University of Northumbria. 2005.
García-Nieto R, Blasco-Fontecilla H, Paz Yepes M, Baca-García E. Translation and validation of the ‘Self-Injurious Thoughts and Behaviours Interview’ in a Spanish population with suicidal behaviour. Rev Psiquiatr y Salud Ment (English Ed). 2013;6:101–8.
Navarro-Mateu F, Morán-Sánchez I, Alonso J, Tormo MJ, Pujalte ML, Garriga A, Aguilar-Gaxiola S, Navarro C. Cultural adaptation of the Latin American version of the World Health Organization Composite International Diagnostic Interview (WHO-CIDI) (v 3.0) for use in Spain. Gac Sanit. 2013;27:325–31.
Contel Guillamón M, Gual Solé A, Colom Farran J. Test para la identificación de transtornos por uso de alcohol (audit): traducción y validación del audit al catalán y castellano. Adicciones. 1999;11:337–47.
Gould MS, Marrocco FA, Kleinman M, Thomas JG, Mostkoff K, Cote J, and Davies M. Evaluating Iatrogenic Risk of Youth Suicide Screening Programs A Randomized Controlled Trial. Jama. 2005;293:504–28.
Gordon R, Angus K. Review of the literature to identify whether encouraging people to talk about suicidal feelings is associated with any known/identified positive or negative outcomes and to identify any similar/relevant awareness raising/social marketing campaigns and their impact. University of Sterling. Institute for Social Marketing, 2007.
Haro JM, Arbabzadeh-Bouchez S, Brugha TS, De Girolamo G, Guyer ME, Jin R, Lepine JP, Mazzi F, Reneses B, Vilagut G, Sampson NA, Kessler RC. Concordance of the Composite International Diagnostic Interview Version 3.0 (CIDI 3.0) with standardized clinical assessments in the WHO World Mental Health Surveys. Int J Methods Psychiatr Res Int J Methods Psychiatr Res. 2006;15:167–80.
Kessler RC, Pennell B-E. Developing and selecting mental health measures. In: Johnson TP, editor. Health Survey Methods. Hoboken: John Wiley & Sons, Inc; 2014. p. 143–69.
Kessler RC, Calabrese JR, Farley PA, Gruber MJ, Jewell MA, Katon W, Keck PE, Nierenberg AA, Sampson NA, Shear MK, Shillington AC, Stein MB, Thase ME, Wittchen U. Composite International Diagnostic Interview screening scales for DSMIV anxiety and mood disorders. Psychol Med. 2013;43:1625–37.
Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, Hergueta T, Baker R, Dunbar GC. The Mini-International Neuropsychiatric Interview (MINI): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998;59:22–33.
The American Association for Public Opinion Research. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 8th ed. 2008.
Groves RM, Heeringa SG. Responsive design for household surveys: Tools for actively controlling survey errors and costs. J R Stat Soc Ser A Stat Soc. 2006;169:439–57.
Lemeshow S, Hosmer DW. A review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol. 1982;115:92–106.
von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol. 2008;61:344–9.
Tourangeau R, Yan T. Sensitive Questions in Surveys. Psychol Bull. 2007;133:859–83.
Craig BJ, Rudd MD. Advances in the Assessment of Suicide Risk. J Clin Psychol. 2006;62:185–200.
We thank Roser Busquets for her continuous administrative and logistic assistance to this project.
Fondo de Investigación Sanitaria, Instituto de Salud Carlos III - FEDER PI13/00343, ISCIII (Río Hortega, CM14/00125), ISCIII (Sara Borrell, CD12/00440); Ministerio de Sanidad, Servicios Sociales e Igualdad, PNSD (Exp. 2015I015); DIUE Generalitat de Catalunya (2014 SGR 748).
The UNIVERSAL study group is formed by
Jordi Alonso, José Almenara, Gabriela Barbaglia, Maria Jesús Blasco, Pere Castellví, Ana Isabel Cebrià, Enrique Echeburúa, Andrea Gabilondo, Álvaro Iruin, Carolina Lagares, Andrea Miranda-Mendizábal, Oleguer Parès-Badell, María Teresa Pérez-Vázquez, José Antonio Piqueras, Miquel Roca, Jesús Rodríguez-Marín, Albert Sesé, Victoria Soto-Sanz, Gemma Vilagut, Margarida Vives
Dr Kessler reported being a consultant for Analysis Group, GlaxoSmithKline Inc, Kaiser Permanente, Merck & Co Inc, Ortho-McNeil Janssen Scientific Affairs, Pfizer Inc, Sanofi Group, Shire US Inc, SRA International, Inc, Takeda Global Research & Development, Transcept Pharmaceuticals Inc, Wellness and Prevention Inc, and Wyeth-Ayerst; serving on advisory boards for Eli Lilly & Company, Mindsite, and Wyeth-Ayerst; and receiving Research support for his epidemiologic studies from Analysis Group Inc, Bristol-Myers Squibb, Eli Lilly & Company, EPI-Q, Ortho-McNeil Janssen Scientific Affairs, Pfizer Inc, Sanofi Group, and Shire US Inc. He reported owning stock in Datastat Inc.
No other disclosures were reported.
The International College Survey initiative originated the conception and design of the study. RCK, RPA, and RB are the leaders of the World Mental Health (WMH) College Surveys Workgroup and have participated in several phases of the study design. J. Alonso, PC, and MJB adapted the project to Spanish context and obtained the funding. J. Almenara, CL, MR, AS, JAP, JRM, EE, AG and AIC have participated in all phases of the adaptation of the project to Spanish context and were involved in obtaining funding. MJB, PC, J. Almenara, CL, MR, AS, JAP, VSS, JRM, EE, AG, AMM are responsible of the implementation of the project in each one of the 5 universities. J. Alonso is responsible to coordinate the project. GV is responsible data management. The initial draft manuscript was written by MJB, GV, and J. Alonso, and all authors (including those mentioned in the UNIVERSAL Study Group) revised and contributed to writing the final manuscript. All authors have approved the manuscript and met the guidelines for authorship.
About this article
Cite this article
Blasco, M.J., Castellví, P., Almenara, J. et al. Predictive models for suicidal thoughts and behaviors among Spanish University students: rationale and methods of the UNIVERSAL (University & mental health) project. BMC Psychiatry 16, 122 (2016). https://0-doi-org.brum.beds.ac.uk/10.1186/s12888-016-0820-y
- Mental health
- University students
- Cohort studies
- Risk factors
- Protective factors
- Predictive modelling