{'fraction_non_alphanumeric': 0.03295831530362037, 'fraction_numerical': 0.01943603057839319, 'mean_word_length': 5.0898111550285465, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Psychological and Social Factors Affecting Internet Searches on Suicide in Korea: A Big Data Analysis of Google Search Trends

Purpose The average mortality rate for death by suicide among OECD countries is 12.8 per 100000, and 33.5 for Korea. The present study analyzed big data extracted from Google to identify factors related to searches on suicide in Korea. Materials and Methods Google search trends for the search words of suicide, stress, exercise, and drinking were obtained for 2004-2010. Analyzing data by month, the relationship between the actual number of suicides and search words per year was examined using multi-level models. Results Both suicide rates and Google searches on suicide in Korea increased since 2007. An unconditional slope model indicated stress and suicide-related searches were positively related. A conditional model showed that factors associated with suicide by year directly affected suicide-related searches. The interaction between stress-related searches and the actual number of suicides was significant. Conclusion A positive relationship between stress- and suicide-related searches further confirmed that stress affects suicide. Taken together and viewed in context of the big data analysis, our results point to the need for a tailored prevention program. Real-time big data can be of use in indicating increases in suicidality when search words such as stress and suicide generate greater numbers of hits on portals and social network sites.


INTRODUCTION
In 2010, the suicide rate in Korea was 33.5 per 100000, far above the average of 12.8 per 100000 reported in Organization for Economic Co-operation and Development (OECD) countries. 1 With an increase of 101.8% between 2000 and 2010, Korea has experienced the highest increase in suicide rate among OECD countries. Additionally, suicide rates among teenagers and the elderly were reported to be extraordinarily high compared to other OECD countries. 2 The reported causes of suicide in Korea include economic issues, family problems, depression, and anxiety counterparts. 20 On the contrary, among lifestyle-related behaviors associated with better health, exercise is reported to be a protective factor for preventing diseases or facilitating recovery from diseases, as well as for maintaining and promoting optimal mental health.
News from newspapers, broadcasts, and the Internet focusing on suicide may popularize suicides. 21 Celebrity suicides, in particular, which usually garner detailed news coverage, can cause the Werther Effect. [21][22][23][24][25] It has been reported that this can increase the risk of suicide up to five times or even 14.3 times. 22,24 In Korea, the number of suicides in March 2005 was 1309, which was after a celebrity had just committed suicide, almost twice that of the previous month (736 cases). 26 According to a study, 23 after a celebrity suicide case in 2008, a much higher number of individuals who had attempted suicide using the same method as the celebrity used were admitted to the emergency room compared to the previous period.
Also, among cases of divorce or bereavement, the suicide rate is higher than that of their counterparts. 27 As a result of analyzing the association of suicide and socio-economic variables such as divorce rate, birth rate, female labor force participation rate, migration, income, education level, etc., a high level of urbanization was shown to be associate with lower rates of suicide. 28,29 As mentioned above, establishment of systematic strategies to prevent suicides, as well as conducting studies that include macroscopic variables (time sequential, spatial) that consider social influences, is needed. It is difficult; however, to respond effectively in the early stages of suicidal behavior, particularly where social and psychological factors have an extensive impact. Thus, it may prove difficult to establish a plan for predicting and managing suicide because of the complexity of suicidal impulses. In such situations, analysis of big data may be effective in managing suicide at the national level. 30,31 Big data can be understood as extremely large data sets, which cannot be collected, stored, managed, and analyzed via conventional approaches by the use of, for instance, database management systems. In public sectors, big data has been used in disease prevention, prediction, treatment, and patient management through sharing genetic and biological resources. 32,33 Over the recent years, the amount of data transmitted via smart devices and social network services has exponentially increased. Currently, data are recognized as a economic assets: 32,34 By analyzing big data, the Obama administration about the future. According to one survey, 15.6% of adults from the general population seriously considered suicide at least once in their lives. 3 In 2007, data obtained from the National Statistical Office reported that 56.5% of teenagers were distressed by heavy study loads and that 95% of employees were suffering from stress. 4 The Korean Statistical Information Service defines suicide as "intentional self-harm." As well, "accidental poisoning" or "poisoning with undetermined intent" can also be deemed to fall under the category of suicide. 5 Rather than being a one-time event, suicide is actually a series of steps or a process that may include "suicidal ideation→suicide planning→suicidal attempts→suicidal behavior", 6 thus intervention to prevent this series of suicidal processes is needed. 5,7 To date, research on suicide has focused on psychological, biological, medical, and socio-environmental factors in order to determine risk factors and causes. As noted above, suicide is generally the result of a compounding of social and psychological stressors associated with depression or alcoholism. 8 Such disorders have been shown to progress negatively over time, resulting in suicide. 9,10 Hence, this study was conducted within a theoretical framework based on the stress-vulnerability model in an attempt to explicate the causes of suicide in Korean adults.
Being exposed to extreme amounts of stress, either acute or chronic, can threaten not only an individual's capabilities or resources, but also the person's well-being; such threats are signs of suicidal ideation and suicidal behaviors. [9][10][11][12][13][14][15][16] Reportedly, suicidal behaviors are a result of an inability to handle what is requested or expected in an individual's life, which stems from the stress that arises from the perception of such a dire reality. 10,17 Even though individuals may experience stress of the same intensity, its effects with regard to suicidal behaviors vary between individuals. Many studies have examined potential factors that may mediate the relationship between stress and suicidal behaviors: cognitive factors, 10,14 such as self-worth, family support, and social support, as well as lifestyle factors, 16,18,19 such as smoking, drinking, physical exercise, and nutrition, have been reported in the literature.
Factors related to lifestyle and problematic drinking behaviors are especially reported to be associated with higher odds of abusing other addictive substances and provoking suicidal impulses in situations where it is difficult to maintain self-control. [18][19][20] Among alcoholics, suicidal attempt rates are 10 times higher than those of their non-alcoholic sion and suicide, and that lifestyle factors related to health have mediating effects on the relationship between stress and suicide. This study, therefore, was designed to analyze the determinants of the number of suicide-related searches in Korea using big data. The institutional review board of Inje University Seoul Paik Hospital approved this research.
This study employed Google search trends, a method that utilizes analyses of search-word inputs by users worldwide to provide standardized statistics for the number of searches on a specific search word carried out in a specific region at a specific time extracted according to a certain conditions. The Google search words used for this study, both English and Korean, included "stress" and "스트레스"; "drinking" and "음주"; "exercise" and "운동"; and lastly, "suicide" and "자살".
By means of a multi-dimensional analysis, an attempt was made to discover whether suicide rate by year, unemployment rate, after reports of a celebrity suicide, and the number of searches by month for stress, drinking, and exercise were compared with the number of searches on suicide (Fig. 1). In conducting the study, the following research questions were assessed: 1) Is there a difference between the amount of stress of the Korean people and the number of suicide-related searches by year?
2) Do monthly factors (stress, drinking, and exercise according to number of related searches) affect suicide-related search numbers?
3) Do monthly factors and yearly factors of Korean people affect suicide-related search numbers? 4) Is there an interaction between monthly factors and yearly factors on the number of suicide-related searches in Korea? 5) Do suicide-related search numbers affect the stress-related search numbers in Korea?

Data analysis
This study analyzed the relationship between the number of searches for the four keywords mentioned above at a primary level (month) and Korea's suicide rate, unemployment rate, and after reports of a celebrity suicide or not at a secondary level (year) from January 1, 2004 to December 31, 2010, building a multi-level model (Fig. 2). In the figure, the peak refers to the point of suicide incident when a famous politician committed suicide in Oct. 2010.
A multi-level model is commonly called a hierarchical linear model (HLM), which is used when predictions are to and other government bodies, as well as even multi-national information technology companies all over the world, are able to generate meaningful and useful information. 34 For example, the National Institutes of Health in the U.S. reduced their expenses by $50 million per year by means of a Pillbox service that employs two-way interaction between manufacturers and users to provide information on various medications at the users' request. Google provides a realtime flu forecast service by analyzing the spread of the flu worldwide through analysis of search words by users on the web. Such applications of big data and the development of analysis methods may allow for more accurate predictions of various aspects of our society. For this reason, big data is recognized as a cost-effective resource. 33 The existing methods for identifying suicide related factors, such as questionnaire surveys and clinical studies, have the advantage of enabling investigation of related variables based on data obtained from an individual; however, these also involve limitations. The magnitude of the association between suicide and these investigated variables remains to be clarified. Suicide is a complex and social phenomenon, and its diffusion effect is macroscopic under both time sequential and spatial aspects, as suggested by the Werther Effect, which highlights the importance of applying big data to the analysis of suicide related risk factors. Nevertheless, no studies have analyzed big data related to suicide. The purposes of this study were to utilize big data obtained from Google statistics and to analyze the determinants of suicide-related searches by means of a multidimensional analysis.

MATERIALS AND METHODS Research design and keyword search
The research design of this study was based on reviewed literature demonstrating that stress is closely related to depres-

Descriptive statistics for major study variables
A descriptive analysis was conducted to test the normality of variables (Table 1). Skewness and kurtosis appeared to meet the normality assumptions. 36 The suicide rate in Korea showed an increasing trend, and the volume of Google searches related to suicide showed a trend similar to the actual suicide rate of Korea. In particular, the volume of suicide-related searches increased in 2005, 2008, and 2010 after reports of celebrity suicides, indicating a risk for copycat suicides (Fig. 3). Table 2 shows the suicide rates and monthly suicide-related search volume (SSV) of Korea and other OECD countries. For Korea, both "suicide" and "자살" were used as search words.

Multi-level model analysis
The results of a multi-dimensional analysis for the determinants of suicide searches are shown in Table 3. By analyzing yearly level variance regarding monthly SSV when independent variables were not entered in the examination of research question 1, Model 1 was employed to test if there is a difference in SSV by year through a multi-level analy-be made of dependent variables not only with primary-level variables but also with higher ranked variables. A multi-level model can reflect all varieties and characteristics of both lower and higher levels of data collection. 35 To estimate parameters in this study, restricted maximum likelihood was used, which considers decreases in the degree of freedom in a fixed effect during the process of estimating the variance in a random effect. 35 For the final estimation of the fixed effect, a robust standard error was applied, which does not assume that the distribution of dependent variables is a normal distribution. Stress, exercise, and drinking-related searches at the primary level were entered as group means and suicide rates at the secondary level as the grand mean. Then, the variables were entered into the model. SPSS software, version 20.0 (SPSS Inc., Chicago, IL, USA), was used for the descriptive statistics analysis, and HLM software, version 7.0 (SSI Inc., Chicago, IL, USA), was used for the multi-level model analysis. This showed that yearly level variance accounted for about 33.2% of the total variance explained regarding monthly SSV; consequently, the monthly level variance was shown to make up about 66.8% of the total variance explained.
sis. As a result of the fixed effect analysis, the probability that the number of Google searches in Korea per month would reach an average of 55.20 times was statistically significant (β=55.20, p<0.001). As a result of the random effect analysis, both the monthly level variance (δ 2 =256.61) and yearly level variance (δ 2 =127.98) appeared to be statistically significant (χ 2 =41.91, p<0.001).
The calculation of the variance ratio of yearly SSV through an intra-class correlation coefficient (ICC), which shows similarity among the lower levels belonging to the same level, yielded the following results:   significant (χ 2 =12.41, p=0.053), and there were differences by year (χ 2 =50.32, p<0.001). Therefore, the necessity of entering yearly variables was supported. In other words, SSV increased as STSV increased monthly, and this effect showed that there were differences by year. The ICC for yearly SSV was calculated as 0.39 and the deviance was 707.12. Model 3 was designed to test research question 3, and analyzed SSV by including both yearly and monthly factors in the model. This model also included DSV and ESV, which were processed as fixed unknowns in the analysis because they were not significantly associated with STSV. As a result of fixed effects regarding the suicide search results, none of the monthly factors was statistically significant. Yearly suicide rate (YSR) appeared to affect SSV to a statistically significant degree (β=3.35, p=0.031). In other words, increases in YSR indicate increases in SSV. The number of suicides by a celebrity for a year showed a statistically significant effect on SSV (β=13.01, p=0.048). However, unemployment rate by year did not. As a result of random effects, although STSV was statistically significant (χ 2 =16.40, p=0.012), it did not show a difference by year (χ 2 =4.83, p=0.183). In the model, deviance was 707.12.

Model 4 was designed to test the interaction of STSV as
This study therefore rejected the null hypothesis of χ 2 , which states that suicide-related searches would vary the same amount across years as the monthly averages for suicide-related searches varied for a single year. In the model, deviance was revealed to be 710.25. Generally, if ICC is greater than 0.05, one can suppose that there is an intergroup variation, and even if ICC is less than 0.05, a multi-level analysis can be conducted when there is an experiential research result regarding intergroup variation. 37 This result supports the idea that analyzing a multi-level model is valid if all the variables are entered at monthly and yearly levels; nevertheless, although SSV is affected by monthly factors, the influence of yearly factors cannot be ignored.
Model 2 was performed to address research question 2. The effects of monthly factors on SSV were estimated through fixed effects, and the random effects were analyzed to see if these individual factors showed differences by year. As a result of fixed effects, drinking-related search volume (DSV) and exercise-related search volume (ESV) appeared not to have any effects on SSV, but stress-related search volume (STSV) appeared to have some effect (β=0.38, p= 0.071). As a result of random effects, STSV was statistically peared to affect stress-related searches (χ 2 =15.29, p=0.018). There was a difference in STSV by year (χ 2 =33.69, p<0.001). The ICC of yearly suicide search numbers was calculated to be 0.28, and the deviance was 761.69.

DISCUSSION
The ultimate goal of this study was to analyze the determinants of searches concerning suicide utilizing big data. To this end, the stress-vulnerability model was employed as a theoretical basis for this study. As shown in this study, the suicide rate of Korea and the volume of suicide-related searches on Google showed similar trends, and further demonstrated that copycat suicides may result from broadcasting celebrity suicides. This finding was consistent with previous studies that have reported the serious impact of suicide-related press reports. 25,27 Recent suicide rates and suicide-related search numbers in the major OECD countries have either remained stable or decreased, whereas Korea's suicide rate has risen to almost three times the average suicide rate of other OECD members. Based on the results of this study, Koreans conduct suicide-related searches on Google 55.2 times a month on average. Moreover, suicide-related searches and stressrelated searches showed a year-by-year difference.
In the analysis to test the effects of stress, drinking, and exercise-related searches on suicide-related searches, stressrelated searches appeared to have the greatest influence on suicide-related searches. In other words, more stress-related searches were associated with more suicide-related searches. This result is consistent with previous studies, reporting that stress directly affects suicide. [12][13][14][17][18][19][20] Therefore, it is a monthly factor and YSR as a yearly factor. The result was statistically significant (β=0.7, p=0.032). This result indicated that there is a difference in the relationship between stress-related search volume and suicide-related search volume according to YSR. In other words, if there are a large number of stress-related searches, the number of suicide-related searches increases as well. This result also demonstrated that YSR has the effect of increasing the number of suicide-related searches. The ICC of yearly SSV was 0.168, and its deviance was 707.33.
Thus far, statistical tests were conducted on factors affecting SSV via a multi-level analysis based on the stressvulnerability model. The fact that stress-related searches affected suicide-related searches was supported in this study, but there remained a need to further define this result and provide evidence for a good circular model: suicide-related searches affect stress-related searches and stress-related searches affect suicide-related searches. Therefore, another analysis was conducted to check for differences by year in relation to the effect of monthly suicide-related searches on stress-related searches (Table 4).
In the fixed effect of Model 1, the total average number of stress-related searches was 113.67, and significant variations existed by year, given the significant variances at monthly and yearly levels (t=22.62, p<0.001). ICC was 25.3%, and there were differences in STSV by year, as there was a significant variance at the year level (χ 2 =30.42, p<0.001).
As a result of the fixed effect analysis regarding STSV in Model 2, STSV was significantly different by year (t=22.62, p<0.001), but SSV did not appear to affect STSV. This supports the validity of the one-way model: that is, stress-related searches increase suicide-related searches, but not vice versa. Yet in the results from the random effect analysis, SSV ap- ety, therefore, classifying data-driven information and securing personal data should be also considered to prepare another societal challenge.
Second, there is an urgent need to further develop a way of analyzing and processing big data in health and welfare sectors. To promote greater development, education programs are needed to train professionals in data management, so called "data scientists," in an effort strengthen their ability to find hidden information from large-scale, unstructured big data. Also, development of clouding computer services for which to store, classify, and analyze non-relational and atypical data should be at the core of this whole process. Additionally, preferential attention should be given to the development of relevant technology that can enable "collection→storage→analysis→deduction" of big data in the sectors of health and welfare.
Third, a policy should be set in place for data security and privacy protection regarding personal information while utilizing big data. However, at this point, relevant laws and systems are next to non-existent and are not even discussed in Korea. Therefore, for the prevention of privacy related crimes such as a cyber-civil-rights violation, strict controls on data processing, information access, and anonymity assurance must accompany the use of big data. To this effect, a policy should be established to ensure the balance of utilizing and protection of big data in health and welfare sectors.
For the prevention of suicide, the policy implications of this study are as follows: first, age-friendly intervention programs for suicide prevention, such as school-wide programs for youth, workplace programs for workers, and agingfriendly programs for the elderly, warrant development. Second, applications such as a "Respect for Life Online Gatekeeper" should be developed to reduce suicide-associated risks as revealed through the analysis of big data and to provide tailored programs in real time if the warning signs of suicide are detected. Third, suicide prediction models by region such as "Respect for Life Prediction System" should be developed to prevent suicide in real time, by analyzing suicidal behaviors by region.
Finally, the limitations of this study in the following: first, rather than analyzing individual characteristics, this study analyzed entire groups as a whole; therefore, applying the results to individuals may result in ecological errors. Second, this study used data obtained from Google search trends statistics, so its representativeness may be questioned. Therefore, it is recommended that big data also be collected through various channels to analyze factors related to search-possible that, when there is a buzz or when many keyword searches related to stress or suicide are on search portals or social network services, suicide impulses could be curtailed by analyzing various types of data, including the age and search patterns of an individual user, in order to provide timely intervention.
Additionally, although monthly factors did not affect the number of suicide-related searches, the actual suicide rate and yearly factors strongly affected it. This result may be attributable to the fact that monthly factors were affected by controlling yearly suicide rate. It also can be interpreted that yearly suicide-related factors directly affected suicide-related searches. The interactions between stress-related searches and suicide rates were significant: a large number of stress-related searches in a year reporting high suicide rates were shown to affect suicide-related searches.
There are few previous studies on suicide risk factors utilizing big data. From this study, it was revealed that suicide searches and suicide rates were interrelated. Therefore, it may be possible to set up a systematic plan for preventing suicide at the governmental level. For instance, in Finland, governmental departments made an comprehensive effort to prevent suicide by implementing a national project (1986)(1987)(1988)(1989)(1990)(1991)(1992)(1993)(1994)(1995)(1996) based on 1397 psychological autopsy reports related to suicide (including medical and social security data, as well as family, friends, and doctor interview materials, etc.). Following its implementation, the suicide rate in Finland was found to be decreasing as follows: 30.3 suicides per 100000 people in 1990, 20.4 suicides in 2004, and 17.3 suicides in 2010. Considering that the decrease in Finland's suicide rates was based on a review of offline reports at the government level, Korea may also now be in a position to utilize big data to set up a nationwide suicide prevention plan. 38 For the health and welfare sector in Korea, governmental and public institutions are already managing many different types of big data; however, their utilization and application are still in the initial steps. To present efficient strategies for preventing suicide and providing tailored service by the use of big data, we suggest the following: first, collaborative and systematic approaches should be developed between governmental departments and private institutes to manage health and welfare-related big data in an integrated way. Currently, in Korea, the National Health Insurance Corporation, the Korean Food & Drug Administration, and other national research institutes manage big data. Such data are mainly collected and stored through web search portals run by private institutes or social network services. In the soci-es on suicide, such as other web search portals, blogs, Internet cafes, various types of social network services, web boards, etc. Third, suicide BUZZ via the Internet shows a tendency to spread rapidly for the first week after its onset and has about the life cycle of three weeks. 39 Employing a time lag of one to three weeks for the analysis is therefore necessary; however, this study did not apply such a time lag. Future research should consider this issue. Fourth, this study analyzed search results obtained from unspecified individuals; therefore, it would be impossible to predict suicidal behavior therefrom. Future research should consider this issue by analyzing social big data including emotions and psychological behaviors related to suicide. Fifth, in this study, unemployment rate by year was not statistically significant in the model. Nevertheless, this result warrants further study.
In conclusion, this study utilized big data to analyze suicide related factors and was ultimately intended to suggest the use of big data, which exists in various forms in different areas of society, to establish suicide prevention plans at the national level. Training talented professionals and preparing related infrastructure should be systematically implemented so that big data can be employed for various purposes in health and welfare sectors in the future.