{'fraction_non_alphanumeric': 0.027610683349374398, 'fraction_numerical': 0.013474494706448507, 'mean_word_length': 5.164256581386726, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Mental Health Surveillance over Social Media with Digital Cohorts

The ability to track mental health conditions via social media opened the doors for large-scale, automated, mental health surveillance. However, inferring accurate population-level trends requires representative samples of the underlying population, which can be challenging given the biases inherent in social media data. While previous work has adjusted samples based on demographic estimates, the populations were selected based on specific outcomes, e.g. specific mental health conditions. We depart from these methods, by conducting analyses over demographically representative digital cohorts of social media users. To validated this approach, we constructed a cohort of US based Twitter users to measure the prevalence of depression and PTSD, and investigate how these illnesses manifest across demographic subpopulations. The analysis demonstrates that cohort-based studies can help control for sampling biases, contextualize outcomes, and provide deeper insights into the data.


Introduction
The ability of social media analysis to support computational epidemiology and improve public health practices is well established (Culotta, 2010;Paul and Dredze, 2011;Salathe et al., 2012;. The field has seen particular success around the diagnosis, quantification and tracking of mental illnesses (Hao et al., 2013;Schwartz et al., 2014;Coppersmith et al., 2014aCoppersmith et al., , 2015aAmir et al., 2017). These methods have utilized social media (Coppersmith et al., 2014b;Kumar et al., 2015;De Choudhury et al., 2016), as well as other online data sources (Ayers et al., 2017(Ayers et al., , 2013(Ayers et al., , 2012Arora et al., 2016), to obtain population level estimates and trends around mental health topics.
Accurately estimating population-level trends requires obtaining representative samples of the general population. However, social media has many well know biases, e.g. young adults tend be over-represented (demographic bias). Yet, most social media analyses tend ignore these issues, either by assuming that all the data is equally relevant, or by selecting data for specific outcome. For example, studying depression from users who talk about depression instead of first selecting a population and then measuring outcomes. Outcome based data selection can also introduce biases, such as over-representing individuals vocal about the topic of interest (self-selection bias). Consequently, trends or insights gleaned from these analyses might not be generalizable to the broader population.
Fortunately, these problems are well understood in traditional health studies, and well-established techniques from polling and survey-based research are routinely used to correct for these biases. For example, medical studies frequently utilize a cohort based approach in which a group is pre-selected to study disease causes or to identify connections between risk factors and health outcomes (Prentice, 1986). We can replicate these universally accepted approaches by conducting analyses over digital cohorts of social media users, characterized with respect to key demographic attributes. In this work, we propose to use such a social media based cohort for the purposes of mental health surveillance. We developed a digital cohort by sampling a large number of Twitter users at random (not based on outcomes), and then using demographic inference techniques to infer key demographics for the users namely, the age, gender, location and race/ethnicity. Then, we used the cohort to measure relative rates of both depression and PTSD, using supervised classifiers for each mental health condition. The inferred de-mographic information allowed us to observe clear differences in how these illnesses manifest in the population. Moreover, the analysis demonstrates how social media based cohort studies can help to control for sampling biases and contextualize the outcomes.

Methodology
We now briefly describe our approach for cohortbased studies over social media. A more detailed description of the proposed methodology will appear in a forthcoming publication. Most works on social media analysis estimate trends by aggregating document-level signals inferred from arbitrary (and biased) data samples selected to match a predefined outcome. While some recent work has begun incorporating demographic information to contextualize analyses (Mandel et al., 2012;Mitchell et al., 2013;Huang et al., 2017Huang et al., , 2019 and to improve representativeness of the data (Coppersmith et al., 2015b;Dos Reis and Culotta, 2015), these studies still select on specific outcomes.
We depart from these works by constructing a demographically representative digital cohort of social media users prior to the analyses, and then conducting cohort-based studies over this preselected population. While a significant undertaking in most medical studies, the vast quantities of available social media data make assembling social media cohorts feasible. Such cohorts can be used to support longitudinal and cross-sectional studies, allowing experts to contextualize the outcomes, produce externally valid trends from inherently biased samples and extrapolate those trends to a broader population. Similar strategies have been utilized in online surveys, which can have comparable validity to other survey modalities simply by controlling for basic demographic features such as the location, age, ethnicity and gender (Duffy et al., 2005).

Building Digital Cohorts
Our cohort construction process entails two key steps: first, randomly selecting a large sample of Twitter users; and second, annotating those users with key demographic attributes. While such attributes are not provided by the API, automated methods can be used to infer such traits from data (Cesare et al., 2017). Following this approach, we develop a demographic inference pipeline to automatically infer age, gender, race/ethnicity and location for each cohort candidate.
Age Identifying age based on the content of a user can be challenging, and exact age often cannot be determined based on language use alone. Therefore, we use discrete categories that provide a more accurate estimate of age: Teenager (below 19), 20s, 30s, 40s, 50s (50 years or older).
Gender The gender was inferred using Demographer, a supervised model that predicts the (binary) gender of Twitter users with features based on the name field on the user profile (Knowles et al., 2016).
Race/Ethnicity The standard formulation of race and ethnicity is not well understood by the general public, so categorizing social media users along these two axes may not be reasonable. Therefore, we use a single measure of multicultural expression that includes five categories: White (W), Asian (A), Black (B), Hispanic (H), and Other.
Location The location was inferred using Carmen, an open-source library for geolocating tweets that uses a series of rules to lookup location strings in a location knowledge-base (Dredze et al., 2013). We use the inferred location to select users that live in the United States.
The age and race/ethnicity attributes were inferred with custom supervised classifiers based on Amir et al. (2017)'s user-level model. The classifiers were trained and evaluated on a dataset of 5K annotated users, attaining performances of 0.28 and 0.41 Average F 1 , respectively. See the supplemental notes for additional details on these experiments 1 .

Mental Health Classifiers
We build on prior work on supervised models for mental health inference over social media data. We focus on two mental health conditions -depression and PTSD -and develop classifiers with the self-reported datasets created for CLPysch 2015 (Mitchell et al., 2015;Coppersmith et al., 2015b). These labeled datasets derive from users that have publicly disclosed on Twitter a diagnosis of depression (327 users) or PTSD (246 users), with an equal number of randomly selected demographically-matched (with respect to age and gender) users as controls. For each user, the asso- ciated metadata and posting history was also collected -up to the 3000 most recent tweets, per limitations of the Twitter API.
The participants of the task proposed a host of methods ranging from rule-based systems to various supervised models (Pedersen, 2015;Preotiuc-Pietro et al., 2015;Coppersmith et al., 2015b). More recently, the neural user-level classifier proposed by Amir et al. (2017) showed not only good performance on this task, but also the ability to capture implicit similarities between users affected by the same diseases, thus opening the door to more interpretable analyses 2 . Hence, we adopt their model for this analysis.

Analysis
We constructed a cohort for our analysis by randomly selecting a sample of Twitter users and processing it with the aforementioned demographic inference pipeline. After discarding accounts from users located outside the United States, we obtained a cohort of 48K Twitter users with the demographic composition shown in Figure 1. Some demographic groups are over-represented (e.g. young adults) while others are grossly underrepresented (e.g. teenagers) which illustrates the need for methodologies that can take these disparities into account.
We then processed the cohort through the mental-health classifiers to estimate the prevalence of depression and PTSD, and examine how these illnesses manifest across the population. The analysis revealed that 30.2% of the cohort members are likely to suffer from depression, 30.8% from 2 a similar finding to Benton et al. (2017) PTSD, and 20% from both. We observe a significant overlap between people affected by depression and PTSD, which is not surprising given that the comorbidity of these disorders is wellknown, with approximately half of people with PTSD also having a diagnosis of major depressive disorder (Flory and Yehuda, 2015).
How do these conditions affect different parts of the population? To answer this question, we looked at the affected users and measured how the demographics of individual sub-populations differ from those of the cohort as a whole. Figures 2 and 3 show the estimates for depression, PTSD and both, controlled for the cohort demographics. We observe large generational differences -PTSD seems to be more prevalent among older people whereas depression affects predominantly younger people. We also observe that in all cases Women are more susceptible than Men, and Blacks and Hispanics are more likely to be affected than Whites. This may represent a bias in the underlying data used to construct the classifiers, or a difference in how social media is used by different demographic groups. For example, models that were trained with a majority of data from White users maybe oversensitive to specific dialects used by other communities.

Discussion
Comparing our estimates with the current statistics provided by the NIH -a prevalence of 6.7% for depression 3 and 3.6% for PTSD 4 -, we can see that ours are much higher. It should be noted however, that the NIH reports refers to Major Depression episodes whereas our classifiers maybe also be sensitive to mild depressions which may never be diagnosed as such. Moreover, these estimates are not directly comparable since the NIH statistics are outdated (the estimates are from 2003 and 2015 for PTSD and depression, respectively) and our cohort was not adjusted to match the demographics of the US population. Nevertheless, it is worth noting that the relative prevalence rates, per demographic group, we obtained correlate with the NIH reports. For example, we observe similar distributions in terms of age and gen- der. However, we found that Blacks and Hispanics are more likely to be affected by mental illnesses, whereas the NIH reports a higher prevalence among Whites. One possible reason for these disparities is that racial minorities are more likely to come from communities with lower education rates and socioeconomic status (SES), and to be in a position where they lack proper health coverage and mental-health care. Reports from the NIH and other US governmental agencies show that 46.3% of Whites suffering from a mental-illness were subjected to some form treatment, but this was case for only 29.8% of Blacks and 27.3% of Hispanics 5 . There may also be a bias in reporting within different racial and ethnic groups, as prevalence estimates can be biased by access to mental health care and social stigma. Recent studies show that factors such as discrimination and perceived inequality have a stronger influence on mentalhealth than it was previously supposed, even when controlling for the SES (Budhwani et al., 2015). Others have found that acute and chronic discrimination causes racial disparities in health to be even more pronounced at the upper ends of the socioeconomic spectrum. One of the reasons being that for Whites, improvements in SES result in improved health and significantly less exposure to discrimination, whereas for Blacks and Hispanics upwards mobility significantly increases the likelihood of discrimination and unfair treatment, 5 https://www.integration.samhsa.gov/ MHServicesUseAmongAdults.pdf as they move into predominantly White neighborhoods and work environments (Colen et al., 2017).
While an in-depth analysis of this issue is beyond the scope of this work, these results suggest that it deserves further investigation. A followup study to investigate the role of discrimination in mental-health could be conducted by adding a model to identify users who reported instances of discrimination and compare the prevalence of mental-illness with a control group.

Conclusions
We have presented the first cohort based study of mental health trends on Twitter. Instead of conducting the analysis over arbitrary data samples selected to match a given outcome, we first developed a digital cohort of social media users char-acterized with respect to key demographic traits. We used this cohort to measure relative rates of depression and PTSD, and examine how these illnesses affect different demographic strata. The ability to disaggregate the estimates per demographic group allowed us to observe clear differences in how these illnesses manifest across different parts of the population -something that would not be possible with typical social media analysis methodologies. This brings social media analysis methodologies closer to universally accepted practices in surveillance based research.
Information about how different subpopulations perceive or are affected by certain health issues, could also improve public health policies and inform intervention campaigns targeted for different demographics. Moreover, the fact that some of our estimates correlate with statistics obtained through traditional methodologies suggests that this might be a promising approach to complement current epidemiology practices. Indeed, this opens the door to more responsive and deliberate public health interventions, and allow experts to track the progress or the effects of targeted interventions, in near real-time.

Privacy and Ethical Considerations
The majority of social media analysis approaches try to extract signals from individual posts and thus do not need to record any personal information. However, as we start moving towards userlevel analyses, we are collecting and storing complete records of social media users communications. Even though this information is publicly available, people might not be consciously aware of the implications of sharing all their data and certainly have not given explicit consent for their data to be analyzed in aggregate. This is even more pertinent for analyses involving sensitive information (e.g. health related issues). As it has been demonstrated by the recent incidents involving companies inadvertently sharing or failing to protect users personal data, there is a serious danger of abuse and exploitation for systems that collect and store large amounts of personal data.
Even though this is in large part an ethical question, there are technical solutions that can be used to partially address this issue. One is to use anonymization techniques to obfuscate any details that allow third parties (even analysts) to identify the individuals that are involved in the study. Another is to store only abstract representationswhich can still be updated and consumed by predictive models -, and discard the actual content. In regards to consent, there are initiatives to support voluntary data donation for research purposes, e.g. the Our Data Helps program 6 .