{'fraction_non_alphanumeric': 0.022795915432187546, 'fraction_numerical': 0.010211419531137638, 'mean_word_length': 4.751447477253929, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Small Town or Metropolis? Analyzing the Relationship between Population Size and Language

The variance in language used by different cultures has been a topic of study for researchers in linguistics and psychology, but often times, language is compared across multiple countries in order to show a difference in culture. As a geographically large country that is diverse in population in terms of the background and experiences of its citizens, the U.S. also contains cultural differences within its own borders. Using a set of over 2 million posts from distinct Twitter users around the country dating back as far as 2014, we ask the following question: is there a difference in how Americans express themselves online depending on whether they reside in an urban or rural area? We categorize Twitter users as either urban or rural and identify ideas and language that are more commonly expressed in tweets written by one population over the other. We take this further by analyzing how the language from specific cities of the U.S. compares to the language of other cities and by training predictive models to predict whether a user is from an urban or rural area. We publicly release the tweet and user IDs that can be used to reconstruct the dataset for future studies in this direction.


Introduction
In recent years, online profiles and posts on social media platforms have provided a wealth of information for researchers to analyze, leading to discoveries about the relationships between the words that people write online and their emotions (Suttles and Ide, 2013), demographics (Rao et al., 2010), values , geographic locations (Han et al., 2014), and more. Through these linguistic associations, we can not only predict certain characteristics about the authors of these messages, but also get a glimpse of how people of different demographics interact with their world and process information. Prior studies have analyzed how location affects the type of language that people use, often looking at text written by authors from different countries when exploring crosscultural differences (Poblete et al., 2011;Garcia-Gavilanes et al., 2013). However, it is not always necessary to look at multiple countries in order to view different cultures. Recent electoral results in the United States exemplify a divide in the political opinions between those living in densely populated areas and those living in rural parts of the country (Scala and Johnson, 2017). This urban-rural ideological divide is intensifying, yet the people living in these areas depend on one another more and more (Lichter and Ziliak, 2017), interacting at the geographical borders, but also online through platforms like Twitter. Can we leverage text analysis tools to investigate how people in these seemingly disparate areas are expressing themselves through the words that they write to each other and about themselves online? In this study, we analyzed Twitter data collected from users that live in urban and rural areas of the United States and look for differences in what categories of words are more likely to be used in one area over the other. Our principal contributions are (1) a new dataset of twitter users from urban and rural areas as categorized by the U.S. Census Bureau along with tweets written by these users, (2) a linguistic analysis of this data, and (3) the proposal of city population size as an additional user-level variable to consider when performing social media studies or modeling user behavior online.

Data
We began with a set of tweets collected using the public Twitter API 1 , sampled from the public Twitter stream over the course of five years from March 2014 to March 2019, with a sample of new tweets collected every four hours during this period. We added tweets to our dataset if they had been written in English, and if the user had provided a location of their account that matches a valid U.S. city as verified through the US census API. 2 We did not use geotagged locations from the tweets directly, instead relying on the locations provided explicitly in users' Twitter bios, meaning that mispelled or unconventional location names were discarded. In this way, these tweets are more likely to match the location of the user's place of residence rather than just a location they are currently visiting. This is important for our analysis, as we are interested in analyzing the content created by users who are actually from various cities, not tweets that happened to have been tweeted by a person visiting those cities. Using the census API, we also collected the populations of each valid location as of the most recent U.S. census in 2010. We consider a tweet to be from a rural area if the population of its author's location was less than 50,000 people. 3 Otherwise, we consider the tweet to be from an urban setting. This cutoff value comes directly from the U.S. government's definitions of urban and rural populations (Ratcliffe et al., 2016). We filtered the set of tweets so that only one tweet per user appeared in our dataset to prevent having certain accounts with many tweets bias the analyses. For the dominance score analysis, tweets and user descriptions were preprocessed by expanding contractions, removing symbols, and tokenizing text. Stopwords were kept in on account of some of the lexical resources that include these words in their word classes. City-level analysis and classification of users included the same preprocessing in addition to stopword removal. In total, the dataset contains over 2.6 million unique users, each with a tweet and self-written description for their account. 4

Methodology
We use a set of six lexical resources containing meaningful word classes in order to identify trends in the language produced by users in areas classified as urban and rural. The frequency that a person uses words from each of these classes allows us to calculate a score that indicates the prevalence of a word class in one group compared to the other.

Word Classes
The six lexicons we consider are the Linguistic Inquiry and Word Count, General Inquirer, Roget, Morality, Values, and Opinion Finder. LIWC The Linguistic Inquiry and Word Count dictionary was constructed in order to identify groups of words related to important psychological phenomena that may be present in writing samples. We utilize the 2015 edition of the LIWC dictionary , which contains over 80 categories comprised of a total of nearly 6,400 words, word stems, and emoticons. General Inquirer (GI) As a resource for automated content analysis, this dictionary (Stone et al., 1966) provides categories of words related to emotional and cognitive states, as well as semantic dimensions and words from the categories outline in the Laswell Dictionary (Namenwirth and Weber, 2016). Roget's Thesaurus (R) This widely popular lexical resource (Roget, 1883) has been used as a resource for natural language processing (Jarmasz, 2012) due to its hierarchical nature and ability to categorize words into broad classes. We use the Open Roget resource (Kennedy and Szpakowicz, 2014) as another source of categories of words to measure. Morality (M) Based on Moral Foundations Theory (Graham et al., 2013), this dictionary allows for the measurement of words that either align with or oppose five main moral foundations, which include care, fairness, in-group loyalty, sanctity, and authority (Garten et al., 2016). Values (V) The hierarchical lexicon for personal values (Wilson et al., 2018) was created by sorting and expanding a set of seed terms collected from surveys of the values of people from around the world. The words in these classes describe the types of things that are important to people in their everyday lives. While the tool allows for the generation of various word classes from the tree structure of the lexicon, we use the author's originally recommended set of 50 value categories. Opinion Finder (OF) This lexicon (Wilson et al., 2005) is used specifically to search for subjectivity in language. All words in the lexicon are grouped into either the Positive class or the Negative class depending on the word's connotation.

Dominance Scores
In order to compare texts written by users from areas classified as urban to text written by users from rural areas, we calculate dominance scores (Mihalcea and Pulman, 2009) for each lexicon category. This allows us to compare the relative difference in usage of words from the lexicon class between the two groups. For a class of words C = W 1 , W 2 , ..., W N , the coverage of the class in a corpus X is the percentage of words in X that belong to class C, normalized by the number of words in X: Consider two corpora: the foreground corpus, F , and the background corpus, B. The dominance score for any class C of F with respect to B is the ratio between the coverage of the corpus F and the coverage of the corpus B.
A score of 1 indicates that both groups use words from the class C at equal rates, while scores higher than 1 indicate a greater usage of the words in F , and scores less than 1 indicate the opposite. In the following sections, we will consider the text written by users from either rural or urban areas as F and the text written by the other users as B.

Results and Analysis
Given the data and approach described above, we analyze the differences in content written by users from urban and rural areas.

Word Class Dominance Scores
We calculate dominance scores of the urban and rural corpus' for each class represented among the six lexicons. the urban corpus is in the foreground and the rural corpus is in the background, and when the rural corpus is in the foreground and the urban corpus is in the background.
Twitter users from urban areas used more words in both their descriptions and tweets that come from the Creativity and Art categories of the Values lexicon. They also included words from the Moral and Advice Values classes in their descriptions more often, suggesting that these users are more likely to share their opinions and moral stances on Twitter than their rural counterparts. They also talk about friends more often than the rural users, who talked more about familial relationships and various members of their families. The users from rural areas were more likely to use words from the LIWC Home class, the Values Family class, and the Values Significant-Other class, suggesting that in addition to family, users from rural areas are more likely to discuss their home life and significant others in their descriptions. The rural population represented in our dataset also uses words from the LIWC, General Inquirer, and Values Religion classes more often in their user descriptions than our urban population, suggesting that Twitter users from rural communities are more likely to identify themselves using religious terms, and are also more likely to tweet about theological concepts.

City-Level Analysis
In order to perform an analysis at the city-level, we split our dataset up into the tweets and corresponding user descriptions from users from each city represented in our data. For this analysis, we only included cities in our analysis represented by at least 50 unique users, leaving us with 1,985 rural areas and 716 urban cities. To generate lexicon scores for a city, we average the scores of all users from that city. Further, we computed an average rural area vector and an average urban city vector by averaging the city-level scores from all cities falling into either classification.
Next, we used cosine similarity to compare each city's vector with the average vector representing all cities in either the urban or rural grouping to find the "most similar" city to these average vectors (Table 3). Interestingly, the rural area most similar to the average is the same for both the average urban city and the average rural area for descriptions and tweets. We suspect that this indicates that the average urban city vector and average rural area vector are very similar to one another, and the vectors for each rural area are less similar to one another than the vectors for each urban city are to each other. Additionally, the urban cities for both tweets and descriptions that are most similar to rural areas are large Midwestern cities that are surrounded by more rural areas.

Classification of Urban/Rural Users
Lastly, we sought to determine whether or not, using linguistic features alone, we could predict whether a user claims to be located in an urban or rural area. To do this, we created a dataset balanced equally between both categories, and we used the FastText classifier (Joulin et al., 2016) with default parameters, training new word embeddings and subwords to be used as features for the classification with cross-entropy loss.
Using only users' tweets, we were able to achieve an accuracy of 55.7%, while using users' descriptions allowed the model to make the prediction with 62.6% accuracy (com-pared to a baseline of 50%). This indicates that there are identifiable linguistic differences between the groups, but the performance is not strong enough to suggest that this model could be used to automatically tag users by their city's population size.

Conclusion
In this paper, we introduced a new set of Twitter data tied to U.S. cities and their corresponding population sizes, we compared dominance scores across a series of lexicons to explore the difference in tweets and self-descriptions written by urban and rural users, and we measured the similarity of individual cities to the average urban and rural areas. We also trained classification models that are able to capture a relationship between population size classification and users' language patterns. Through our analyses, we provided insights into how users present themselves and communicate online based on the size of the cities in which they reside.