{'fraction_non_alphanumeric': 0.01486988847583643, 'fraction_numerical': 0.0015932023366967605, 'mean_word_length': 5.5874125874125875, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Rapid geotagging and disambiguation of social media text via an indexed gazetteer

Microblogging services like Twitter afford opportunities for real time determination of situation awareness during crises as people report, via their statuses, information about events on the ground. An important component of the information included in a tweet are mentions of place names that may be sites of damage, injuries, or relief efforts. Methods for extracting these place names and determining the actual location being referenced are an essential part of the suite of tools required for automated extraction of situation awareness from tweets. Extracting and disambiguating place name mentions from text have been areas of extensive research. Twitter, however, presents challenges given the 140 character restriction on status and the informal, abbreviated language that are a norm in this communication channel. Named entity recognizers, which are dependent on labeled training data, may not be useful in this medium for extracting location mentions because the typical training domains for these taggers are absent the noise found in Twitter statuses. Additionally, the contextual information that is necessary for disambiguating place names is not always present. In this paper, we demonstrate a new technique, RapidGeo, for extracting and disambiguating place names from a location specific Twitter feed using an unsupervised technique for tagging location mentions and relying on the known geographic context of the feed for disambiguation. Our location tagging technique performs much better than an off-the-shelf named entity recognizer and we achieve reasonable precision in disambiguating extracted place names. We argue that such fast, high precision, unsupervised approaches are needed when important, actionable information is required from noisy data sources such as Twitter.