{'fraction_non_alphanumeric': 0.01652892561983471, 'fraction_numerical': 0.0, 'mean_word_length': 5.727777777777778, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Capacity Planning and Headroom Analysis for Taming Database Replication Latency: Experiences with LinkedIn Internet Traffic

Internet companies like LinkedIn handle a large amount of incoming web traffic. Events generated in response to user input or actions are stored in a source database. These database events feature the typical characteristics of Big Data: high volume, high velocity and high variability. Database events are replicated to isolate source database and form a consistent view across data centers. Ensuring a low replication latency of database events is critical to business values. Given the inherent characteristics of Big Data, minimizing the replication latency is a challenging task. In this work we study the problem of taming the database replication latency by effective capacity planning. Based on our observations into LinkedIn's production traffic and various playing parts, we develop a practical and effective model to answer a set of business-critical questions related to capacity planning. These questions include: future traffic rate forecasting, replication latency prediction, replication capacity determination, replication headroom determination and SLA determination.