{'fraction_non_alphanumeric': 0.009097270818754374, 'fraction_numerical': 0.0, 'mean_word_length': 5.271929824561403, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Atypical event and typical pattern detection within complex systems

Algorithms have been developed to find typical patterns and atypical events within complex data systems. A software package called The Morning Report was developed in which these algorithms were applied to digital flight data for commercial airlines. These systems contain many sets of data with hundreds of variables being measured over time, generally resulting in many gigabytes of data to be analyzed. Using statistical and mathematically based algorithms this software identifies atypical flights as well as which flight parameters and which flight phases are atypical. These algorithms also cluster the flights into a finite number of distinct patterns. This clustering allows the flight analyst to focus on atypical flights as well as the typical flight patterns discovered, removing the need to explore each flight separately. The Morning Report software was designed to run each night, producing a report in the morning. This report identifies the characteristics of only the newly added flights, but it uses data from previous flights to help establish the baseline. The report consists of interactive analysis tools that allow for plotting of significant flight parameters for each atypical flight as compared to the typical flights, as well as plots that contrast a flight pattern of interest to any other flight pattern or to all patterns combined