{'fraction_non_alphanumeric': 0.016727272727272726, 'fraction_numerical': 0.0, 'mean_word_length': 5.115555555555556, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Exploratory Analysis of Retail Sales of Billions of Items

This paper describes several different approaches to analysis of a data set collected over the past year from a retail grocery chain containing hundreds of stores. Each record in the data set represents an individual item processed by an individual checkout laser scanner at a particular store at a particular time on a particular day. Each record contains additional information such as store department, price, etc. together with identifying information such as the particular checkout scanner and, for some transactions, customer identification. The total data set contains billions of items which can be aggregated into hundreds of millions of transactions for millions of repeat customers. In order to get some insights in the data, we used several different approaches including some statistical analysis, some machine learning, and some data mining methods. Some of these have simply focused on ascertaining the “quality” of the data while others have been more narrowly focused on simple questions like “which pairs of items are most frequently purchased together” or “what is the relationship between basket size and number of baskets”. The sheer size of the data set has forced us to go beyond usual data mining methods and utilize Meta-Mining: the post processing of the results of basic analysis methods.