{'fraction_non_alphanumeric': 0.017107309486780714, 'fraction_numerical': 0.008553654743390357, 'mean_word_length': 4.771300448430493, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
DATA PREPARATION FOR MINING COMPLEX PATTERNS IN LARGE SPATIAL DATABASES TECHNICAL REPORT 576

The aim of the thesis is to design an efficient algorithm for data preparation in large spatial databases for the purpose of data mining. With respect to finding complex spatial patterns, the raw data needs to be in the form converted into a set of cliques. In our case the raw data was a 1% sample from the Sloane Digital Sky Survey database which contains 818 Gigabytes of astronomical information. The following steps are required to extract cliques from the raw data: (1) The three-dimensional spatial coordinate of each galaxy was computed using a combination of information given in the database and Hubble's law. The set of galaxies were represented as a graph where the nodes are the galaxies and edges represent the distance between them. (2) A new heuristic based on the divide and conquer strategy was designed to efficiently extract all the maximal cliques from the graph. The full general problem of extracting a maximal clique from a graph is known to be NP-Hard. The proposed algorithm (GridClique) was compared with a naïve brute force approach to benchmark its efficiency. We demonstrate that GridClique performs 80% more efficiently than the Naïve approach on the SDSS data.