{'fraction_non_alphanumeric': 0.014669926650366748, 'fraction_numerical': 0.0, 'mean_word_length': 5.109452736318408, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Super Rack: Reusing the Results of Queries in MapReduce Systems

Over the last few years, Apache MapReduce has become a prevailing framework for large scale data processing. Instead of writing MapReduce programs which are too obscure to express, many developers usually adopt high level query languages, such as Hive or Pig Latin, to finish their complex queries. These languages automatically compile each query into a workflow of MapReduce jobs, so they much facilitate querying and managing large datasets residing in a distributed environment. One option to speed up the execution of workflows is to save the results produced previously and reuse them in the future if needed. In this paper we present Super Rack, assisting by shared storage devices, each workflow can store its results in Super Rack, so the incoming query can reuse these results in order to avoid the redundant computation and fasten the execution. We propose several novel techniques to improve the access and storage efficiency of the previous results. We also evaluate Super Rack to exhibit its feasibility and effectiveness. The experimental results show that our solution outperforms Hive significantly under TPC-H benchmark and real life workloads.