{'fraction_non_alphanumeric': 0.020551967116852612, 'fraction_numerical': 0.005284791544333529, 'mean_word_length': 5.129496402877698, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Leveraging Memory Level Parallelism Using Dynamic Warp Subdivision

SIMD organizations have shown to allow high throughput for data-parallel applications. They can operate on multiple datapaths under the same instruction sequencer, with its set of operations happening in lockstep sometimes referred to as warps and a single lane referred to as a thread. However, ability of SIMD to gather from disparate addresses instead of aligned vectors means that a single long latency memory access will suspend the entire warp until it completes. This under-utilizes the computation resources and sacrifices memory level parallelism because threads that hit are not able to proceed and issue more memory requests. Eventually, the pipeline may stall and performance is penalized. Therefore, we propose warp subdividing techniques that dynamically construct run-ahead “warp-splits” from threads that hit the cache so that they can run ahead and prefetch cache lines that may be used by others that fall behind. Several optimization strategies are investigated and we evaluate the techniques over two types of memory systems: a bulk-synchronous cache organization and a coherent cache hierarchy. The former has private caches communicating with the main memory with coherence taken care of by global barriers; the latter has private caches coherently sharing an inclusive, on-chip last level cache (LLC). Experiments with eight data-parallel benchmarks show our technique improves performance on average by 15% on the bulk-synchronous cache organization with a maximum speedup of 1.6X, and 17% on a coherent cache hierarchy with a maximum speedup of 1.9X. This can be achieved with an area overhead of less than 2%.