{'fraction_non_alphanumeric': 0.02072538860103627, 'fraction_numerical': 0.004029936672423719, 'mean_word_length': 5.297101449275362, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Framework for Effective Scheduling of Data-Parallel Applications in Grid Systems

Grid systems – a unified collection of resources connected by a network – have potential to deliver high performance for many applications and many system users. Achieving high performance in a grid system requires effective resource scheduling. The heterogeneous and dynamic nature of the grid, as well as the differing demands of applications run on the grid, makes grid scheduling complicated. Existing schedulers in wide-area heterogeneous systems require a large amount of information about the application and the grid environment to produce reasonable schedules [Lo88][Wei95][Wei00]. However, this information may not be available, may be too costly to collect, or may increase the run-time overhead of the scheduler such that the scheduler is rendered ineffective. We believe that no one scheduler is appropriate for all grid systems. Instead, grid systems require a scheduling framework that produces reasonable schedules for a variety of applications, and allows for more sophisticated scheduling mechanisms to be added to the framework as they are needed. We propose a scalable, extensible framework for statically scheduling parallel applications in a grid system. By default, a scheduler built within the framework will require only a small amount of information about the application and the state of the network. We then evaluate a scheduler implemented within the framework, and show that it produces reasonable task assignments for a variety of data-parallel applications without consuming much run-time overhead. 1 In this context, a static schedule is produced before job execution, and is not modified during the course of execution.