{'fraction_non_alphanumeric': 0.029523809523809525, 'fraction_numerical': 0.006666666666666667, 'mean_word_length': 5.146198830409356, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Cloud-Based Parallel Tiling Algorithm for Large Scale Remote Sensing Datasets

Tiled remote sensing data is essential for web-based remote sensing applications, widely applied in online map services, cloud-based remote sensing processing, etc. However, most existing tiling algorithms focus on tiling single remote sensing images with stand-alone machines. As the volume of remote sensing data increases, the demand for tiling high-resolution and large-scale remote sensing datasets increases dramatically. In this research, we propose a cloud-based parallel tiling algorithm for large-scale remote sensing datasets. A three-step processing flow is designed to implement the tiling of datasets composed of a set of images. Furthermore, three types of cloud-based storage are adopted to improve the efficiency of data extraction, namely, cloud storage, block storage, and NoSQL. We experimented with the proposed algorithm for tiling a national-scale 2 m resolution remote sensing dataset. The whole process took about 25.7 hours with up to 180 cores.