{'fraction_non_alphanumeric': 0.032105131830657906, 'fraction_numerical': 0.020710305248274143, 'mean_word_length': 5.2818704284221525, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
GPU UTILIZATION IN GEOPROCESSING BIG GEODATA: A REVIEW

The expansion of data collection from remote sensing and other geographic data sources, as well as from other technology such as cloud, sensors, mobile, and social media, have made mapping and analysis more complex. Some geospatial applications continue to rely on conventional geospatial processing, where limitation on computation capabilities often lacking to attain significant data interpretation. In recent years, GPU processing has improved far more GIS applications than using CPU alone. As a result, numerous researchers have begun utilising GPUs for scientific, geometric, and database computations in addition to graphics hardware use. This paper summarizes parallel processing concept and architecture, the development of GPU geoprocessing for big geodata ranging from remote sensing and 3D modelling to smart cities studies. This paper also addresses the GPU future trends advancement opportunities with other technologies, machine learning, deep learning, and cloud-based computing.


INTRODUCTION
Graphic Processing Unit (GPU) hardware has obtained significant advancement in geospatial applications in recent years. Such progress has been primarily fuelled by the expansion of data collection increments (remote sensing, point clouds, laser scanning, sensors). The development of big geodata geoprocessing using GPU parallel computation enables the computer capacity to accelerate the processing efficiently and dynamically.
The widely availability of location-based data now becoming a growing service and have revolutionized in every industry. The datasets collection comes in variety of technology such as satellite, Unmanned Aerial Vehicle (UAV), Light Detection and Ranging (LiDAR), geotagging or even from web-scrapping. The application of these geospatial intelligence in e-commerce services for example, helped expanding e-commerce sites for better location visualization, analytical and operational decisionsupport systems (Chowhan, 2017). This continuously data collection process has made the growth of data storage and gave birth to a relatively new term called big geospatial data.
Big geodata analytics receives a lot of attention these days because it allows users to analyse massive amounts of geospatial data. Big geodata refers to spatial datasets that exceed the capacity to be processed by traditional computing systems (J. Lee & Kang, 2015). Handling these scalable datasets demands complex computational developments and robust computing technologies in order to accomplish the tasks efficiently (Adam et al., 2014). Various ways have been prepared by researchers to process these big geodata, and some have invested a lot of money and time to solve the challenges of processing this data such as High-Performance Computing (HPC), cloud computing and even GPU. Li (2020), reviewed HPC can be a new framework paradigm to accommodate this big geodata challenges. In fact, this HPC has been used for two decades for the management of this geospatial data (Clarke, 2003;S. Wang & Armstrong, 2003). The expansion computing technologies from serial computation using Central Processing Units (CPU) to HPC earned the benefits in geospatial field, particularly for big geodata. However, the biggest challenge of maintaining this HPC technology is very costly (Netto et al., 2018) and often this technology is played by large giant companies such as Google, and Amazon. To handle this cost issue, the deployment of Graphic Processing Units (GPU) is much more treasured as this hardware proposes the similar parallelism architecture as HPC, and as we aware very consumer cost friendly (Hasan & Chakraborty, 2021;Jagtap & Rao, 2020) and easily accessible (Chen et al., 2014).
The remaining section is organized as follows: Section 2 provides an overview of GPU developments for big geodata varies from the remote sensing, 3D modelling and smart cities. Section 3 presents the general concepts of parallel geoprocessing architecture, GPU for big geodata, existing GPU programming model and the best practice for speedup performance evaluation technique, Amdahl's Law. Section 4 summaries the overall paper and future trends GPU utilization.

RELATED WORKS ON GPU DEVELOPMENTS FOR BIG GEODATA
The increasing potential usage of GPU hardware has demonstrated the GPU's capacity to give faster applications and excellent task-based parallelism that provides a significant increase in processing performance. This potential has been used in a variety of fields, including cryptocurrencies such as Bitcoin to mine digital money (Alkaeed et al., 2020;Iyer & Pawar, 2018), DNA analysis and modelling (Ahmed et al., 2020), large-scale simulations (Saprykin et al., 2019;Vu & Tan, 2019), large-scale tasks with combination of machine learning (Abadi et al., 2016;Nie et al., 2018) and of course, big geodata (Breunig et al., 2020;Zhou et al., 2019).
The current challenges of increasing geospatial datasets magnitude has encouraged researchers all over the world to contribute their simulation approaches and techniques for processing these scalable datasets derived from a variety of high resolution information sources, such as satellite imageries, remote sensing products, aerial photographs, and web scrapping location extraction (Hoang & Mothe, 2018). The research development can be seen in a few applications such as satellite imageries processing, building conversation, and modelling.

Remote Sensing
The use of GPU in remote sensing has presented significant computing problems as remote sensing datasets have grown in term of capacity. Traditional geoprocessing methods are no longer capable of supporting these large datasets. (P. Liu et al., 2014). Previously, the utilization GPU functions included satellite imageries classification (Sharma et al., 2020), real-time radiometric correction (Fang et al., 2014), soil parameter inversion (Yin et al., 2020), noise removal (Granata et al., 2020) and hyperspectral image classification (Yusuf & Alawneh, 2018). Some of these applications are being optimised using NVIDIA's application programming interface (API), Compute Unified Device Architecture (CUDA) (Fang et al., 2014;Sharma et al., 2020;Yin et al., 2020) and OpenCL (Granata et al., 2020), an open-source API used for NVIDIA or AMD manufactured GPU. These studies displayed satellite image processing able to demonstrate a good flexibility to GPU computational elements.
Using GPU to accelerate satellite image processing has a huge influence on the remote sensing industry. Sharma et al., (2020) investigated the feasibility of employing GPU to accelerate batch processing of spatial raster data. They concluded that the GPU is capable of drawing conclusions about its applicability in solving various problems related to geoinformation and its efficiency processes by using neutral network training for segmenting images of 10 classes that included ground, non-ground, and manmade features of 1601 images. Li et al., (2019) have created a parallel computing toolkit for terrain analysis that makes use of GPUs to accelerate the processing of large-scale LiDAR datasets when compared to the CPU alone, the experiment showed good speedup ratios performance with a maximum memory size of 8 GB. The toolkit can analyse points and doing statistical calculations. The toolkit demonstrated considerable time savings and addressed how computing costs may be lowered by making it available through GPU, particularly for LiDAR geoprocessing and other derived products.

3D Modelling
Condorelli et al., (2020) proposed a method for automatically recognising lost architectural heritage in video footage to build a metrically validated 3D model. The experiment focuses on reducing human effort in the detection of the searched object and increasing the operator's efficiency in the archive. Machine Learning techniques were recognised as viable solutions to reduce the time required to search for monuments in historical video footage, and the usage of GPU has increased the computational efficiency of such algorithms.

Smart Cities
3D point cloud technology enables the automated (Paffenholz et al., 2021) and extremely detailed digital capture of real-world (Y. Li et al., 2021) settings such as assets (S. Y. Lee et al., 2013), sites area (Shih & Wang, 2014), cities (Buyuksalih et al., 2019;Park & Guldmann, 2019), and vegetation (Mesas-Carrascosa et al., 2020;Yang et al., 2015); the obtained 3D point clouds represent an important type of geospatial data that categorized and utilised in wide range of geoinformation applications and systems (Ariff et al., 2020;Discher et al., 2018;Richter et al., 2015). Ariff et al., (2020) experimented 3D model meshing of 2.77 GB Putrajaya city using two different GPU, NVIDIA GEOFORCE GTX 1070 and NVIDIA GEOFORCE GTX 850M and they concluded that a graphic card with a higher graphic and computing performance is essential for smoother and faster visualization of 3D model and shorter time processing and lesser technical issue.
Many smart city applications, such as flood risk monitoring (Jing et al., 2019;W. Li et al., 2020), emergencies response (Alazawi et al., 2014), and solar energy potential (Amado & Poggi, 2012; Pilipczuk, 2020) of parcels and rooftops, require 3D maps to contain the height dimension. Making a city smart begins with the construction of a detailed 3D map of the entire city. This data magnitude can be enhanced by applying and utilized GPU for geospatial data processing for better decision-making.
Satellites imageries processing (Yusuf & Alawneh, 2018), coordinate transformations (Chavez, 2016;J. Li et al., 2017;Zhu & Wu, 2021), building modelling (Ma et al., 2008;Schweizer, 2015), and other GIS and remote sensing applications can benefit from the parallelism provided by GPGPU programming. For a history of parallel solutions for remote sensing applications spanning from multi-core CPU implementations to early GPU systems Christophe et al., (2011) attained up to 10 to 400 performance gains for processing several gigabytes of remote sensing images by considering the data scalability and data streaming using two programming model, CUDA and OpenCL investing a USD200 GPU hardware.
Aside from raster processing, vector processing has gained popularity in recent years. Since LiDAR technology has become more popular in recent years, the necessity for vector data processing for both raw and processed vector data cannot be overlooked. Mobile Laser Scanning (MLS) and Terrestrial Laser Scanning (TLS) sensors able to emits millions of pulses per seconds and this made data collecting denser and more accurate. H. Wang et al., (2017) used a heterogeneous CPU and GPU to accelerate a massive point cloud to run a hybrid parallel Spatial Interpolation framework on their systems. They recognise that adopting this CPU-GPU hybrid accommodation will be more practicable for processing big point cloud collections in future.

Concept of Parallel Architecture
The hype and expansion of parallel processing came in response to process growing frequencies of the data and the alternative to improve the number of processors is no longer practical after a certain point. Therefore, the researchers try to tackle and address the issues by designing the parallel systems with the aim to produce more processing efficiency using multiple cores (Surakhi et al., 2018), manage the power consumption (Garcia et al., 2018;Jin et al., 2017) and CPU overheating (Jin et al., 2017;Rajachandrasekar et al., 2012).
Parallel processing is the use of several processing cores to handle a job or tasks at the same time, whereas sequential processing is the use of a central processing unit (CPU) to carry out just one action at a time (Memon et al., 2017). Traditionally, software has been developed sequentially, which is a simpler technique but as the data magnitude keep increasing, the software become severely constrained by the processor's speed and capacity to execute each sequence of instructions (Rahmawati, 2009).  The larger tasks are broken down into smaller tasks or calculations, which are individualistic and normally identical tasks that are executed at the same time by multiple processors communicating via machine shared memory, and the results of all small tasks are merged as the final product upon completion (Pervan & Knezovic, 2020). Giant corporations utilise many approaches for big scale structure and high-performance computation including distributed computing, multi-core computing, cluster computing, and grid computing, where this structure may handle the continuous usage (Kayum & Rogowski, 2020) and high-capacity machines (Pavilion & Performance, 2019).
The essential of utilizing parallel processing for geospatial analysis functions is undoubtedly relevant gradually over time due to circumstances of datasets size and the efficiency for high performance processing. The challenges to process big geodata still underdeveloped by per function basis and not fully enhanced by geospatial researchers' and community due to hardware complexity and underutilized (Kang et al., 2018;J. Lee & Kang, 2015;Stojanovic & Stojanovic, 2013).
Previous study on processing big geodata, has proposed to utilise parallel processing for high efficiency large-scale geospatial processing (Zhou et al., 2019) and data memory management (Doraiswamy & Freire, 2020). Armstrong (2020) suggested by employing parallel processing to handle geographical information big datasets, the potential to stimulate the computing standard in spatial problem solving, analysis, and modelling will be clear developmental path in geospatial research.
There are two types of parallel programming commonly used in GIS applications: data parallelism and tasks parallelism. Both types can be used simultaneously to increase parallelism, which is referred to as hybrid parallelism (H. Wang et al., 2017). The general tasks and data parallelism is shown in Figure 3 below. In Table 1 shows the comparison summary between task parallelism and data parallelism.

GPU for Big Geodata
The potential of using GPU is seen able to assist and expand the big geodata processing. Eckart et al., (2016) presented the potential of GPU parallelization during GTC Silicon Valley-2019 for the use in large-scale 3D point clouds processing and beneficial for real-time computation over tens of thousands of points. The use of GPGPU to show significant impact on improving data processing efficiency in terms of time consuming (Gao et al., 2019) and data processing management (Yao & Li, 2018;. The continuous demand for faster and more efficient development via graphics drives appears to bode well for the future. Due to a few constraints, GPU implementation is still underdeveloped for existing GIS processing products and tools. Stojanovic & Stojanovic (2013) identify the increment geographic data collection have gathering the impacts of analyses and geoprocessing complexity and computational intensity in many GIS application domains due to a lack of parallel GIS algorithms, application libraries, and toolkits in parallel architectures. With the evolution of GPU production, manufacturers launched a range of models with varying total CUDA cores and specialties; as a result, periodic driver updates are required to ensure the drivers operate at their full capacity. Additionally, as bitcoin mining has grown in popularity, miners have discovered that using GPUs to mine the digital currency enables them to expedite their mining, which has impacted the market's GPU pricing as a result of the increasing demands.
The initial application of GPU is mostly been utilised to render the gaming pipeline (Mišić et al., 2012) so that the images visualizing more realistic, and the image texture is more smoother (Crow, 2004). The largest manufacturers of GPU hardware that are in the market right now are NVIDIA and AMD. Each GPU model released has different cores, depending on the price. For the low-end NVIDIA GPU has about 700 CUDA cores while the high end has more than 4000 CUDA cores (NVIDIA, 2021). Performance for each model is affected by the number of cores (K. H.  and GPU frequency clocks speed (H. J. .

Figure 2.
Comparisons between CPU core and GPU core (Source: Google). Figure 2 shows the CPU architecture structure is built to handle various types of tasks and functions in series while the GPU is more data intensive and processed simultaneously. Measuring performance in sequential programming is considerably less complicated and essential than benchmarking in parallel computing since it generally involves finding bottlenecks in the system (Chandrashekhar et al., 2021). To bypass bottleneck issue in sequential processing, transferring data across the memory hierarchy is remarkably helping to perform parallel processing onto data science, machine learning, and artificial intelligence. While parallel computing is more difficult and has a higher initial cost, the benefit of being able to solve a problem quicker, frequently surpasses the expenses of purchasing parallel computing hardware. Benchmarks in parallel computing may be accomplished using execution time (Hoefler & Belli, 2015;Lam et al., 2014), performance regression Nagasaka et al., 2010) and measuring techniques, such as statistical (Y. Zhang et al., 2011) and repetitions transfer cycle (Bridges et al., 2016).

GPU Programming Model
Many GPU programming approaches, and data structures are shared by programming model such as CUDA, created and maintained as a product by NVIDIA (NVIDIA, 2021), and Open Computing Language (OpenCL), an open-source language supervised by the Khronos Group (OpenCL, 2021). Because CUDA are limited to NVIDIA hardware, OpenCL is more often used for cross-platform development. However, to attain the same degree of speed as a functionally similar CUDA application, OpenCL programming language may require more manual tuning (Filipovič et al., 2017;Petrovič et al., 2020;Thoman et al., 2011;van Werkhoven, 2019). Many mainstream programming languages, including C, C++, and Python, have CUDA and OpenCL library bindings supported (Jacob et al., 2010;Sarkar, 2007).
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVI-4/W3-2021 Joint International Conference Geospatial Asia-Europe 2021 and GeoAdvances 2021, 5-6 October 2021, online

Speedup Performance Evaluation
Measuring the level of performance in parallel computing architecture, the best practice is to utilize Amdahl's Law formula is always be used., Amdahl's Law calculates how much a computation can be speeded up by running the computation in parallel. Amdahl's Law is named after Gene Amdahl who proposed the formula in 1967 (Amdahl, 1967).
(1) Where, Slatency = the theoretical speedup of the execution of the whole task s = the speedup of the part of the task that benefits from improved system resources p = the proportion of execution time that the part benefiting from improved resources originally occupied However, the number of simultaneous processes practically grows in proportion to the complexity of the task because the load is fixed according to Amdahl's Law, the execution time decreases as the number of processors rises. Amdahl's law is useful in situations when response speed is important (Rocha et al., 2016).
In relation to time that takes to execute a given routine, the CPU outperforms the GPU for smaller workloads. As the magnitude of the workload and the number of datasets arise, the execution time of the GPU begins to decrease in comparison to that of the CPU. Such as for 100% occupancy, the GPU did the addition work faster than the CPU alone. In terms of throughput, or the number of activities executed per second, GPU has always outperformed CPU in many applications. As GPU architecture will remain to progress, GPUs will be fully utilized for throughput rather than latency.

CONCLUSION
This paper has discussed the development of GPUs for big geodata, ranging from remote sensing to 3D modelling and smart cities. There are some researchers have seized the graphics hardware platform to accelerate their geoprocessing functions and tools and the results promising a positive performance efficiency. In this paper also introduces the fundamental ideas of parallel geoprocessing architecture, GPU for big geodata, existing GPU programming models, and the best practise for speedup performance evaluation, Amdahl's Law. At the end of the paper, a remarkable view on future trends of the GPU utilisation in other platform such as machine learning, deep learning, TPU, AI, cloud computing and the beneficial use in geospatial and remote sensing aspect.
Processing big scale datasets is commonly used as an evaluation criterion since it shows how quickly a parallel application can solve a problem. However, for enormous issues like processing satellite imageries or other big geodata, without proper data and tasking management assigned during processing may weaken the efficiency results and allowing you to assess how well a parallel programme can scale a massive geospatial processing.
GPUs processing are fast because their high-bandwidth features and technology performed arithmetic processing substantially faster than conventional CPUs. The primary function of GPUs is to do the calculations required to render three-dimensional (3D) computer graphics. The GPU's ability to process massive datasets is consistent with the nature of machine learning. Processing large scale of datasets is essentially what Machine Learning (ML) and Deep Learning (DL) does, which is why GPUs are advantageous for machine learning jobs. TensorFlow and Pytorch are two libraries that make advantage of GPUs that users can now manipulate the data frames and perform machine learning algorithms on GPUs as well.
The emergence of artificial Intelligence (AI) and autonomous systems have brought breakthroughs in processing capacity, whether GPUs, TPUs, or CPUs, and algorithmic advancements. Google introduced their first Tensors Processing Units (TPU) processing, ASIC, as a solution for their datacentre's computation demands. The heterogenous between CPUs, GPUs, TPUs could enhance more on cost-performance processing.
With the rise of cloud computing, several businesses have offered a cloud-based service as a service. Where users are not required to run a processing on their machine using hardware or software. As a result, significant resources are spent on idle GPUs, while insufficient GPUs are used to run jobs. Manifold is one of the successful companies providing on-the-cloud GPU processing services where the users can automatically run their code on Manifold SQL server and experience the embarrassingly parallelism and performance acceleration.
As GPU advancements become more prevalent in the near future, it is reasonable to anticipate an increase in the number of solutions and geoprocessing tools that leverage GPU platform implementations in geospatial and remote sensing applications. This is likely to broaden the range and complexity of available geoprocessing tools, as well as facilitate geospatial and remote sensing projects that involve large collections of spatial datasets. The integration of artificial intelligence and geospatial technologies has created tremendous unimaginable potential. The advancement of AI, machine learning, deep learning, and other technologies enables researchers and authorities to make more precise town development decisions by deploying predictive town planning in all aspects and future insights and proactive planning for Covid-19 pandemic by harvesting and archiving geo-located crowdsourcing, human movement mapping/tracking, and projection of spatial and temporal trends. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVI-4/W3-2021 Joint International Conference Geospatial Asia-Europe 2021 and GeoAdvances 2021, 5-6 October 2021, online