{'fraction_non_alphanumeric': 0.023519612898108118, 'fraction_numerical': 0.0067028913655653825, 'mean_word_length': 4.9819691577698695, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Hybrid Deep Learning Model Based Indoor Positioning Using Wi-Fi RSSI Heat Maps for Autonomous Applications

: Positioning using Wi-Fi received signal strength indication (RSSI) signals is an effective method for identifying the user positions in an indoor scenario. Wi-Fi RSSI signals in an autonomous system can be easily used for vehicle tracking in underground parking. In Wi-Fi RSSI signal based positioning, the positioning system estimates the signal strength of the access points (APs) to the receiver and identiﬁes the user’s indoor positions. The existing Wi-Fi RSSI based positioning systems use raw RSSI signals obtained from APs and estimate the user positions. These raw RSSI signals can easily ﬂuctuate and be interfered with by the indoor channel conditions. This signal interference in the indoor channel condition reduces localization performance of these existing Wi-Fi RSSI signal based positioning systems. To enhance their performance and reduce the positioning error, we propose a hybrid deep learning model (HDLM) based indoor positioning system. The proposed HDLM based positioning system uses RSSI heat maps instead of raw RSSI signals from APs. This results in better localization performance for Wi-Fi RSSI signal based positioning systems. When compared to the existing Wi-Fi RSSI based positioning technologies such as ﬁngerprint, trilateration, and Wi-Fi fusion approaches, the proposed approach achieves reasonably better positioning results for indoor localization. The experiment results show that a combination of convolutional neural network and long short-term memory network (CNN-LSTM) used in the proposed HDLM outperforms other deep learning models and gives a smaller localization error than conventional Wi-Fi RSSI signal based localization approaches. From the experiment result analysis, the proposed system can be easily implemented for autonomous applications.


Introduction
Indoor positioning using Wi-Fi received signal strength indication (RSSI) signals is a promising positioning technique when the inertial measurement unit (IMU) sensor based [1] or the camera [2] based positioning systems face localization challenges. In Wi-Fi RSSI signal based indoor positioning, the positioning system effectively utilizes the indoor Wi-Fi access points (APs) signal strength for localization. The RSSI raw signals from the APs give the user distance information from the APs to the receiver using free-space path loss (FSPL) model [3]. The conventional Wi-Fi RSSI based localization approaches such as trilateration [4], fingerprint [5] and weighted path loss [6] algorithms use the user distance information to estimate the user's positions in the indoor environments. However, the raw RSSI signals from APs are susceptible to signal fluctuations, multipath effects, shadow fading, and signal blockage. To solve these challenges, we propose an indoor positioning system, which uses the RSSI heat maps instead of raw RSSI signals to estimate the user's positions in indoor environments. As compared to the conventional localization approaches which use raw RSSI signals, the proposed RSSI heat maps based positioning technique reduces the challenges in indoor localization. The proposed positioning system uses a hybrid deep learning model (HDLM) that comprises a convolutional neural network and long short-term memory network (CNN-LSTM). In HDLM, the CNN layers used for feature extraction of input data, are combined with LSTM layers to support sequence prediction. CNNs are used in modelling problems with spatial image inputs and give accurate image classification results. LSTMs on the other hand are used in modelling tasks related to sequences and give the best prediction results in temporal sequence tasks. To perform the tasks which need a sequence of images to predict user positions, leveraging advantages from both ends makes HDLM the best choice for our application. Further more, when compared to individual CNN and LSTM model based indoor positioning systems, the proposed HDLM based positioning system enhances the indoor localization performance and achieves reasonable localization results. The major application areas of the proposed system are object localization in airports, people tracking in shopping malls, navigation in public transport, robot localization, vision-and-language navigation [7][8][9], visual tracking [10][11][12] and autonomous car parking. The proposed positioning system can be easily implemented in these applications at low cost and provides reasonable localization accuracy for real time localization.
Deep learning based Wi-Fi RSSI signal based positioning systems have been analyzed using several localization approaches and the systems show significant improvement in localization performance [13][14][15][16]. The existing Wi-Fi RSSI based localization technologies use models such as K-nearest neighbors (KNN), artificial neural network (ANN), support vector regression (SVR), extreme learning machine (ELM), CNN and LSTM for localization. These models give accurate localization results for Wi-Fi RSSI signal based systems. However, hybrid models such as the CNN-LSTM model have better localization performance and reduced localization error compared to other deep learning models. In this paper, we investigate the significance of the hybrid model for indoor localization using Wi-Fi RSSI heat maps. The main contributions of the paper are summarized as follows: • We formulated a Wi-Fi RSSI signal based positioning system using four APs and collected the RSSI values using an Android based smartphone. The smartphone uses an application which shows the RSSI signal strengths of the particular location in the experiment area.

•
We generated a database of Wi-Fi RSSI heat maps from the RSSI data. The heat maps indicate the RSSI signal strength from APs to the receiver for a particular location in the experiment area. The proposed HDLM uses the generated heat maps for localization.

•
We implemented HDLM using CNN-LSTM. The model takes RSSI heat maps as the input and predicts the user positions. The results from the proposed HDLM approach have better localization accuracy and less error for localization compared to conventional localization approaches.
The rest of the paper is organized as follows-Section 2 presents a discussion of related work on the Wi-Fi RSSI signal based positioning systems and the existing challenges of Wi-Fi RSSI signal based positioning systems. In Section 3, we proposed a hybrid deep learning model based on an indoor positioning system and explained the HDLM architecture used in the proposed system. The experiment and result analysis of the proposed HDLM is discussed in Section 4 and we conclude our paper in Section 5.

Related Work
Indoor localization using Wi-Fi RSSI signal has been studied in the past as well as in recent times based on RSSI signal strength [17], Wi-Fi channel state information (CSI) [18], combined RSSI and CSI signals [19] and time of arrival of signals [20]. In this section we focus on the previous work on the deep learning based Wi-Fi RSSI based localization approaches and the advantages and limitations of the existing systems.
In the existing Wi-Fi RSSI signal based indoor positioning systems the RSSI signals always suffer from severe multipath effects caused by the signal's reflection, refraction, diffraction and collision. To address this problem, an ANN based localization system is proposed in [21]. The results from [21] show that using an ANN architecture in the presence of multipath effects from the indoor environments significantly improves the localization performance of the Wi-Fi RSSI signal based positioning systems. However, the unpredictable behavior of the ANN network reduces the trust on the localization results therefore requiring further improvements. To make a reliable localization system, a deep neural network (DNN) based localization system is introduced in [22][23][24][25]. The DNN based localization system improves the indoor position accuracy and the system can easily identify the user positions in complex indoor environments. However, the difficulties of training the DNN and large amount of training data are the major challenges of the DNN based localization systems. An alternative approach to improve the localization performance is to introduce a CNN. A CNN based localization, which uses RSS time-series of a wireless local area network (WLAN) access points is introduced in [26]. The CNN based Wi-Fi system provides accurate building and floor prediction for indoor localization. However, the large memory requirements due to activations at each network layer increase the system cost and it is difficult to implement a CNN based localization system with limited resources. To make an accurate and advanced Wi-Fi RSSI signal based localization system, a new time-series semi-supervised learning algorithm is proposed in [27]. The experimental results from [27] show that the semi-supervised learning algorithm gives accurate position results than conventional Wi-Fi RSSI signal based positioning systems. To reduce the calibration efforts of Wi-Fi fingerprint maps, a Wi-Fi localization system which uses the ELM is explained in [28,29]. The fast learning speed of ELM reduces the time and manpower costs for the Wi-Fi fingerprint maps and improves the localization performance. However, the localization results from the ELM based system are not precise and accurate as compared to deep learning based systems. A localization system which utilizes the advantages of recurrent neural network (RNN) and LSTM models are presented in [30][31][32]. These systems improved the localization performance and gave significant localization results for real time implementation. A deep learning model based Wi-Fi indoor localization system using RSSI and channel state information (CSI) is implemented in [33] for better localization performances. The deep learning model based localization approaches discussed here achieved superior performance compared to conventional Wi-Fi RSSI signal based localization approaches. However, the deep learning model based Wi-Fi localization systems use raw RSSI data from APs for localization. The raw RSSI data from the APs is not stable in a particular indoor location and the RSSI values can easily fluctuate with indoor channel conditions. To overcome these challenges of RSSI signals, we proposed a Wi-Fi RSSI signal based positioning system, which uses RSSI heat maps instead of RSSI raw values. The results from our experiments show that the proposed Wi-Fi RSSI heat map based positioning system using HDLM gives the best localization accuracy from all conventional deep learning approaches.

Proposed Hybrid Deep Learning Model Based Indoor Positioning System
The proposed localization approach uses RSSI heat maps and HDLM to identify the user's position. Figure 1 shows the block diagram of the proposed HDLM based localization approach.
From Figure 1, the proposed system starts with the RSSI data collection. The collected RSSI data is then converted into heat maps. For the heat map generation, we used a 'plotly' (https://plotly.com/matlab/) graphing library for MATLAB [34]. The 'plotly' graphing library has the option to add the Wi-Fi RSSI values and it generates the corresponding heat maps. The generated heat maps are used for the training and testing of the HDLM. The CNN model is used for the feature extraction and the LSTM model is to support location prediction. A CNN model consists of two parts: feature extraction part and the classification part. The feature extraction part utilizes the convolutional + pooling layers to perform the feature extraction from heat maps. The CNN model used in the proposed system extracts the features from the input heat maps such as the RSSI colour pattern. The RSSI colour pattern of each heat map is different for each location and the model utilizes this feature for classification. In the case of the LSTM network, the model uses the output of the CNN model and predicts the user positions.

Proposed HDLM Model
The proposed HDLM model consists of two parts. The first part is the CNN model followed by an LSTM model. Figure 2 shows the proposed HDLM.
CNN Model: The most popular neural network used for image classification is the CNN model [35]. The most attractive and unique feature of the CNN model is that it gives accurate classification results. This model is used for image recognition [36], image classifications [37], object detections [38] and recognizing faces [39]. In the CNN model, data features are extracted from local connection and layer-by-layer calculation. The global information is synthesized through the full layer connection. The CNN model consists of the convolutional layer (Conv2D), pooling layer (Max pooling) and the fully connected layer. Figure 3 shows the structure of the CNN model.
From Figure 3, the convolutional layer operates the convolution process with input data and each convolution kernel extracts the input data features. The weight sharing method in the convolution operation helps to reduce the number of parameters and decreases the training complexity of the neural network. The weight sharing process increases the training speed of the network, and the pooling layer in the CNN model reduces the dimension and size of the input data [40].  LSTM Model: LSTM model [41] is an improved version of the RNN [42] and is capable of learning the sequence prediction problem such as predicting the user positions. The model solves the gradient disappearance and gradient explosion problems in RNN. The unique features of LSTM model shows that the model is suitable for processing longterm sequence data and solves long-term dependence. The basic LSTM neutral network consists of a memory unit and three gates which control the memory unit. The three gates are referred to as input gate (IG), output gate (OG) and forget gate (FG). In the LSTM neural network, gates are the structures that determine the selective passage of data. If the sigmoid function gives the output as 0, the network discards the data completely. When the output is 1, the network passes the data. The data processes in the three gates are summarized as follows: (i) Forget Gate (FG): The FG is used to determine the data discard process. This gate decides which data should to be discarded or not. The output from the FG is defined as were σ is the sigmoid function, w FG and U FG are the weight vectors for forget gate, b FG is the offset and h t−1 is the output of the previous layer with the current input x t .
(ii) Input Gate (IG): The IG's role in the LSTM neural network is to update based on the existing information. In IG, we run the sigmoid function and estimate the value of IG t . IG t decides which value will enter into the network, and based on the tanh function a candidate value vectorC t is obtained. The obtainedC t is then multiplied with IG t and added to the state C t . The expressions used in the IG are as follows: (iii) Output Gate (OG): In this gate, the network estimates the output of the information of the current point. After running the sigmoid function to get output, OG t and identifies which part is the output, then cell state output C t is processed by the tanh function to obtain a value between -1 and 1. Finally, the obtained value is multiplied with OG t to get the ultimate output and the expressions are as follows:

Hybrid Deep Learning Model:
The hybrid deep learning model takes the advantages of the CNN and LSTM models and improves the system performances [43]. In the hybrid deep learning model, we implemented a hybrid network and it uses a CNN model followed by a LSTM model. The hybrid deep learning model block from the Figure 1 shows the structure of the hybrid network. The first part of the hybrid model is the CNN which is used for the data input and feature extraction. In CNN, the input is the feature graph and which is arranged in time series. The CNN model uses two convolutional (Conv2D) and pooling (MaxPooling2D) layers. After the convolutional and pooling layer, the CNN model uses a fully connected layer (Dense) which extracts the characteristics of the data as a one-dimensional vector array. The second part of the hybrid model is the LSTM section which takes the outputs from the CNN as input. In the LSTM, our model uses one LSTM layer and a fully connected layer and the outputs user position values.

Localization Process
The localization algorithm in the proposed system effectively used the HDLM and the process is shown in Figure 4.
In Figure 4, the localization algorithm first uses the training RSSI heat maps for CNN and LSTM models' training. After the weight initialization of the models the system uses testing heat maps for localization. In the testing phase, the CNN extracts the RSSI information from the spatial domain and passes it to the LSTM model. The

Experiment Setup and Result Analysis
To validate the performance of the proposed HDLM based indoor positioning system, we carried out the experiments in our university building corridor with four APs. The Smartphone, APs and experiment rooms used for the RSSI data collection is shown in Figure 5. During the experiments, the RSSI data was collected by a user with the help of an Android smartphone. The experiments were carried out strictly along the skeleton path. The user held the smartphone in his hand and walked in the skeleton path. In the experiment area, we selected the training locations based on the user movements. We assumed the user has a constant walking speed with a 0.5 m step length. We used the starting point of the user as the first training location and added each training location with 0.5 m differences. The locations from two adjacent training locations are used as the testing places. For the training and testing data collection, we used an Android 4.4.2 (KitKat) platform on a LG G3 smartphone with Snapdragon 801 processor and 3 GB RAM. The skeleton map of the experiment is shown in Figure 6. For the experiments, we used our three research lab rooms and we divided the lab rooms into 30 training and test locations. The user started walking from room 509 into room 511 and finished his walk in room 512. From the 30 training and testing locations, we generated 3000 RSSI heat maps for model training and 1500 heat maps for model testing. Figure 7 shows the generated RSSI heat maps. In RSSI heat maps, the four-square boxes indicate the 4 APs RSSI signal strength at a particular location and the x and yaxis are the APs real coordinates. The four rectangles with different box size show the RSSI signal characteristics from each APs. To classify the APs signal strength and identify the signal strengths of each APs, the heat map uses different size of rectangular boxes. The different size of the rectangular boxes indicates the APs number starting from AP1 to AP4 based on the largest to smallest box sizes. From the heat maps, it can be seen that the RSSI pattern is different for each location and the unique RSSI pattern of a particular location in the experiment area can be used for user position estimation. The proposed model uses the unique features of the heat maps for model training and testing.
To analyze the performance and accuracy of the proposed HDLM, we started our simulation with hyperparameter tuning. In hyperparameter tuning, we train the model with different hyperparameter settings and obtained the optimum values which gives best model performance. Table 1 summarizes the hyperparameter values used in the proposed model. To evaluate the impact of the proposed HDLM based approach, first we estimated the deep learning model accuracy and compared the model accuracy with CNN [44], LSTM [45] and ELM [46] models. For the model accuracy comparison, we used the same experiment setup for ELM, CNN and LSTM based approaches. Figure 8 shows the accuracy comparison of the proposed HDLM with other deep learning models. From Figure 8, the proposed HDLM model gives better accuracy for RSSI heat maps than other models. The proposed model effectively utilizes the CNN and LSTM model features and improves the model's performance. The ELM model gives poor performance for RSSI heat maps and it is not a suitable model for our application.
Next, we analyzed the training and testing time of the proposed model for RSSI heat map data. The training and testing time are an influencing factor for real time indoor localization. The model should predict the user's position accurately without any delay problems. Table 2 summarizes the training and testing time of the proposed and other models. From Table 2, we can see that the training and testing time of the proposed HDLM model is still much higher than the other models. However, when we consider the localization performance of the proposed HDLM, these computational times are acceptable for indoor localization [47]. The ELM model is capable of fast training and testing, however the localization results from the ELM model are not suitable for our localization purpose.
The performance of the proposed localization system changes with the number of APs used in the experiments. The APs number is an influencing parameter to determine the system's localization performance. To analyze the effect of the APs number for localization, we did the experiment with varying the APs number from 2 to 4 and Figure 9 shows the results. From Figure 9, the results show that the mean localization error decreases with increase in the number of APs. The performance of the proposed approach maintains the localization accuracy in all situations and gives best results. The results from LSTM model shows better performance than CNN and ELM models and also maintains a reasonable localization accuracy. The ELM model based localization approach shows the worst performance and the system shows a high margin of localization error when we use a lower number of APs. From the result analysis, it can be seen that the proposed approach shows best localization performance in all situations.
To prove that the proposed system outperforms the conventional approaches we estimated the average localization error, mean localization error and probability distribution of the localization error. The proposed HDLM based localization approach is compared with conventional Wi-Fi localization approaches such as fingerprint algorithm, trilateration algorithm and Wi-Fi fusion algorithm [48]. For the performance comparison, we used the same experiment setup such as smartphone, APs and experiment rooms. Figure 10 shows the average localization error results from the proposed HDLM based localization approach and conventional localization approaches. From Figure 10, the proposed HDLM based localization approach gives the best localization results when compare to other conventional Wi-Fi localization approaches. In the conventional Wi-Fi localization approaches, the Wi-Fi fusion algorithm shows better performance than trilateration and fingerprint approaches. From the average localization error results, we estimated the mean, maximum, minimum and standard deviation of localization errors and Table 3 summarizes the results. From Table 3, the proposed localization approach gives least localization error than conventional localization approaches. The mean localization error result from the Wi-Fi fusion approach outperforms the Wi-Fi fingerprint and trilateration approaches. From the mean localization error results, it is clear that the trilateration approach is not a suitable approach for our localization purpose.
The localization performance of the proposed approach was also validated and analyzed by the probability distribution of localization error and Figure 11 shows the results from all approaches.  From Figure 11, the proposed method shows least mean and standard deviation of localization error than conventional Wi-Fi localization approaches. The localization error results indicate that the proposed localization system is a good choice for indoor localization with minimum localization error. From all the experiments and result analysis, we demonstrated the superior performance of the proposed HDLM based indoor localization and this approach has high influence for estimating the user positions in indoor scenarios.

Conclusions
In this paper, we proposed a HDLM, a CNN-LSTM based localization system for indoor localization with Wi-Fi RSSI heat maps. We experimentally verified the feasibility of using Wi-Fi RSSI heat maps for indoor localization. We then presented the HDLM based localization system, where we trained the HDLM using Wi-Fi RSSI heatmaps, and then used newly generated Wi-Fi RSSI heat maps to estimate the user positions. The localization results from our experiments show that the proposed HDLM based localization system achieved reasonable localization accuracy for indoor localization. Through extensive experimentation, we validated the superior performance of our proposed system to conventional Wi-Fi based localization systems.

Conflicts of Interest:
The authors declare no conflict of interest.