{'fraction_non_alphanumeric': 0.03193345301003767, 'fraction_numerical': 0.01006295668883214, 'mean_word_length': 5.22964246338006, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Multiple Source Framework for the Identification of Activities of Daily Living Based on Mobile Device Data

The monitoring of the lifestyles may be performed based on a system for the recognition of Activities of Daily Living (ADL) and their environments, combining the results obtained with the user agenda. The system may be developed with the use of the off-the-shelf mobile devices commonly used, because they have several types of sensors available, including motion, magnetic, acoustic, and location sensors. Data acquisition, data processing, data fusion, and artificial intelligence methods are applied in different stages of the system developed, which recognizes the ADL with pattern recognition methods. The motion and magnetic sensors allow the recognition of activities with movement, but the acoustic sensors allow the recognition of the environments. The fusion of the motion, magnetic and acoustic sensors allows the differentiation of other ADL. On the other hand, the location sensors allows the recognition of ADL with large movement, and the combination of these sensors with the other sensors increases the number of ADL recognized by the system. This study consists on the comparison of different types of ANN for choosing the best methods for the recognition of several ADL, which they are implemented in a system for the recognition of ADL that combines the sensors data with the users agenda for the monitoring of the lifestyles. Conclusions point to the use of Deep Neural Networks (DNN) with normalized data for the identification of ADL with 85.89% of accuracy, the use of Feedforward neural networks with non-normalized data for the identification of the environments with 86.50% of accuracy, and the use of DNN with normalized data for the identification of standing activities with 100% of accuracy, proving the reliability of the framework presented in this study.


Introduction
A multiple source framework for the identification of Activities of Daily Living (ADL) [1], proposed in [2][3][4], may be developed using data acquired from the several sensors available in off-the-shelf mobile devices, e.g., the accelerometer, the gyroscope, the magnetometer, the microphone, and the Global Positioning System (GPS) receiver. Sensors available in the off-the-shelf mobile devices allows the capture of several parameters in order to enable the automatic identification of the activities with movement as well as the environmental characteristics during the data acquisition [5]. This framework is one of the several modules for the development of a personal digital life coach [6].
The development of the framework for the identification of ADL and their environments started with previous studies using several sensors, such as the accelerometer, the gyroscope, the magnetometer, and accelerometer, the correlation between axis of the accelerometer, and the distance, location and speed acquired from the GPS receiver [26]. The reported accuracies were 92.4% with J48 decision tree, 91.7% with MLP, and 84.3% with LR [26].
In [27], the accelerometer and GPS receiver are used for the recognition of standing, driving, walking, running, going upstairs, going downstairs, riding an elevator, and cycling, using Bayesian networks. The input features used are mean, variance, spectral energy and spectral entropy from the accelerometer, and the location retrieved from the GPS receiver, reporting an accuracy around 95% [27].
Other studies have been implemented for the recognition of ADL using the GPS receiver and other sensors available in off-the-shelf mobile devices. The authors of [28] tested the use of SVM and HMM method based on the data acquired from the accelerometer, gyroscope and GPS receiver, recognizing, standing, walking, running and sitting activities with a reported reliable accuracy.
In [29], the travelling by car, cycling, and travelling by train activities were recognized based on the International Road Index (IRI) and angle of slope measured by the data acquired from the accelerometer, gyroscope and GPS receiver with a reliable accuracy.
The authors of [30] used the accelerometer and the GPS receiver for the recognition of lying, sitting, standing and falling activities using the Signal Magnitude Area (SMA), Signal Magnitude Vector (SMV) and Tilt Angle (TA) from the accelerometer data, and the distance, location and speed retrieved from the GPS receiver, reporting results with a reliable accuracy.
In [31], the GPS receiver, Wi-Fi Positioning System (WPS), GSM Positioning System (GSMPS), and accelerometer are used for the recognition of standing, sitting, lying, walking, jogging, cycling, travelling by bus, travelling by train, travelling by taxi and travelling by car, using several features, including the numbers of peaks, number of troughs, sum of peaks, sum of troughs, different between the maximum peak and the maximum trough, difference between the maximum and the minimum peak values every 2 seconds, and difference between the maximum and the minimum trough values every 2 seconds from the accelerometer data, and the location retrieved from the GPS receiver, Wi-Fi Positioning System (WPS), GSM Positioning System (GSMPS), reporting results with reliable accuracy and with less 53% of battery energy spent than other methods.
The authors of [32] used only the GPS receiver for the recognition of working, attending lectures, shopping, swimming, training in a gym, playing team sports, visiting friends, going to a pub, eating, going to the cinema, going to a concert, going to the theatre, visiting a doctor and going to church, using the density and time-based methods available in the OpenStreetMap (OSM) platform that uses the Points of Interest (POI) as features. Based on different levels of threshold, the accuracies reported by the authors were between 72.2% and 95.4% with density-based method, and between 66.1% and 69.6% with timebased method [32].
Following the analysis of the studies available in the literature, the table 1 shows the ADL recognized with GPS receiver and other sensors, verifying that the standing, walking, sitting, running, going upstairs, going downstairs, and driving/travelling (i.e., car, train, bus, taxi) are the most recognized ADL.  Table 2 shows the features extracted from the data acquired from the GPS receiver, verifying that the location, speed, and distance are the most used features, where the location provide the geographic information of the place where the activity is performed, and speed and distance help to measure the intensity of the activity.  , table 3 shows the methods already implemented for the recognition of the ADL, concluding that the most used methods are SVM, MLP, and decision trees. In the most used methods, implemented in more than 4 studies, the method that reports the best average accuracy in the recognition of ADL is the MLP, with an average accuracy equals to 93.53%.

Methods
The development of a multiple source framework [2][3][4] for the recognition of ADL includes the methods presented in previous studies [7][8][9] and the methods developed in this study that includes several modules, these are: • Data acquisition module, where performed with a mobile application; • Data processing module, which is forked in data cleaning and feature extraction methods; • Recognition module, which is forked in data fusion and artificial intelligence methods.
As this study is focused on the differentiation of standing activities (i.e., watching TV, sleeping, and driving), the data acquisition process in the section 3.1; Section 3.2 presents the data processing methods; And, the methods for the recognition of ADL are presented in the section 3.3.

Data Acquisition
A mobile application for the Android operating systems [33,34] was developed and installed in a BQ Aquarius device [35], which saves the data acquired from the several sources in text files, including the data acquired from the accelerometer, the magnetometer, the gyroscope, the microphone, and the GPS receiver. The data is acquired with the mobile application, in a background process, from all sources during 5 seconds with a frequency of 5 minutes, and each capture is labeled by the user in order to identify the ADL and/or environment. For the testing stage, we choose people with distinct lifestyles and aged between 16 and 60 years old, which performed the defined activities with the mobile device in the pocket. The final framework for the recognition of ADL will allows the recognition of several ADL, such as walking, going upstairs, sleeping, driving, going downstairs, and standing, and the recognition of environments, such as bar, classroom, gym, kitchen, library, street, hall, watching TV and bedroom. However, the focus of this study is the recognition of standing activities, such as sleeping, driving, and watching TV. The acquired data is stored in the ALLab MediaWiki [36].

Data Processing
This section will present the data processing techniques for the recognition of standing activities proposed in this paper, where the data cleaning methods are presented in the section 3.2.1, and, in the section 3.2.2, the extraction of the features from the accelerometer, gyroscope, magnetometer, and GPS receiver are fused with the environment recognized with the method defined in [9].

Data Cleaning
This study makes use of the accelerometer, gyroscope and magnetometer data, applying the low pass filter [37] for the reduction of the effects of the environmental noise and invalid data. This study does not use the microphone data, but it only uses the environment recognized in the study [9], and the data cleaning is not needed for this type of input. Finally, the data acquired from the GPS receiver does not also need data cleaning methods.

Feature Extraction
Based in our previous studies [7][8][9] and the most features extracted in the studies available in the literature, after the data cleaning process applied to the data acquired from the accelerometer, gyroscope, magnetometer, GPS receiver and environment recognized for the differentiation of standing activities, the features extracted from these sensors should be the 5 greatest distances between the maximum peaks, the Average of the maximum peaks, the Standard Deviation of the maximum peaks, the Variance of the maximum peaks, the Median of the maximum peaks, the Standard Deviation of the raw signal, the Average of the raw signal, the Maximum value of the raw signal, the Minimum value of the raw signal, the Variance of the of the raw signal, the Median of the raw signal, the distance travelled, and the environment recognized.

Identification of Activities of Daily Living with Data Fusion
Next to our previous studies [7][8][9], this study aggregates the features defined in the section 3.2.2 in several datasets for the differentiation of standing activities. In the section 3.3.1, the features extracted from the accelerometer data, the distance travelled and the environment recognized are fused in different datasets. The features extracted from the accelerometer and magnetometer sensors' data, the distance travelled and the environment recognized are fused in the different datasets presented in the section 3.3.2. In section 3.3.3, the features extracted from the accelerometer, magnetometer and gyroscope sensors' data, the distance travelled and the environment recognized are fused in different datasets. Finally, the artificial intelligence methods for the recognition of standing activities with the datasets previously defined are presented in the section 3.3.4.

Data fusion of the environment recognized with the Accelerometer and Location sensors for the recognition of standing activities
Regarding the features extracted from each standing activity, five datasets have been constructed with features extracted from the accelerometer data acquired during the performance of the three standing activities, having 2000 records from each activity. This method allows the distinction between sleeping, driving and watching TV. The datasets defined are: • Dataset 1: Composed by 5 greatest distances between the maximum peaks, Average of the maximum peaks, Standard Deviation of the maximum peaks, Variance of the maximum peaks, Median of the maximum peaks, Standard Deviation of the raw signal, Average of the raw signal, Maximum value of the raw signal, Minimum value of the raw signal, Variance of the of the raw signal, and Median of the raw signal, extracted from the accelerometer sensor, the distance travelled extracted from the data acquired from the GPS receiver, and the environment recognized with the method proposed in [9]; • Dataset 2: Composed by Average of the maximum peaks, Standard Deviation of the maximum peaks, Variance of the maximum peaks, Median of the maximum peaks, Standard Deviation of the raw signal, Average of the raw signal, Maximum value of the raw signal, Minimum value of the raw signal, Variance of the of the raw signal, and Median of the raw signal, extracted from the accelerometer sensor, the distance travelled extracted from the data acquired from the GPS receiver, and the environment recognized with the method proposed in [9]; • Dataset 3: Composed by Standard Deviation of the raw signal, Average of the raw signal, Maximum value of the raw signal, Minimum value of the raw signal, Variance of the of the raw signal, and Median of the raw signal, extracted from the accelerometer sensor, the distance travelled extracted from the data acquired from the GPS receiver, and the environment recognized with the method proposed in [9]; • Dataset 4: Composed by Standard Deviation of the raw signal, Average of the raw signal, Variance of the of the raw signal, and Median of the raw signal, extracted from the accelerometer sensor, the distance travelled extracted from the data acquired from the GPS receiver, and the environment recognized with the method proposed in [9]; • Dataset 5: Composed by Standard Deviation of the raw signal, and Average of the raw signal, extracted from the accelerometer sensor, the distance travelled extracted from the data acquired from the GPS receiver, and the environment recognized with the method proposed in [9].

Data fusion of the environment recognized with the Accelerometer, Magnetometer and Location sensors for the recognition of standing activities
Regarding the features extracted from each standing activity, five datasets have been constructed with features extracted from the accelerometer and magnetometer sensors' data acquired during the performance of the three standing activities, having 2000 records from each activity. This method allows the distinction between sleeping, driving and watching TV. The datasets defined are: • Dataset 1: Composed by 5 greatest distances between the maximum peaks, Average of the maximum peaks, Standard Deviation of the maximum peaks, Variance of the maximum peaks, Median of the maximum peaks, Standard Deviation of the raw signal, Average of the raw signal, Maximum value of the raw signal, Minimum value of the raw signal, Variance of the of the raw signal, and Median of the raw signal, extracted from the accelerometer and magnetometer sensors, the distance travelled extracted from the data acquired from the GPS receiver, and the environment recognized with the method proposed in [9]; • Dataset 2: Composed by Average of the maximum peaks, Standard Deviation of the maximum peaks, Variance of the maximum peaks, Median of the maximum peaks, Standard Deviation of the raw signal, Average of the raw signal, Maximum value of the raw signal, Minimum value of the raw signal, Variance of the of the raw signal, and Median of the raw signal, extracted from the accelerometer and magnetometer sensors, the distance travelled extracted from the data acquired from the GPS receiver, and the environment recognized with the method proposed in [9]; • Dataset 3: Composed by Standard Deviation of the raw signal, Average of the raw signal, Maximum value of the raw signal, Minimum value of the raw signal, Variance of the of the raw signal, and Median of the raw signal, extracted from the accelerometer and magnetometer sensors, the distance travelled extracted from the data acquired from the GPS receiver, and the environment recognized with the method proposed in [9]; • Dataset 4: Composed by Standard Deviation of the raw signal, Average of the raw signal, Variance of the of the raw signal, and Median of the raw signal, extracted from the accelerometer and magnetometer sensors, the distance travelled extracted from the data acquired from the GPS receiver, and the environment recognized with the method proposed in [9]; • Dataset 5: Composed by Standard Deviation of the raw signal, and Average of the raw signal, extracted from the accelerometer and magnetometer sensors, the distance travelled extracted from the data acquired from the GPS receiver, and the environment recognized with the method proposed in [9].

Data fusion of the environment recognized with the Accelerometer, Magnetometer, Gyroscope and Location sensors for the recognition of standing activities
Regarding the features extracted from each standing activity, five datasets have been constructed with features extracted from the accelerometer, magnetometer and gyroscope sensors' data acquired during the performance of the three standing activities, having 2000 records from each activity. This method allows the distinction between sleeping, driving and watching TV. The datasets defined are: • Dataset 1: Composed by 5 greatest distances between the maximum peaks, Average of the maximum peaks, Standard Deviation of the maximum peaks, Variance of the maximum peaks, Median of the maximum peaks, Standard Deviation of the raw signal, Average of the raw signal, Maximum value of the raw signal, Minimum value of the raw signal, Variance of the of the raw signal, and Median of the raw signal, extracted from the accelerometer, magnetometer and gyroscope sensors, the distance travelled extracted from the data acquired from the GPS receiver, and the environment recognized with the method proposed in [9]; • Dataset 2: Composed by Average of the maximum peaks, Standard Deviation of the maximum peaks, Variance of the maximum peaks, Median of the maximum peaks, Standard Deviation of the raw signal, Average of the raw signal, Maximum value of the raw signal, Minimum value of the raw signal, Variance of the of the raw signal, and Median of the raw signal, extracted from the accelerometer, magnetometer and gyroscope sensors, the distance travelled extracted from the data acquired from the GPS receiver, and the environment recognized with the method proposed in [9]; • Dataset 3: Composed by Standard Deviation of the raw signal, Average of the raw signal, Maximum value of the raw signal, Minimum value of the raw signal, Variance of the of the raw signal, and Median of the raw signal, extracted from the accelerometer, magnetometer and gyroscope sensors, the distance travelled extracted from the data acquired from the GPS receiver, and the environment recognized with the method proposed in [9]; • Dataset 4: Composed by Standard Deviation of the raw signal, Average of the raw signal, Variance of the of the raw signal, and Median of the raw signal, extracted from the accelerometer, magnetometer and gyroscope sensors, the distance travelled extracted from the data acquired from the GPS receiver, and the environment recognized with the method proposed in [9]; • Dataset 5: Composed by Standard Deviation of the raw signal, and Average of the raw signal, extracted from the accelerometer, magnetometer and gyroscope sensors, the distance travelled extracted from the data acquired from the GPS receiver, and the environment recognized with the method proposed in [9].

Artificial Intelligence
The main focus of this study is related to the fusion of the location sensors with the other motion, mechanical and acoustic sensors available in the off-the-shelf mobile devices, based on the literature review presented in the section 2, verifying that the ANN are one of the most used methods that achieves the best accuracy in the differentiation of standing activities using the features extracted from the location sensors.
The main question of this paper will answer in the section 4 is related to the research about the accuracies obtained with different types of neural networks and the datasets defined in the section 3.3.1, 3.3.2 and 3.3.3, for the recognition of standing activities, which are extracted from the data available in [36]. For this purpose, we implemented three types of neural networks with different frameworks, these are: • MLP with Backpropagation, applied with Neuroph framework [16]; • Feedforward Neural Network with Backpropagation, applied with Encog framework [17]; • Deep Neural Networks (DNN), applied with DeepLearning4j framework [18]. The implementation were performed with normalized and non-normalized data, where, for the use of the MLP with Backpropagation and the Feedforward Neural Network with Backpropagation, the MIN/MAX normalizer [38] was implemented, and, for the use of the DNN method, the normalization with mean and standard deviation [39] and the application of the L2 regularization [40] were performed. The normalization of the datasets was implemented in order to verify if the normalization increases the reliability and accuracy of the methods using neural networks and to discover the best method for the implementation in the framework for the recognition of ADL and their environments, increasing the accuracy in the recognition of standing activities.
In addition of the implementation of different types of neural networks, we tested the creation of the methods with different numbers of maximum training iterations, attempting to discover if the number of training iterations influence the accuracy of the method developed, testing with 1M, 2M and 4M training iterations.
The artificial intelligence methods defined for the implementation of the framework for the recognition of the ADL and their environments should be based on our previous studies [7][8][9], concluding that DNN using normalized data and L2 regularization is the best method for the recognition of ADL with movement, Feedforward Neural Network with Backpropagation using non-normalized data is the best method for the recognition of the environments, and, at the end of this study, the best method for the recognition of standing activities should be presented, based on the datasets defined in the section 3.3.1, 3.3.2 and 3.3.3.

Results
This section is focused on the presentation of the results obtained with the method of recognition of standing activities (i.e., watching TV, sleeping and driving) using accelerometer, gyroscope, magnetometer, microphone, and GPS receiver, presenting the main achievements different sensors, these are the use of accelerometer, environment previously recognized and distance calculated from the GPS receiver (section 4.1), the use of accelerometer, magnetometer, environment previously recognized and distance calculated from the GPS receiver (section 4.2), and, finally, the use of accelerometer, magnetometer, gyroscope, environment previously recognized and distance calculated from the GPS receiver (section 4.3).

Identification of the standing activities with the environment and the Accelerometer and Location sensors' data
Following the artificial intelligence methods and frameworks presented in the section 3.3.4, and the datasets presented in the section 3.3.1, we implemented the MLP with Backpropagation, Feedforward Neural Network with Backpropagation, and DNN methods for the identification of the best method for the recognition of standing activities. The datasets used for the training and testing phases are composed by the features extracted from the data acquired from the standing activities previously defined (i.e., watching TV, sleeping and driving) in a total of 6000 records equally distributed for the proposed standing activities.
Firstly, the results of the implementation of the MLP with Backpropagation using the Neuroph framework are presented in the figure 1, verifying that the results have reliable accuracy with all datasets. With non-normalized data ( figure 1-a), the results achieved are around 100%, except with the dataset 1. And, with normalized data ( figure 1-b), the results obtained are always around 100% with all datasets. In table 4, the maximum accuracies achieved with the different types of neural networks are presented with the relation of the different datasets used for the environment recognized, distance travelled and the accelerometer data, and the maximum number of iterations, verifying that the use of all neural networks achieves reliable results.  Regarding the results obtained, in the case of the use of the environment recognized, the distance travelled and the accelerometer data in the module for the recognition of standing activities implemented in the framework for the identification ADL and their environments, the type of neural networks that should be used is a DNN with normalized data, because the results obtained are always 100%.

Identification of the standing activities with the environment and the Accelerometer, Magnetometer and Location sensors' data
Following the artificial intelligence methods and frameworks presented in the section 3.3.4, and the datasets presented in the section 3.3.2, we implemented the MLP with Backpropagation, Feedforward Neural Network with Backpropagation, and DNN methods for the identification of the best method for the recognition of standing activities. The datasets used for the training and testing phases are composed by the features extracted from the data acquired from the standing activities previously defined (i.e., watching TV, sleeping and driving) in a total of 6000 records equally distributed for the proposed standing activities.
Firstly, the results of the implementation of the MLP with Backpropagation using the Neuroph framework are presented in the figure 4, verifying that the results have reliable accuracy with all datasets. With non-normalized data ( figure 4-a), the results achieved are around 100%, except with the dataset 1. And, with normalized data ( figure 4-b), the results obtained are always around 100% with all datasets. In table 5, the maximum accuracies achieved with the different types of neural networks are presented with the relation of the different datasets used for the environment recognized, the distance travelled, and the accelerometer and magnetometer sensors' data, and the maximum number of iterations, verifying that the use of all neural networks achieves reliable results.  Regarding the results obtained, in the case of the use of the environment recognized, the distance travelled, and the accelerometer and magnetometer sensors' data in the module for the recognition of standing activities implemented in the framework for the identification ADL and their environments, the type of neural networks that should be used is a DNN with normalized data, because the results obtained are always 100%.

Identification of the standing activities with the environment and the Accelerometer, Magnetometer, Gyroscope and Location sensors' data
Following the artificial intelligence methods and frameworks presented in the section 3.3.4, and the datasets presented in the section 3.3.3, we implemented the MLP with Backpropagation, Feedforward Neural Network with Backpropagation, and DNN methods for the identification of the best method for the recognition of standing activities. The datasets used for the training and testing phases are composed by the features extracted from the data acquired from the standing activities previously defined (i.e., watching TV, sleeping and driving) in a total of 6000 records equally distributed for the proposed standing activities.
Firstly, the results of the implementation of the MLP with Backpropagation using the Neuroph framework are presented in the figure 7, verifying that the results have reliable accuracy with all datasets. With non-normalized data (figure 7-a), the results achieved are around 100%, except with the dataset 1. And, with normalized data ( figure 7-b), the results obtained are always around 100% with all datasets. In table 6, the maximum accuracies achieved with the different types of neural networks are presented with the relation of the different datasets used for the environment recognized, the distance travelled, and the accelerometer, magnetometer and gyroscope sensors' data, and the maximum number of iterations, verifying that the use of all neural networks achieves reliable results.  Regarding the results obtained, in the case of the use of the environment recognized, the distance travelled and the accelerometer, magnetometer and gyroscope sensors' data in the module for the recognition of standing activities implemented in the framework for the identification ADL and their environments, the type of neural networks that should be used is a DNN with normalized data, because the results obtained are always 100%.

Discussion
This paper finishes the research about the development of a framework for the recognition of ADL and their environments [2][3][4] using all sensors available in the off-the-shelf mobile devices, which started with the studies [7,8] that uses the motion and magnetic sensors (i.e., accelerometer, gyroscope and magnetometer) available in off-the-shelf mobile devices for the recognition of ADL with movement, recognizing running, standing, going upstairs, going downstairs, and walking activities. Next, the study [9] focused on the recognition of the users' environment, recognizing bar, classroom, gym, kitchen, library, street, hall, watching TV, and bedroom, and allowing the differentiation between the most common standing activities (i.e., watching TV and sleeping). Finally, this study presents a method that increases the number of standing activities differentiated in [9], recognizing watching TV, driving and sleeping. Therefore, based on the literature reviews presented in the previous studies [7][8][9] and the literature review presented in this study, the table 7 summarizes the most recognized ADL and environments and the figure 10 presented the ADL and environments recognized with the proposed framework for the recognition of ADL and their environments using the motion, magnetic, acoustic and location sensors.  Regarding the results obtained with the table 7 and the figure 10, there are 11 ADL/environments recognized in the literature, but there are no studies using all sensors available in the off-the-shelf mobile devices. Firstly, regarding the results obtained with the accelerometer, our framework are able to recognize 5 of 6 ADL recognized in the literature (83%). Secondly, regarding the results obtained with the accelerometer, magnetometer and gyroscope sensors, our framework are also able to recognize 5 of 6 ADL recognized in the literature (83%). Thirdly, regarding the results obtained with the accelerometer, magnetometer, gyroscope and microphone, our framework are able to recognize 4 of 8 ADL/environments recognized in the literature (50%). Finally, regarding the results obtained with the accelerometer, magnetometer, gyroscope, microphone and GPS receiver, our framework are able to recognize 6 of 7 ADL recognized in the literature (86%). However, the proposed framework uses all sensors and the studies available in the literature use only a subset of sensors available in the off-the-shelf mobile devices, recognizing the most common ADL and environments. Thus, our final framework are able to recognized several ADL, such as going downstairs, going upstairs, running, walking, standing, running, sleeping and driving, and several environments, such as bard, classroom, gym, kitchen, library, street, hall, watching TV and bedroom.
The proposed framework for the recognition of ADL and their environments is composed by several modules, these are data acquisition, data processing, data fusion, and artificial intelligence methods, presented in the figure 11. Based on our previous studies [7,8], the data should be acquired and filtered for the extraction of the relevant features for the recognition of ADL (i.e., running, walking, standing, going upstairs and going downstairs), which are fused, normalized and, the L2 regularization is applied for the recognition of the ADL using DNN method. And, based in our previous study [9], the data should be acquired and filtered for the extraction of the relevant features for the recognition of the environments (i.e., bar, classroom, gym, kitchen, library, street, hall, watching TV and bedroom), which are fused and the Feedforward neural network with Backpropagation will be applied. The study [9] is also focused on the differentiation of the standing activities, where the data is acquired and filtered for the extraction of the relevant features for the recognition of these types of activities (i.e., sleeping and watching TV), which are fused, normalized and, the L2 regularization is applied for the recognition of the standing activities using DNN method. This study increases the number of standing activities recognized by the framework, where the data is acquired and filtered for the extraction of the relevant features for the recognition of these types of activities (i.e., sleeping, standing and watching TV), which are fused, normalized and, the L2 regularization is applied for the recognition of the standing activities using DNN method. The workflow of the framework for the recognition of ADL and their environments is refined in the figure 12, where the method for each stage are representing and the method applied should be a function of the number of sensors available in the off-the-shelf mobile device used, recognizing 7 ADL and 9 environments with reliable accuracy. Firstly, based on the our previous studies [7,8], the framework for the recognition of ADL and their environments starts with the recognition of the most common ADL, reporting that the best accuracy is around 89.51% using the DNN method with normalization of the data and the application of L2 regularization with data acquired from all sensors available in the off-the-shelf mobile device.
Secondly, based on the study [9], the framework for the recognition of ADL and their environments is able to recognize several environments with Feedforward neural networks with non-normalized data using the microphone data, reporting that the best accuracy is around 86.50%.
Based on this study, the final step of the framework for the recognition of ADL and their environments is related to the recognition of standing activities, reporting that the best accuracy is around 100% using the DNN method with normalization of the data and the application of L2 regularization with data acquired from all sensors available in the off-the-shelf mobile device.
In the table 8, the accuracy of the framework for the recognition of ADL and their environments is defined, verifying that the average accuracy of the framework for all devices with different combinations of sensors is 91.27%. In conclusion, the framework for the recognition of ADL and their environments allows the recognition of the ADL and environments previously defined, but with the GPS receiver is possible to infer the geographic location of the user, and, with the users' agenda available in the mobile device is possible to discover the lifestyle, monitoring the activities and combining with the agenda, handling to help in the control of the daily tasks, the monitoring of the activities of elderly people, and the monitoring of the physical training. The outputs of the framework are: • The ADL performed; • The environment, where ADL was performed; • The geographic location of the user; • The validation with the users' agenda as feedback.

Conclusions
The hardware of the off-the-shelf mobile devices includes several sensors that are able to handle the recognition of ADL and their environments [1] in a framework, presented in [2][3][4], that combines the data acquired from all sensors available in the off-the-shelf mobile devices, but the developed framework adapts their functionalities with based on the number of sensors available in the device used. This paper finished the definition of the methods that should be used in the different stages of the identification of the ADL and their environments. The framework starts with the data acquisition, data cleaning and feature extraction methods, and, at the first stage of the recognition of ADL, the framework uses the DNN method for the recognition of walking, running, standing, going downstairs, and going upstairs. In the second stage, the environments are recognized with the Feedforward neural networks with Backpropagation, these are bar, classroom, gym, kitchen, library, street, hall, watching TV, and bedroom. Finally, in the third stage, the framework uses DNN method for the recognition of standing activities, these are watching TV, sleeping, and driving.
The recognition of the ADL and environments is based on the features extracted from the accelerometer, gyroscope, magnetometer, microphone and GPS receiver. The features extracted from the accelerometer, gyroscope, and magnetometer sensors' data are the 5 greatest distances between the maximum peaks, the Average of the maximum peaks, the Standard Deviation of the maximum peaks, the Variance of the maximum peaks, the Median of the maximum peaks, the Standard Deviation of the raw signal, the Average of the raw signal, the Maximum value of the raw signal, the Minimum value of the raw signal, the Variance of the of the raw signal, and the Median of the raw signal. The features extracted from the accelerometer, gyroscope and magnetometer sensors' data are the 26 MFCC coefficients, the Standard Deviation of the raw signal, the Average of the raw signal, the Maximum value of the raw signal, the Minimum value of the raw signal, the Variance of the of the raw signal, and the Median of the raw signal. The features extracted from the GPS receiver are the distance travelled and the location.
The development of the framework for the recognition of ADL and their environments is based on the comparison of three types of neural networks, these are MLP with Backpropagation using the Neuroph framework [16], the Feedforward Neural Network with Backpropagation using the Encog framework [17], and the DNN using DeepLearning4j framework [18]. Depending on the number of sensors available in the mobile device, the first stage related to the recognition of common ADL has an average accuracy of 87.50%, the second stage related to the recognition of environments has an average accuracy of 86.50%, and the third stage related to the differentiation of standing activities has an average accuracy of 100%, concluding that the average accuracy of the framework is 91.27%.
Thus, the framework proves their reliability in the recognition of the recognition of ADL and their environments, where the expected outputs are the ADL and environment recognized, the geographic location, and their relation with the users' agenda.
As future work, this framework should be implemented in a mobile application in order to start the creation of the personal digital life coach [6]. The data related to this research is available in a free repository [36].