{'fraction_non_alphanumeric': 0.02754968174529906, 'fraction_numerical': 0.011466886934551284, 'mean_word_length': 4.830594900849858, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Imputation Analysis for Time Series Air Quality (PM10) Data Set: A Comparison of Several Methods

Good quality data is important to guarantee for the best quality results of research analysis. However, the quality of the data often being impacted by the existence of missing values that bring bad implication on the accuracy of analysis and subsequently lead to biased results. In air quality data set, missing values problem often caused by various reasons, for example machine malfunction and errors, computer system crashes, human error and insufficient sampling used. In the case for time series modelling, complete series of data is very important to enable for the model construction. This paper aims to highlight a systematic statistical procedure and analysis on how to investigate the performance of several missing values imputation methods to solve for the problem of missing value existence when data are time series. The knowledge could help researchers to implement a comprehensive procedure in deciding a type of imputation method that suits with their data. A case study was conducted using real data set from Shah Alam air quality monitoring station. The results have shown that the missing data at the monitoring station is completely at random (MCAR). Among six imputation methods compared and based on the performance of indicators such as RMSE, MAE, AI and R2 it is shown that imputation using Kalman Filter using ARIMA model is the best appropriate method for the data set.


Introduction
Time series is a sequence of measurements of some quantity of interest taken at different time points consecutively. Most of the time, the measurements are observed at constant spaced time intervals which producing a univariate discrete-time time series. The ultimate objective of univariate time series analysis is to find the dynamic dependence of yi on its past values {yt-1, yt-2,...} for the purpose of forecasting and trend investigation. However, there is a problem that might arise in gathering time series data which is the existence of missing values. Missing values occur when the values are unavailable due to several reasons. Consequently, missing values will cause the problem in the modelling of time series predictive model due to unavailable values. According to [1,9], missing values has become a common problem in almost all research studies including environmental and air pollution studies.
In particular, air quality monitoring is carried out to provide data for the use of the environmentalists and researchers to study air pollution problem as well as to predict its future occurrence. Thus, analysis of the recorded data plays an important role to provide information for mitigation and adaptation purposes. However, such analysis is complicated by the frequently large proportions of observations missing from the data. Missing values in air quality data set often reported to be due to malfunctioned equipment, routine maintenance, and human errors or even the system might stay off-line for several days. There are three major problems that may arise when dealing with incomplete data [2,14,15]. The first problem is there is a loss of information due to missing values and, as a consequence, a loss of efficiency. Second, there are several complications related to data handling, computation and analysis, due to irregularities in data structure and the impossibility of using standard software [13]. Lastly, the results may be biased due to systematic differences between observed and unobserved data. For the case of Time series data, missing value problem will prevent time series analysis to be conducted if it is not being treated. This is because time series analysis requires complete series of time-point data [3].Therefore, missing values need to be replaced with reasonable values. In statistics this process and technique of replacing with reasonable values is called imputation. Up to date, only a limited number of studies have taken a closer look at the special case of univariate time series imputation [4]. Due to the missing values problems, the estimation of missing values becomes the first priority in the data preparation process. Various replacement methods have been used to tackle the problem and are profoundly discussed in the literature.
Generally, methods available for imputing missing values can be divided into two main categories of experimentation which are single imputation and multiple imputations. Single imputation works by filling in one value for each missing one; they have many appealing features, because standard completedata methods can be applied directly and because imputation needs to be carried out only once. While multiple imputation generates multiple simulated values for each missing value, in order to reflect the uncertainty attached to missing data. Besides that, these methods require a full specification of the distributional form of data in order to derive the conditional distribution of the missing data given the observed data. The impact of missing data on the result of statistical analysis depends on the mechanism that made the data to be missing and on the way the data analyst deals with them as well as the choice of an appropriate method for handling missing data including the pattern of misingness and the missing mechanism. [5] had classified missing data mechanism can be in three categories including missing at random (MAR), missing completely at random (MCAR) and lastly missing not at random (MNAR). For most air quality data set, usually the missing data mechanism of air quality data is assumed to be MAR [6,7] discusses on the methods applicable to air quality data sets in the context of whether the air quality data is univariate and multivariate forms of data representation. The imputation methods applied includes the univariate, multivariate as well as hybrid. In a context of univariate data, linear, spline and nearest neighbourhood interpolation has been applied. While, for multivariate, Regression-based imputation (REGEM), nearest neighbour (NN), Self-organizing map (SOM) and multi-layer perception (MLP) are used. For multivariate air quality data in Malaysia, [8] had applied mean and medium substitution method; expectation maximization (EM) method, singular value decomposition (SVD), and K-nearest neighbour (KNN) method for multivariate air quality data sets. Their results showed that EM, KNN and SKNN methods were consistently superior irrespective of the different air quality monitoring stations and percentage of missingness. Another study by [1] has shown that EM imputation was the best method when compared to mean substitution and hot deck methods. [2] has found mean -beforeafter method is the best method compared to several Interpolation methods (i.e. Linear, Quadratic and Cubic) when applied for univariate time series air quality (PM10) data set in Seberang Perai, Penang, Malaysia. Other than that, a machine learning approach was also found actively considered for imputing missing values in multivariate environmental data set [17,18,19]. A recent study by [20] has incorporated a multiple imputation approach in solving missing values for multiple time series medical data set using Amelia package in R. The developed technique incorporates prior or bound argument in the imputation approach.
Noticeably, from the literatures, most of the studies conducting missing values imputation methods were in the context of multivariate air quality data set in comparison to univariate perspectives. On top of that, often in practical application, with the same kind of data set, researchers used a chosen technique that they feel appropriate based on the literature review obtained without knowing whether the method is really appropriate for their data set. Preliminary investigation to confirm about the type of missing mechanism is also often being neglected. Thus, for the better results of estimation accuracy of the imputed values when applying a chosen imputation method, this paper discusses on a comprehensive procedure and experimentation to prior investigate the type of missing mechanism and to analyse the performance of several considered imputation techniques. Since in Malaysia, the most indicative and significant element of air pollutant is PM10, hence, this study will be specified into missing values imputation methods in the context of univariate PM10 time series data for experiment illustration. The knowledge could help researchers to decide on the best imputation method for their data set.

Missing Value Investigation Process and Framework
After the data set to be studied was in hand at phase 1, investigation on the appropriate imputation technique for a data set should begin with the knowledge on the kind of missing mechanism whether the missingness is MAR, MCAR or MNAR. Based on this knowledge, next step investigation is the searching for the available techniques in the literature that have been used within the same kind and type of data set. These investigation activities are considered as the preliminary analysis (phase 2). When several techniques under MAR per say, has been identified, now the third step is to determine which one is the best technique. A comparison analysis must be conducted comprehensively and is conducted using missing data imputation analysis (phase 3). The determination of which method is the best can be based on the performance of the methods using several indicators such as Root Mean Square Error (RMSE), coefficient of determination (R 2 ), Mean Absolute Error (MAE), Agreement Index (AI) (phase 4) and etc. The framework of the investigation procedure is shown in figure 1.  Figure 2 describes the process which is generally comprises of four important stages of analysis within the missing data imputation analysis phase (phase 3). The first stage is obtaining benchmark data. What is benchmark data and how to obtain the data? Bench mark data is a real observed or recorded data set. It is a set of complete data taken as a sample from the population of studied data and is made as a reference data set [9]. The data set is used to assess the performance of imputation method based on the amount of difference between the observed and the imputed values (i.e. MAE) as well as their degree of relationship (i.e. R 2 and AI). After obtaining the benchmark data, simulation process will take place in stage 2. This simulation process will produce several sets of simulated data of several patterns of missingness. Missing data pattern can be generated according to various criteria including the amount or percentage of data being missing, for example 5%, 10% and 15%, the size of the sequence of the missingness and whether the missingness is random or not random depending on the prior analysis conducted in figure 1. Next, the chosen imputation methods are implemented to impute the missing observations in the simulated data sets at stage 3 and the performance of the methods is assessed in stage 4 using several indicators [9,10]. For the case study conducted in this paper, the performance indicators involved are RMSE, MAE, R 2 and AI. Let Y denotes the observed value that is assumed missing and Y-hat ( ŷ ) is the estimated or imputed value then the formula for RMSE, MAE, R 2 and AI [15,16] are as follows.

Imputation Methods
Missing values imputation analysis are conducted using several imputation techniques for univariate data set including the methods that have been used by previous researchers for Malaysia air quality data set such as Linear Interpolation (M1), Spline Interpolation (M2), Exponential Moving Average (M3) and mean before after (M6). Two other methods including Kalman Filter) using ARIMA Model (M5) and Random replacement (M4) are introduced to be applied for the data set. The analysis is conducted using R software. The mathematical models and algorithm of the techniques can be obtained from the following review sources; [2,7,8,11].

Linear Interpolation Method (M1)
When two data points are connected with a straight line, hence the interpolation function is as follow: t is the independent variable, ti (i=0,1,2,…) is a known value of the independent variable, and bi are unknown coefficients. Then from (4), and b1 = y(t1) -y(t0)/t1 -t0 (7) in which in this case ymissing= y1, in this case y1 will be the imputed value.

Spline Interpolation (M2)
Given a tabulated function yk = y(tk), k = 0, 1, …, N, a spline is a polynomial between each pair of tabulated points, but one whose coefficients are determined "slightly" non-locally. The non-locality is designed to guarantee global smoothness in the interpolated function up to some order of derivative. We focus on one particular interval (tk, tk+1). The linear spline interpolation in that interval gives the interpolation formula y = Ay(tk) + By(tk+1) where

Exponential Moving Average (M3)
A moving average not just a regular mean, as new values become available, the oldest data points must be dropped from the set and new data points must come in to replace them. Thus, the data set is constantly "moving" to account for new data as it becomes available. This method of calculation ensures that only the current information is being accounted for. The exponential moving average is a type of moving average that gives more weight to recent observed value in an attempt to make it more responsive to new information. Below is the formula for imputation by exponential moving average:  6 is a function that takes a SEED (number) as input and returns a random value from a uniform distribution between 0 and 1.

Kalman Filter Using ARIMA Model (M5)
In a state space model, we have a (potentially unobserved) state variable, αt, and measurements, yt. Let Yt-1 be all measurable ({y1, y2, ...., yt-1}) variables up to time t -1. The state space is characterized by State Equation, describing the evolution state: And measurement Equation, describing how the measurable variables relate to state variables.
The Kalman Filter is a recursive algorithm for obtaining predictions of future observations and quickly provides the one step ahead prediction errors and the variances. For an ARIMA model, Kalman filter can be used for the following task: 1. Construct the conditional distribution of yt and αt given all past observations |yt-1. We know that it is normal, Kalman filter calculates the conditional mean yt|t-1,αt|t-1 and conditional variances. 2. Calculate a likelihood of { y1, y2, ...., yT} 3. Construct the distribution of α given all observations of y, αt|yt 4. Calculate yt|t-1, which is the best linear forecast of yt given |Yt-1 even if error are non-normal.

Mean Before After Method (M6)
Replacing the missing value by averaging both before and after values of that specific missing value. yimputed = 0.5*(yt-1 + yt+1), (17) Given that yt is the missing value.

Application and Case Study
For an illustration purposes, a case study analysis is conducted using series of four months period of hourly recorded PM10 data ( June 2015 -September 2015) from Shah Alam air quality monitoring station. The data were obtained from the Department of Environment (DOE), Putrajaya Malaysia. The original data of the Air Quality Data (PM10) from June 2015 to September 2015 from Shah Alam Station is a daily hourly recorded PM10 data which is also categorized as Univariate Time Series Data which consists of missing values. From this dataset, a complete case data of 1009 observations (consist of 42 days of hourly data) which cover from July 2015 up to August 2015 were selected for simulation. This data set is treated as benchmark data and will be used as reference data set. For the missing imputation analysis, several pattern of missing data set were simulated. In simulation, we make the comparison of the errors and performances for each of the methods listed above based on the missing percentage of 5%, 10% and 15% by generating 200 simulated patterns for each missing percentage.

Results of Case Study Analysis
Overall, based on the results in figure 3, it shows that the error (MAE and RMSE) would be increased and the measure of performances (R 2 and d) decreased as the percentages of simulated missing data increases. This was justified by the statement stated [12,16] that the validity of the estimates would be decreased when the missing values increased.
M4 shown the highest RMSE among the six methods of imputation. M4 gave the highest error in the range of 34.90 and 35.38. Error in M4 shown a big difference compared to other methods which probably will make M4 is the least preferable for imputing missing values. M5 has the least MAE of the range of 7.07 and 7.16. Then it followed by M1 and M3 which has MAE of the range of 7.14 and 7.32.  Based on the summary result in table 1, the performances are parallel according to its percentage of missingness. The best imputation method for estimating the simulated missing data is M5 (Kalman Filter using ARIMA Model) method. This is because M5 give the smallest values of MAE and RMSE and the highest values for R 2 and Agreement Index (AI) in almost all parameters and percentages of the simulated missing data. Hence, M5 is the best method used for imputing missing values in this study. should be prevented since this method might give a greatest error and produced biasness. M1, M3 and M5 are the methods that seems to be the good methods among the other methods. As shown in the figure above, these methods produced a fairly good values of RMSE from the range of 9 to 11 on average. However, among these three methods, it can be seen that M5 gave the least value of RMSE and should be considered as the best method with least error. Mean Absolute Error (MAE) for M4 are very high compare to the remaining five methods regardless of its percentage of missing values. The boxplots for M1, M3, and M5 looked alike and the values behave the same within the range of 6.5 and 8. These three methods can be considered the good methods so far. Therefore, we should look a bit details on how these three differ between each other. For M5, it is clearly shown that it's median for MAE (average of 7) are the lowest among those three methods. M1, M3 and M5 are the methods that seem to be the good methods among other methods. As shown in the figure above, these methods produced a quite good values of Agreement Index (AI) from the range of 0.90 to 0.95 on average. However, among these three methods, it can be seen that M5 gave the highest value of Agreement Index (AI) and should be considered as the best method with a good performance in terms of Agreement Index (AI) which is approaching to 1.
By referring to figure 5, Linear Interpolation, Exponential Moving Average and Kalman Filter using ARIMA Model are the methods that can be considered as good methods. These three methods are good because the scatterplots explained that almost all points aligned in a line which are having a strong positive relationship between imputed values and observed values and consistent compared to the other three methods used. Therefore, these three methods of imputation (Linear Interpolation, Exponential Moving Average and Kalman Filter using ARIMA Model) are recommended since they imputed those missing values with the values that approaching the same to its actual values.
. Figure 5. The relationship between the observed and estimated value based on scatter plot.

Conclusion
The main aim of this paper is to discuss and highlight the framework and the step by step approach needed in the investigation of missing values problem focuses on time series data set. To illustrate the methodology, a case study to assess and compare several imputation methods for PM10 time series from Shah Alam Air Quality Monitoring Station were conducted. The experimentation in determining the best appropriate methods were conducted involving Linear Interpolation, Spline Interpolation, Exponential Moving Average, Random Sample, Kalman Filter using ARIMA Model and Mean Before and After. The results of the analysis show that Kalman Filter using ARIMA Model is the the best imputation method for PM10 time series data at Shah Alam air quality monitoring station. However, this identified method cannot be generalized to other locations. The same experiment and analysis will need to be conducted for the data set at different location. This is because, the data might have different behaviour due to different background and would be influenced by different pattern of meteorological factors and geography.