Detection of Anomalies in a Time Series Data using InfluxDB and Python
  Analysis of water and environmental data is an important aspect of many
intelligent water and environmental system applications where inference from
such analysis plays a significant role in decision making. Quite often these
data that are collected through sensible sensors can be anomalous due to
different reasons such as systems breakdown, malfunctioning of sensor
detectors, and more. Regardless of their root causes, such data severely affect
the results of the subsequent analysis. This paper demonstrates data cleaning
and preparation for time-series data and further proposes cost-sensitive
machine learning algorithms as a solution to detect anomalous data points in
time-series data. The following models: Logistic Regression, Random Forest,
Support Vector Machines have been modified to support the cost-sensitive
learning which penalizes misclassified samples thereby minimizing the total
misclassification cost. Our results showed that Random Forest outperformed the
rest of the models at predicting the positive class (i.e anomalies). Applying
predictive model improvement techniques like data oversampling seems to provide
little or no improvement to the Random Forest model. Interestingly, with
recursive feature elimination, we achieved a better model performance thereby
reducing the dimensions in the data. Finally, with Influxdb and Kapacitor the
data was ingested and streamed to generate new data points to further evaluate
the model performance on unseen data, this will allow for early recognition of
undesirable changes in the drinking water quality and will enable the water
supply companies to rectify on a timely basis whatever undesirable changes
abound.
