Near real-time map building with multi-class image set labelling and
  classification of road conditions using convolutional neural networks
  Weather is an important factor affecting transportation and road safety. In
this paper, we leverage state-of-the-art convolutional neural networks in
labelling images taken by street and highway cameras located across across
North America. Road camera snapshots were used in experiments with multiple
deep learning frameworks to classify images by road condition. The training
data for these experiments used images labelled as dry, wet, snow/ice, poor,
and offline. The experiments tested different configurations of six
convolutional neural networks (VGG-16, ResNet50, Xception, InceptionResNetV2,
EfficientNet-B0 and EfficientNet-B4) to assess their suitability to this
problem. The precision, accuracy, and recall were measured for each framework
configuration. In addition, the training sets were varied both in overall size
and by size of individual classes. The final training set included 47,000
images labelled using the five aforementioned classes. The EfficientNet-B4
framework was found to be most suitable to this problem, achieving validation
accuracy of 90.6%, although EfficientNet-B0 achieved an accuracy of 90.3% with
half the execution time. It was observed that VGG-16 with transfer learning
proved to be very useful for data acquisition and pseudo-labelling with limited
hardware resources, throughout this project. The EfficientNet-B4 framework was
then placed into a real-time production environment, where images could be
classified in real-time on an ongoing basis. The classified images were then
used to construct a map showing real-time road conditions at various camera
locations across North America. The choice of these frameworks and our analysis
take into account unique requirements of real-time map building functions. A
detailed analysis of the process of semi-automated dataset labelling using
these frameworks is also presented in this paper.
