STFNets: Learning Sensing Signals from the Time-Frequency Perspective
  with Short-Time Fourier Neural Networks
  Recent advances in deep learning motivate the use of deep neural networks in
Internet-of-Things (IoT) applications. These networks are modelled after signal
processing in the human brain, thereby leading to significant advantages at
perceptual tasks such as vision and speech recognition. IoT applications,
however, often measure physical phenomena, where the underlying physics (such
as inertia, wireless signal propagation, or the natural frequency of
oscillation) are fundamentally a function of signal frequencies, offering
better features in the frequency domain. This observation leads to a
fundamental question: For IoT applications, can one develop a new brand of
neural network structures that synthesize features inspired not only by the
biology of human perception but also by the fundamental nature of physics?
Hence, in this paper, instead of using conventional building blocks (e.g.,
convolutional and recurrent layers), we propose a new foundational neural
network building block, the Short-Time Fourier Neural Network (STFNet). It
integrates a widely-used time-frequency analysis method, the Short-Time Fourier
Transform, into data processing to learn features directly in the frequency
domain, where the physics of underlying phenomena leave better foot-prints.
STFNets bring additional flexibility to time-frequency analysis by offering
novel nonlinear learnable operations that are spectral-compatible. Moreover,
STFNets show that transforming signals to a domain that is more connected to
the underlying physics greatly simplifies the learning process. We demonstrate
the effectiveness of STFNets with extensive experiments. STFNets significantly
outperform the state-of-the-art deep learning models in all experiments. A
STFNet, therefore, demonstrates superior capability as the fundamental building
block of deep neural networks for IoT applications for various sensor inputs.
