SoundDet: Polyphonic Moving Sound Event Detection and Localization from
  Raw Waveform
  We present a new framework SoundDet, which is an end-to-end trainable and
light-weight framework, for polyphonic moving sound event detection and
localization. Prior methods typically approach this problem by preprocessing
raw waveform into time-frequency representations, which is more amenable to
process with well-established image processing pipelines. Prior methods also
detect in segment-wise manner, leading to incomplete and partial detections.
SoundDet takes a novel approach and directly consumes the raw, multichannel
waveform and treats the spatio-temporal sound event as a complete
"sound-object" to be detected. Specifically, SoundDet consists of a backbone
neural network and two parallel heads for temporal detection and spatial
localization, respectively. Given the large sampling rate of raw waveform, the
backbone network first learns a set of phase-sensitive and frequency-selective
bank of filters to explicitly retain direction-of-arrival information, whilst
being highly computationally and parametrically efficient than standard 1D/2D
convolution. A dense sound event proposal map is then constructed to handle the
challenges of predicting events with large varying temporal duration.
Accompanying the dense proposal map are a temporal overlapness map and a motion
smoothness map that measure a proposal's confidence to be an event from
temporal detection accuracy and movement consistency perspective. Involving the
two maps guarantees SoundDet to be trained in a spatio-temporally unified
manner. Experimental results on the public DCASE dataset show the advantage of
SoundDet on both segment-based and our newly proposed event-based evaluation
system.
