Deep Music Information Dynamics
  Music comprises of a set of complex simultaneous events organized in time. In
this paper we introduce a novel framework that we call Deep Musical Information
Dynamics, which combines two parallel streams - a low rate latent
representation stream that is assumed to capture the dynamics of a thought
process contrasted with a higher rate information dynamics derived from the
musical data itself. Motivated by rate-distortion theories of human cognition
we propose a framework for exploring possible relations between imaginary
anticipations existing in the listener's mind and information dynamics of the
musical surface itself. This model is demonstrated for the case of symbolic
(MIDI) data, as accounting for acoustic surface would require many more layers
to capture instrument properties and performance expressive inflections. The
mathematical framework is based on variational encoding that first establishes
a high rate representation of the musical observations, which is then reduced
using a bit-allocation method into a parallel low rate data stream. The
combined loss considered here includes both the information rate in terms of
time evolution for each stream, and the fidelity of encoding measured in terms
of mutual information between the high and low rate representations. In the
simulations presented in the paper we are able to juxtapose aspects of
latent/imaginary surprisal versus surprisal of the music surface in a manner
that is quantifiable and computationally tractable. The set of computational
tools is discussed in the paper, suggesting that a trade off between
compression and prediction are an important factor in the analysis and design
of time-based music generative models.
