Flat latent manifolds for music improvisation between human and machine
  The use of machine learning in artistic music generation leads to
controversial discussions of the quality of art, for which objective
quantification is nonsensical. We therefore consider a music-generating
algorithm as a counterpart to a human musician, in a setting where reciprocal
improvisation is to lead to new experiences, both for the musician and the
audience. To obtain this behaviour, we resort to the framework of recurrent
Variational Auto-Encoders (VAE) and learn to generate music, seeded by a human
musician. In the learned model, we generate novel musical sequences by
interpolation in latent space. Standard VAEs however do not guarantee any form
of smoothness in their latent representation. This translates into abrupt
changes in the generated music sequences. To overcome these limitations, we
regularise the decoder and endow the latent space with a flat Riemannian
manifold, i.e., a manifold that is isometric to the Euclidean space. As a
result, linearly interpolating in the latent space yields realistic and smooth
musical changes that fit the type of machine--musician interactions we aim for.
We provide empirical evidence for our method via a set of experiments on music
datasets and we deploy our model for an interactive jam session with a
professional drummer. The live performance provides qualitative evidence that
the latent representation can be intuitively interpreted and exploited by the
drummer to drive the interplay. Beyond the musical application, our approach
showcases an instance of human-centred design of machine-learning models,
driven by interpretability and the interaction with the end user.
