Actions Speak Louder than Listening: Evaluating Music Style Transfer
  based on Editing Experience
  The subjective evaluation of music generation techniques has been mostly done
with questionnaire-based listening tests while ignoring the perspectives from
music composition, arrangement, and soundtrack editing. In this paper, we
propose an editing test to evaluate users' editing experience of music
generation models in a systematic way. To do this, we design a new music style
transfer model combining the non-chronological inference architecture,
autoregressive models and the Transformer, which serves as an improvement from
the baseline model on the same style transfer task. Then, we compare the
performance of the two models with a conventional listening test and the
proposed editing test, in which the quality of generated samples is assessed by
the amount of effort (e.g., the number of required keyboard and mouse actions)
spent by users to polish a music clip. Results on two target styles indicate
that the improvement over the baseline model can be reflected by the editing
test quantitatively. Also, the editing test provides profound insights which
are not accessible from usual listening tests. The major contribution of this
paper is the systematic presentation of the editing test and the corresponding
insights, while the proposed music style transfer model based on
state-of-the-art neural networks represents another contribution.
