Dance Dance Convolution
  Dance Dance Revolution (DDR) is a popular rhythm-based video game. Players
perform steps on a dance platform in synchronization with music as directed by
on-screen step charts. While many step charts are available in standardized
packs, players may grow tired of existing charts, or wish to dance to a song
for which no chart exists. We introduce the task of learning to choreograph.
Given a raw audio track, the goal is to produce a new step chart. This task
decomposes naturally into two subtasks: deciding when to place steps and
deciding which steps to select. For the step placement task, we combine
recurrent and convolutional neural networks to ingest spectrograms of low-level
audio features to predict steps, conditioned on chart difficulty. For step
selection, we present a conditional LSTM generative model that substantially
outperforms n-gram and fixed-window approaches.
