Deep Autotuner: A Data-Driven Approach to Natural-Sounding Pitch
  Correction for Singing Voice in Karaoke Performances
  We describe a machine-learning approach to pitch correcting a solo singing
performance in a karaoke setting, where the solo voice and accompaniment are on
separate tracks. The proposed approach addresses the situation where no musical
score of the vocals nor the accompaniment exists: It predicts the amount of
correction from the relationship between the spectral contents of the vocal and
accompaniment tracks. Hence, the pitch shift in cents suggested by the model
can be used to make the voice sound in tune with the accompaniment. This
approach differs from commercially used automatic pitch correction systems,
where notes in the vocal tracks are shifted to be centered around notes in a
user-defined score or mapped to the closest pitch among the twelve
equal-tempered scale degrees. We train the model using a dataset of 4,702
amateur karaoke performances selected for good intonation. We present a
Convolutional Gated Recurrent Unit (CGRU) model to accomplish this task. This
method can be extended into unsupervised pitch correction of a vocal
performance, popularly referred to as autotuning.
