Quantifying the Effects of Prosody Modulation on User Engagement and
  Satisfaction in Conversational Systems
  As voice-based assistants such as Alexa, Siri, and Google Assistant become
ubiquitous, users increasingly expect to maintain natural and informative
conversations with such systems. However, for an open-domain conversational
system to be coherent and engaging, it must be able to maintain the user's
interest for extended periods, without sounding boring or annoying. In this
paper, we investigate one natural approach to this problem, of modulating
response prosody, i.e., changing the pitch and cadence of the response to
indicate delight, sadness or other common emotions, as well as using
pre-recorded interjections. Intuitively, this approach should improve the
naturalness of the conversation, but attempts to quantify the effects of
prosodic modulation on user satisfaction and engagement remain challenging. To
accomplish this, we report results obtained from a large-scale empirical study
that measures the effects of prosodic modulation on user behavior and
engagement across multiple conversation domains, both immediately after each
turn, and at the overall conversation level. Our results indicate that the
prosody modulation significantly increases both immediate and overall user
satisfaction. However, since the effects vary across different domains, we
verify that prosody modulations do not substitute for coherent, informative
content of the responses. Together, our results provide useful tools and
insights for improving the naturalness of responses in conversational systems.
