The Potential of Using Vision Videos for CrowdRE: Video Comments as a
  Source of Feedback
  Vision videos are established for soliciting feedback and stimulating
discussions in requirements engineering (RE) practices, such as focus groups.
Different researchers motivated the transfer of these benefits into crowd-based
RE (CrowdRE) by using vision videos on social media platforms. So far, however,
little research explored the potential of using vision videos for CrowdRE in
detail. In this paper, we analyze and assess this potential, in particular,
focusing on video comments as a source of feedback. In a case study, we
analyzed 4505 comments on a vision video from YouTube. We found that the video
solicited 2770 comments from 2660 viewers in four days. This is more than 50%
of all comments the video received in four years. Even though only a certain
fraction of these comments are relevant to RE, the relevant comments address
typical intentions and topics of user feedback, such as feature request or
problem report. Besides the typical user feedback categories, we found more
than 300 comments that address the topic safety, which has not appeared in
previous analyses of user feedback. In an automated analysis, we compared the
performance of three machine learning algorithms on classifying the video
comments. Despite certain differences, the algorithms classified the video
comments well. Based on these findings, we conclude that the use of vision
videos for CrowdRE has a large potential. Despite the preliminary nature of the
case study, we are optimistic that vision videos can motivate stakeholders to
actively participate in a crowd and solicit numerous of video comments as a
valuable source of feedback.
