Real-time Ultrasound-enhanced Multimodal Imaging of Tongue using 3D
  Printable Stabilizer System: A Deep Learning Approach
  Despite renewed awareness of the importance of articulation, it remains a
challenge for instructors to handle the pronunciation needs of language
learners. There are relatively scarce pedagogical tools for pronunciation
teaching and learning. Unlike inefficient, traditional pronunciation
instructions like listening and repeating, electronic visual feedback (EVF)
systems such as ultrasound technology have been employed in new approaches.
Recently, an ultrasound-enhanced multimodal method has been developed for
visualizing tongue movements of a language learner overlaid on the face-side of
the speaker's head. That system was evaluated for several language courses via
a blended learning paradigm at the university level. The result was asserted
that visualizing the articulator's system as biofeedback to language learners
will significantly improve articulation learning efficiency. In spite of the
successful usage of multimodal techniques for pronunciation training, it still
requires manual works and human manipulation. In this article, we aim to
contribute to this growing body of research by addressing difficulties of the
previous approaches by proposing a new comprehensive, automatic, real-time
multimodal pronunciation training system, benefits from powerful artificial
intelligence techniques. The main objective of this research was to combine the
advantages of ultrasound technology, three-dimensional printing, and deep
learning algorithms to enhance the performance of previous systems. Our
preliminary pedagogical evaluation of the proposed system revealed a
significant improvement in flexibility, control, robustness, and autonomy.
