Generating and Estimating Nonverbal Alphabets for Situated and
  Multimodal Communications
  In this paper, we discuss the formalized approach for generating and
estimating symbols (and alphabets), which can be communicated by the wide range
of non-verbal means based on specific user requirements (medium, priorities,
type of information that needs to be conveyed). The short characterization of
basic terms and parameters of such symbols (and alphabets) with approaches to
generate them are given. Then the framework, experimental setup, and some
machine learning methods to estimate usefulness and effectiveness of the
nonverbal alphabets and systems are presented. The previous results demonstrate
that usage of multimodal data sources (like wearable accelerometer, heart
monitor, muscle movements sensors, braincomputer interface) along with machine
learning approaches can provide the deeper understanding of the usefulness and
effectiveness of such alphabets and systems for nonverbal and situated
communication. The symbols (and alphabets) generated and estimated by such
methods may be useful in various applications: from synthetic languages and
constructed scripts to multimodal nonverbal and situated interaction between
people and artificial intelligence systems through Human-Computer Interfaces,
such as mouse gestures, touchpads, body gestures, eyetracking cameras,
wearables, and brain-computing interfaces, especially in applications for
elderly care and people with disabilities.
