{'fraction_non_alphanumeric': 0.026168110284250336, 'fraction_numerical': 0.006343784311333415, 'mean_word_length': 4.89748201438849, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Towards Dialogue-based Navigation with Multivariate Adaptation driven by Intention and Politeness for Social Robots

Service robots need to show appropriate social behaviour in order to be deployed in social environments such as healthcare, education, retail, etc. Some of the main capabilities that robots should have are navigation and conversational skills. If the person is impatient, the person might want a robot to navigate faster and vice versa. Linguistic features that indicate politeness can provide social cues about a person's patient and impatient behaviour. The novelty presented in this paper is to dynamically incorporate politeness in robotic dialogue systems for navigation. Understanding the politeness in users' speech can be used to modulate the robot behaviour and responses. Therefore, we developed a dialogue system to navigate in an indoor environment, which produces different robot behaviours and responses based on users' intention and degree of politeness. We deploy and test our system with the Pepper robot that adapts to the changes in user's politeness.


Introduction
The perception of politeness of a user can be a reflection of their patience during interaction. In addition to other factors such as the robot appearance, robot behaviour is a crucial aspect for their acceptance. Hence, politeness cues are intimately related to the dynamics of behavior and interaction [4,7,12,23]. It is useful for adapting to the dynamic tension [20] that occurs as a user tries to maintain a sufficient degree of politeness while interacting with the robot. For example, sentence-initial you or an action directive verb can be impolite "You need to show..." or "Show me the...", whereas sentence-medial you or sentenceinitial could or would often indicates the politeness like in these sentences "Could you show me..." or "Would you take me to...".  Multivariate adaptive and affective dialogue systems based on linguistic features have been subject to previous research [1,8,22]. The effect of politeness on the conversation is prominent and it has been researched in the sociolinguist community [7,12,20]. The effect of such a feature on human-robot interaction (HRI) has been a subject of study with various aspects: an impolite vs. a polite robot playing a game [5], in determining social robot acceptance with multi-cultural background people [21], making robots sociable and to achieve safe HRI [8]. Hence, a robot that can recognize the intention of the user during interaction should also adapt to the human's linguistic behavioural changes. For example, different sociolinguistic features such as politeness, emotion, sentiment, etc. represent the user's behavioural dynamics. If the user's utterance is impolite, then he/she might be in a hurry and vice versa. In such cases, the robot might need to change its behaviour or even alter some actions or speed up or down the movements. We develop a modular dialogue system (DS) that can process such features and make a robot to adapt accordingly. The natural language understanding part of the DS uses recurrent neural networks (RNNs) [25,26] and the Snips library [6] for extracting the structured information from the user input. The politeness detection is learned from the human-annotated corpus and fine-tuned for the scenario-specific data. The navigation of the Pepper robot is achieved by using the NAOqi framework. The robot behaviour and responses are driven by dialogue flow module using the intention and politeness. The main contributions of the present work towards bridging the gap between sociolinguistic research and HRI community are: -developing a dialogue-based navigation for incorporating politeness, and -incorporating sociolinguistic features for robotic behavioural modelling.
To the best of our knowledge, our system is the first dialogue-based navigation that incorporates politeness as an important social cue to drive the robot behaviour and responses.

Approach
We propose a dialogue system which takes into account the degree of politeness as a factor that affects the conversation flow and the robot behaviour. The dialogue system takes the intention and other information, for example, slotvalue pairs (see details in Section 3.1) into account to understand the input utterance [25]. However, in the proposed model, the dialogue system processes sociolinguistic features to make inferences regarding the input utterance. The overall architecture is shown in Figure 1. The proposed system is customizable to any extent as the robot is controlled using a client-server architecture. The state and motion managers are wrapped into an application programming interface (API) as a server [10] and communicated via the dialogue flow module. The dialogue system can be accessed also if the robot is not connected to the server.

Natural Language Understanding
The input speech from a user is converted into text using the speech recognition module (from NAOqi). The language understanding module takes the converted input utterance forms a symbolic representation and provides the degree of politeness. The dialogue act recognition module is used to extract the symbolic representation and politeness detection module to detect the degree of politeness of that input utterance.

Dialogue Act Recognition Module
The dialogue act (DA) recognition is a crucial process in the dialogue system. The task is to decode the input utterance and form the symbolic representation, such as dialogue acts and slot-value pairs. For example, the utterance "could you please show me the retail department" can be decoded as {da : TakeToPlace, room : retail} where da represents the dialogue act or intention, room is a slot and retail its value. We created a dataset for the given scenario to be able to drive the conversation (some examples are given in Table 1). The following methods are used in conjunction for robustness by validating one another based on heuristics of their confidence values: (1) Dialogue act recognition using RNNs: The architecture is shown in Figure 2, where RNNs are used in a hierarchical fashion to learn the dialogue acts and slot-value pairs [3,14,26]. The RNNs are better at encoding the contextual and sequential information in the utterance [14]. The dialogue acts are classified with the first layer of the RNN, preserving the utterance representation utt rep1. utt rep1 is then used to recognize slots on the next layer of the RNN, producing a new utterance representation utt rep2. The values of slots are learned with next layer of the RNN using utt rep2 and the detected slot as a switch (∈) for the belonging values learned in this layer (see the output in Figure 5 for better understanding). We fit the model to the data and use the trained model for inference [3]. (2) Snips Natural Language Understanding (NLU) Engine: Snips NLU Engine 3 is an open source Python library that uses two approaches: (a) a deterministic parser and (b) a probabilistic parser [6]. The deterministic parser is basically a pattern matching mechanism which uses regular expressions to parse the input utterance. The probabilistic parser uses logistic regression for intent classification and conditional random fields (CRFs) for slot filling. For the given input utterance, the engine provides the intention and slot-value pairs.

Politeness Detection Module
The politeness detection module takes the input utterance as an input and computes its degree of politeness. An RNN is used to learn the degree of politeness from Stanford Politeness Corpus 4 [7]. We fine-tune the trained model for the dataset (mentioned in the previous section) that is created for the particular scenario to minimize uncertainty in prediction. The degree of politeness (DoP) varies from 1 to -1 (very polite to very impolite). For the sake of conceptual and computational simplicity, we discretized them into categories: polite (1), neutral (0) and impolite (-1); see the examples below:

Dialogue Flow
The dialogue flow is a central engine of the system which communicates with most of the modules. It is implemented as a main function to drive the DS. A rule-based and probabilistic belief tracking or dialogue state tracking model could be used to maintain the dialogue flow [25]. We used a rule-based model where the dialogue flow module keeps track of the input dialogue acts and DoP and send them to the response manager to fetch responses. The complete state loop has a queue to store the context information of the preceding utterances. It is helpful to trigger new dialogue acts based on the context information. For example, if the last dialogue act is TakeToPlace, it triggers a new dialogue act called FinishedOne to inform the system that the last action was finished and asks the user if he/she wishes to visit the next place. Another loop keeps track of whether the user accepts or rejects the proposal using Accept and Reject dialogue acts. If one of the dialogue acts appears, the robot takes the user to the next location until either the list of locations is finished or the user rejects to visit the next place.

Response Manager
The response manager is responsible for picking up the right response for the given intention and degree of politeness. Pre-defined response templates are stored in a data file that is accessed continuously during the interaction.

Robot Platform
Pepper is a 1.2 meter tall omnidirectional wheeled humanoid robot platform capable of exhibiting body language, perceiving and interacting with its surround- ings, and move autonomously. Due to its 17 joints and 20 degrees of freedom (DoF) kinematic configuration and edgeless design, the system is suitable for safe HRI [18]. The platform is equipped with a large variety of sensors and actuators that ensure safe navigation and a high degree of expressiveness: LED's are distributed across the head (eyes and ears) and torso (shoulders) to support non-verbal communication by modifying colour and intensity. The microphones and speakers allow verbal interaction as well as environmental awareness. Sensing components include three laser sensors, two sonars and two infrared sensors located in the robot's base, as well as two cameras and a three-dimensional camera located in the head. Finally, the platform is powered by an Atom processor with a 1.91 GHz quad-core unit that allows the NAOqi SDK to orchestrate the different hardware elements as well as their access from other APIs.

State Manager
In order to produce the physical and verbal responses in accordance with the degree of politeness exhibited during the interaction, a behavioural model inspired by the valence and arousal model [2] has been designed. The model is given the discrete DoP computed from the last utterance being [1, 0, -1] and maps the cumulative sum of the politeness of the previous and current utterances to different actuators. In this way, a variability is provided to every single social cue that can vary in order to fit the interaction needs.
The actuators used to externalize the robot's change of state are the LED's color [17], head pitch orientation [15], voice pitch [13] and navigation speed, and are mapped following the intuition (shown in Figure 3). For example, a user repetitively polite during the whole interaction will experience a decrement in the navigation speed of the robot, a head position oriented towards the user, green coloured eyes and a slightly higher voice pitch.

Motion Manager
The motion manager is responsible for navigation and can be operated in the following modes: Tele-operation In this mode, the Pepper robot could be teleoperated with the help of the NAOqi framework using the moveToward function from ALMotion service and the keys on the keyboard are used for moving or stopping the robot.

Scripted Navigation
The scripted navigation is achieved by commanding a robot to move to the specific positions/places with the known distances in the environment. This is also achieved with NAOqi framework using moveTo command from ALMotion service. We specify how far the robot has to move (in meters) and the orientations (in radians) it has to take during motion.
Navigation: Mapping and Planning This module requires the use of the Robot Operating System (ROS), an open source middle-ware framework. To fit our need for navigation, we have adopted the following approach for generating and post-processing the map. The current readings of the Pepper's depth image are converted into virtual laser data, using the package depthimage to laserscan [19,24]. An offline map (shown in Figure 4, and post-processed for testing purposes) can be acquired using gmapping (laser-based SLAM) [11]. Then, the localization is performed using Adaptive Monte Carlo Localization (acml ) [9]. Finally, the navigation uses a global planner with a map with inflated obstacles (costmap) and a local costmap with observations from the virtual laser data. The Dialog Flow requests a location from the API server (on the robot using a virtual machine) using an ID and this one sends the coordinates to the ROS navigation stack to execute the path.

Experiments and Results: A Real-World Scenario
The task is to navigate in the given environment to show a user the different departments. Our tour scenario in the lab consists of four departments: retail, education, tourism and healthcare as shown in the map in Figure 4. The robot is a guide which takes the user to the particular department using verbal interaction as mentioned in Section 3.2. When the user asks the robot, the input utterance gets processed by the DA recognition module which produces the result as shown in Figure 5. The politeness detection module provides the DoP of that utterance. The dialogue flow communicates this information with all the managers. The robot adapts its behaviour such as speeding up or down while navigating to the locations and changing the pitch of speech, changing the pitch angle of the head. We tested our system on the Pepper robot with different users, expressing different levels of politeness. The behavioural changes and adaptation to speed change based on a change in DoP are shown in Figure 6. The robot behavioural adapts to the human being polite; the robot slows down and spends more time with the user. When the user is impolite, the robot speeds up and executes motion faster. The proposed behaviour of the robot for different situations shown in the figure is mainly to demonstrate the developed system and the efficacy of the proposed framework. The results indicate that the system is able to consider the linguistic features to modulate the navigation behaviour of the robot in a coherent theoretical and functional framework. As aforementioned, to the best of our knowledge such a framework and implementation in a practical situation is one of the first attempts of its kind. However, it is important to mention that the  validation of the hypotheses about the most appropriate behaviours of the robot is not within the scope of this paper and it will require further investigation and user studies. As mentioned in the conclusion such studies are one of the next steps to utilize the framework for different situations. The demonstration video and dialogue logs of the generated graphs in Figure 6 are available at the SECURE EU Project website: https://secure-robots.eu/fellows/bothe/ secondment-project/

Conclusions and Future Work
We developed a dialogue-based navigation system for integrating intention and politeness features for multivariate adaptation of the robot. We successfully deployed and tested our system on the robot with different levels of politeness. Currently, our work does not elicit the causal explanation for the behaviour and the multivariate adaptation of the robot. However, our experimental framework opens up a new challenge for the study of the effect of politeness in humanrobot social interaction. We strongly believe that our work will be helpful in bridging the gap between sociolinguistic research and the HRI community. This research shall also be helpful in targeting the deployment of social-service robots with adaptation to sociolinguistic features such as politeness. In this work, the behaviours are based on previous research [8,16,21,23]. The validation of the system is crucial and it will be addressed in future work through user studies.