{'fraction_non_alphanumeric': 0.014476614699331848, 'fraction_numerical': 0.0, 'mean_word_length': 5.330985915492958, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Generation of Robot Motions from Environmental Sounds Using Inter-modality Mapping by RNNPB

This paper proposes mapping between different sensory modalities for a robot system to generate motion expressing auditory signals or sounds from the movements of objects. Since all correspondences between auditory signals and visual signals in the world are hard to memorize, the ability to generalize is indispensable. We adopted a neural circuit model called RNNPB, which has good generalization ability, for the learning model. We implemented the proposed system on the robot “Keepon.” We taught it horizontal reciprocating or rotating motions with fricative sounds and falling or overturning motion with the sounds of collision by manipulating a box object. Keepon behaved not only from learned events but also from unknown events. It could also generate various sounds according to observed motions.