{'fraction_non_alphanumeric': 0.019425019425019424, 'fraction_numerical': 0.0, 'mean_word_length': 5.313725490196078, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
PRRM: An Efficient Framework for Learning Multi-step Robotic Manipulation Tasks

Multi-step tasks in cluttered environment are extremely tough for robotic manipulation. Such a task interweaves high-level reasoning that estimates the stage of the current state in achieving the overall goal and low-level reasoning that determines which action will make progress. We propose a PRRM framework, which is modular and loop-closed, realizing the learning of multi-step manipulation tasks through self-supervised learning. The framework involves an object detection module used to provide guidance for action selection. We introduce a vision-based Action Projection Network (AcProNet) that maps visual observation to the execution values of action candidates, trained by a deep Q learning method. We define a reward function that the reward weights of different actions can be adjusted according to the goals of tasks. We further introduce a policy that determines the ultimate action from action candidates related to the results of detection module. We demonstrate the effectiveness of our framework by completing simulated trials of several multi-step tasks. Experimental results show that our framework can learn complex behaviors in a cluttered environment, and achieve a good performance.