{'fraction_non_alphanumeric': 0.037029879948791034, 'fraction_numerical': 0.012488224353245248, 'mean_word_length': 4.807266096226679, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Multi-skill Mobile Manipulation for Object Rearrangement

We study a modular approach to tackle long-horizon mobile manipulation tasks for object rearrangement, which decomposes a full task into a sequence of subtasks. To tackle the entire task, prior work chains multiple stationary manipulation skills with a point-goal navigation skill, which are learned individually on subtasks. Although more effective than monolithic end-to-end RL policies, this framework suffers from compounding errors in skill chaining, e.g., navigating to a bad location where a stationary manipulation skill can not reach its target to manipulate. To this end, we propose that the manipulation skills should include mobility to have flexibility in interacting with the target object from multiple locations and at the same time the navigation skill could have multiple end points which lead to successful manipulation. We operationalize these ideas by implementing mobile manipulation skills rather than stationary ones and training a navigation skill trained with region goal instead of point goal. We evaluate our multi-skill mobile manipulation method M3 on 3 challenging long-horizon mobile manipulation tasks in the Home Assistant Benchmark (HAB), and show superior performance as compared to the baselines.


Introduction
Building AI with embodiment is an important future mission of AI. Object rearrangement [2] is considered as a canonical task for embodied AI. The most challenging rearrangement tasks [1,3,4] are often long-horizon mobile manipulation tasks, which demand both navigation and manipulation abilities, e.g., to move to certain locations and to pick or place objects. It is challenging to learn a monolithic RL policy for complex long-horizon mobile manipulation tasks, due to challenges such as high sample complexity, complicated reward design, and inefficient exploration. A practical solution to tackle a long-horizon task is to decompose it into a set of subtasks, which are tractable, short-horizon, and compact in state or action spaces. Each subtask can be solved by designing or learning a skill, so that a sequence of skills can be chained to complete the entire task [5][6][7][8]. For example, skills for object rearrangement can be picking or placing objects, opening or closing fridges and drawers, moving chairs, navigating in the room, etc.
Achieving successful object rearrangement using this modular framework requires careful subtask formulation such that skills trained for these subtasks can be chained together effectively. We define three desirable properties for skills to solve diverse long-horizon tasks: achievability, composability, and reusability. Note that we assume each subtask is associated with a set of initial states. Then, achievability quantifies the portion of initial states solvable by a skill. A pair of skills are composable if the initial states achievable by the succeeding skill can encompass the terminal states of the preceding skill. This encompassment requirement is necessary to ensure robustness to mild compounding errors.  Figure 1: 1a provides an overview of our multi-skill mobile manipulation (M3) method. The inactive part of the robot is colored gray. Previous approaches exclusively activate either the mobile platform or manipulator for each skill, and suffer from compounding errors in skill chaining given limited composability of skills. We introduce mobility to manipulation skills, which effectively enlarges the feasible initial set, and a region-goal navigation reward to facilitate learning the navigation skill. 1b illustrates one task (SetTable) in the Home Assistant Benchmark [1], where the robot needs to navigate in the room, open the drawers or fridge, pick multiple objects in drawers or fridge and place them on the table. Best viewed in motion at the project website 1 .
However, trivially enlarging the initial set of a subtask increases learning difficulty and may lead to many unachievable initial states for the designed/learned skill. Last, a skill is reusable if it can be directly chained without or with limited fine-tuning [6,8]. According to our experiments, effective subtask formulation is critical though largely overlooked in the literature.
In the context of mobile manipulation, skill chaining poses many challenges for subtask formulation. For example, an imperfect navigation skill might terminate at a bad location where the target object is out of reach for a stationary manipulation skill [1]. To tackle such "hand-off" problems, we investigate how to formulate subtasks for mobile manipulation. First, we replace stationary (fixed-base) manipulation skills with mobile counterparts, which allow the base to move when the manipulation is undertaken. We observe that mobile manipulation skills are more robust to compounding errors in skill chaining, and enable the robot to make full use of its embodiment to better accomplish subtasks, e.g., finding a better location with less clutter and fewer obstacles to pick an object. We emphasize how to generate initial states of manipulation skills as a trade-off between composability and achievability in Sec 4.1. Second, we study how to translate the start of manipulation skills to the navigation reward, which is used to train the navigation skill to connect manipulation skills. Note that the goal position in mobile manipulation plays a very different role from that in point-goal [9,10] navigation. On the one hand, the position of a target object (e.g., on the table or in the fridge) is often not directly navigable; on the other hand, a navigable position close to the goal position can be infeasible due to kinematic and collision constraints. Besides, there exist multiple feasible starting positions for manipulation skills, yet previous works such as [1] train the navigation skill to learn a single one, which is selected heuristically and may not be suitable for stationary manipulation. Thanks to the flexibility of our mobile manipulation skills, we devise a region-goal navigation reward to handle those issues, detailed in Sec 4.2.
In this work, we present our improved multi-skill mobile manipulation method M3, where mobile manipulation skills are chained by the navigation skill trained with our region-goal navigation reward. It achieves an average success rate of 63% on 3 long-horizon mobile manipulation tasks in the Home Assistant Benchmark [1], as compared to 50% for our best baseline. Fig 1 provides an overview of our method and tasks. Our contributions are listed as follows: 1. We study how to formulate mobile manipulation skills, and empirically show that they are more robust to compounding errors in skill chaining than stationary counterparts; 2. We devise a region-goal navigation reward for mobile manipulation, which shows better performance and stronger generalizability than the point-goal counterpart in previous works; 3. We show that our improved multi-skill mobile manipulation pipeline can achieve superior performance on long-horizon mobile manipulation tasks without bells and whistles, which can serve as a strong baseline for future study.
2 Related Work

Mobile Manipulation
Rearrangement [2] is "to bring a given physical environment into a specified state". We refer readers to [2] for a comprehensive survey. Many existing RL tasks can be considered as instances of rearrangement, e.g., picking and placing rigid objects [11,12] or manipulating articulated objects [13,14]. However, they mainly focus on stationary manipulation [11][12][13] or individual, shorthorizon skills [14]. Recently, several benchmarks like Home Assistant Benchmark (HAB) [1], ManipulaTHOR [3] and ThreeDWorld Transport Challenge [4], are proposed to study long-horizon mobile manipulation tasks. They usually demand that the robot rearranges household objects in a room, requiring exploration and navigation [15,16] between interacting with objects entirely based on onboard sensing, without any privileged state or map information.
Mobile manipulation [17] refers to "robotic tasks that require a synergistic combination of navigation and interaction with the environment". It has been studied long in the robotics community. [18] provides a summary of traditional methods, which usually require perfect knowledge of the environment. One example is task-and-motion-planning (TAMP) [19][20][21]. TAMP relies on well-designed state proposals (grasp poses, robot positions, etc.) to sample feasible trajectories, which is computationally inefficient and unscalable for complicated scenarios.
Learning-based approaches enable the robot to act according to visual observations. [22] proposes a hierarchical method for mobile manipulation in iGibson [23], which predicts either a high-level base or arm action by RL policies and executes plans generated by motion-planning to achieve the action. However, the arm action space is specially designed for a primitive action pushing. [24] develops a real-world RL framework to collect trash on the floor, with separate navigation and grasping policies. [3,18] train an end-to-end RL policy to tackle mobile pick-and-place in ManipulaTHOR [3]. However, the reward function used to train such an end-to-end policy usually demands careful tuning. For example, [18] shows that a minor modification (a penalty for disturbance avoidance) can lead to a considerable performance drop. The vulnerability of end-to-end RL approaches restricts scalability. Most prior works in both RL and robotics separate mobile the platform and manipulator, to "reduce the difficulty to solve the inverse kinematics problem of a kinematically redundant system" [25,26]. [27] trains an end-to-end RL policy based on the object pose and proprioception to simultaneously control the base and arm. It focuses on picking a single object up in simple scenes, while our work addresses long-horizon rearrangement tasks that require multiple skills.
[1] adopts a different hierarchical approach for mobile manipulation in the HAB [1]. It uses taskplanning [28] to generate high-level symbolic goals, and individual skills are trained by RL to accomplish those goals. It outperforms the monolithic end-to-end RL policy and the classical sense-plan-act robotic pipeline. It is scalable since skills can be composited to solve different tasks, and benefit from progress in individual skill learning [12,14]. Moreover, different from other benchmarks, the HAB features continuous motor control (base and arm), interaction with articulated objects (opening drawers and fridges), and complicated scene layouts. Thus, we choose the HAB as the platform to study long-horizon mobile manipulation.

Skill Chaining for Long-horizon Tasks
[1] observes that sequentially chaining multiple skills suffers from "hand-off" problems, where a preceding skill terminates at a state that the succeeding skill has either never seen during training or is infeasible to solve. [5] proposes to learn a transition policy to connect primitive skills, but assumes that such a policy can be found through random exploration. [8] regularizes the terminal state distribution of a skill to be close to the initial set of the following skill, through a reward learned with adversarial training. Most prior skill chaining methods focus on fine-tuning learned skills.
In this work, we instead focus on subtask formulation for skill chaining, which directly improves composability and reusability without additional computation.

Home Assistant Benchmark (HAB)
The Home Assistant Benchmark (HAB) [1] includes 3 long-horizon mobile manipulation rearrangement tasks (TidyHouse, PrepareGroceries, SetTable) based on the ReplicaCAD dataset, which contains a rich set of 105 indoor scene layouts. For each episode (instance of task), rigid objects from the YCB [29] dataset are randomly placed on annotated supporting surfaces of receptacles, to generate clutter in a randomly selected scene. Here we provide a brief description of these tasks.
TidyHouse: Move 5 objects from starting positions to goal positions. Objects and goals are located in open receptacles (e.g., table, kitchen counter) rather than containers. Complex scene layouts, diverse receptacles, dense clutter all pose challenges. The task implicitly favors collision-free behavior since a latter target object might be knocked out of reach when a former object is moved by the robot. PrepareGroceries: Move 2 objects from the fridge to the counters and move an object from the counter to the fridge. The fridge is fully open initially. The task requires picking and placing an object in a cluttered receptacle with restricted space. SetTable: Move a bowl from a drawer to a table, and move a fruit from the fridge to the bowl on the table. Both the drawer and fridge are closed initially. The task requires interaction with articulated objects as well as picking objects from containers.
All the tasks use the GeometricGoal [2] specification (s 0 , s * ), which describes the initial 3D (centerof-mass) position s 0 of the target object and the goal position s * . For example, TidyHouse is specified

Subtask and Skill
In this section, we present the definition of subtask and skill in the context of reinforcement learning. A long-horizon task can be formulated as a Markov decision process (MDP) 1 defined by a tuple (S, A, R, P, I) of state space S, action space A, reward function R(s, a, s ), transition distribution P (s |s, a), initial state distribution I. A subtask ω is a smaller MDP (S, A ω , R ω , P, I ω ) derived from the original MDP of the full task. A skill (or policy), which maps a state s ∈ S to an action a ∈ A, is learned for each subtask by RL algorithms.
[1] introduces several parameterized skills for the HAB: Pick, Place, Open fridge, Close fridge, Open drawer, Close drawer, Navigate. Each skill takes a single 3D position as input, either s 0 or s * . See Appendix C for more details. Here, we provide a brief description of these skills. Note that s 0 is constant per episode instead of a tracked object position. Hence, the target object may not be located at s 0 at the beginning of a skill, e.g., picking an object from an opened drawer. Next, we will illustrate how these skills are chained in the HAB.

Skill Chaining
Given a task decomposition, a hierarchical approach also needs to generate high-level actions to select a subtask and perform the corresponding skill. Task planning [28] can be applied to find a sequence of subtasks before execution, with perfect knowledge of the environment. An alternative is to learn high-level actions through hierarchical RL. In this work, we use the subtask sequences generated by a perfect task planner [1]. Here we list these sequences, to highlight the difficulty of tasks 2 .

Subtask Formulation and Skill Learning for Mobile Manipulation
Following the proposed principles (composability, achievability, reusability), we revisit and reformulate subtasks defined in the Home Assistant Benchmark (HAB) [1]. The core idea is to enlarge the initial states of manipulation skills to encompass the terminal states of the navigation skill, given our observation that the navigation skill is usually more robust to initial states. However, manipulation skills (Pick, Place, Open drawer, Close drawer) in [1], are stationary. The composability of a stationary manipulation skill is restricted, since its feasible initial states are limited due to kinematic constraints. For instance, the robot can not open the drawer if it is too close or too far from the drawer. Therefore, these initial states need to be carefully designed given the trade-off between composability and achievability, which is not scalable and flexible. On the other hand, the navigation skill, which is learned to navigate to the start of manipulation skills, is also restricted by stationary constraints, since it is required to precisely terminate at a small set of "good" locations for manipulation. To this end, we propose to replace stationary manipulation skills with mobile counterparts. Thanks to mobility, mobile manipulation skills can have better composability without sacrificing much achievability.
For example, a mobile manipulator can learn to first get closer to the target and then manipulate, to compensate for errors from navigation. It indicates that the initial states can be designed in a more flexible way, which also enables us to design a better navigation reward to facilitate learning.
In the context of mobile manipulation, the initial state of a skill consists of the robot base position, base orientation, and joint positions. For simplicity, we do not discuss the initial states of rigid and articulated objects in the scene, which are usually defined in episode generation. Moreover, we follow previous works [1,8] to initialize the arm at its resting position and reset it after each skill in skill chaining. Such a reset operation is common in robotics [21]. Each skill is learned to reset the arm after accomplishing the subtask as in [1]. Furthermore, for base orientation, we follow the heuristic in [1] to make the robot face the target position s 0 or s * .

Manipulation Skills with Mobility
We first present how initial base positions are generated in previous works. For stationary manipulation, a feasible base position needs to satisfy several constraints, e.g., kinematic (the target is reachable) and collision-free constraints. [1] uses heuristics to determine base positions. For Pick, Place without containers (fridge and drawer), a navigable position closest to the target position is selected. For Pick, Place with containers, a fixed position relative to the container is selected. For Open, Close, a navigable position is randomly selected from a handcrafted region relative to each container. Noise is added to base position and orientation in addition, and infeasible initial states are rejected by constraints. See Fig 2 for examples.
The above example indicates the difficulty and complexity to design feasible initial states for stationary manipulation. One naive solution is to enlarge the initial set with infeasible states, but this can hurt learning as shown later in Sec 5.4. Besides, rejection sampling can be quite inefficient in this case, and [1] actually computes a fixed number of feasible initial states offline.
Manipulation Skills with Mobility. To this end, we propose to use mobile manipulation skills instead. The original action space (only arm actions) is augmented with base actions. We devise a unified and efficient pipeline to generate initial base positions. Concretely, we first discretize the floor map with a resolution of 5 × 5cm 2 , and get all navigable (grid) positions. Then, different candidates are computed from these positions based on subtasks. Candidates are either within a radius (e.g., 2m) around the target position for Pick, Place, or a region relative to the container for Open, Close. Finally, a feasible position is sampled from the candidates with rejection and noise. Compared to stationary manipulation, the rejection rate of our pipeline is much lower, and thus can be efficiently employed on-the-fly during training. See

Navigation Skill with Region-Goal Navigation Reward
The navigation skill is learned to connect different manipulation skills. Hence, it needs to terminate within the set of initial achievable states of manipulation skills. We follow [1] to randomly sample a navigable base position and orientation as the initial state of navigation skill. The challenge is how to formulate the reward function, which implicitly defines desirable terminal states. A common navigation reward [9] is the negative change of geodesic distance to a single 2D goal position on the floor. [1] extends it for mobile manipulation, which introduces the negative change of angular distance to the desired orientation (facing the target). The resulting reward function, r t (s, a), for state s and action a is the following (Eq 1): , g) is the geodesic distance between the current base position x base t and the 2D goal position g. d geo where θ t is the current base orientation, and θ * is the target orientation. Note that the 2D goal on the floor is different from the 3D goal specification for manipulation subtasks. This reward has several drawbacks: 1) A single 2D goal needs to be assigned, which should be an initial base position of manipulation skills. It is usually sampled with rejection, as explained in Sec 4.1. It ignores the existence of multiple reasonable goals, introduces ambiguity to the reward (hindering training), and leads the skill to memorize (hurting generalization). 2) There is a hyperparameterD, which defines the region where the angular term ∆ ang is considered. However, it can lead the agent to learn the undesirable behavior of entering the region with a large angular distance, e.g., backing onto the target.
Region-Goal Navigation Reward. To this end, we propose a region-goal navigation reward for training the navigation skill. Inspired by object-goal navigation [30], we use the geodesic distance 3 between the robot and a region of 2D goals on the floor instead of a single goal. Thanks to the flexibility of our mobile manipulation skills, we can simply reuse the candidates (Sec 4.1) for their initial base positions as the navigation goals. However, these candidates are not all collision-free. Thus, we add a collision penalty r col = λ col C t to the reward, where C t is the current collision force and λ col is a weight. Besides, we simply remove the angular term, and find that the success reward is sufficient to encourage correct orientation. Our region-goal navigation reward is the following (Eq 2):

Experimental Setup
We use the ReplicaCAD dataset [1] and the Habitat 2.0 simulator [1] for our experiments. The ReplicaCAD dataset contains 5 macro variations, with 21 micro variations per macro variation 4 . We hold out 1 macro variation to evaluate the generalization of unseen layouts. For the rest of the 4 macro variations, we split 84 scenes into 64 scenes for training and 20 scenes to evaluate the generalization of unseen configurations (object and goal positions). For each task, we generate 6400 episodes (64 scenes) for training, 100 episodes (20 scenes) to evaluate cross-configuration generalization, and another 100 episodes (the hold-out macro variation) to evaluate cross-layout generalization. The robot is a Fetch [31] mobile manipulator with a 7-DoF arm and a parallel-jaw gripper.
Observation space: The observation space includes head and arm depth images (128 × 128), arm joint positions (7-dim), end-effector position (3-dim) in the base frame, goal positions (3-dim) in both base and end-effector frames, as well as a scalar to indicate whether an object is held. The goal position, depending on subtasks, can be either the initial or desired position of the target object.
We assume a perfect GPS+Compass sensor and proprioceptive sensors as in [1], which are used to compute the relative goal positions. For the navigation skill, only the head depth image and the goal position in the base frame are used.
Action space: The action space is a 10-dim continuous space, including 2-dim base action (linear forwarding and angular velocities), 7-dim arm action, and 1-dim gripper action. Grasping is abstract as in [1][2][3]. If the gripper action is positive, the object closest to the end-effector within 15cm will be snapped to the gripper; if negative, the gripper will release any object held. For the navigation skill, we use a discrete action space, including a stop action, as in [1,32]. A discrete action will be converted to continuous velocities to move the robot, while arm and gripper actions are masked out.
Hyper-parameters: We train each skill by the PPO [33] algorithm. The visual observations are encoded by a 3-layer CNN as in [1]. The visual features are concatenated with state observations and previous action, followed by a 1-layer GRU and linear layers to output action and value. We use 64 parallel environments and train each skill for 100M steps with 16 CPU cores and 1 2080Ti GPU. Each skill is trained with 3 different seeds. See Appendix C.1 for more details.
Metrics: Each HAB task consists of a sequence of subtasks to accomplish, as illustrated in Sec 3.3. The completion of a subtask is conditioned on the completion of its preceding subtask. We report progressive completion rates of subtasks, and the completion rate of the last subtask is thus the success rate of the full task. For each evaluation episode, the robot is initialized at a random base position and orientation without collision, and its arm is initialized at the resting position. The completion rate is averaged over 9 different runs (3 seeds for RL training multiplied by 3 seeds for initial states).

Baselines
We denote our method by M3, short for a multi-skill mobile manipulation pipeline where mobile manipulation skills (M) are chained by the navigation skill trained with our region-goal navigation reward (R). We compare our method with several RL baselines. All baselines follow the same experimental setup in Sec 5.1 unless specified. We refer readers to [1] for a task-and-motion-planning (TAMP) baseline, which is shown to be inferior to the skill chaining pipeline emphasized in this work. Stationary manipulation skills and point-goal navigation reward are denoted by S and P. baseline does not use the region-goal navigation reward. It demonstrates the effectiveness of proposed mobile manipulation skills. Note that the point-goal navigation reward is designed for the start of stationary manipulation skills. Since it does not require the policy to memorize ambiguous goals, the induced skill shows better generalizability, especially in the cross-layout setting (55.0% for M3 vs.36.2% for M+P).

Ablation Studies
We conduct several ablation studies to show that mobile manipulation skills are more flexible to formulate than stationary ones, and to understand the advantage of our navigation reward.
Can initial states be trivially enlarged? We conduct experiments to understand to what extent we can enlarge the initial states of manipulation skills given the trade-off between achievability and composability. In the S(L)+P experiment, we simply replace the initial states of stationary manipulation skills with those of mobile ones. The success rates of stationary manipulation skills on subtasks drop by a large margin, e.g., from 95% to 45% for Pick on TidyHouse. Fig 5 shows that S(L)+P (37.7%/18.1%) is inferior to both S+P (57.4%/31.1%) and M+P (64.9%/36.2%). It indicates that stationary manipulation skills have a much smaller set of feasible initial states compared to mobile ones, and including infeasible initial states during training can hurt performance significantly. Besides, we also study the impact of initial state distribution on mobile manipulation skills in Appendix F.
Is the collision penalty important for the navigation skill? Our region-goal navigation reward benefits from unambiguous region goals and the collision penalty. We add the collision penalty to the point-goal navigation reward (Eq 1) in S+P(C) and M+P(C) experiments.  (57.4%/31.1%) and M+P(C) (67.9%/49.2%) vs.M+P (64.9%/36.2%). A collision-aware navigation skill can avoid disturbing the environment, e.g., accidentally closing the fridge before placing an object in it. Besides, M+P(C) is still inferior to our M3 (71.2%/55.0%). It implies that reducing the ambiguity of navigation goals helps learn more robust and generalizable navigation skills.

Conclusion and Limitations
In this work, we present a modular approach to tackle long-horizon mobile manipulation tasks in the Home Assistant Benchmark (HAB), featuring mobile manipulation skills and the region-goal navigation reward. Given the superior performance, our approach can serve as a strong baseline for future study. Besides, the proposed principles (achievability, composability, reusability) can serve as a guideline about how to formulate meaningful and reusable subtasks. However, our work is still limited to abstract grasp and other potential simulation defects. We leave fully dynamic simulation and real-world deployment to future work.
Autonomous mobile robots that improve productivity.

A Overview
Compared to the original implementation [1], our implementation benefits from repaired assets (Sec B), improved reward functions and better training schemes (Sec C). Other differences include observation and action spaces. We introduce in observations the target positions in the base frame in addition to those in the end-effector frame. The arm action is defined in the joint configuration space (7-dim) rather than the end-effector Euclidean space (3-dim with no orientation).

B Dataset and Episodes
[1] keeps updating the ReplicaCAD dataset. The major fix is "minor furniture layout modifications in order to better accommodate robot access to the full set of receptacles" 5 . The agent radius is also decreased from 0.4m to 0.3m to generate navigation meshes with higher connectivity. Besides, [1] also improves the episode generator 6 to ensure stable initialization of objects. Those improvements eliminate most unachievable episodes in the initial version. The episodes used in our experiments are generated with the ReplicaCAD v1.4 and the latest habitat-lab 7 .
For TidyHouse, each episode includes 20 clutter objects and 5 target objects along with their goal positions, located at 7 different receptacles (chair, 2 tables, tv stand, two kitchen counters, sofa). For PrepareGroceries, each episode includes 21 clutter objects located at 8 different receptacles (the 7 receptacles used in TidyHouse and the top shelf of the fridge) and 1 clutter object located at the middle shelf of the fridge. 2 target objects are located at the middle shelf, and each of their goal positions is located at one of two kitchen counters. The third target object is located at one of two kitchen counters, and its goal position is at the middle shelf. SetTable generates episodes similar to PrepareGroceries, except that two target objects, bowl and apple, are initialized at one of 3 drawers and at the middle fridge shelf respectively. Each of their goal positions is located at one of two tables.

C Skill Learning
Each skill is trained to accomplish a subtask and reset its end-effector at the resting position. The robot arm is first initialized with predefined resting joint positions, such that the corresponding resting position of the end-effector is (0.5, 1.0, 0.0) in the base frame 8 . The initial end-effector position is then perturbed by a Gaussian noise N (0, 0.025) clipped at 0.05m. The base position is perturbed by a Gaussian noise N (0, 0.1) truncated at 0.2m. The base orientation is perturbed by a Gaussian noise N (0, 0.25) truncated at 0.5 radian. The maximum episode length is 200 steps for all the manipulation skills, and 500 steps for the navigation skill. The episode terminates on success or failure. We use the same reward function for both stationary and mobile manipulation skills, unless specified.
For all skills, d o ee is the distance between the end-effector and the object, d r ee is the distance between the end-effector and the resting position, d h ee is the distance between the end-effector and a predefined manipulation handle (a 3D position) of the articulated object, d g a is the distance between the joint position of the articulated object and the goal joint position.
stands for the (negative) change in distance between a and b. For example, ∆ o ee is the change in distance between the end-effector and the object. I holding indicates if the robot is holding an (correct) object or handle. I succ indicates the task success. C t refers to the current collision force, and C 1:t stands for the accumulated collision force.
The 7-dim arm action stands for the delta joint positions added to the current target joint positions of the PD controller. The input arm action is assumed to be normalized to [−1, 1], and will be scaled by 0.025 (radian). The 2-dim base action stands for linear and angular velocities. The base movement in the Habitat 2.0 is implemented by kinematically setting the robot's base transformation. The collision between the robot base and navigation meshes is taken into consideration. The input base action is assumed to be normalized to [−1, 1], and will be scaled by 3 (navigation skill) or 1.5 (manipulation skills). For the navigation skill, we follow [1] to use a discrete action space and translate the discrete action into the continuous one. Concretely, the (normalized) linear velocity from -0.5 to 1 is discretized into 4 choices ({−0.5, 0, 0.5, 1}), and the (normalized) angular velocity from -1 to 1 is discretized into 5 choices (({−1, −0.5, 0, 0.5, 1}). The stop action corresponds to the discrete action representing zero velocities.

Pick(s 0 )
• Objective: pick the object initialized at s 0 • Initial base position (noise is applied in addition): -Stationary: the closest navigable position to s 0 -Mobile: a randomly selected navigable position within 2m of s 0 • Reward: I pick indicates whether the correct object is picked and I wrong indicates whether a wrong object is picked. r t = 4∆ o ee I !holding + I pick + 4∆ r ee I holding + 2. • Reward: I open = g − q a > 0.15, where q a is the joint position (radian) of the fridge. To avoid the robot from penetrating the fridge due to simulation defects, we add a collision penalty but excludes collision between the end-effector and the fridge.

C.1 PPO Hyper-parameters
Our PPO implementation is based on the habitat-lab. The visual encoder is a simple CNN 9 . The coefficients of value and entropy losses are 0.5 and 0 respectively. We use 64 parallel environments and collect 128 transitions per environment to update the networks. We use 2 mini-batches, 2 epochs per update, and a clipping parameter of 0.2 for both policy and value. The gradient norm is clipped at 0.5. We use the Adam optimizer with a learning rate of 0.0003. The linear learning rate decay is enabled. The mean of the Gaussian action predicted by the policy network is activated by tanh. The (log) standard deviation of the Gaussian action, which is an input-independent parameter, is initialized as −1.0. Fig 6 shows training curves of skills.

C.2 Other Implementation Details
The PPO algorithm implemented by the habitat-lab does not distinguish the termination of the environment (MDP) and the truncation due to time limit. We fix this issue in our implementation. Furthermore, we separately train all the skills for each HAB task to avoid potential ambiguity. For example, the starting position of an object in the drawer is computed when the drawer is closed at the beginning of an episode. However, the skill Pick needs to pick this object up when the drawer is open and the actual position of the object is different from the starting position. It is inconsistent with other cases when the object is in an open receptacle or the fridge. We observe such ambiguity can hurt performance. See Fig 6 for all task-specific variants of skills.

D Monolithic Baseline
For the monolithic baseline, a monolithic RL policy is trained for each HAB task. During training, the policy only handles one randomly selected target object, e.g., picking and placing one object in TidyHouse. During inference, the policy is applied to each target object. We use the same observation space, action space and training scheme as those for our mobile manipulation skills. The main challenge is how to formulate a reward function for those complicated long-horizon HAB tasks that usually require multiple stages. We follow [1] to composite reward functions for individual skills, given the sequence of subtasks. Concretely, at each time step during training, we infer the current subtask given perfect knowledge of the environment, and use the reward function of the corresponding skill. To ease training, we remove the collision penalty and do not terminate the episode due to collision. Besides, we use the region-goal navigation reward for the navigation subtask. Thanks to our improved reward functions and better training scheme, our monolithic RL baseline is much better than the original implementation in [1]. However, although able to move the object to its goal position, the policy never learns to release the object to complete the subtask Place during training. It might be due to exploration difficulty since Place is the last subtask in a long sequence and previous subtasks all require the robot not to release. To boost its performance, we force the gripper to release anything held at the end of execution during evaluation.

E.1 Sequential Skill Chaining
For evaluation, skills are sequentially executed in the order of their corresponding subtasks, as described in Sec 3.3. The main challenge is how to terminate a skill without privileged information. Basically, each skill will be terminated if its execution time exceeds its max episode length (200 steps for manipulation skills and 500 steps for the navigation skill). The termination condition of Pick is that an object is held and the end-effector is within 15cm of the resting position, which can be computed based on proprioception only. The gripper is disabled to release for Pick. The termination condition of Place is that the gripper holds nothing and the end-effector is within 15cm of the resting position. The gripper is disabled to grasp for Place. Besides, anything held will be released when Place terminates. For Open and Close, we use a heuristic from [1]: the skill will terminate if the end-effector is within 15cm of the resting position and it has moved at least 30cm away from the resting position during execution. Navigate terminates when it calls the stop action. Furthermore, since the manipulation skills only learn to reset its end-effector, we apply an additional operation to reset the whole arm after each skill. This reset operation is achieved by setting predefined joint positions as the target of the robot's PD controller.

E.2 Progressive Completion Rate
In this section, we describe how progressive completion rates are computed. The evaluation protocol is the same as [1] (see its Appendix F), and here we phrase it in a way more friendly to readers with little knowledge of task planning and Planning Domain Definition Language (PDDL). To partially evaluate a HAB task, we divide a full task into a sequence of stages (subgoals). For example, TidyHouse can be considered to consist of pick_0, place_0, pick_1, etc.Each stage can correspond to multiple subtasks. For example, the stage pick_i includes Navigate(s i 0 ) and Pick(s i 0 ). Thus, to be precise, the completion rate is computed based on stages instead of subtasks. We define a set of predicates to measure whether the goal of a stage is completed. A stage goal is completed if all the predicates associated with it are satisfied. The predicates are listed as follows: • holding(target_obj|i): The robot is holding the i-th object. During evaluation, we evaluate whether the current stage goal is completed at each time step. If the current stage goal is completed, we progress to the next stage. Hence, the completion rate monotonically decreases. Listings 1, 2, 3 present the stages defined for each HAB task and the predicates associated with each stage. Note that the stage goal place_i only indicates that the object has been released at its goal position, but the placement can be unstable (e.g., the object falls down the table), which can lead to the failure of the next stage. Besides, due to abstract grasp, it is difficult to place the object stably since the pose of the grasped object can not be fully controlled. Therefore, we modify the objective of SetTable to make the task achievable given abstract grasp. Concretely, instead of placing the fruit in the bowl, the robot only needs to place the fruit picked from the fridge at a goal position on the table.

F More Ablation Studies
In this section, we study the impact of different initial state distributions on mobile manipulation skills. We enlarge initial states by changing the distributions of the initial base position (the radius around the target) and orientation. For reference, the maximum radius around the target is set to 2m in the main experiments (Sec 5). Several experiments are conducted: M(S)+R, M(L1)+R, M(L2)+R, M(L3)+R. M(S)+R, M(L1)+R and M(L2)+R stand for the experiments where the maximum radii around the target are set to 1.5m, 2.5m and 4m respectively. M(L3)+R keeps the radius as 2m, but samples the initial base orientation from [−π, π], instead of using the direction facing towards the target. Fig 7  shows the quantitative results. Enlarging the initial states in general leads to performance degradation. Compared to M3 (71.2%/55.0%), M(L1)+R (67.4%/49.7%) and M(L3)+R (67.5%/46.4%) show moderate performance drop. M(L2)+R (55.2%/38.9%) shows the largest performance drop, which indicates that mobile manipulation skills are not able to handle long-range navigation yet. Moreover, M(S)+R (69.5%/52.1%) performs on par with M3. It implies that there usually exists a "sweet spot" of the initial state distribution for mobile manipulation skills as a trade-off between achievability and composability.