{'fraction_non_alphanumeric': 0.03243876646946183, 'fraction_numerical': 0.010231787133098974, 'mean_word_length': 4.580542264752791, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Spot the Difference: A Novel Task for Embodied Agents in Changing Environments

Embodied AI is a recent research area that aims at creating intelligent agents that can move and operate inside an environment. Existing approaches in this field demand the agents to act in completely new and unexplored scenes. However, this setting is far from realistic use cases that instead require executing multiple tasks in the same environment. Even if the environment changes over time, the agent could still count on its global knowledge about the scene while trying to adapt its internal representation to the current state of the environment. To make a step towards this setting, we propose Spot the Difference: a novel task for Embodied AI where the agent has access to an outdated map of the environment and needs to recover the correct layout in a fixed time budget. To this end, we collect a new dataset of occupancy maps starting from existing datasets of 3D spaces and generating a number of possible layouts for a single environment. This dataset can be employed in the popular Habitat simulator and is fully compliant with existing methods that employ reconstructed occupancy maps during navigation. Furthermore, we propose an exploration policy that can take advantage of previous knowledge of the environment and identify changes in the scene faster and more effectively than existing agents. Experimental results show that the proposed architecture outperforms existing state-of-the-art models for exploration on this new setting.


I. INTRODUCTION
Imagine you have just bought a personal robot, and you ask it to bring you a cup of tea. It will start roaming around the house while looking for the cup. It probably will not come back until some minutes, as it is new to the environment. After the robot knows your house, instead, you expect it to perform navigation tasks much faster, exploiting its previous knowledge of the environment while adapting to possible changes of objects, people, and furniture positioning. Embodied AI has recently gained a lot of attention from the research community, with amazing results in challenging tasks such as visual exploration [1], [2], [3] and navigation [4], [5], [6], [7]. However, in the current setting, the environment is completely unknown to the agent as a new episode begins. We believe that this choice is not supported by real-world experience, where information about the environment can be stored and reused for future tasks. As agents are likely to stay in the same place for long periods, such information may be outdated and inconsistent with the actual layout of the environment. Therefore, the agent also needs to discover those differences during navigation. In this paper, we introduce a new task for Embodied AI, which we name Spot the Difference. In the proposed setting, the agent must identify all the differences between an outdated map of the environment and its current state -a challenge that combines visual exploration using monocular images and embodied navigation with spatial reasoning. To succeed in this task, the agent needs to develop efficient exploration policies to focus on likely changed areas while exploiting priors about objects of the environment. We believe that this task could be useful to train agents that will need to deal with changing environments. Recent work on Embodied AI has tackled the training of embodied agents capable of navigating and locating objects [8], [5], [6], [9]. One of the key factors for success in the field consists in building map representations in which knowledge about the environment can be stored while the agent proceeds [10], [4]. However, the dominant training and evaluation protocol involves an agent initialized from scratch that sees the environment for the first time [11]. Another line of work [12], [13], [1], [14], [15], [3], instead, introduces a mapping phase of the environment to increase the performance on both exploration and down-stream tasks. Unfortunately, if the environment changes over time, the agent needs to rebuild a full representation from scratch and cannot count on an efficient policy to update its internal representation of the environment. In this work, we simulate the natural evolution of an environment and design a specific policy to navigate in changing environments seamlessly.
Due to the high cost of 3D acquisitions from the real world, the existing datasets of photorealistic 3D spaces [16], [17] do not contain different layouts for the same environment. In this paper, we create a reproducible set-up to generate alternative layouts for an environment. We semi-automatically build a dataset of 2D semantics occupancy maps in which the objects can be removed and rearranged while the area and the position of architectural elements do not change (Fig. 1). In the proposed setting, the agent is deployed in an interactive 3D environment and provided with a map from our produced dataset, which represents the information retained while performing tasks in a past state of the environment.
To train agents that can deal with changing environments efficiently, we develop a novel reward function and an approach for navigation aiming at finding relevant differences between the previous layout of the environment and the current one. Our method is based on the recent Active Neural SLAM paradigm proposed in [10] and [4]. Differently from previous proposals, though, it can read and update the given map to identify relevant differences in the environment in the form of their projections on the map. Our dataset and architecture can be employed with the Habitat simulator [18], a popular research platform for Embodied AI that renders photorealistic scenes and that enables seamless sim-to-real deployment of navigation agents [19], [20]. Experimental results show that our approach performs better than existing state-of-the-art architectures for exploration in our newly-proposed task. We also compare with different baselines and evaluate our agent in terms of percentage of area seen, percentage of discovered differences, and metric curves at varying exploration time budgets. The new dataset, together with our code and pretrained models, is available at this link.

II. RELATED WORK
Current research directions on Embodied AI for navigation agents can be categorized according to the quantity of knowledge about the environment provided to the agent prior to performing the task [11]. The first direction focuses on the scenario in which the agent is deployed in a completely new environment for which it has no prior knowledge [21], [10], [14]. Running exploration in parallel with a targetdriven navigation task resulted in an effective approach to solve the latter (e.g., object-goal navigation [5] and pointgoal navigation [4]). Other directions consider the case in which the agent can exploit pre-acquired information about the environment [22], [23] when performing a navigation task. Such pre-acquired information can be either partial [24], [25], [26] or complete [12], [8], [1]. A major limitation of such approaches is that the obtained map representation is assumed to conform perfectly with the environment where the downstream task will be performed.
In this work, we explore a fourth direction, in which the pre-acquired map provided to the agent is incomplete or incorrect due to changes occurred in the environment over time. Common strategies to deal with changing environments  Validation  42  9  160  320  MP3D Test  42  14  260  610  Gibson Validation  20  5  130  450 entail disregarding dynamic objects as landmarks when performing SLAM [27], [28] and applying local policies to avoid them when navigating [29]. An alternative strategy is learning to predict geometric changes based on experience, as done in [30], where the environment is represented as a traversability graph. The main limitation of this strategy is its computational intractability when considering dense metric maps of wide areas, as in our case.

III. PROPOSED SETTING
In the first part of this section, we introduce a new task for embodied agents, named Spot the Difference. We then describe the newly-proposed dataset that we create to enable this setting. Finally, we propose an architecture for embodied agents to tackle the defined task.

A. Spot the Difference: Task Definition
At the beginning of an episode, the agent is spawned in a 3D environment and is given a pre-built occupancy map M , representing its spatial knowledge of the environment, i.e. a previous state of the environment that is now obsolete: where m ij represents the probability of finding an obstacle at coordinates (i, j). The task entails exploring the current environment to recognize all the differences with respect to the state in which M was computed, in the form of free and occupied space. To accomplish the task, the agent should build a correct occupancy map of the current environment starting from M , recognizing and focusing on parts that are likely to change (e.g., the middle of wide rooms rather than tight corridors). For every episode of Spot the Difference, the agent is given a time budget of T time-steps. At time t = 0, the agent holds the starting map representation M . At each time-step t, the map is updated depending on the current observation to obtain M t . Whenever the agent discovers a new object or a new portion of free space, the internal representation of the map changes accordingly. The goal is to gather as much information as possible about changes in the environment by the end of the episode. To measure the agent performance, we compare the final map M T produced by the agent with the ground-truth occupancy map M * . In this sense, the paradigm we adopt is the one of knowledge reuse starting from partial knowledge.

B. Dataset Creation
Semantic Occupancy Map. Given a 3D environment, we place the agent in a free navigable location with heading θ = 0°(facing eastward). We assume that the input consists of a depth image and a semantic image and that the camera intrinsics K are known. To build the Semantic Occupancy Map (SOM) of an environment, we project each semantic pixel of the acquired scene into a 2-dimensional top-down map: given a pixel with image coordinates (i, j) and depth value d i,j , we first recover its coordinates (x, y, z) with respect to the agent position. Then, we compute the corresponding (u, v) pixel in map through an orthographic projection, using the information about the agent position and heading: We perform the same operation after rotating the agent by ∆ θ = 30°until we perform a span from 0°to 180°. To cover the whole scene, we repeat this procedure placing the agent at a distance of 0.5m from the previous capture point, following the axis directions. The agent elevation is instead kept fixed. During this step, we average the results of subsequent observations of overlapping portions of space.
After the acquisition, we obtain a SOM with C channels, where each pixel corresponds to a 5cm×5cm portion of space in the 3D environment. For each channel c ∈ {0, ..., C}, the map values represent the probability that the corresponding portion of space is occupied by an object of semantic class c. Multiple Semantic Occupancy Maps for the Same Environment. The SOMs obtained in the previous step can be seen as one possible layout for the corresponding 3D environments. In order to create a dataset with different states (i.e. different layouts) of the same environment, instead of manipulating the real-world 3D scenes (changing the furniture position, removing chairs, etc.), we propose to modify the SOM to create a set of plausible and different layouts for the environment.
First, we isolate the objects belonging to each semantic category by using an algorithm for connected component labeling [31], [32], [33]. Then, we sample a subset of objects to be deleted from the map and a subset of objects to be re-positioned in a different free location of the map. During sampling, we consider categories that have a high probability of being displaced or removed in the real world and ignore non-movable semantic categories such as fireplaces, columns, and stairs. After this step, we obtain a new SOM representing a possible alternative state for the environment, which could be very different from the one in which the 3D acquisition was taken. Sample manipulated maps can be found in Fig. 1. Dataset Details. To generate alternative SOMs, we start from the Matterport 3D (MP3D) dataset of spaces [16], which comprises 90 different building scans, and is enriched with dense semantic annotations. We consider each floor in the building and compute the SOM for that floor. For each map, we create 10 alternative versions of that same environment. In this step, we discard the floors that have few semantic objects (e.g., empty rooftops) or that are not fully navigable by the agent. As a result, we retain 249 floors belonging to 81 different buildings, thus generating a total of 2490 different semantic occupancy maps for these floors. Finally, we split the dataset into train, validation, and test subsets.
As an additional test bed, we also build a set of out-ofdomain maps (13 floors from 5 spaces) taken from the Gibson dataset [17], enriched with semantic annotations from [34], and manipulated as done for the MP3D dataset. For each SOM, multiple episodes are generated by selecting different starting points. More information about our dataset can be found in Table I and in the supplementary material.

C. Agent Architecture
Our model for embodied navigation in changing environments comprises three major components: a mapper module, a pose estimator, and a navigation policy (which, in turn, consists of a global policy, a planner, and a local policy). An overview of the proposed architecture is shown in Fig. 2 and described below, while additional details can be found in the supplementary material. Although the data we provide is enriched with semantic labels, our agent does not make use of such information directly. This is in line with current state-ofthe-art architectures for embodied exploration that we choose as competitors.
Mapper. The mapper module takes as inputs an RGB observation o r t and the corresponding depth image o d t , representing the first-person view of the agent at time-step t, and outputs the agent-centric occupancy map v t of a V × V region in front of the camera. Each pixel in v t corresponds to a 25mm × 25mm portion of space and consists of two channels containing the probability of that cell being occupied and explored, respectively. As a first step, we encode o r t using the first two blocks of ResNet-18 pre-trained on ImageNet, followed by a three-layer CNN. We project the depth image o d t using the camera intrinsics [12] and obtain a preliminary map for the visible occupancy. We name the obtained feature representationsô r t andô d t , respectively. We then encode the two feature maps using a U-Net [35]: and decode the 2 × V × V matrix of probabilities as: where µ and φ represent the learnable parameters in the U-Net encoder and decoder, respectively, and σ is the sigmoid activation function. The computed agent-centric occupancy map v t is then registered in the global occupancy map M t−1 coming from the previous time-step to obtain M t . To that end, we use a geometric transformation to project v t in the global coordinate system, for which we need a triple (x, y, θ) corresponding to the agent position and heading in the environment. This triple is estimated by a specific component that tracks the agent displacements across the environment, as discussed in the following paragraph. Pose Estimator. The agent can move across the environment using three actions: go forward 0.25m, turn left 10°, turn right 10°. Since each action may produce a different outcome because of physical interactions with the environment (e.g., bumping into a wall) or noise in the actuation system, the pose estimator is used to estimate the real displacement made at every time-step. We estimate the agent displacement (∆x t , ∆y t , ∆θ t ) at time-step t by using two consecutive RGB and depth observations, as well as the agent-centric occupancy maps (v t−1 , v t ) computed by the mapper at t − 1 and t. The actual agent position (x t , y t , θ t ) is computed iteratively as: We assume that the agent starting position is the triple (x 0 , y 0 , θ 0 ) = (0, 0, 0). Global Policy, Planner, and Local Policy. The sampling of atomic actions for the exploration relies on a three-component hierarchical policy. The first component is the global policy, which samples a long-term global goal on the map. The global policy outputs a probability distribution over discretized locations of the global map. We sample the global goal from this distribution and then transform it in (x, y) global coordinates. The second component is a planner module, which employs the A* algorithm to decode a local goal on the map. The local goal is an intermediate point, within 0.25m from the agent, along the trajectory towards the global goal. The last element of our navigation module is the local policy, which decodes the series of atomic actions taking the agent towards the local goal. In particular, the local policy is an RNN decoding the atomic action a t to execute at every time-step. The reward r local t given to the local policy is proportional to the reduction in the Euclidean distance d between the agent position and the current local goal: Following the hierarchical structure, a global goal is sampled every N time-steps. A new local goal is computed if a new global goal is sampled, if the previous local goal is reached, or if the local goal location is known to be not traversable. Exploiting Past Knowledge for Efficient Navigation. The global policy is trained using a two-term reward. The first term encourages exhaustive exploration and is proportional either to the increase of area-coverage [12] or to the increase of anticipated map accuracy as in [4]. Intuitively, the agent strives to maximize the portion of the seen area and thus maximizes the knowledge gathered during exploration. Moreover, since we consider a setting where a significant amount of knowledge is already available to the agent, we add a reward term to guide the agent towards meaningful points of the map. These correspond to the coordinates where major changes are likely to happen.
Given the occupancy map of the agent at time t, M t , the true occupancy map for the same environment M * , and a time budget of T time-steps for exploration, we aim to minimize the following, for 0 < t ≤ T : In other words, we want to maximize the number of pixels in the online reconstructed map M t that the agent correctly shifts from free to occupied (and vice-versa) during exploration. This leads to the reward term for difference discovery: The proposed reward term is designed to encourage navigation towards areas in the map that are more likely to contain meaningful differences (e.g., rooms containing more objects that can be displaced or removed from the scene). Additionally, an agent trained with this reward will tend to avoid difficult spots that are likely to produce a mismatch in terms of the predicted occupancy maps. This is because errors in the mapping phase would result in a negative reward.
To train our model, we combine a reward promoting exploration and the more specific reward on found differences to exploit semantic clues in the environment: where r exp is the reward term encouraging task-agnostic exploration (such as coverage-based or anticipation-based rewards, as described in the next section), and β 1 and β 2 are two coefficients weighing the importance of the two elements.

IV. EXPERIMENTS AND RESULTS
In this section, we detail our experimental setting and show experimental results for our new proposed task. Further analysis can be found in the supplementary material. Evaluation Metrics. To evaluate the performance in Spot the Difference, we consider three main classes of metrics. First, we consider the percentage of navigable area in the environment seen by the agent during the episode (Seen[%]). Then, we evaluate the percentage of elements that have been correctly detected as changed in the occupancy map (Acc.) and the pixel-wise Intersection over Union for the changed occupancy map elements (IoU). Besides, we evaluate the task as a twoclass problem and compute the IoU score for objects that were added in place of free space (IoU + ) and for objects that were deleted during the map creation (IoU − ). In addition, to evaluate the performance independently from the exploration capability, we propose to compute the metrics only on the portion of space that the agent actually visited (mAcc., mIoU, mIoU + , and mIoU − ) Implementation Details. We conduct our experiment using Habitat [18], a popular platform for Embodied AI in photorealistic indoor environments [17], [16]. The agent observations are 128 × 128 RGB-D images from the environment. The learning algorithm adopted for training is PPO [36]. The learning rate is 10 −3 for the mapper and 2.5 × 10 −4 for the other modules. Every model is trained for ≈ 6.5M frames using Adam optimizer [37]. A global goal is sampled every N = 25 time-steps. The local and global policies are updated, respectively, every N and 20 × N time-steps, and the mapper is updated every 4 × N time-steps. The size of the local map is V = 101, while the global map size is set to W = 2001 for the MP3D dataset and to W = 961 for the Gibson dataset. The global policy action space size G is 240. The reward coefficients {β 1 , β 2 } are set to {1, 10 −2 } and {1, 10 −1 } when the exploration reward is based on coverage and anticipation reward, respectively. The length of each episode is fixed to T = 1000 time-steps. Competitors and Baselines. We consider the following competitors and variants of the proposed method on two different setups: one where the agent position is predicted by the agent (as in Eq. 5), and one where it has access to oracle coordinates: Difference Reward (DR): an exploration policy that maximizes the correctly predicted changes between M and M * . This corresponds to setting β 1 = 0 and β 2 = 1 in Eq. 9.
Coverage Reward (CR): an agent that explores the environment with an exploration policy that maximizes the covered area and builds the occupancy map as it goes, as in [4]. Anticipation Reward (AR): an agent that explores the environment with an exploration policy that maximizes the covered area and the correctly anticipated values in the occupancy map built as it goes, from [4]. Our proposed approach consists of an agent trained with the combination of the difference reward with the coverage reward (CR+DR) or with the anticipation reward (AR+DR). Occupancy Anticipation (OccAnt): we also compare with the agent presented by Ramakrishnan et al. [4] using the available pre-trained models, referenced to as OccAnt. Note that OccAnt was trained on the Gibson dataset for the standard exploration task and without any prior map. Thus, it is not directly comparable with the other methods considered. We include it to gain insights into the performance of an off-the-shelf state-of-the-art agent on our task. Results on MP3D dataset. As a first testbed, we evaluate the different agents on the MP3D Spot the Difference test set. We report the results for this experiment in Table II.
We observe that the agent combining a reward based on coverage and our reward based on the differences in the environment (CR+DR) performs best on all the pixel-based metrics and places second in terms of percentage of seen area. It is worth noting that, even if the results in terms of the area seen are not as high as the ones obtained by the CR agent, the addition of our Difference Reward helps the agent to focus on more relevant parts, and thus, it can discover more substantial differences. Additionally, predictions are more accurate and more precise, as indicated by the 4.7% and 6.6% improvements in terms of Acc. and IoU with respect to the CR competitor. Instead, a reward based on differences alone is not sufficient to promote good exploration. In fact, although the DR agent outperforms the CR and AR agents on some metrics, our reward alone does not provide as much improvement as when combined with rewards encouraging exploration (as for CR+DR and AR+DR). Even in the oracle localization setup, the CR+DR agent achieves the best results. Interestingly, the gap with the CR agent decreases to 1.1% and 0.1% in terms of Acc. and IoU, respectively. This is because our CR+DR agent learns to sample trajectories that can be performed more efficiently and without accumulating a high positioning error. For this reason, the performance boost given by the oracle localization is lower. For both setups, our CR+DR agent outperforms the state-of-the-art OccAnt agent for exploration on all the metrics.
Finally, in Fig. 3, we plot different values of Acc. and IoU over different time-steps during the episodes. This way, we can evaluate the whole exploration trend, and not only its final point. We can observe that the proposed models incorporating the difference reward outperform the competitors. In particular, the CR+DR agent scores first by a significant margin. The performance gap can be noticed even in the first half of the episode and tends to grow with the number of steps. Results on Gibson dataset. The environments from the Gibson dataset [17] are generally smaller than those in MP3D, and thus, they can be explored more easily and exhaustively. We report the results for this experiment in Table III. Also in this experiment, the CR+DR agent performs best on all the metrics but the percentage of the area seen. Although CR+DR explores 3.8% of the environment less than the CR agent, it still overcomes the competitor by 2.5% and 7.7% in terms of Acc. and IoU. The AR+DR agent is the second-best in terms of Acc.. The OccAnt agent, instead, is competitive in terms of area seen but achieves low Acc. and IoU metrics. Qualitative Results. In Fig. 4, we report some qualitative results. Starting from the left-most column, we present the starting map given to the agent as the episode begins, the results achieved by the CR agent, those of the proposed CR+DR agent, and the ground-truth map. The differences that the agents have correctly identified during the episode are highlighted in red. As it can be seen, the CR+DR agent can identify more differences than the CR counterpart, even in small environments (top row). As the size of the environments grows (bottom row), the performance gap increases and the CR+DR agent outperforms its competitor.

V. CONCLUSION
In this work, we proposed Spot the Difference: a new task for navigation agents in changing environments. In this novel setting, the agent has to find all variations that occurred in the environment with respect to an outdated occupancy map. Since current datasets of 3D spaces do not account for such variety, we collected a new dataset containing different layouts for the same environment. We tested two state-of-the-art exploration agents on this task and proposed a novel reward term to encourage the discovery of meaningful information during exploration. The proposed agent outperforms the competitors and can identify changes in the environment more efficiently. We believe that the results presented in this paper motivate further research on this new proposed setting for Embodied AI.
ACKNOWLEDGMENT This work has been supported by the "European Training Network on PErsonalized Robotics as SErvice Oriented applications" (PERSEO) MSCA-ITN-2020 project (G.A. 955778).

A. Additional Implementation Details
Semantic Classes Division. The generation of semantic maps for each floor of each scene produces 2001 × 2001 × 43 maps for the MP3D dataset and 961 × 961 × 21 maps for the Gibson dataset. The last channel of every map registers the explorable space, so it is ignored for the creation of the dataset and is concatenated, as it is, to the manipulated map obtained at the end of the semi-automatic dataset creation process.
We divide the semantic channels of the maps depending on the possible actions performable on the connected components in that channel. We identify four types of classes: No Operation, Removal, Displacement, and Overlap Removal. A list of semantic categories with their classification is reported in Table VI for the MP3D dataset and in Table VII for the Gibson dataset. No Operation classes are left untouched, and correspond to non movable objects, such as wall, stairs and columns; the connected component of the Removal classes can be removed; those in the Displacement classes can be either removed or relocated in other free spaces in the environment; and Overlap Removal components are removed if connected components removed or displaced in other channels overlap with them, e.g., if a sofa is removed, every instance of cushion overlapping with that sofa will be removed as well because it is supposed to be on top of it.
In Fig. 9, we report some additional examples of manipulated semantic maps with relative difference maps obtained by applying our semi-automatic procedure.
Train/Val/Test Splits Division. We use the same scene partitioning adopted by the existing datasets for embodied exploration and PointGoal navigation on Matterport3D and Gibson Tiny [16], [17].
Episode Creation. For the creation of the episodes of our dataset we use the starting positions of the exploration dataset for MP3D, and of the PointGoal navigation dataset for Gibson Tiny. After the episodes located in floors with few semantic objects or that are not fully navigable by the agent are discarded, we associate one of the alternative versions of the ground-truth semantic map to each episode. For the validation and test splits of the MP3D dataset and the validation split of the Gibson dataset we create new episodes with random sampled starting positions so that the number of episodes on every floor is at least 10 and fix the number of episodes per floor to a multiple of 10. We report a detailed list of scans, selected floors and number of episodes per scan in Tables IX, X, XI, and XII. Pose Estimator. The pose estimator takes as input two consecutive RGB and depth observations, consisting in two pairs . Additionally, it accepts as input the agent-centric occupancy maps (v t−1 , v t ) computed by the mapper at t − 1 and t. For each modality, we encode information using a CNN followed by a fully-connected layer. We call these intermediate representationsō r t ,ō d t , andv t . Then, we compute a first estimate of the relative displacement in terms of (x, y, θ) coordinates and heading for each modality: with ∈ {ō r t ,ō d t ,v t }. We stack the vectors computed in Eq. 10 to obtain a 3 × 3 matrix G. Finally, we compute the agent displacement at time-step t (∆x t , ∆y t , ∆θ t ) as: where G i indicates the i-th row of the G matrix, MLP is a three-layer fully-connected network, and [·, ·, ·] denotes tensor concatenation.
Global Policy. An enriched occupancy map M + t ∈ [0, 1] 4×W ×W is obtained by stacking the occupancy map, the map of visited states, and the one-hot representation of the agent location (x t , y t ). Then, we compute two versions of M + t : one by cropping the map to an agent-centered G × G area, and the other by max-pooling the map to the same spatial resolution. The 8-channel tensor obtained by concatenating these two versions of M + t is fed to the global policy. The global policy consists of a CNN that outputs a probability distribution over the G × G global action space. We sample the global goal from this distribution, and then transform it in (x, y) global coordinates. List of Hyperparameters. In Table VIII, we list the hyperparameters used to train our agents. These hyperparameters are shared among all the agents. Agent-depending values are specified in the main paper. Training setup. We train every agent using 16 NVIDIA V100 GPUs in parallel, distributed on 4 different nodes with 4 GPUs each. On every node, we run 8 Habitat environments, for a total of 32 environments in parallel. To coordinate the interaction among nodes and train our models, we use PyTorch. Each train took about 48 hours using this setup.

B. Additional Experimental Results
Results on MP3D Validation Set. We report the quantitative results on the MP3D validation set in Table IV. Discussion for these experiments is analogous to the one presented for the experiments on the MP3D test set in the main paper.

Results on Gibson Validation Set with Oracle Localization.
We report the quantitative results on the Gibson validation set using the oracle localization setup in Table V. In this setting, the agent using only the proposed difference reward (DR) performs the best on almost all the metrics. We can conclude that, for small environments, and given an optimal localization system, our reward alone is sufficient to surpass the competitors on Spot the Difference. Analysis at Different Time-Steps. In Fig. 5 and Fig. 6, we report the plots of different values of Acc. and IoU for the MP3D and Gibson validation sets, respectively. In these plots, we show how Acc. and IoU vary at different time-steps during the episodes for the various methods.  Qualitative Results. We include some additional qualitative results for the proposed Spot the Difference task in Fig. 8.
In particular, we compare the CR agent with the CR+DR counterpart on different episodes with different map size and complexity. For each episode, we report the starting map given to the agent, the reconstructed map collected after exploration, and the ground-truth map of the actual state of the environment. We display the spotted differences in red while keeping the undiscovered elements in light gray. Dark gray denotes unchanged elements of the environment. The proposed CR+DR agent can discover a larger set of differences, thus achieving better results. Moreover, we report sample exploration trajectories for the CR and the CR+DR agents with estimated localization in Fig. 7. These confirm the competitive exploration capabilities of our proposed agent.

C. Discussion and Future Directions
We present a method that exploits outdated information about the current environment to improve the exploration capabilities of the agent. However, the focus of this work is on pure occupation, ignoring semantic information. For future work, we expect to include semantic reasoning into the agent's pipeline. We assume that additional information could boost the performance. With the proposed dataset, we enable a series of possible embodied tasks that imply dynamic environments and incorporate available past knowledge.    Fig. 9. Generation of alternative states of an environment: original and sample manipulated semantic maps with relative difference maps.