{'fraction_non_alphanumeric': 0.027405541561712845, 'fraction_numerical': 0.012795969773299748, 'mean_word_length': 5.064925145126795, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 1, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
SPEED+: Next-Generation Dataset for Spacecraft Pose Estimation across Domain Gap

Autonomous vision-based spaceborne navigation is an enabling technology for future on-orbit servicing and space logistics missions. While computer vision in general has benefited from Machine Learning (ML), training and validating spaceborne ML models are extremely challenging due to the impracticality of acquiring a large-scale labeled dataset of images of the intended target in the space environment. Existing datasets, such as Spacecraft PosE Estimation Dataset (SPEED), have so far mostly relied on synthetic images for both training and validation, which are easy to mass-produce but fail to resemble the visual features and illumination variability inherent to the target spaceborne images. In order to bridge the gap between the current practices and the intended applications in future space missions, this paper introduces SPEED+: the next generation spacecraft pose estimation dataset with specific emphasis on domain gap. In addition to 60,000 synthetic images for training, SPEED+ includes 9,531 hardware-in-the-loop images of a spacecraft mockup model captured from the Testbed for Rendezvous and Optical Navigation (TRON) facility. TRON is a first-of-a-kind robotic testbed capable of capturing an arbitrary number of target images with accurate and maximally diverse pose labels and high-fidelity spaceborne illumination conditions. SPEED+ is used in the second international Satellite Pose Estimation Challenge co-hosted by SLAB and the Advanced Concepts Team of the European Space Agency to evaluate and compare the robustness of spaceborne ML models trained on synthetic images.


INTRODUCTION
Autonomous navigation about a noncooperative Resident Space Object (RSO) is an enabling technology for future inorbit servicing and space logistics missions, such as refueling space assets [1] and active debris removal [2]. As these missions involve interaction with a noncooperative target at close proximity, accurate determination and tracking of the target's pose (i.e., position and orientation) with respect to the servicer spacecraft is vital for safe docking and capture. Moreover, driven by the limited on-board power and computational capabilities, a monocular camera is a favored choice of sensor as opposed to more complex systems such as stereovision, Range Detection and Ranging (RADAR) or Light Detection and Ranging (LIDAR). Recently, Machine Learning (ML) techniques based on Convolutional Neural Networks (CNN) have been explored in applications of vision-based spacecraft pose estimation and navigation [3][4][5][6][7][8][9][10]; however, it is impractical to acquire large-scale imagery of the interested target spacecraft with accurate pose labels in space to train CNN models. This is the main reason that CNN models have not been deployed in space so far. In fact, spaceborne ML models up to date have been trained and, most importantly, validated virtually exclusively on synthetic imagery. While synthetic imagery is easy to mass-produce and annotate for training, it is prone to performance degradation when tested on the target spaeborne imagery as CNNs overfit to the features specific to the synthetic imagery. The phenomenon is known as domain gap in literature and is an active field of research in machine learning [11,12].
While it is possible to train a more domain-invariant CNN model using methods such as domain randomization [13,14] and texture randomization [4,15], its use in space missions requires a validation of performance in a highly representative space environment prior to mission deployment. Some authors resort to existing spaceborne images captured from previous missions to evaluate the robustness of CNNs [3,4,8,10]. However, for a single target, these images lack diversity in terms of quantity, pose distribution and variability of environmental factors, which severely restrict the comprehensive evaluation of the model's robustness for future missions or for future phases of the same mission. Moreover, these images mostly lack accurate pose labels, which has forced the previous evaluations to be strictly qualitative [8] or based on hand-labeled pose annotations [10]. Therefore, an alternative approach is to instead physically re-create and simulate the space environment using an on-ground Hardware-In-the-Loop (HIL) robotic testbed which is capable of capturing an arbitrary number of images of the target mockup model and annotating accurate pose labels for each image sample. Then, these HIL images can be used as a weaker surrogate of the target spaceborne images to evaluate on-ground the robustness of a CNN model on an image domain different from the synthetic training images.
To the best of the authors' knowledge, the only publicly available benchmark dataset on spacecraft pose estimation with fully labeled images from multiple sources is the Spacecraft PosE Estimation Dataset (SPEED) [16] which consists of 15,300 images of the Tango spacecraft from the PRISMA mission [17,18]. Specifically, the dataset comprises 15,000 synthetic images rendered with OpenGLbased Optical Stimulator (OS) camera emulator software [19,20] of the Stanford's Space Rendezvous Laboratory (SLAB) multi-Satellite Software Simulator (S 3 ) [21] and 300 simulated images captured from the robotic Testbed for Rendezvous and Optical Navigation (TRON) at SLAB. The dataset was made available as part of the Satellite Pose Estimation Competition (SPEC2019) co-hosted by SLAB and the Advanced Concepts Team (ACT) of the European Space Agency (ESA). Throughout the competition, the submitted entries were evaluated and ranked based on their performances on the synthetic test set images. While the models could be evaluated on the 300 simulated images as well, their restricted pose and illumination configurations fall short to perform a comprehensive analysis of the model's robustness across domain gap.
In order to overcome the aforementioned challenges and facilitate the study of ML robustness for future space missions, this paper presents SPEED+, the next-generation spacecraft pose estimation dataset with specific emphasis on bridging the domain gap. As shown in Figure 1 In summary, the contributions of SPEED+ are as follows: • SPEED+ is a first-of-a-kind dataset for vision-only spacecraft pose estimation and relative navigation with emphasis on domain gap between synthetic training images and Hardward-In-the-Loop (HIL) test images from a robotic simulation testbed.
• The test images of SPEED+ extend the full orientation space and distance up to 10 m with realistic re-creation of Earth albedo and direct sunlight present in spaceborne imagery. SPEED+ provides unique and unprecedented quantity and quality of mockup spacecraft images, which allow comprehensive evaluation of robustness of spaceborne ML models under a wide range of high-fidelity simulation of environmental settings. • The SPEED+ dataset is used in the second international Satellite Pose Estimation Competition (SPEC2021) with emphasis on robustness of spaceborne ML models across domain gap. The baseline studies in this work exhibit and justify the challenges associated with bridging the performance gap between the synthetic training and HIL test images.

Datasets for spacecraft pose estimation
Apart from SPEED, a number of other datasets for spacecraft pose estimation and spaceborne computer vision problems have been made public as well. For example, URSO [8] consists of synthetic and spaceborne images of Soyuz and Dragon spacecraft rendered on Unreal Engine 4, but the labels for spaceborne images are missing. Dung et al. [22] introduced a dataset comprising about 3,000 synthetic and spaceborne images of random satellites with bounding box and segmentation labels. Other authors have also developed their own datasets, such as synthetic images of the Envisat spacecraft rendered with Blender [7,9] and Cygnus spacecraft rendered with Blender and its Cycles rendering engine [10].

Testbeds for spaceborne optical navigation
Testbeds to simulate vision-based closed-loop navigation and control algorithms in general employ air-bearing platforms on a flat epoxy or granite floor with thrusters and actuators, such as ASTROS at the Georgia Institue of Technology [23], POSEIDYN at the Naval Postgraduate School [24], and M-STAR at the California Institute of Technology [25]. While  Train  12000  5  47966  --Validation  --11994  --Test  2998  300  -6740  2791 these testbeds excel at simulating the spacecraft maneuver commands, their capabilities for creating a large-scale annotated dataset have not been showcased. In an effort to do away with reliance on synthetic imagery, Pasqualetto Cassinis et al. [9] have recently used the GRASL facility at ESTEC to generate 100 simulated HIL images of the 1:25 mockup model of the Envisat satellite. However, as the target mockup is mounted onto a static tripod, the configurable pose distribution is severely restricted. To the best of the authors' knowledge, there is no publicly available dataset that consists of more than just hundreds of HIL or spaceborne images of a single target with accurate pose labels. This is not sufficient for a comprehensive evaluation of the robustness of a spaceborne ML model.

Datasets for domain gap
A number of datasets with emphasis on domain gap are available for terrestrial computer vision applications, such as Office-31 [26], Syn2Real [12,27], and DomainNet [28] for various tasks such as classification, object detection and semantic segmentation. Perhaps the application most similar to the spaceborne navigation is the semantic segmentation for autonomous driving, in which the large-scale synthetic images from gaming engines such as GTA V [29] or SYNTHIA [30] are used for training, whereas real street images available from KITTI [31] or Cityscapes [32] are used for testing. Evidently, none of these images contain unique, challenging visual features and characteristics encountered in spaceborne imagery.

SPEED+ OVERVIEW
SPEED+ is the next-generation dataset for spacecraft pose estimation with specific emphasis on the model robustness across the domain gap. While the presented SPEED+ consists of images of the Tango spacecraft from the PRISMA mission [17,18], the spacecraft model can be seamlessly exchanged in the future, and the presented methodology of data collection is general. SPEED+ consists of three different domains of imageries from two distinct sources. The first source is the OpenGL-based Optical Stimulator (OS) [19,20]  In order to reproduce realistic space environment, the albedo light boxes and the sun lamp are carefully designed and calibrated to accurately capture the illumination characteristics in space. Specifically, the light boxes [34] are designed and calibrated to provide a uniform maximum radiance of 14 W/m 2 sr, which corresponds to the mean radiance from the Earth for an albedo coefficient of 0.3 [35] and a solar irradiance of 1366 W/m 2 [36]. The sun lamp, on the other hand, consists of a metal halide arc lamp and a paraboloidal mirror, which together are designed to produce a collimated beam with 1.0 solar constant in space (nominal 1357 W/m 2 ) and a spectral response close to 6000 K. The readers are referred to Beierle [20] for more details.

GENERATING SPEED+ IMAGES
This section first describes the process of collecting and postprocessing the HIL images of SPEED+ 2 . Then, it ends with the brief description of creating the synthetic imagery of SPEED+.

TRON Facility Overview
The TRON facility at SLAB is an 8 × 3 × 3 (m) simulation room which consists of two 6 Degrees-Of-Freedom (DOF)  KUKA robot arms. One robot arm is fixed to the ground and holds a lightweight, reduced-scale mockup model of the target object, whereas the other arm holds the camera and can move along the ceiling-mounted linear rail across the room. The movements of both objects are tracked using two independent measurement sources: 12 Vicon Vero cameras [37] around the facility which track the infrared (IR) markers attached to both objects, and the KUKA system's internal telemetry of the end-effectors of both robots. The measurements from both sources are used to jointly calibrate the facility, which allows the user to recover the true pose label of the target with respect to the camera from available measurements of an arbitrary sample with millimeter position and sub-degree orientation accuracies at close range. The reader is referred to Park et al. [33] for details on the facility and its calibration procedure.
In order to emulate a high-fidelity spaceborne illumination setting, the facility contains 10 Earth albedo lightboxes [34] and a metal halide arc lamp. The configuration of the lightboxes is visualized in Figure 2(a) along with the IDs associated with each lightbox (i.e., L1 ∼ L10). Figure 2(a) also shows that the sun lamp is placed at three distinct locations at varying angles with respect to the ground-fixed robot arm (S1 ∼ S3), with the lamp facing directly towards the spacecraft mockup model. Figures 2(b), (c) show both the lightboxes and the sun lamp in operation. The applicable components of the facility are painted black, and all deactivated sources of illumination are covered with light-absorbing black commando cloths to maximally suppress the reflection and ambient light.

Collecting SPEED+ HIL Images
The HIL images of SPEED+ involve a half-scale mockup model of the Tango spacecraft from the PRISMA mission [17,18]. The creation of the SPEED+ HIL imageries begins with generating a total of 10,000 pose labels, which are then translated into the commands of both robots. In theory, there is a countless number of pairs of robot commands that will achieve the same prescribed pose label. In order to simplify the process of commands generation and data collection, the target model is commanded at a constant location within the facility regardless of its orientation. Moreover, the camera and the target are commanded such that the camera's boresight is always directed towards the target's location along −xaxis as visualized in Figure 2(a). The position labels are generated in compliance with these operational constraints, while ensuring (1) the camera's pose is reconfigurable with the ceiling-mounted robot arm, and (2) the target's center is visible within the image frame of the camera. On the other hand, the orientation labels are simply sampled from the uniform distribution in the SO(3) space using the subgroup algorithm [38]. The mockup model of the Tango spacecraft is manufactured with two mounting holes at opposite sides, so that depending on the direction of the camera's viewpoint  with respect to the model, it can be mounted on either side to ensure the robot arm does not obstruct its view in any way.
With all pose labels translated into the robot arm commands, they are divided into 10 partitions of random 1,000 images based on different illumination configurations shown in Table  2. It indicates that the configurations 1 ∼ 7 employ different sets of light boxes, whereas the configurations 8 ∼ 10 employ the sun lamp at different locations along the facility as marked in Figure 2(a). Naturally, the 70% of the simulated images for configurations 1 ∼ 7 are placed under lightbox domain, and the remaining 30% under sunlamp domain. During the data collection, the intensity of each activated lightboxes is randomly selected from a distribution that is representative of various orbit altitudes for each sample in lightbox domain, whereas the sun lamp is left at constant intensity facing the same direction for sunlamp domain. Figure 3 depicts the pose distributions of the pose labels for the HIL images, which confirms that for both lightbox and sunlamp datasets, the position and orientation labels are well sampled from the uniform distributions of the 3D Cartesian and SO(3) spaces, respectively. This is a big improvement compared to the severely restricted pose distribution of SPEED real images as visualized in Figure 4. Moreover, Figure 3 shows that for sunlamp domain, the direction of incident light is well-distributed with respect to the mockup target, illuminating it from all possible angles.

Post-Processing
As shown in Figure 5, the raw images include not only the target mockup model of interest but also other components of the facility, such as the robot arm, Vicon cameras, activated lightboxes, etc. While a robust ML model should be able to perform the task of spacecraft pose estimation regardless of the presence of these components, in the setting of the competition, they may pose risks of containing information that can be extracted and exploited to aid the given task. In order to eliminate such possibility altogether, these raw images are post-processed to mask out the unrelated items in the image while simultaneously striving to emulate as closely as possible the visual properties of the spaceborne imagery.
The post-processing procedure for lightbox images loosely follows that of the synthetic imagery of SPEED detailed in [3,39]. First, the image backgrounds are simply masked out using the binary masks rendered based on the pose labels. Then, either random starfields rendered with the OS software or random patches of the Earth images captured from the Himawari-8 geostationary meteorological satellite [40] are substitued to the backgrounds, as shown in Figure 5. Specifically, an Earth image with incident sunlight direction that approximately matches the active lightbox configuration is chosen for background substitution. Finally, Gaussian blurring and zero-mean Gaussian white noise are added to emulate the depth of field and additional thermal noise, respectively. Specifically, blurring helps mitigate the sharp boundaries between the satellite foreground and the starfield/Earth background.
On the other hand, the sunlamp raw images contain interesting properties unavailable in lightbox raw images, such as patterned flare introduced by the sun lamp and intense surface glow due to high reflectivity and overexposure of the camera (see Figure 5). In many cases, the glow is too strong such that it obfuscates the shape and boundaries of the target model, effectively acting as an occlusion noise in the image. Therefore, applying the same post-processing as to lightbox images using binary masks will not only eliminate challenging visual effects created by the sun lamp, but it will also give away the exact silhouette of the target model that is otherwise unavailable in the raw images. To mitigate such issues, the post-processing of sunlamp images instead blurs out all the unnecessary components in the image except the general area surrounding the model and the sun lamp, as shown in Figure 5. The overall pipeline is shown in Figure 6, which involves the approximate mask for all the components that must be removed (m X ) and the target model and its surrounding glow effects (m T ). Then, given the original image (I), the final result is equivalent to the output of the following MATLAB command, regionfill(I, m X −m T ), which effectively blurs out the foreground of the given mask.
The final results of the HIL images inevitably include some artifacts that are unique to lightbox and sunlamp images: the infrared markers and the mounting holes, which are not present in the synthetic domain. As these components do not affect the pose labels and are indispensable for the data collection, they are retained in both HIL images. The other artifact unique to sunlamp images is the shape of the surface glow, which is a byproduct of the post-processing step that aims to distinguish the glow effects from the target's surface and the robot arm holding the target. Though somewhat unnatural, the irregular shape of the glow effect is retained as long as it does not add any information about the target's shape. The examples of these artifacts are shown in Figure 8 of Appendix D.

Manual Removal of Samples
At the end of the post-processing, some samples are manually removed from both lightbox and sunlamp images, leaving total 6,740 and 2,791 images for respective image domains. The majority of rejected lightbox images are due to non-negligible misalignment between the spacecraft and its binary mask when Earth images are inserted as the background. While the pose labels estimated by the facility have millimeterlevel and sub-degree-level position and orientation accuracies at close range, it is possible that there may be up to a centimeter-level position error beyond close range due to imperfect calibration of the facility [33]. For example, a position error induced by the misaligned camera boresight would scale linearly with the distance to the target, so 0.1 • error would cause 1.75 cm position error at 10 m distance. The misalignment of the binary mask then creates a gap between the spacecraft and the Earth background which could aid the pose estimation task. Therefore, any images with such misalignment observed at more than one edge of the spacecraft are manually discarded by the authors.
For sunlamp images, the samples are rejected based on two criteria. One is the case in which the model's surface reflection is so severe that a human cannot even approximate the pose of the target in an image. The second is when the post-processing step, in an attempt to retain the shape of the surface glow, fails to discriminate and mask the reflection caused by the robot arm holding the model. Some of the rejected samples are visualized in Figure 9 of Appendix D.

Generating SPEED+ Synthetic images
Once the outliers are rejected from lightbox and sunlamp domains, the synthetic imagery is created based on the observed pose distribution of the complete HIL imageries. Specifically, the distribution of the position labels encompasses those of the HIL imageries with the separation along z-axis sampled from U(2.25, 10.0) [m]. The images are rendered using the camera's intrinsic parameters estimated from the calibration of TRON, which are specified in Appendix B. Then, similar to the SPEED synthetic dataset, random Earth backgrounds are inserted to half of the entire synthetic images. Finally, Gaussian blur and noise are added, similar to lightbox images.

EXPERIMENTS
This section provides a few baseline performance studies, starting with the synthetic-only experiments (i.e., trained only on synthetic training set) to provide a lower-bound performance on both HIL domains. A number of CNNs with different pose estimation architectures are tested on HIL images to demonstrate the domain gap between the training and test images. Then, two different domain adaptation and randomization algorithms are applied to one of the CNNs as a benchmark. The oracle performances are also approximated by in-domain training (i.e., train and test on the same HIL domain), where the performance is approximated via 5-fold cross validation, i.e., averaged from testing on one of the 5 equal-sized randomly partitioned sets after being trained on the rest. Thus, the oracle represents the upper-bound performance achievable by the given CNN model on the HIL domains. The purpose of these studies is to characterize the domain gap between the synthetic and HIL images and to show that the HIL images and labels from the robotic testbed do retain learnable features. Finally, the same model and the domain randomization algorithm are evaluated on the prisma25 dataset, which consists of 25 flight images of the same Tango spacecraft from the PRISMA mission [18]. Despite the limited availability of prisma25, the comparative study between SPEED+ HIL and flight images gives insight to the applicability of HIL images as a surrogate for testing.

Pose estimation CNNs
First, the baseline performance study uses three different pose estimation CNNs: the Keypoint Regression Network (KRN) [4] and Spacecraft Pose Network (SPN) [3], which are the two baseline models for SPEED, and HigherHRNet [41], the state-of-the art model for the bottom-up human joint detection task. These networks employ different strategies for spacecraft pose estimation. For example, SPN simultaneously predicts N closest discrete attitude classes and the relative weights associated with each class, which are then averaged to compute the final attitude. Then, based on the target's model, predicted attitude and bounding box, SPN solves for the relative position by minimizing the residuals between the bounding box and the extremal points of the projected target model using a Gauss-Newton algorithm. On the other hand, KRN directly regress the locations of 11 pre-designated keypoints of the Tango spacecraft's corners and antennae. Then, given the known 2D-3D keypoint correspondences, a Perspective-n-Point (PnP) problem is solved to output the complete 6D pose [42]. Finally, HigherHRNet is trained to detect the same 11 keypoints as in KRN, but it returns heatmaps associated with each keypoints. Then, PnP is solved for complete 6D pose.
Note that both KRN and SPN depend on a separate object detection network to detect and crop the bounding box around the spacecraft prior to running the pose estimation networks. In this study, the first-stage object detection network is skipped to simplify the study, especially since the problem of detecting a single known object is a much easier problem than the 6D pose estimation. Instead, the ground-truth bounding boxes computed from the pose labels are used to crop the images around the target during pre-processing.

Algorithms for domain gap
This study adopts two algorithms that represent two distinct approaches to bridging the domain gap: unsupervised domain adaptation [11], which utilizes unlabeled images from the target domain to bridge the feature-level discrepancies, and domain randomization [13], which instead randomizes various aspects of the input images such that the target image would be considered as another randomized version of the source image. The domain adaptation method used in this study is the Domain-Adversarial Neural Network (DANN) [44,45], which employs an adversarial training using a domain classifier to train a domain-invariant feature extractor. For domain randomization, the style augmentation method by Jackson et al. [43] is used to randomize the spacecraft's texture using the Style Transfer network [46]. The style transfer is performed online during the data augmentation phase for half of the Table 3. synthetic-only performances of the baseline models, tested on synthetic validation, lightbox and sunlamp images.

Metrics
The pose accuracies are measured using the position error (E t ), orientation error (E q ), and pose error (E pose ) respectively defined as follows, where (t, t) respectively denote the predicted and ground-truth relative position vector, <q, q > denote the inner-product of the predicted and ground-truth quaternion vectors, and the pose error is the sum of the position error normalized by the magnitude of the ground-truth position vector and the orientation error in radians. The pose error, also known as the SPEED score, was the official competition metric used in SPEC2019, chosen as it has shown to properly balance the contribution of the position and orientation errors [5].
In SPEC2021, a modified pose error is used to account for the errors associated with the pose labels of the HIL images. Specifically, from the calibration results of the TRON facility [33], the following best-performance thresholds are determined: θ q = 0.169 • for orientation and θ t = 2.173 mm/m for normalized translation. Then, if the predicted orientation and translation are both below respective thresholds, the pose error is considered perfect, i.e., for each sample,

Results
The results of the baseline models are shown in Table 3. First, it indicates significant domain gap between the synthetic and HIL images, especially for SPN and KRN, as evidenced by the performance drop during testing. The degraded performance on lightbox images indicates that, despite milder illumination condition compared to sunlamp images, the difference in spacecraft's texture between computer graphics and a real, physical model is significant enough to prohibit successful pose prediction. As expected, the performance is much worse when tested on sunlamp images, as they involve much more challenging illumination effects characterized by high contrast and camera overexposure. Interestingly, the performance drop is much smaller for HigherHRNet, which hints that the strategy of heatmap prediction is inherently more robust to domain gap than attitude classification and direct keypoints regression. Table 4 shows the performances of KRN after domainbridging algorithms are applied. First, the oracle performance corroborates the overall accuracy of the pose labels of HIL images, and that they do retain the features learnable by the CNNs to perform the pose estimation task. This serves to justify the use of lightbox and sunlamp images as the test images in SPEC2021. The results of domain adaptation using DANN and domain randomizing using style augmentation showcase the challenges associated with bridging the domain gap between synthetic and HIL images of SPEED+, and that more sophisticated domain-bridging algorithms and hyperparameter tuning will be required to further bridge the gap than a straightforward application of an existing approach.
Finally, Figure 7 exhibits the cumulative distribution of KRN's performance on HIL and prisma25 test sets after 5 synthetic-only training sessions with different random seeds. Its visualization confirms the challenges associated with sunlamp images compared to those of lightbox for vanilla models trained exclusively on synthetic images. Moreover, despite the limited size of prisma25, Figure 7 also demonstrates that the distribution curves of SPEED scores on prisma25 more or less overlap with those of the SPEED+ HIL images. The coinciding performance trends between SPEED+ HIL and spaceborne images imply a similar level of domain gap against the synthetic domain and the viability of the HIL images as surrogates of the actual flight images in the robustness analysis of CNNs.

DISCUSSIONS
This section describes a few limitations and planned future improvements regarding the current SPEED+ and the provided baseline performance studies.

Dataset Limitations
In its currently released form, SPEED+ consists of images of a single known target. However, this paper demonstrates that, given a mockup model of a target, the TRON facility can generate a large number of target HIL images and accurate pose labels with minimal human intervention. Therefore, future sequels of SPEED+ will include more targets to facilitate the study of spaceborne ML techniques beyond the application of pose estimation and navigation about a single known target. Moreover, the KUKA robot arms will undergo a pending update to enable more accurate pose labeling with less human intervention for sample rejection.

Relation to Real Mission Constraints and Flight Images
In real missions, the servicer spacecraft typically does not have access to the target's images until rendezvous. However, in SPEC2021, the unlabeled images from lightbox and sunlamp are available to the public. Such operational constraints are not placed in SPEC2021 to further engage the community's participation: the participants are encouraged to both use and ignore the given unlabeled target images in training robust ML models.

Baseline Studies
To emphasize, the goal of the provided baseline performance studies is not to showcase the best CNN model architecture and training algorithm for the task at hand, but to characterize and justify the SPEED+ dataset and its suitability for SPEC2021 and robustness studies in spaceborne ML and vision applications. A more comprehensive analysis on the factors contributing to robust ML models will be conducted once the competition concludes.

CONCLUSIONS
This paper presents SPEED+, the next-generation dataset for spacecraft pose estimation and navigation. Compared to its predecessor, SPEED+ focuses on domain gap between training synthetic images and test images captured from a robotic testbed with high-fidelity spaceborne-like illumination conditions. SPEED+ serves to test and compare the robustness of different ML models as part of the international Satellite Pose Estimation Competition that begins in October 2021. This paper introduces and characterizes the SPEED+ dataset using existing pose estimation CNNs and well-established domain adaptation and randomization algorithms.
The presented SPEED+ is an important step toward enabling autonomous proximity operations in space using machine learning techniques to support various future mission concepts, such as refueling defunct space assets and active debris removal. This will help ensure sustained access to near-Earth space, which is a finite resource that needs to be properly understood and managed for the future of mankind.

Kelvins Platform
SPEC2021 is hosted on the Kelvins platform of ESA, which has hosted the first SPEC (SPEC2019) and many other spacerelated challenges. The platform includes a link to the dataset, the description of its structure, a leaderboard, and a page for public discussions about the dataset and the competition. More information can be found at kelvins.esa.int/ pose-estimation-2021/.
The SPEED+ dataset is publicly released through the Stanford Digital Repository with a unique DOI [47], and the link to the dataset is available on Zenodo as well through the SPEC2021 platform. The dataset is released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 (CC BY-NC-SA 4.0) license.

Baseline Models
The SPN [3] and KRN [4] baseline models, along with DANN [44,45] and style augmentation [43], are available in the following GitHub repository: github.com/tpark94/ The camera is calibrated prior to its use, and its calibrated parameters are shown in Table 5.
The radial and tangential distortion parameters follow the conventions used in OpenCV. The specified parameters are provided in the camera.json file.

C. TRAINING DETAILS
This section describes the training details for the baseline studies. All methods are implemented with PyTorch v1.8.0 and trained on an NVIDIA GeForce RTX 2080 Ti 12GB GPU.

KRN [4]
First, the training of KRN [4] largely follows the original work, except it instead uses the AdamW [48] optimizer with β 1 = 0.9, β 2 = 0.999 for backpropagation. When training on synthetic training images, KRN is trained with batch size of 48 for 50 epochs and initial learning rate of 0.001 which decays by the factor of 0.95. When training on the HIL images, the number of training epochs and the decay factor are adjusted according to the number of the available training images.

SPN [3]
The implementation of SPN follows the original tensorflow version of the author, which differs from the descriptions in Sharma & D'Amico [3]. In this work, the backbone is the first 5 layers of ImageNet-pretrained AlexNet [49], which takes 227 × 227 images as inputs. The Branches 2 and 3 of SPN, along with the backbone, are trained together simultaneously. The labels are generated from 5,000 attitude classes uniformly sampled from the SO(3) space, and at training and inference, 5 neighboring attitude classes are predicted.

HigherHRNet [41]
The network is directly taken from the official repository 3 and simplified for the application of single-object pose estimation. The exact same data augmentation pipeline for KRN is used here. The input image is resized to 480 × 320. The keypoint location is taken to be the pixel with the highest intensity in the predicted heatmaps. Then, EPnP [42] is used to solve for 3 https://github.com/HRNet/HigherHRNet-Human-Pose-Estimation DANN [44,45] The domain classifier of DANN are attached to the Mo-bileNetv2 [50] feature extractor of KRN, which ends with a feature map of 320 × 7 × 7. Here, after the gradient reversal layer, the domain classifier with two convolution layers is added. Specifically, the first layer is a 1 × 1 convolution operation to 1280 channels followed by a ReLU activation and an average pooling layer. Then, the second layer uses another 1 × 1 convolution to end with just 1 channel which is then classified to either source or target domain classes. The training of KRN with DANN follows the training routine of KRN above. Style augmentation [43] The pre-trained style transfer network is directly taken from the official repository 4 . Then, in order to facilitate online stylization at the data augmentation stage, the mean embedding of the SPEED+ synthetic training set is interpolated with a random style embedding from R 100 with α = 0.5. The training with style transfer network follows the training routine of KRN above.

D. VISUALIZATION
Retained Artifacts in HIL Images Figure 8 visualizes the retained artifacts unique to HIL images.

BIOGRAPHY[
Tae Ha Park is a Ph.D. candidate in the Space Rendezvous Laboratory, Stanford University. He graduated from Harvey Mudd College with a Bachelor of Science degree (2017) in engineering. His research interest is in the development of machine learning techniques and GN&C algorithms for spaceborne computer vision tasks, specifically on robust and accurate determination of the relative position and attitude of arbitrary resident space objects using monocular vision. Potential applications include space debris removal and refueling of defunct geostationary satellites with unprecedented autonomy and safety measures.
Marcus Märtens graduated from the University of Paderborn (Germany) with a Masters degree in computer science. He joined the European Space Agency as a Young Graduate Trainee in artificial intelligence where he worked on multi-objective optimization of spacecraft trajectories. He was part of the winning team of the 8th edition of the Global Trajectory Optimization Competition (GTOC) and received a HUMIES gold medal for developing algorithms achieving human competitive results in trajectory design. The Delft University of Technology awarded him a Ph.D. for his thesis on information propagation in complex networks. After his time at the network architectures and services group in Delft (Netherlands), Marcus rejoined the European Space Agency, where he works as a research follow in the Advanced Concepts Team. While his main focus is on applied artificial intelligence and evolutionary optimization, Marcus has worked together with experts from different fields and authored works related to neuroscience, cyber-security and gaming.
Gurvan Lecuyer graduated from the National Institute of Applied Science of Rennes (France) with a Master's degree in Computer Engineering. He then completed a Ph.D. in signal and image processing at the University of Rennes (France), working on an artificial intelligence algorithm to accelerate the annotation of surgical videos. After that, Gurvan Lecuyer joined the European Space Agency as a Research Fellow in the Advanced Concept Team and Φ-lab to work on artificial intelligence, focusing on the application for Earth Observation.
Dario Izzo graduated as a Doctor of Aeronautical Engineering from the University Sapienza of Rome (Italy). He then took a second master in Satellite Platforms at the University of Cranfield in the United Kingdom and completed his Ph.D. in Mathematical Modelling at the University Sapienza of Rome where he lectured classical mechanics and space flight mechanics. Dario Izzo later joined the European Space Agency and became the scientific coordinator of its Advanced Concepts Team.