{'fraction_non_alphanumeric': 0.0234090514999133, 'fraction_numerical': 0.011184324605514133, 'mean_word_length': 5.036106750392465, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Unsupervised Distribution Learning for Lunar Surface Anomaly Detection

In this work we show that modern data-driven machine learning techniques can be successfully applied on lunar surface remote sensing data to learn, in an unsupervised way, sufficiently good representations of the data distribution to enable lunar technosignature and anomaly detection. In particular we train an unsupervised distribution learning neural network model to find the Apollo 15 landing module in a testing dataset, with no dataset specific model or hyperparameter tuning. Sufficiently good unsupervised data density estimation has the promise of enabling myriad useful downstream tasks, including locating lunar resources for future space flight and colonization, finding new impact craters or lunar surface reshaping, and algorithmically deciding the importance of unlabeled samples to send back from power- and bandwidth-constrained missions. We show in this work that such unsupervised learning can be successfully done in the lunar remote sensing and space science contexts.


Introduction, Motivations
The search for so called technosignatures in our Solar System and beyond has gained new attention recently [1]. Here, technosignatures are physical properties or effects that provide scientific evidence of past or present extraterrestrial technology [2]. According to NASA the search for technosignatures should be performed in parallel to the search for biosignatures in the field of astrobiology, as technosignatures could reveal the existence of intelligent life elsewhere in the universe [1]. One area in the field of technosignature research is the search for non-terrestrial artifacts in our Solar System, particularly on the surfaces of planets and moons [e.g., 1,3,4]. [5] suggested to start a systematic search for ground-based technosignatures on the lunar surface. Besides technosignatures, there are other objects of scientific relevance on the lunar surface, such as fresh impact craters, etc. However, traditional search methods involve inspection of remote sensing data by human operators, which is a slow, biased, and inefficient process. In addition, the physical shape, form, and material of technosignatures is completely unknown and signatures could easily be missed by operators. However, there is a wealth of remote sensing data available, meaning that the bottleneck of technosignature search is the human component.
In recent years, numerous powerful machine learning methods have been developed, such as vision methods [7] and explicit data distribution learners, including variational autoencoders (VAEs) [8,9], flow based models [10], and auto-regressive models [11]. Our motivation is to utilize such machine learning-driven approaches to automate technosignature search and to prove the potential of explicit data distribution learners to extract scientifically relevant results from space exploration data. For technosignature search, a data and ML-driven approach would provide unique advantages: 1) large data sources enable us to scan the surfaces of entire planets and moons, 2) machine learning allows us to remove the human in the loop and to automate the scanning process, and 3) the utilization of unsupervised methods allows us to scan for anomalies without a priori knowledge about the physical state and form of the to-be-detected targets. For this proof of concept, the Moon is an ideal test bed, as there is a wealth of satellite data available and as numerous exploration missions have left non-natural relics on the surface, such as the Apollo and Luna missions, that can serve as ground truth for model validation and testing.

Data
The detection of objects with unknown shapes and sizes on the surface of a planet or moon requires imagery with two main characteristics, 1) sufficient spatial resolution and 2) global coverage. On the Moon, only one sensor has both characteristics, the Narrow Angle Camera onboard NASA's Lunar Reconnaissance Orbiter (LRO). Since LRO's launch in 2009, NAC covered the entire surface of the Moon multiple times, returning more than 1.6 million optical images with a spatial resolution ranging from 0.5 to 1.5 m/pixel. These NAC images can be retrieved from the Planetary Data System (PDS) in a large variety of formats and processing levels. While there are many lunar global datasets available, the NAC image stack is the only data with spatial resolutions sufficient for the detection of small geomorphological and human-made objects. For this study, we used the uppermost layer of the pre-calibrated pyramid tif files (ptifs) that can be retrieved from the LROC image archive. These images provide the full spatial resolution (in the top layer), but are reduced in size, as they have been reduced to 8bit. The reduced size allows for optimized file download, handling, and processing times.
In a first step we collected all available NAC imagery over the Apollo 15 and 17 landing sites with spatial resolutions higher than 0.8 m/pixel. We then tiled these images in patches of 64x64 pixels, resulting in 255,000 NAC tiles. This dataset had a stride of thirty-two pixels, so that each pixel was included, on average, in four images. All used patches cover an area of approximately sixty-four million square meters and include the Apollo 15 and 17 landing sites.

Model
For our unsupervised data distribution learner, we used a variational autoencoder (VAE). [13,14] This autoencoder has an encode (decode) module layers eight layers deep that alternated between convolution (transposed convolution) and batch norm layers. By the end of the encoding layer, the data undergoes a learned transformation from its native 3 channels x 64 pixels x 64 pixels image space size to 128 filters each of size 16 pixels x 16 pixels. We used a default informational bottleneck latent code dimension size of 2 6 , but varied this down to 2 3 and up to 2 11 to control model capacity, as well as training time, inference time and memory size, between experiment runs. We used the reparametrization trick to allow differentiability through the stochastic sampling process of the VAE. The loss bound that we sought to minimize included the calculation of the KL divergence between the distribution of latent codes and a multivariate Gaussian prior with dimension that is the size of the informational bottleneck latent code dimension. We also used the approach of the beta-VAE literature [12]to upweight the divergence term of the loss to promote disentangled learned representations. The anomaly score of an image relative to our trained model is the L 2 reconstruction loss between the image and the VAE's reconstruction of the image. A promising anomaly score, not yet fully evaluated, is the norm of the µ vector in the VAE's code of that image, which can be seen as a distance of the image's latent code from the mode of the latent code distribution, and hence a quantitative degree of anomaly-ness.

Metrics, Evaluating Lunar Surface Anomalies
For evaluating anomaly detection on the lunar surface, we took NAC image patches with human-made artifacts as positive anomalous samples for our experiments, and all image samples with no known human-made artifacts negative non-anomalous samples. Examples of these human-made artifacts in our study were lunar lander modules left from the American Apollo 15 and 17 missions. For model evaluation metrics, we used the average precision (AP) of our model's precision recall curves, precision at total recall (PaTR), defined as PaTR = precision at the model's threshold where the model returns all positive samples, and efficiency over manual vetting (EoMV), defined as EoMV = PaTR precision of the naive random anomaly scorer .
The motivation for EoMV is from considering it as the fraction of images needed to be vetted by a human using a random search over the ratio of images needed to be vetted by a human using our model to present images by descending anomaly score order.

Software, Hardware
We used PyTorch for training, JupyterLab with Python to coordinate the experiments, and the seaborn statistical visualization python package to view and plot results. We used an NVIDIA GeForce GTX 1070 and Intel Core i7 system with 512 GB SSD for training and validation. hyper-parameters were learned one the train set above, and evaluated on our train set in a one-shot fashion, with results reported below. Extra care was taken to avoid hyperparameter leakages between optimizing our approach on the train set and evaluating performance on our train set, due to the acutely limited nature of human lunar technosignatures.

Lunar VAE Landing Site Validation
For our smallest dataset containing 200 patches, our method achieved an AP and PaTR of 1.00 with minimal tuning by returning all positive samples before all negative samples. Hence we moved to a more challenging and diverse validation dataset. For our larger validation dataset of 2000 patches, in Figure 1 we plot the precision-recall curve with metrics for validation on our Apollo 17 train set, and the distribution of anomaly scores of all samples on the horizontal axis versus only positive samples on the horizontal axis. We achieve an AP of 0.25, PaTR of 0.333, resulting in an EoMV of 0.333 0.003 = 111. Note the unusual shape of the PR curve, which was due to a clustering of the positive sample anomaly scores into one mode.

Lunar VAE Landing Site Test
In Figure 2 we plot the precision-recall curve with metrics for validation on our Apollo 17 train set, and the distribution of anomaly scores of all samples on the horizontal axis versus only positive

Evaluation of Learned Features
In Figure 3, we plot two different input images each presented twice (top row) with their corresponding reconstructed images through the VAE's forward pass (bottom row). We note the stochastic nature of VAE generation. In the same figure we plot a sample walk through the learned latent variable space of our trained model on the Apollo 17 landing site. We consider the relative quality of reconstruction and the smoothness of interpolation between the two data samples at upper-left and lower-right hand corners as evidence that the unsupervised generative model was able to learn a good representation of the underlying lunar dataset, in addition to the performance of the learned distribution for technosignature detection.

Conclusion
In this work we show that modern data-driven machine learning techniques can be successfully applied on lunar surface data to learn, in an unsupervised way, sufficiently good representations of the distribution of lunar surface data to enable lunar technosignature detection. In particular we have trained an unsupervised distribution learning model to find the landing module of the Apollo 15 landing site in a testing dataset, with no specific model or hyperparamter tuning . Good data density estimation has myriad applications in lunar and space sciences, including finding known missions with unknown landing sites, discovering non-publicly disclosed landing sites, either by governmental or non-governmental organizations, technosignatures in other signal domains, locating lunar resources for future space flight and colonization, locating new impact craters or lunar surface reshaping, e.g. when applying this to temporal stacks of images, and deciding the importance of unlabeled samples to send back from power-and bandwidth-constrained missions. We hope this current work stimulates and enables future work towards these goals.