{'fraction_non_alphanumeric': 0.03026862185310148, 'fraction_numerical': 0.021671424863742537, 'mean_word_length': 4.834752981260648, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 1, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The CosmoQuest Moon Mappers Community Science Project: The Effect of Incidence Angle on the Lunar Surface Crater Distribution

The CosmoQuest virtual community science platform facilitates the creation and implementation of astronomical research projects performed by citizen scientists. One such project, called Moon Mappers, aids in determining the feasibility of producing crowd-sourced cratering statistics of the surface of the Moon. Lunar crater population statistics are an important metric used to understand the formation and evolutionary history of lunar surface features, to estimate relative and absolute model ages of regions on the Moon's surface, and to establish chronologies for other planetary surfaces via extrapolation from the lunar record. It has been suggested and shown that solar incidence angle has an effect on the identification of craters, particularly at meter scales. We have used high-resolution image data taken by the Lunar Reconnaissance Orbiter's Narrow-Angle Camera of the Apollo 15 landing site over a range of solar incidence angles and have compiled catalogs of crater identifications obtained by minimally trained members of the general public participating in CosmoQuest's Moon Mappers project. We have studied the effects of solar incidence angle spanning from approximately 27.5 deg to approximately 83 deg (extending the incidence angle range examined in previous works), down to a minimum crater size of 10 m, and find that the solar incidence angle has a significant effect on the crater identification process, as has been determined by subject matter experts in other studies. The results of this analysis not only highlight the ability to use crowd-sourced data in reproducing and validating scientific analyses but also indicate the potential to perform original research.


INTRODUCTION
A fundamental tool of planetary geology is the statistics of impact craters. The size-frequency distribution of craters is the only 1 method available on solid bodies other than Earth to model surface ages (Bell 2020;Neukum et al. 2001). On the Moon, calibrations provided by Apollo and Luna samples allow us to calculate aplazas@astro.princeton.edu 1 Other than in situ methods, e.g., Farley et al. (2014). the lunar crater calibration function, which relates absolute age of the sample with the spatial density of craters larger than 1 km (Neukum & Ivanov 1994). Since many surfaces require measuring craters either smaller than 1 km due to their small area or youth, or measuring only larger craters due to their older age and crater saturation at smaller sizes, a model crater size-frequency distribution (SFD) must be used to estimate the 1 km and larger crater population. However, ensuring that the craters measured are a representative sample of the craters on arXiv:2110.13404v2 [astro-ph.EP] 26 Jan 2022 that surface is not as straightforward as it might seem.
Identification of impact craters, with statistics typically given in the form of an SFD, is not trivial. Factors such as the large number of craters of different diameters on the lunar surface and the individual variations in crater identification from different researchers make this a challenging task.
Another important parameter that plays a central role when measuring the SFD of craters on the lunar surface is the solar incidence angle (Young 1975;Wilcox et al. 2005), defined as the angle between the normal to the surface and the position vector of the Sun on the sky. Images that are taken under different illumination conditions highlight different characteristics on the lunar surface ( Fig. 1). Topographic features of craters such as rims generate more pronounced shadows at higher incidence angles (i.e., with the solar illumination nearer the local horizon), facilitating the identification of craters in some cases (Head & Lloyd 1972;Soderblom 1972;Young 1975), although potentially hiding small craters under the shadows of larger craters in other cases (Moore 1972). On the other hand, rays formed from ejecta of primary and secondary craters could have a distinct brightness in optical light, so higher sun that emphasizes these brightness differences may aid in identification of those craters (Neish et al. 2013;Elliot et al. 2018), while making difficult the identification of other craters (Wilcox et al. 2005).
The precise determination of the crater equilibrium state-in which craters smaller than the equilibrium diameter are being produced at the same rate at which they are being destroyed-is fundamental to the accuracy of crater-based age determinations and regolith depth estimations (Shoemaker & Morris 1970;Ostrach et al. 2011;Xiao & Werner 2015). The equilibrium diameter can be identified as a break in the slope in a cumulative SFD plot (Gault 1970;Schultz et al. 1977). Hirabayashi et al. (2017) demosntrate, for example, that the the Apollo 15 landing site is in equilibrium at crater sizes below approximately 100 m.
Previous work (e.g., Soderblom (1972); Wilcox et al. (2005)) suggests that the determination of the equilibrium state is affected considerably by the different crater number counts found as a consequence of different solar incidence angles, a hypothesis that is also consistent with the analysis of lunar data by Ostrach et al. (2011). Schultz et al. (1977) (Fig. 3) have also demonstrated the relative difference between crater counts as a function of solar incidence angles.
In this paper, we use the high resolution data of the Apollo 15 landing region imaged by the Lunar Reconnaissance Orbiter (Robinson et al. 2010) Narrow-Angle Camera (LROC NAC) (Chin et al. 2006) to study the effect of solar incidence angle on crater counts in overlapping regions from different images. The LROC camera system has imaged the lunar surface several times at sub-meter resolution over a wide range of illumination angles.
Lunar craters are marked by minimally-trained nonsubject matter volunteers via the Moon Mappers 2 community science portal, a part of the CosmoQuest 3 project  ) and M109215691LE over the same lunar region and at different solar incidence angles with respect to the normal (77.5 • and 27.5 • , respectively). Corresponding craters are marked with numbers by one of the authors (L.L). The scale is such that the diameter of crater number 2 corresponds to approximately 90 m.Crater detection and identification is affected by the different illumination angles. (Gay et al. 2014;Robbins et al. 2012). Robbins et al. (2014) have demonstrated that community science volunteers produce, on average, results comparable to those by subject-matter experts when identifying lunar craters. Earlier works by the CosmoQuest team analyzing similar data annotated by the public have corroborated the dependence of crater counts on solar incidence angle (Antonenko et al. 2013;Grier et al. 2018).
The paper is organized as follows. In §2 we describe the LROC data used for this analysis. Section §3 reviews the CosmoQuest project and the Moon Mappers interface, detailing the crater-marking process by the vol-unteers, and the analysis performed on these data to determine the craters SFDs. We report our results in §4, and conclude in §5.

DATA
The data used for this analysis consist of seven overlapping images acquired by the LROC NAC. The overlap region used between the images has an estimated area of 14.76 km 2 and encompasses the Apollo 15 landing site. The images have an average scale of ≈ 0.5 m per pixel, and they were imaged at solar incidence angles of 27.5 • , 38.0 • , 58.0 • , 77.5 • , and 83 • (Fig. 2; some NAC left-right pairs are included, which is why there are only five incidence angles despite our use of seven images). This range of incidence angles extends the angle ranges examined in previous works (e.g., Wilcox et al. (2005);Ostrach et al. (2011)). Many studies-some similar in nature to our analysis-have been conducted within this region of the lunar surface; it serves as an ideal location to test the effects of incidence angle on crowd-sourced crater identifications and measurements. Other community science projects, such as Moon Zoo, performed similar analyses to the one presented in this work using images from different Apollo mission landing sites (Joy et al. 2011;Bugiolacchi et al. 2016).
All image data were downloaded from the Planetary Data Systems (PDS) 4 LROC image node. The images were processed using the Integrated Software for Images and Spectrometers (ISIS) 5 software package and projected to a local Mercator projection. Our region of interest focuses on most of the area between 26.09 • to 26.21 • latitude and 3.56 • to 3.70 • longitude, which roughly translates to a grid of 3.64×4.24 km 2 . As a single image easily blankets most of the target region, even at full screen resolution, the imaged area is far too large to identify craters at the ten-meter scale. Furthermore, attempting to identify craters present within an image would have the appearance of being a daunting task. To make the identification of craters manageable and enhance the feasibility of identifying craters at the minimum desired crater diameter, images were sliced into sub-images 450 pixels across and tall, with overlaps. To ensure accuracy in the georeferencing of positions within the images, all images were uploaded into the Java Mission-planning and Analysis for Remote Sensing (JMARS) (Christensen et al. 2009) software package to estimate the selenographic corner coordinates of the images. The tables in the appendix show the crater population results used in this work.

METHODS
In recent years, astronomical surveys and projects have steadily increased their capabilities to produce large quantities of data. As new computational and analysis techniques advance to meet the new challenges that arise, it has also been recognized that involving the general public in astronomical research through community science is also a valuable method for data analysis and verification (e.g., Raddick et al. (2010); Marshall et al. (2015); Bugiolacchi et al. (2016)).
Founded in 2012, CosmoQuest is a virtual research facility that engages the general public in community science projects related to astronomy (Robbins et al. 2012;Gugliucci et al. 2014b,a;Gay et al. 2014). CosmoQuest's custom web interface has hosted a suite of projects that have not only allowed volunteers to assist in the mapping of craters and other features on the Moon, Mars, Mercury, Vesta, and Bennu, but has also increased the general public's exposure to the various topologies that these worlds exhibit.
One of the original community projects produced by CosmoQuest is Moon Mappers (Robbins et al. 2012). Volunteers gain access to the online interface of Moon Mappers through CosmoQuest's Moon Mappers web portal, which also provides tutorials on how to use the interface and educational materials (about the impact of solar incidence on lunar crater identifications, in this case) through blog posts and videos. 6,7 CosmoQuest's volunteers are also engaged with the project via social media and, in some cases, through more direct communication channels such as discord, Twitch, and email.
The Moon Mappers interface ( Fig. 3) is an application specifically designed to facilitate the identification and measurement of craters within the selected LROC NAC images (or master images). CosmoQuest's participants have used Moon Mappers to identify and measure more than 1 million lunar craters (Antonenko et al. 2013), with a large majority of the crater counts being collected within the first year of Moon Mappers official launch. All images displayed within the interface are set to 450×450 square pixel sub-images of the master image. Most of the sub-images have the same pixel scale as the main images; however, some sub-images feature a zoomed-out view to allow for crater identifications at larger diameters. For this analysis, we only use sub-images that are at the original scale. The images presented to the volunteers have a linear stretch with a range of ± 0.5 % (users did not have the ability to change the stretch).
To be an active participant in the project, i.e., for a user's annotations to be recorded, volunteers must be authenticated and have completed the Moon Mappers tutorial, which covers all essential information and steps to identify craters. The identification of craters is accomplished by selecting the interface's circle annotation tool and drawing circles that match the rim and are centered on craters identified by the user within a given image (Fig. 3). A minimum threshold diameter of 18 pixels has been imposed to ensure that volunteers are able to visually identify craters with ease (determined through usability testing), restrict the number of crater identifications to a light workload per image, and establish a lower limit, of D ≥ 10 m (at the average resolution of the NAC images), on the size of craters to be identified (Robbins et al. 2014). User-circled annotations that are less than the threshold diameter will appear red and are automatically discarded by the interface. Annotations that are equal to or greater than the threshold are displayed in green and can be modified (resized, re-positioned, or erased). Upon moving to the next image, all identified craters are saved in a MySQL database that can be accessed by project staff for later analysis. We require each . Data associated with image names ending with "LE", "RE", and "LERE" correspond to crater data derived from the left, right, and both NAC cameras, respectively. Crater number 2 of Figure 1 with an approximate diameter of 90 m is located at coordinates (longitude, latitude) ≈ (3.66 • , 26.14 • ) .
450×450 square pixel sub-image to be annotated by 15 different volunteers in order for it to be considered complete. For calibration and quality assurance purposes, at random intervals the volunteers are shown an image that has already been marked by an expert. The volunteers are given feedback on their scores compared to the experts to encourage quality work. In addition, this calibration process is used to assign confidence intervals to a particular volunteer's work during crater clustering (Robbins et al. 2014). A 3D clustering algorithm (Robbins et al. 2014) is used to combine the initial markings and assign a unique marking per crater (demanding a minimum of N≥9 marks per crater) that will be saved to produce the final crater catalog (Fig. 4).

Preliminary analysis of crater data
The first year of crater annotations made by Moon Mappers volunteers on our previous work (Antonenko et al. 2013;Grier et al. 2018) yielded over 1.1 million crater identifications and diameter measurements. Upon using the 3D clustering algorithm, the crowd-sourced crater identifications translated to a catalog of tens of thousands of physical craters. Having amassed such a large database of craters, a statistically significant preliminary investigation into the effects of solar incidence angle on crater identification could be done. At each incidence angle, all craters that had been identified After completing a brief tutorial on how to use the tool, the interface guides volunteers to mark craters by drawing circles around the crater rim. The tool guides them to only select craters of a certain minimum size. The interface also has a bar on the right that contains examples of craters in all stages of degradation to assist volunteers in crater identification. The images presented to the volunteers have a linear stretch with a range of ± 0.5 % (users did not have the ability to change the stretch).
were used to construct cumulative size-frequency distributions (CSFD; histograms of the crater diameters that are summed from large to small diameters). The results showed that when the Sun was higher in the sky, such as ∼ 30 • , volunteers had more difficulty finding craters than when the Sun was lower in the sky, such as ∼ 80 • . Quantitatively, this discrepancy in the perceived crater population within the same region resulted in approximately four times fewer crater identifications in the crater population enumerated at the lowest solar incidence angle relative to the crater population observed at the highest solar incidence angle. The analysis indicated that the ideal incidence angle for crater identification lies within ∼ 58 • < i < 77 • .
The major conclusion drawn from the preliminary analysis agreed with expectations, i.e., that the solar incidence angles closer to zenith makes the identification and measurement of craters difficult. For the results of the preliminary analysis to be valid, two assumptions must be true: 1.) the crater population throughout the Apollo 15 landing region is consistent, and 2.) the catalog produced from Moon Mappers volunteers has high completeness at each solar incidence angle used for the analysis (i.e., at a given angle, volunteers have identified as many craters as a professional crater counter would have identified). Therefore, in this work we have restricted the analysis to regions on the lunar surface that are shared in common between pairs of LROC NAC images taken at different solar incidence angles. We have used the crowd-sourced crater identifications within these regions to derive and compare resulting cratering statistics, as this approach allows us to safely ignore the aforementioned assumptions.

Crater analysis in overlapping regions
While the master images generally cover the same area, the overlap between images at different incidence angles is not complete. To ensure that the non-overlapping areas would not bias comparisons made between different sets of calculated cratering statistics derived from images at different incidence angles, we first identified the overlapping region between a given set of master images at two different incidence angles (Fig. 5).
The overlap is determined at the 450×450 square pixel scale so as to only include sub-images that had been examined by Moon Mappers volunteers. To find the pairs of overlapping sub-images, we began by first determining the corner coordinates of each sub-image. This was accomplished by making use of the fact that all of the master images' corner coordinates have been defined in selenographic coordinates. The relationship between a given master image's four corner pixel coordinates and corresponding selenographic coordinates allows for a unique mapping between the two coordinate systems through the calculation of the projective transformation matrix. We calculated the transformation matrix for each master image and then used the transformation matrices to convert the sub-images' four corner pixel coordinates to selenographic coordinates. With the selenographic coordinates of the corners of each sub-image known, we then employed the ray-casting algorithm, also referred to as the crossing number algorithm, to determine which sub-images overlap by checking if any of the vertices of one sub-image were within the bounding region of another sub-image. The Southerland-Hodgman algorithm (Sutherland & W 1974) was then used to determine the coordinates of the boundary of the overlap region between pairs of sub-images that were determined to have overlap.
Visual inspection of the master images indicated that the overlap region between different pairs of master images can differ in either size or geographical location. This fact was accentuated after computationally identifying the overlap regions between the master images. As we wish for our statistical inferences to be based on crater data obtained within the same region imaged at two incidence angles, we elected to calculate statistical results for each pair of master images. This task required us to first identify all user annotations contained within a given overlap region, and record those annotations to produce a catalog of user annotations. Given that a single crater can have multiple user annotations, we utilized a modified density-based spacial clustering of applications with noise (DBSCAN) algorithm-the aforementioned 3D clustering algorithm-to compile a crater catalog of single craters associated with clusters of user annotations in the user annotations catalog.
Lastly, we constructed cumulative size-frequency dis-tributions and relative size-frequency distributions ("R" plots) for each pair of incidence angles. We bin and construct the distributions following the conventions in Crater Analysis Techniques Working Group et al. (1979).

RESULTS
Figs. 6 and 7 show the resulting CSFDs and "R" plots for different pairs of incidence angles. Volunteer identifications and measurements of craters are consistent within uncertainties for incidence angle differences ≤ 10.5 • as shown by cumulative and relative SFDs. We begin to see noticeable deviations between cumulative and relative SFDs where the incidence angle difference is ≥ 19.5 • . Specifically, the deviations are small at smaller crater diameters and gradually increase in magnitude with increasing crater diameter.
This trend becomes more pronounced with increased difference in incidence angle. For comparisons where the difference in incidence angle is ≥ 50 • the CSFD's and "R" plots appear to be fairly distinct distributions in favor of more craters being identified at the higher incidence angle.
The relative comparison between low and high incidence angles becomes more informative under the assumption that the high-angle images (≥ 77 • ) provide a better crater-mapping surface and are therefore more representative of the actual crater population in this image. Lower angles (≤ 58 • ) are presumptively more susceptible to recognition challenges, i.e., they are more susceptible to observational or perceptual variables rather than actual surface conditions, and therefore rendering the images analyzed under these conditions potentially less optimal for science investigations.
Following Robbins et al. (2018), we also fit the unbinned crater size data to a truncated Pareto distribution. 8 The slopes of the distribution are reported in the legend of Fig. 6 for each incidence angle and field. We note that the fitted slopes are < −3, which would indicate the areas are still in production. However, it is known that the Apollo 15 sites are in equilibrium, with cumulative slopes in the derived data of approximately −2 or shallower (e.g., Hirabayashi et al. (2017) and the results from CosmoQuest's previous study in Robbins et al. (2014)). The results in this study show a general agreement with our previous results in Robbins et al. (2014) at high angles within the errors bars, however, the results at lower angles (< 58 • ) are not in good agreement at larger diameters, which could contribute to steepen method that fits a power-law. Each panel depicts the difference in the CSFD and for craters identified within the same area imaged at two different incidence angles. Colors and plotting style indicate which distribution was derived from the larger incidence angle (dashed and triangles) and smaller incidence angle (solid and squares). The slope value for a fit to a truncated Pareto distribution of the unbinned diameter data is also shown in the legend for each incidence angle.
the slopes in this case. A factor that could affect identification of larger craters at low solar incidence angle is the roughness of lunar topography (e.g., Kreslavsky et al. (2013) have analyzed the roughness of the lunar surfaces at large range of scales). At high solar incidence angles (lunar evening/morning), the general background roughness of the lunar surface is comparable to small craters, and therefore more of them might be missed under these conditions. On the other hand, at low solar incidence angles (near lunar noon) the almost absence of shadows and more presence of reflective differences is on the scale of smaller craters and the eye of the non-subject matter experts latches onto those features, preferentially finding them. However, in large craters, these prominent topographic cues are not present, and instead, the eyes of the volunteers are drawn to features within the craters that are at the same scale as smaller craters elsewhere.

CONCLUSIONS
The results we have obtained from the data confirm that Moon Mappers volunteers identified more lunar craters when the Sun is lower on the horizon (high incidence angle) on overlapping regions. We conclude that for a given area on the lunar surface, imaged at two different incidence angles, one should expect that the overall number of craters identified at the higher incidence angle will be greater than the overall number of craters identified at the lower incidence angle. Furthermore, a larger difference in incidence angle should generally result in a larger difference in the overall number of craters identified at the two levels of surface illumination. In particular, we find that there is practically no difference when the sun is very low on the horizon (77.5 • vs 83.0 • ) or high in the sky (27.5 • vs 38.0 • ), but we found progressively fewer craters were found as sun angle decreases between ≈ 40 • and ≈ 75 • . Moving from a ≈ 75 • -80 • incidence angle to a mere 30 • -40 • cuts crater detections by a maximum of ≈ 3.5x (see e.g., the cumulative and relative counts from Table 3 in the appendix). We find that -"R" plots. Each panel depicts the difference in the "R" and for craters identified within the same area imaged at two different incidence angles. Colors and plotting style indicate which distribution was derived from the larger incidence angle (dashed and triangles) and smaller incidence angle (solid and squares).
this difference decreases for the smallest craters, possibly because ≈ 10-20 pixel craters are easier to interpret as depressions regardless of sun angle than the larger ones.
Compared to our previous the results in Robbins et al. (2014), the results of this present study show an agreement consistent with the error bars, except for smaller scales, where the data presented in this study tends to be higher, and at larger diameters for smaller incidence angles. We point out, however, that our study used a larger area compared to Robbins et al. (2014) (approximately 2.5x larger area; see tables in the appendix), which could also contribute to the mismatch at these scales. In the case that there were a systematic offset towards smaller biases in the CosmoQuest's Moon Mappers data and under the assumption that is is uniform across different incidence angles, our study would still be valid given that its main point is to analyze incidence angle differences via crowd sourcing.
In order to complement the observational analysis presented here, an additional venue of study would be the study of the topographical characteristics of the nondetected craters at low incidence angles, compared to those detected, via complementary data such as the Apollo 15 landing site Digital Terrain Models for areas that overlap. In addition, further work could include the analysis of the magnitude of incidence angle effects at size ranges larger than those considered in this study (≥ 100 m, e.g., 200 m -2 km) where the population is not expected to be fully degraded and the impact of incidence angle should therefore diminish. This type of study would also allow to gauge the interplay between observational and topographical/geological factors.
Our results validate and verify what many professional crater counters' analyses have been conveying for a few decades now. For a cratered region of a planetary surface that has been imaged at two considerably different incidence angles, one expects to be able to identify more craters in the image captured at the higher incidence angle. This effect is qualitatively well understood in terms of the brightness variations caused by variable illumination of different slopes: at lower incidence angles with respect to the normal (when the sun is high overhead), craters are more difficult to distinguish, given that in these circumstances albedo is more prominent and topographic features are more difficult to identify (Wilhelms et al. 1987). We highlight that the results of this analysis were generated with contributions from volunteers of the general public. This study underscores the value in having the public as science collaborators by confirming previous work and helping refine confidence levels, and fundamentally demonstrates the viability of public participation in authentic scientific investigations in future studies

ACKNOWLEDGMENTS
The authors would like to thank the many Cos-moQuest volunteers whose hard work and dedication made this study possible.
We thank the anonymous referees for their suggestions, which have improved the clarity and readability of the paper. Funding for CosmoQuest's Moon Mappers project and research staff was provided through NASA cooperative agreements NNX16AC68A and NNX17AD20A, grants NNX09AD34G and NNX12AB92G, and contracts through the Lunar Reconnaissance Orbiter education and public outreach office. This publication was made possible through additional support from the NASA SSERVI VORTICES project, via CAN NNA14AB02A. This research has made use of NASA's Astrophysics Data System Bibliographic Service, and of the following software: NumPy (van der Walt et al. 2011), Matplotlib (Hunter 2007, and SciPy (Jones et al. 2001).  one gives the lower edge value of each bin. Bin widths conform to 2 1 2 bin intervals. Column two gives the surface area on which the data were measured. The next three columns are the cratering statistics for the cumulative size-frequency distributions. Each column has been split in two to show values derived from the overlap region for M109215691RE and M111578606RE, respectively. The third column is the cumulative number of craters. The fourth column corresponds to the density, which is in units of cumulative number of craters per square kilometer. The fifth column yields the corresponding 1σ uncertainty for each density value. The remaining three columns are the cratering statistics for the relative size-frequency distributions. Each column has been split in two to show values derived from the overlap region for M109215691RE and M111578606RE, respectively. The sixth column is the number of craters per bin. The seventh column corresponds to the relative frequency of craters per bin, which is a unitless quantity. The eighth column yields the 1σ uncertainty for each R value. The R value is calculated with respect to the geometric mean of the bin, rather than the bin minimum diameter.    1.14 × 10 −2 /2.54 × 10 −2 3.81 × 10 −3 /5.67 × 10 −3 5.66 × 10 −2 5.12 5/7 0.98/1.37 0.44/0.52 4/6 1.02 × 10 −2 /1.52 × 10 −2 5.08 × 10 −3 /6.22 × 10 −3 8.00 × 10 −2 5.12 1/1 0.2/0.2 0.2/0.2 1/1 5.08 × 10 −3 /5.08 × 10 −3 5.08 × 10 −3 /5.08 × 10 −3