{'fraction_non_alphanumeric': 0.02653531754349193, 'fraction_numerical': 0.013162858939425696, 'mean_word_length': 4.877309682187731, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
An initial exploration of vicarious and in-scene calibration techniques for small unmanned aircraft systems

The use of small unmanned aircraft systems (sUAS) for applications in the field of precision agriculture has demonstrated the need to produce temporally consistent imagery to allow for quantitative comparisons. In order for these aerial images to be used to identify actual changes on the ground, conversion of raw digital count to reflectance, or to an atmospherically normalized space, needs to be carried out. This paper will describe an experiment that compares the use of reflectance calibration panels, for use with the empirical line method (ELM), against a newly proposed ratio of the target radiance and the downwelling radiance, to predict the reflectance of known targets in the scene. We propose that the use of an on-board downwelling light sensor (DLS) may provide the sUAS remote sensing practitioner with an approach that does not require the expensive and time consuming task of placing known reflectance standards in the scene. Three calibration methods were tested in this study: 2-Point ELM, 1-Point ELM, and At-altitude Radiance Ratio (AARR). Our study indicates that the traditional 2-Point ELM produces the lowest mean error in band effective reflectance factor, 0.0165. The 1-Point ELM and AARR produce mean errors of 0.0343 and 0.0287 respectively. A modeling of the proposed AARR approach indicates that the technique has the potential to perform better than the 2-Point ELM method, with a 0.0026 mean error in band effective reflectance factor, indicating that this newly proposed technique may prove to be a viable alternative with suitable on-board sensors.


INTRODUCTION
Recent innovation in remote aircraft design has enabled remote sensing practitioners to work at resolutions and under conditions originally thought to be impractical with traditional remote sensing platforms such as satellites and manned aircrafts. Known as sUAS (small unmanned aerial systems), these platforms have become widespread throughout the consumer market leading to an explosion of sUAS being used for untested remote sensing applications. 1 With a sUAS ready to fly at a moments notice, remotely sensed images are able to be captured at low altitudes (below 400ft) with atmospheric conditions ranging from ideal clear sunny days to overcast cloudy days. These images need to undergo proper calibration, otherwise incorrect results and conclusions will be drawn from them. With this newfound ease of collection, we must not forget the importance of the lessons learned from traditional remote sensing practices.
Remotely sensed images are affected by a variety of factors. These factors include, but are not limited to, date/time, geographic location, and atmospheric conditions. To mitigate these components and produce accurate results, calibration needs to take place. As an example, Figure 1 displays the need for calibration. Radiance images normalized by the exposure and gain, of the same scene on a cloudy and sunny day, do not appear the same, yet when converted to reflectance, the images appear invariant to illumination conditions. A traditional agricultural remote sensing metric, Normalized Difference Vegetation Index (NDVI), 2 calculated using both radiance and reflectance are presented in Figure 2.  Figure 2: NDVI maps derived with radiance imagery under (a) cloudy and (b) sunny conditions. NDVI maps derived using reflectance maps for the (c) cloudy and (d) sunny conditions are shown. The imagery was collected using a MicaSense RedEdge sensor for an altitude of 375ft on November 2, 2017 (cloud) and November 8, 2017 (sunny). Note the difference in the radiance NDVI of the grass in the lower left corner of the NDVI maps which became consistent in the reflectance images.
The purpose of this study is to compare the accuracy of the widely used 2-Point and 1-Point ELM calibration techniques to a newly proposed At-altitude Radiance Ratio technique.

Empirical Line Method
The Empirical Line Method (ELM) is one of the most well known methods of converting radiance to reflectance for any remotely sensed image. By placing suitably sized calibration panels in the scene, a linear relationship can be determined between digital count and reflectance. The linear relationship can be determined if the reflectance of the calibration panel(s) is known at the time of data collection (to best account for directional effects). For results that best approximate the linear relationship, two panels are recommended (one bright and one dark) and have the properties of homogeneousness and are parallel to the ground. 3,4 In 2008, Baugh and Groeneveld empirically proved the ELM. 5 Other researchers have implemented the ELM techniques differently. Wang et al. 6 showed results that the relationship of raw digital number and the natural log of the surface reflectance is linear. Furthermore, they state that the y-intercept is the minimum reflectance of the surface, and only one data point (panel) is needed to compute the digital number (digital count) to reflectance look up table. Ultimately, their results show that there is no statistically significant difference between the measured and predicted reflectance values.

MicaSense RedEdge
MicaSense offers remote sensing imagery solutions for agricultural use. Their RedEdge multispectral camera is a five band camera that captures multiple filtered images simultaneously. They offer a DLS (downwelling light sensor) that measures the downwelling irradiance for the same spectral bands that can be mounted on a sUAS platform. Our experiment utilizes this sensor to measure the lighting conditions for any given image and is integral to the proposed calibration technique. Table 1 displays the five MicaSense RedEdge spectral bands along with their respective wavelengths and bandwidth (FWHM). 7 A picture of the MicaSense RedEdge camera and the DLS can be seen in Figure 3.  The metadata of each captured image contains fixed values, for each band, to convert the digital count values to radiance (W/m 2 /nm/sr). These values include a vignette function, radiometric calibration coefficients, sensor gain, exposure time and black level values.
To model the MicaSense RedEdge's output correctly, the relative spectral response (RSR) functions of the camera was computed using spectral power data measured using a Newport Model 74004-1 monochromator. Figure 4 displays these RSR functions for the five spectral channels of the MicaSense RedEdge camera.

Sensor Reaching Radiance
The spectral radiance reaching a sensor 8 can be expressed as: where, E solar (λ) is the spectral exoatmospheric solar irradiance, σ is the solar elevation angle, τ 1 (λ) is the spectral transmission from space to the target, τ 2 (λ) is the spectral transmission from target to sensor, ρ(λ) is the spectral directional reflectance function for the target, ρ d (λ) is the spectral diffuse reflectance for the target, L ↓solar (λ) is the solar scattered downwelling sky radiance propagating on to the target, L ↑solar (λ) is the solar scattered spectral path radiance generated in the path between target and the sensor, and L a (λ) is the scattered photons from background objects near the target of interest.
Band effective radiance, L s,i , can be computed by integrating the spectral radiance given by Equation 1 over the spectral bandpass defined by the RSR functions for each of the five bands of the MicaSense RedEdge sensor using where RSR i (λ) is the relative spectral response for the i th band.

METHODOLOGY
The first method is a 2-Point ELM which applied a look up table (LUT), computed from two in scene calibration panels, to convert digital count to reflectance. The second method is a 1-Point ELM that applied a radiance to reflectance factor to every pixel in the image (after converting the image from digital count to radiance). Both ELM methods were applied to five images that were captured during a sUAS flight on November 8, 2017 at 375ft.
Afterwards, a newly proposed technique, which we refer to as At-altitude Radiance Ratio (AARR), is tested on the same five images. This approach utilized the recorded DLS spectral irradiance values and divides the converted radiance image by the converted DLS radiance values. The reflectance factor errors from all three of these methods were computed and compared against one another.

2-Point Empirical Line Method (2-Point ELM)
The first method tested was the 2-Point ELM. Raw digital counts were normalized by their gain and exposure time. The normalized digital count in each band, DC norm,i , is given by where DC raw,i denotes the raw digital count of each pixel, t min denotes the shortest exposure time the sensor system is capable of (0.066ms), and t is the exposure time of the current frame. Similarly, g min represents the minimum gain, or ISO, possible (1x or 100) and g is the gain used in capturing the current frame. n is the bit depth of the desired normalized digital count dynamic range and m is the sensor bit depth.
A LUT is calculated for each band by relating the normalized pixel value of a calibration panel to the measured reflectance of that panel. One bright and one dark reflectance calibration panel were used in the scene. From these two points, ELM was conducted by finding the slope and bias which converts normalized pixel value to reflectance ρ i described by where b 1,i is the slope, ρ bright,i is the band effective reflectance of the bright panel for band i, ρ dark,i is the band effective reflectance of the dark panel for band i, DC norm,bright,i is the normalized digital count of the bright panel for band i, and DC norm,dark,i is the normalized digital count of the dark panel for band i.
A unique series of spectral LUTs is only valid for a single illumination level. Illumination conditions can vary greatly during a single flight. By producing several LUTs, under varying cloud conditions, illumination variation can be handled. For this study, images were selected where the two calibration panels were located nadir to the sensor (within a 10 • radius of the image center), and a series of spectral LUTs was produced for each of those images. These LUTs were paired with the DLS measurements (a five element array) recorded with their respective image.
When converting an image to reflectance, The LUT with the smallest Euclidean distance between it's paired DLS measurement and the DLS measurement of the current image is applied to the image. Euclidean distance is described by where d(DLS img , DLS LU T k ) is the Euclidean distance between the DLS measurement of the image being converted to reflectance, DLS img , and the DLS measurement of the current LUT, DLS LU T k . K is the number of LUTs for the current sUAS flight, and i denotes the spectral band of DLS measurement. This ultimately results in a reflectance image that has been adjusted for variation in illumination.
The second method tested was a 1-point ELM using code provided by MicaSense. 9 Here, vignetting correction, dark level bias, and radiometric calibration are also accounted for in the conversion of digital counts to radiance, L s,i as where a 1,i ,a 2,i and a 3,i are the radiometric calibration coefficients, g i is the sensor gain, p and p BL,i are the normalized raw pixel value and normalized black level value, respectively, and t i is the exposure time of the image.
The vignette model applies a light sensitivity fall-off correction to all pixels in the image. Pixels that fall further from the vignette center have a larger factor applied. Equations 9-11 show how the vignette model is applied to each pixel in the image.
where c x and c y represent the vignette center, k 0 through k 5 are polynomial correction coefficients extracted from the image metadata, I(x, y) is the normalized pixel intensity from Equation 8. At this point, the radiance L s,i can be found at any pixel within an image, and the ELM method described in the previous section can then be applied in the same fashion, except the dark reflectance point is replaced by the origin (L s,i = 0 and ρ i = 0). This means the slope computed by ELM is a radiance to reflectance factor.

At-altitude Radiance Ratio (AARR)
To test the efficacy of the MicaSense RedEdge's DLS, a new method for converting raw digital counts into reflectance was examined. The images were converted to radiance the same way as the 1-Point ELM method (vignetting correction, radiometric calibration, dark level bias, gain and exposure time normalization). The DLS spectral irradiance was extracted from the image's metadata, and converted to radiance (divided by π). These two values were ratioed using Equation 13 to produce reflectance images for each spectral band. This AARR reflectance equation can be rewritten using the terminology from the sensor reaching radiance equation, as shown in Equation 14.
Others have applied similar techniques for conversion to reflectance. Lekki et al obtained reflectance by taking the ratio of the water leaving radiance and the downwelling irradiance, 10 while Ortiz et al took the ratio of the HSI2 (John Glenn Research Centers Hyperspectral Imager) radiance and the upward looking Analytical Spectral Devices (ASD) FieldSpec that they had fixed on their aircraft, which was flying at traditional remote sensing altitudes. 11 Our method differs because our sUAS was flying at a max height of 375ft, and the MicaSense RedEdge DLS recorded single irrradiance values per band, not a full downwelling irradiance spectra.

Modeled At-altitude Radiance Ratio (M-AARR)
To demonstrate the theoretically achievable results by the AARR method, simulations were accomplished using Spectral Science Incorporated's moderate resolution atmospheric transmission (MODTRAN) code, specifically, MODTRAN4. MODTRAN is designed to model atmospheric propagation over the wavelength range for 0.2µm to 100µm. 12 With the ability to model any combination of atmosphere, day, time or location, MODTRAN gives users the ability to simulate countless scenarios. For the purposes of this paper, the focus was primarily on matching the atmospheric conditions of November 8, 2017 in Henrietta, NY, USA (43.041099 N, 77.698343 W).
To run MODTRAN, an input file (named 'tape5') with all the simulation parameters needs to be provided. Each MODTRAN run solves the radiative transfer equation and provides accurate modeling of the atmosphere given the inputted parameters. 12 Therefore, a script was written to modify the input file based on the given parameters. Some of modified parameters were: target, background and sensor altitude. A full list of selected parameters can be seen in Appendix B. In addition, MODTRAN allows users to input their own reflectance curves to help generate models of measured reflectances. The mean ground reference reflectance curves for three targets (grass, concrete, and asphalt) was computed and added to the spectral albedo (named 'spec alb.dat') file, which can be called by MODTRAN for modeling particular types of targets. This allowed for the modeling of the sensor reaching radiance of the targets that were measured in the field.
Once finished, MODTRAN provides an output file (named 'tape7.scn'), which contains spectral transmittance, radiance and irradiance components. Some of the computed variables are ground reflected radiance (GRND RFLT), direct solar radiance (DRCT RFLT), total radiance seen by the sensor (TOTAL RAD) and the solar scattered radiance (SOL SCAT). For the purposes of this study, we used both the total radiance, and the ground reflected radiance variables. The total radiance is represented fully in Equation 1 as L s (λ) and the ground reflected radiance is equivalent to the denominator of Equation 14.
To model the data that was seen by the camera, as well as the DLS, two separate MODTRAN runs were executed for every combination of simulation parameters. The first run was a measurement of the sensor reaching radiance. For this, the camera was pointed directly down at the target on the ground. The total radiance from this simulation was used as the sensor reaching radiance. The second run was to simulate the DLS downwelling radiance. This was important to model correctly, because depending on the atmospheric conditions, the sunlight can be between 10 and 100 times the magnitude of the skylight. 13 To model the DLS downwelling radiance, a constant 100% reflector was placed at the original sensor altitude, and the sensor altitude was raised by 1 meter. The ground reflected radiance of the 100% reflector was recorded and used as the DLS downwelling radiance. By simulating the reflected radiance of a constant 100% reflector, at the original sensor altitude, this value is equivalent to the downwelling radiance onto the DLS, because all of the sun and skylight are being reflected back up to the new sensor position (1m above).
To compute the at-altitude reflectance, the total radiance from MODTRAN run one was divided by the ground reflected radiance from MODTRAN run two. This can be seen in Equation 15, using the MODTRAN terminology, and in Equation 16, using the sensor reaching radiance terminology.
where ρ(λ) is the reflectance of the target, T OT AL RAD 1 is the total radiance outputted from MODTRAN in the first run and GRN D RF LT 2 is the ground reflected radiance outputted from MODTRAN in the second run. These two simulations are visualized in Figure 5 below.

Data Collection
For data collection, permission was given to fly our sUAS at the Henrietta Fire District Training Center (Station Number 6, 43.041099 • N, 77.698343 • W). In the scene, two sets of calibration panels were deployed (one set in direct sunlight and the other in shadow), and two sets of colored felts (red, green, blue and brown) were deployed, among other in scene targets. The tower casts a large shadow onto the ground as well, which will be helpful later when shadow detection and reflectance calibration of shadowed pixels is attempted in a follow up study. An example of a raw digital count orthomosaiced image from these collects is shown in Figure 6. The small unmanned aircraft system used for data collection was a DJI Matrice 100 quadcopter, which can be seen in Figure 7.  Spectra Vista Corporation's (SVC) HR-1024i spectroradiometer was utilized for collecting ground reference data. With a spectral range of 350-2500nm, and use of 100% linear array detectors, the HR-1024i provides excellent wavelength stability, and produces highly accurate spectra. 14 To ensure the highest accuracy possible, a reference spectra was captured of a Spectralon panel before each measurement was taken. The spectra used for ground reference measurements can be found in Appendix A. Figure 8 shows an example spectra measurement in the field.

RESULTS
A table is presented with the band integrated ground reference reflectance factors for the three ground targets (grass, asphalt and concrete). After all five sUAS images were converted to reflectance using 2-Point ELM, 1-Point ELM, and AARR, the computed reflectance factors of the three targets (grass, asphalt, and concrete) were compared against the band integrated ground reference reflectance factors. The average error and standard deviations, across the five images, were computed. Figures 9-11 display the computed statistics.

M-AARR
For the purposes of this preliminary study, the only model simulated was designed to mimic the November 8, 2017 atmospheric conditions in Henrietta, NY. The model parameters for this simulation can be seen in Appendix B. Below are a set of figures that modeled the sensor reaching radiances, the downwelling radiances, and target reflectances vs the sensor altitudes. The six altitudes selected for these simulations were: 2, 150, 225, 300, 375, and 5000ft. 150, 225, 300 and 375ft were all selected because those were the heights of the sUAS data collections at the Henrietta Fire Training Center. 2ft was selected to test the M-AARR technique against the ground reference (these two should have been more or less identical), and 5000ft was simulated to show how the AARR method performs for traditional remote sensing.     For comparison purposes, the average absolute reflectance factor error across all channels and targets for each method was computed. 2-Point ELM, 1-Point ELM, AARR, and M-AARR produced 0.0165, 0.0343, 0.0287, and 0.0026 reflectance factor error, respectively.
While the standard deviations are much higher in the AARR, this is to be expected because of the metadata variables that were utilized for this technique. Among these metadata variables were the gain and exposure time which are discrete values that are set, for each band of the image, automatically, as the images are captured from the RedEdge camera. The gain can only be one of four values, while the exposure time can be one of 21 values. If these variables are not quantized, the results would be more representative of the conditions they were captured under and therefore, more accurate. ELM has a much lower standard deviation because the calibration panels used to compute the point slope formulas also underwent the same exposure and gain correction as the targets in the scene.
Overall, 2-Point ELM performed the best across all five bands and three targets. While AARR did not perform as well as the 2-Point ELM, it outperformed the 1-Point ELM. In addition, the M-AARR results showed that with the proper setup and instrumentation, AARR reflectance factor error could be lowered significantly. Therefore, given the proper instrumentation, sUAS imagery (to 400ft) can produce very reliable reflectance maps without the use of calibration panels.

FUTURE WORK
The initial study was only accomplished for a small set of parameters. The atmosphere, visibility, day and time were all purposefully selected to best model the data that was collected with our sUAS. We plan to expand the modeling section of our study to assess the feasibility of implementing AARR technique under various conditions, which include: This upcoming in-depth study will be aimed at providing an indication of quality of performance under various atmospheric conditions. This will provide sUAS users with an understanding of the reflectance accuracy they could produce without in-scene calibration panels. All of these simulations will be modeled using MODTRAN and analyzed using the M-AARR technique.
In addition to the extra modeling that will be accomplished, all the captured images from the 12 sUAS flights at the Henrietta Fire District Training Center will be analyzed using 2-Point ELM, 1-Point ELM and AARR. This will provide us with more reliable reflectance factor error statistics, as the averages will be across hundreds of images.
Finally, speaking with instrumentation manufacturers could help them in producing better cameras and downwelling light sensors which in turn would produce more reliable data for all sUAS users.

CONCLUSIONS
While 2-Point ELM produced the lowest mean error in band effective reflectance factor (0.0165), this preliminary study has shown that the AARR method is a reliable technique for producing reflectance imagery from sUAS captured images (0.0287 mean error). Under the assumption that the in-depth study produces similar results, for a wide range of atmospheric conditions, sUAS users will not need to place calibration panels or conduct ELM to convert their images to reflectance. This will save the sUAS remote sensing practitioner significant time during ground collects as they will not need to place calibration panels or collect ground reference spectra. In addition, sUAS users will also save time during data processing, because they would not have to go through the burden of creating and applying LUTs to convert the images to reflectance.