{'fraction_non_alphanumeric': 0.024170520812386888, 'fraction_numerical': 0.009249949728534084, 'mean_word_length': 4.94595887135342, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Advancements in the ADAPT Photospheric Flux Transport Model

Global maps of the solar photospheric magnetic flux are fundamental drivers for simulations of the corona and solar wind and therefore are important predictors of geoeffective events. However, observations of the solar photosphere are only made intermittently over approximately half of the solar surface. The Air Force Data Assimilative Photospheric Flux Transport (ADAPT) model uses localized ensemble Kalman filtering techniques to adjust a set of photospheric simulations to agree with the available observations. At the same time, this information is propagated to areas of the simulation that have not been observed. ADAPT implements a local ensemble transform Kalman filter (LETKF) to accomplish data assimilation, allowing the covariance structure of the flux transport model to influence assimilation of photosphere observations while eliminating spurious correlations between ensemble members arising from a limited ensemble size. We give a detailed account of the implementation of the LETKF into ADAPT. Advantages of the LETKF scheme over previously implemented assimilation methods are highlighted.


Introduction
The Sun's dynamic magnetic field, from Coronal holes to strong photospheric magnetic fields within active regions, play a central role in driving Earth's heliosphere. Additionally, the solar photosphere represents the driving boundary conditions for Corona and Solar wind models. Therefore, accurately predicting the Solar photospheric magnetic field provides a better understanding and specification of Earth's space weather environment.
Typically, solar magnetic fields can be directly recorded on only half the solar surface, e.g. National Solar Observatory (NSO) Global Oscillation Network Group [15] and Helioseismic and Magnetic Imager (HMI) [29] onboard the Solar Dynamics Observatory (SDO).
Additional efforts are being made to use helioseismolgy technique to estimate active regions on the back of the Sun by analyzing acoustic waves propagating from the far side of the Sun to the near Earth side [24]. The Solar Terrestrial Relations Observatory (STEREO) mission [32] from NASA has provided 360 • observations from the Sun, making it the first ever data set that provides an almost complete nowcast of the Sun. Additionally, methods of reconstructing the Sun's coronal dynamics from observation have been advancing [9]. Although many of these observational data sets provide an accurate estimation of the current state of the Sun, they cannot provide a forecast of the Sun by themselves. Only a physics-based model, which is well calibrated, can provide such a forecast.
Data assimilation methods are techniques that fuse information from observational data into a physics-based model in order to align the model with current physical conditions and improve forecasts [10,22]. Assimilation methods are starting to be implemented into solar physics models to enable a better nowcast and forecast from a wide range of models [7]. For example, [9] utilize the ensemble Kalman filter, along with a tomography method, to reconstruct the Solar corona, [23] also utilized the Kalman filter to forecast the solar cycles, [5] utilizes the four-dimensional variational method (4D-Var) to predict Solar flares, [21] developed a variational data assimilation technique for the Sun using an αΩ dynamo model, and [33] use a three-dimensional variation (3D-Var) data assimilation methodology for a two-dimensional convection flow. These illustrate just a few examples of a wide range of problems from Solar physics that utilize data assimilation methods. Nevertheless, despite the interest in Solar forecasting with data assimilation a reliable forecast of the global photosphere using real magnetic flux observations combined with an underlying physical model remains a novel research area with many open questions.
The Air Force Data Assimilative Photospheric Flux Transport (ADAPT) Model incorporates the various data assimilation techniques, including an ensemble Kalman filter, with a photospheric magnetic flux transport model. The ADAPT model is calibrated with observations by using an ensemble of simulations from the Worden-Harvey model (WH) [37] to represent the distribution of possible Solar photospheric states under the processes of differential rotation, meridional flow, supergranular diffusion, and Gaussian distributed background flux. The ensemble information is then used in the ensemble Kalman filter (EnKF) data assimilation method to adjust the ADAPT model using observational data. The adjustment is made by comparing the covariance structure of the model, as computed from the ensemble samples, with an observation and its observational noise. In areas where the ensemble shows a diffuse distribution, representing lack of determination in the model, observations have a greater impact on the ensemble. Specifically ADAPT implements a localized version of ensemble Kalman filtering, which assimilates each observation within a local region of the model spatial field. This localization allows approximation of the ADAPT spatial covariance structure to effect the assimilation of new observations more effectively. At the same time the localization in the data assimilation eliminates erroneous long distance correlations occurring from the small sample sizes used to estimate the ADAPT model's covariance structure.
The paper is outlined as follows. In section 2 we give a brief description of the WH model and detail the implementation of the data assimilation scheme. This is followed by a comparison of different data assimilation methods available in ADAPT in section 3. The paper concludes, section 4, with a discussion of future directions for the ADAPT model.

Worden-Harvey Model
The Worden-Harvey (WH) photospheric flux transport model includes the processes of differential rotation, meridional flow, supergranulation, and random background flux. It does not model the emergence or disappearance of large scale active regions. These come into our model only through data assimilation of observations. Mechanisms for the creation and destruction of large scale active regions remains an important research topic [8,1,40] but a definitive model has yet to emerge.

Differential Rotation
The rate of rotation of the photospheric flux is dependent on latitude. This phenomenon is referred to as differential rotation. In the WH model the sidereal rotation rate, at latitude θ, is given by ADAPT uses parameters empirically arrived at by Snodgrass [31],

Meridional Flow
Unlike the differential rotation rate the meridional flow, the poleward flow, is more difficult to estimate from observations. This is due to the flow rate being slow enough that it is difficult to distinguish from other transport processes like supergranular diffusion. Moreover, there is evidence of significant meridional flow rate variation over the course of the solar cycle. Thus, there is no generally accepted latitudinal profile for the stationary meridional flow rate. The ADAPT model uses the profile, attributed to [35], implemented in the original WH model given by

Supergranular Diffusion
Supergranulation refers to the horizontal motion of large scale convective active regions. The average lifetime of these active regions is estimated to be between 20 hours and 2 days [30,26] with an average radius of 12, 000-16, 000 km.
ADAPT uses random attractors to simulate the diffusion and transport of super granules. This is implemented by first creating a random attractor matrix equal to the size of the number of pixels in the ADAPT synoptic map. In the attractor matrix each pixel is assigned an attractor value in the interval [1, ∞) by taking the inverse of a uniform random number on [0, 1]. The attractor value is then scaled by cos(θ), with latitude θ, to give a uniform attraction value per unit area [37]. For each pixel a search area is defined using the average radius of a supergranule, in ADAPT this is taken to be 13, 500 km. For each pixel the largest attractor value in the search area is found and the longitudinal and latitudinal differences between the pixel and the pixel associated with the maximum attractor value are calculated. These are then scaled by a Gaussian distributed random variable with mean µ = 1.0 and standard deviation σ = 1 3 0.5. The scaled latitude and longitude differences then determine the transport of the pixel's flux value over a one day period.
Lastly, the above simulation of supergranular diffusion causes too rapid of dissipation for large active regions [25,37]. Thus, the supergranular diffusion process is shut off for field strengths greater than 25 G.

Background Flux
As noted in the original WH model, [30] points out that the photospheric magnetic flux would disappear due to random cancellations in 2-3 days if the total magnetic flux was not renewed regularly. For this reason the ADAPT framework has daily random background flux emergence. This is accomplished by taking Gaussian distributed random flux at each pixel with mean zero and absolute mean value of 1.8 G in each day of the simulation. This value of random flux emergence maintains a constant level of total flux in the synoptic map [37].

LETKF Data Assimilation
The local ensemble transform Kalman filter (LETKF) has been implemented for ADAPT's data assimilation. Our treatment of the LETKF follows the work of Hunt [20,14]. We emphasize the steps in the LETKF and the localization particular to ADAPT.

Transform
In the LETKF we denote each realization of the forecast synoptic map from the ADAPT model as a column vector, x f . Here the pixel values of the synoptic map are taken as entries of the column vector. We denote the ensemble mean of these column vectors byx f . We then form the forecast ensemble matrix, X f , with columns consisting of the discrepancy vector between an ADAPT realization and the ensemble mean, x f −x f . For each pixel in the rows of X f we define a set of local observations, described below. The observation operator is applied to each of the ensemble members to form an ensemble of observations. This ensemble of observations has members, denoted y f , with meanȳ f . We then form the ensemble observation matrix, Y f , with columns consisting of the discrepancy between an ensemble observation and the ensemble observation mean, y f −ȳ f . The transformation in the local ensemble transform Kalman filter is to view the realizations and local observations of the ADAPT model as Gaussian random variables [20]. For These Gaussian random variables preserve the mean and covariance structure of the original ensemble and ensemble observations as sampled from the ADAPT model. Now the LETKF performs data assimilation in ω-space using Y f as the observation operator. After assimilation has been performed it is easy to transform back to the ADAPT ensemble space through multiplication by X f .

Inflation
It is often the case that the photospheric observations fall far enough away from the entire ADAPT ensemble, X f , that the observations are ignored and the simulations diverge from observations [34]. To remedy this problem we implement inflation of the ensemble. This artificially spreads out the ADAPT ensemble to envelope a wider range of possible photospheric maps thereby increasing the likelihood of a portion of the ensemble members being near the observations. We adjust each forecast ensemble member and observation ensemble member using the transformationx 3) for ρ > 0 [22,11]. The data assimilation is then performed using this adjusted set of ADAPT realizations which has the effect of giving more weight to the observations. The larger the inflation factor ρ is chosen the more observations are favored over model forecasts [22,11]. A careful choice of ρ is important to balance the weight given to the observations and ADAPT ensemble. In instances when the inflation is large along with observational noise localized ensemble divergence can still develop, see Figure 2.

Analysis Ensemble
The actual photosphere observations being assimilated will be denoted by y obs and the observational error or noise will be assumed to have covariance matrix R. In ω-space the analysis mean and covariance is then given by the usual Kalman update equations [22,11] Where k is the size of the ADAPT ensemble.
This is transformed to the ADAPT ensemble space using X f as an operator. Thus, the analysis mean and covariance in ensemble space are [20] x a =x f + X fωa (2.7) One then must make a choice for the updated analysis ensemble members, denoted x a . These must have mean and sample covariance satisfying equations (2.7) and (2.8). ADAPT uses the square root filter method to ensure this [6,11,22]. Namely, we set the analysis ensemble in ω-space to be Ω a = [(k − 1)P a ] 1 2 (2.9) so the analysis ensemble matrix in ADAPT becomes X a = X f Ω a . The individual analysis ensemble members, x (i) a , are then formed using each column of Ω a , x (i) a =x a + X f Ω (i) a , i = 1, 2, . . . , k. (2.10) The square root in (2.9) is the symmetric square root obtained through the singular value decomposition of (k − 1)P a . This preserves continuity during localization [20,14,6,34].

Photosphere Localization
In the above analysis scheme it is possible to perform the data assimilation one pixel at a time by taking the forecast ensemble matrix X f to be a row vector of the ensemble discrepancies at a single pixel. One can then iterate over all the pixels in the ADAPT forecast to generate an analysis ensemble.
For the assimilation of an individual pixel one can either use the full observations to form the observation ensemble discrepancy matrix, Y f , or only the part of the observations that are highly correlated to that pixel's value [19,13,36,28,27]. Only including the observations highly correlated with the pixel value being analyzed reduces the effect of spurious correlations that arise in the observation ensemble due to the small ensemble size [19,13,36,28,27].
In ADAPT the effects of differential rotation and meridional flow for the photospheric flux give a natural local region centered on each pixel that is highly correlated with that pixel's current value. This region has an ellipsoid shape with axes aligned with the Solar longitude and latitude. Due to the combined effects of differential rotation and meridional flow the local ellipse is taken to have a constant radius in the latitudinal direction and a longitudinal radius that decreases away from the equator.
To describe the local observation region let the (i, j) th synoptic pixel value of an ensemble member be denoted by x ij f . The forecast ensemble matrix for this pixel is the row vector X ij f made up of the different ensemble members (i, j) th pixel values minus their average. The (i, j) th pixel has a corresponding Solar latitude and longitude (θ i , φ j ) ∈ − π 2 , π 2 × [0, 2π). For each synoptic pixel value we define a local region of observation, O ij , based on the location (θ i , φ j ). During the analysis computation any pixel locations falling inside O ij contribute to the columns of the local observation ensemble matrix, Y loc f . Any observations with locations in O ij make up the local observation vector, y loc obs , for the (i, j) th pixel. Now equations (2.5)-(2.9) can be used with the localized ensemble and observations to compute the analysis ensemble for the (i, j) th pixel value.
ADAPT sets the local observation region, O ij , to an ellipse with its major and minor axes aligned with latitude and longitude. The ellipses' latitudinal radius is fixed at r θ = π 45 = 4 • and the longitudinal radius, r φ (θ), is dependent on the latitude from the Solar equator. Since differential rotation causes correlations over longer longitudinal distances near the equator r φ (θ) is set to reach its maximum at θ = 0 and decrease linearly as the latitude approaches the poles. We set

Observation Covariance
The observational error covariance matrix R is specified through the synoptic observations [16,17,18]. However, only the observation standard deviation at each pixel is given so we assume that R is diagonal and the observational noise is not spatially correlated. This observational error is greater near the Solar limbs and the poles where a direct line of site is not available for observations.

Data Assimilation Comparison
The main difference between the LETKF data assimilation implementation and the older [3,4,2] ENLS data assimilation schemes used is how much the ADAPT ensemble is adjusted to agree with the observations. With the ENLS the spatial correlation structure of the ADAPT ensemble is not taken into account. This causes observations to be trusted far more than the ADAPT model forecast and therefore the ADAPT forecast is nearly discarded during the ENLS assimilation. In sections of the observation region near the central meridian, where observational noise is low, this can be acceptable. However, near the limbs of the observation region noise is considerable and discarding the model forecast is not desirable.
Though inclusion of some of the spatial covariance structure in the ADAPT assimilation step, through the use of Kalman filtering techniques, has advantages we will show that a pure implementation of the ensemble transform Kalman filter (ETKF) has many drawbacks due to spurious correlations introduced through small ensemble sample size. Localization of the ETKF alleviates these spurious correlations and provides a useful compromise between the ENLS and ETKF methods. We will show how ADAPT with the standard ensemble transform Kalman filter restricts the variance away from observations too much, severely reducing the variance in the ADAPT ensemble. This causes ensemble collapse which effectively eliminates the assimilation of observations.
To evaluate performance of multiple data assimilation methods researchers often use a root mean square error (RMSE) approach [14]. The RMSE is calculated by taking the squared difference of the mean ensemble value and the current observation at each pixel. These squared differences are then averaged over the observation region and the square root of the result is the RMSE. A comparison of the RMSE time series for different ADAPT data assimilation schemes is shown in Figure 1. One can see that one method does not outperform the others, in terms of RMSE, 100% of the time. However, RMSE does not account for how much one data assimilation method preserves the physical model after adjustment. We argue, below, that this is the main advantage of using the LETKF for ADAPT's data assimilation.

ETKF vs. LETKF
In situations, such as Solar photosphere models, where the dimension of the simulation state space is high small ensemble size will give rise to spurious correlations [19,13,36,28,27].
In the case of the Solar photosphere these occur over long distances and thus severely restrict the analysis ensemble's pixel-wise standard deviation both near and far away from observations, see Figure 2.
On the other hand the local ensemble transform Kalman filter (LETKF) only compares each pixel's observation with a model ensemble of pixels nearby, as described above. This eliminates the propagation of strong correlations over long distances due to the small ensemble size [19,13,36,28,27]. We can see, in Figure 2, that the pixel-wise standard deviation is only severely restricted in the interior of the observation region.
The main effect of the variance reduction, in terms of accuracy of the data assimilation, is how much the observations are taken into account when adjusting the photosphere ensemble. The contrast is highlighted by observing one assimilation step for a large active region using the two methods, Figure 3. The mean shape of the active region is almost unaffected by the observations for the ETKF but is noticeably influenced by observations when the LETKF is utilized.

ENLS vs. LETKF
The ensemble least squares (ENLS) data assimilation that ADAPT has used in the past suffers from the opposite problems that hinder the ETKF. With the ENLS observations are assimilated into the ADAPT ensemble pixel by pixel without taking sampled spatial correlations into account. Only pixel-wise standard deviations are considered, resulting in local distortion of coherent structures, such as large active regions, in the photospheric magnetic flux present in the ensemble. This is due to noise in magnetic flux observations that is not spatially correlated and therefore reduces spatial correlations in the observation.
Overall the result of the ENLS data assimilation is to assign a much greater weight to the observations than the model. This reduces the information gained by including the Worden-Harvey model for photospheric flux transport. By observing one assimilation step, for the same active region portrayed in Figure 3, we can see how the ENLS favors the observed magnetic flux more than the ADAPT ensemble model structure. In Figure  4 the ENLS data assimilation step drastically modifies the shape of the active region in its analysis ensemble whereas the LETKF blends the information from the Worden-Harvey model and the observations, maintaining the structure of the active region.  Figure 3: Comparison of ETKF and LETKF assimilation effects: The top row represents one step of the ETKF assimilation. Bottom row is one step of the LETKF assimilation. The first column is the mean forecast, second column is the observation, and the last column is the analyzed ADAPT ensemble mean. We can see that the ETKF essentially ignores the observation while the LETKF blends the observation and forecast ensemble.

Discussion
The ADAPT photospheric forecasting capability continues to improve. This will lead to more timely boundary conditions for Coronal and Solar wind models which drive near Earth space weather forecasting. The data assimilation portion of the ADAPT framework now has the ability to preserve ensemble variance near observations as well as far from observations. This enables a more realistic probable range of predictions. However, to notice this improvement one must be sure to evaluate the structure of the entire ensemble forecast as opposed to only comparing the ensemble mean with observed data.
The current ADAPT framework, using the LETKF, does a noticeably better job at balancing the spatial propagation of information away from the point of observation. Ensemble Kalman filtering, as opposed to ensemble least squares filtering, also preserves the variance in the ensemble near data much more. This allows for a more diffuse model forecast in regions where observations have not yet been made which increases the chance of the ensemble range capturing the true photospheric flux in these regions.
At the same time, a problem in photosphere forecasting noted previously [25,39] is loss of balance in magnetic flux when assimilating large solar active regions on the boundary of the observation region. When this happens the observations only observe one polarity  Figure 4: Comparison of ENLS and LETKF assimilation effects: Top row represents one assimilation step using ENLS while the bottom row utilizes LETKF. The columns are the same as in Figure 3. The ENLS algorithm performs a pixel-by-pixel assimilation and therefore ignores spatial correlations in the ensemble. This leads to discarding the ADAPT ensemble forecast in favor of the observations. Since the LETKF takes into account local spatial correlations large structures, such as this active region, are preserved more.
of what should be a coupled polarity active region. Since the WH model does not include active region creation, for emerging active regions, the ADAPT ensemble can not have members that include the opposite polarity region outside of the observation domain. If a robust mathematical/statistical model existed for active region generation allowing ADAPT to generate its own active regions would alleviate this conflict. Since such a model has yet to be created another option is to incorporate preservation of local magnetic flux balances into the ADAPT assimilation scheme. Such balances would incorporate knowledge of Solar active region structure [39].
A further improvement that we plan on pursuing for ADAPT is to incorporate smooth spatial damping of correlations in the local data assimilation regions as observations become farther from the pixel being analyzed [12]. In the current LETKF implementation, observations on the boundary of the local ellipse have the same weight as observations over the pixel being analyzed. This is known to cause discontinuities along the edge of the local data assimilation region [20,38]. By adding a distance dependent weighting to the observations within the local ellipse this problem can be eliminated.