{'fraction_non_alphanumeric': 0.03542128271153961, 'fraction_numerical': 0.014551769567039942, 'mean_word_length': 4.537887580832366, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 18, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
PCA Rerandomization

Mahalanobis distance between treatment group and control group covariate means is often adopted as a balance criterion when implementing a rerandomization strategy. However, this criterion may not work well for high-dimensional cases because it balances all orthogonalized covariates equally. Here, we propose leveraging principal component analysis (PCA) to identify proper subspaces in which Mahalanobis distance should be calculated. Not only can PCA effectively reduce the dimensionality for high-dimensional cases while capturing most of the information in the covariates, but it also provides computational simplicity by focusing on the top orthogonal components. We show that our PCA rerandomization scheme has desirable theoretical properties on balancing covariates and thereby on improving the estimation of average treatment effects. We also show that this conclusion is supported by numerical studies using both simulated and real examples.


Introduction
Randomized experiments have long been regarded as the gold standard to measure the effect of an intervention, because randomization can reduce the potential bias of estimates by balancing the covariate distributions between treatment groups on average. However, when pure (complete) randomization is implemented in practice, it often yields unbalanced allocations, so that the groups should be rerandomized before the experiment is actually conducted. Although rerandomization has been discussed earlier (Fisher, 1926;Cox, 2009;Worrall, 2010), its formal theoretical framework was not well established until the publication of Morgan and Rubin (2012). Using Mahalanobis distance as the balance criterion for treatment-control experiments, rerandomization was shown to improve the covariate balance and the precision of estimated treatment effects. Following the work of Morgan and Rubin (2012), effort has been made to extend or modify such rerandomization schemes. For example, Morgan and Rubin (2015) proposed a rerandomization strategy for covariates with different tiers of anticipated importance with respect to the outcome variable. The extension of rerandomization to a 2 K factorial design was developed by Branson et al. (2016) based on a real example with educational data. Zhou et al. (2018) considered rerandomization for experiments with sequentially enrolled units. Li et al. (2018 investigated the asymptotic properties of the standard treatment effect estimator for the treatment-control settings and 2 K factorial designs respectively.  further established asymptotic properties for the combination of rerandmization and regression adjustment. All the aforementioned works use Mahalanobis distance as the balance measure owing to its several appealing characteristics. First, it is invariant to any affine transformation of the original covariates. Second, Morgan and Rubin (2012) showed that, not only can Mahalanobis distance guarantee the unbiasedness of the treatment effect estimator as well as the balance of covariate means between equal-sized treatment and control groups, but it also reduces an equal percent of sampling variance for each covariate and principal component. Apart from rerandomization, Mahalanobis distance is also widely applied in matching methods for observational studies (Rubin, 1973a(Rubin, ,b, 1979(Rubin, , 1980Rosenbaum and Rubin, 1985;Stuart, 2010).
Despite the advantages discussed above, the full-rank Mahalanobis distance may not work well for rerandomization with the high-dimensional data (Branson and Shao, 2021).
Because it is difficult to equally balance a large number of principal components with different magnitudes of sampling variances. In related work, Morgan and Rubin (2015) proposed balancing the covariates hierarchically using prespecified tiers of importance of covariates related to the outcome. Branson and Shao (2021) pointed out that it might be difficult to specify the relative importance for a large number of covariates a priori. They proposed including a ridge term in Mahalanobis distance, which they showed puts more emphasis on the top principal components. However, the ridge rerandomization of Branson and Shao (2021) relies on complicated Monte Carlo integration and constraint optimization to determine the value of the ridging parameter. Rather than using Mahalanobis distance, Johansson and Schultzberg (2020) proposed a different rank-based balance measure for rerandomization, but their heuristic metric is designed for longitudinal data where the pre-experimental outcomes are available to estimate the relative importance of each covariate. Moreover, the theoretical properties are have not yet been developed under their proposed balance metric.
Here, we propose using only the top principal components from a principal component analysis (PCA) to calculate Mahalanobis distance in the associated subspace and then perform rerandomization. Our PCA rerandomization can be viewed as using a lower-dimensional alternative to the full-rank Mahalanobis distance. We show that because PCA rerandom-ization only reduces the variance of the selected top components, it thereby imposes more shrinkage on them relative to full-rank rerandomization given the same acceptance probability. Moreover, the lower dimensional orthogonality of top principal components simplifies the covariance matrix into a diagonal matrix and thus improves the computational efficiency when calculating Mahalanobis distance.
We establish theory for PCA rerandomization, including the sampling distribution of the modified balance criterion and the variance reduction properties for the standard treatment effect estimator and covariate mean differences compared with the complete randomization.
Practically, despite using PCA, our method is as easy to implement as the original rerandomization without cumbersome parameter specification or increased computation involved in the ridge rerandomization.
In Section 2, we review the rerandomization framework based on Mahalanobis distance (Morgan and Rubin, 2012). We present the details of PCA rerandomization and its theoretical properties in Section 3. Section 4 reports the results of numerical experiments, which show the potential desirable performance of our proposed method compared with other randomization schemes. Section 5 concludes with a discussion.

Rerandomization with Mahalanobis Distance
Let X = (x 1 , . . . , x n ) ∈ R n×d be the fixed covariate matrix representing n trial participants with x i ∈ R d . For simplicity, assume that all x i 's are standardized to have zero mean and unit variance: X 1 n = 0, where 1 n ∈ R n is the n-component vector with all components equal to 1. We focus here on treatment-versus-control experiments, and define W = (W 1 , . . . , W n ) ∈ {0, 1} n to be the random indicator vector of allocations, where W i = 1 means the ith unit is assigned to the treatment group, whereas W i = 0 means the ith unit is assigned to the control group. In the theoretical developments, for simplicity, we further impose the constraint on W such that treatment and control groups are the same size, (1 − W i ) = n/2. (2.1) Given W and X, letx which denote the mean vector of the covariates of the treatment and control groups respectively. According to the potential outcomes framework (Neyman, 1923;Rubin, 1974;Imbens and Rubin, 2015), each unit is associated with two potential outcomes y i (1) or y i (0) corresponding to the situation where the unit is assigned to the treatment group or the control group. Only one outcome can be observed after the allocation, which can be written as . The standard goal of causal inference is to estimate the sample average treatment effect (SATE), The standard simple estimator of SATE is the mean difference of the observed outcomes between the two observed groups, where y = (y 1 , . . . , y n ) . In randomized experiments, more balanced covariates across treatment and control groups generally lead to a more precise estimatorτ . Traditional pure (or complete) randomization only balances covariates between the treatment and control groups on average, and thus a particular realized allocation can be unbalanced, thereby adversely affecting statistical inference. Morgan and Rubin (2012) formally established the rerandomization framework using Mahalanobis distance, M , as the balance measure, where and Σ = cov(x T −x C |X) = 4cov(X)/n is the covariance matrix ofx T −x C with respect to all W satisfying condition (2.1), and cov(X) = X X/(n − 1) is the sample covariance matrix of X. When Σ is singular, the pseudo-inverse is applied to calculate the distance. Rather than performing one single randomization, basic rerandomization continues generating feasible allocation vectors W until the corresponding M is smaller than a predefined threshold a where v a = P (χ 2 d+2 ≤ a)/P (χ 2 d ≤ a) ∈ (0, 1). Similarly, if there exists a linear relationship between the outcome y and covariate x, the variance reduction for the estimated treat- where R 2 is the multiple squared correlation between y and x.
Branson and Shao (2021) developed ridge rerandomization by including a ridge term when calculating Mahalanobis distance, where I d denotes the d-dimensional identity matrix; they showed that such ridging automatically assigns more weight to the top components and can provide better balance in highdimensional/high-collinearity cases. We propose using the top principal components rather than original covariates (i.e., all principal components) to perform the rerandomization. Our lower-dimensional method inherits most of the theoretical properties of rerandomization and can be more convenient to implement without possibly cumbersome specifications of tuning parameters as in the ridge rerandomization.

Rerandomization using Principal Components
Define X = UDV to be the singular value decomposition of X, where U ∈ R n×p and V ∈ R d×p correspond to the matrices of the left and right singular vectors, with U U = V V = I p and p = min{n, d}; D = diag{σ 1 , . . . , σ p } is a diagonal matrix composed of nonnegative singular (i.e., eigen) values σ 1 ≥ · · · ≥ σ p > 0. Here, we focus on the common case is the first k elements of the ith row of Z, and U k ∈ R n×k is the first k columns We calculate Mahalanobis distance based on the top k principal components, and Σ z = C n Z k Z k = C n D 2 k with C n = 4/(n 2 − n). For selecting a treatment assignment, we proceed by generating W until the criterion M k ≤ a k is reached, which is referred to as PCA rerandomization, or more precisely PCA-k rerandomization.
For a particular randomized allocation, letz T,j andz C,j be the mean values of the jth principal component for the treatment and control groups respectively. Let s 2 j denote the sample variance of the jth component. It can be shown that which is the sum of standardized mean difference of the top k principal components. Note that Z can be obtained from the affine transformation on covariates Z = XV. Following the affinely invariant property of Mahalanobis distance, it can be shown that where the former is the original Mahalanobis distance and the latter corresponds to the ridge rerandomization. Therefore, PCA rerandomization is a truncated version of the original Furthermore, M λ is a weighted version of M , with weight σ 2 j /(σ 2 j + λ/C n ) attached to all components, i.e., there is no dimension reduction. Therefore, ridge rerandomization can be viewed as a smooth counterpart to PCA rerandomization, which uses a binary weight, Although there are two tuning parameters for PCA rerandomization, they are easy to specify. As in the routine PCA, the number of top components k can be determined using the percent of cumulative variation that is explained by the selected principal components, where γ k ∈ (0, 1) is a prespecified constant. Given k, we determine the threshold a k via the acceptance probability p a k analogous to the full-rank rerandomization, where the distribution of M k |X is χ 2 k .

Theoretical Properties of PCA Rerandomization
Given k and a k , several statistical properties can be derived for the PCA rerandomization, and we defer the corresponding technical details to the Appendix. PCA rerandomization balances the covariates between the treatment and control groups on average, and additionally leads to the unbiased estimation of τ .
Theorem 1. Given a constant a k > 0, According to the definition of M k , it has the same value for both allocations W and 1 n − W for any given threshold a k > 0. Furthermore, we assume that equation (2.1) holds in PCA rerandomization. Therefore, the unbiasedness forx T −x C andτ follows Theorem 2.1 and Corollary 2.2 in Morgan and Rubin (2012).
The characteristic of covariate balance in Theorem 1 can be further extended to the unobserved covariates, as implied by Corollary 2.2 of Morgan and Rubin (2012). In addition to removing the conditional bias, PCA rerandomization tends to make the difference of covariate means andτ more concentrated. Before showing such results, we first provide the Theorem 2. Let k denote the number of selected top principal components.
Under PCA rerandomization, M k thus follows a chi-squared distribution with the degree of freedom being the same as the number of selected top principal components. The original rerandomization yields M |X ∼ χ 2 d , because it uses all d components. One major application of Theorem 2 is to specify the threshold a k through the probability p a k such that p a k = P (χ 2 k ≤ a k ). Smaller degree of freedom yields a smaller threshold given the same acceptance probability p a = p a k , so we have a k < a for PCA rerandomization relative to the threshold in pure rerandomization. Moreover, it is easier for our method to determine a k than ridge rerandomizaiton, which has to specify the threshold from a mixture distribution. Given the distribution of M k and the threshold a k , we can quantify how much variance is reduced for the covariance matrix ofx T −x C through the following theorem.
Theorem 3. Given the top k principal components and the threshold Theorem 3 states the obvious fact that PCA rerandomization only reduces the variation of top k selected principal components, for which the percent reduction in variance (PRV) is This strategy automatically identifies and balances the most variable subspace of original covariates. Furthermore, we find that v a k in PCA rerandomization is at most its rerandomization counterpart v a given the same acceptance probability.
From Morgan and Rubin (2012), the covariance reduction for rerandomization is Therefore, smaller v a k means that we reduce more variance along the top principal axes than pure rerandomization. In contrast to ridge rerandomization, our scheme only reduces the same percent of variance for the selected k components, whereas ridge rerandomization reduces for all components but with different percents, with top components receiving more where 0 < ξ λ,1 ≤ · · · ≤ ξ λ,d are constants defined in Theorem 4.2 of Branson and Shao (2021) for a given λ. The aforementioned discussion focuses on the PRV on principal components, whereas the following corollary further provides the PRV for the jth entry of the covariate mean differencex T,j −x C,j under PCA rerandomization.
Corollary 1. Given the conditions in Theorem 3 and the (j, j)th element of Σ, Σ jj > 0, To assess how PCA rerandomization can improve the estimation of τ , we follow Morgan and Rubin (2012) to assume that the treatment effect is additive: where β 0 + x i β is the projection of the outcome y i onto the subspace spanned by (1, X), and ε i is the residual of y i orthogonal to the linear subspace spanned by X. We further assume thatτ , the estimator of τ , follows a normal distribution given X, as in Morgan and Rubin (2012).
Theorem 4. Given the conditions in Theorem 3, if the data arise from (3.4) andτ is normally distributed given X, then for any β ∈ R d , This theorem shows that PCA rerandomization never harms the precision of the treatment effect estimatorτ relative to pure rerandomization, although it requires an additional constraint β ∈ C to guarantee a strict sampling variance reduction. Note that each column of V corresponds to a direction of the principal component. The constraint intuitively means that the coefficients β should not fall into the subspace spanned by the last d − k components. As shown in the proof in the Appendix, the sampling variance reduction ofτ is related to that of covariates. In particular, we have for all rerandomization schemes, Therefore, the variance reduction ofτ is governed by all principal components for pure rerandomization and ridge rerandomization due to the fact that cov( 4 Numerical Studies

Simulation Settings
Our Monte Carlo study can be compactly described as a 4 × 4 × 3 × 3 × 2 × 2 × 2 factorial design. The factors can be divided into two categories at the design stage of causal inference following the setups in Rubin (1979) and Gutman and Rubin (2013. The first type of factor includes the characteristics of covariate distributions, sample size and rerandomization schemes, which are explicitly known to the investigator or can be estimated without using the outcome data, and thus are essentially known to the investigator at the design stage. The second type of factor contains the information about the response surfaces including the variance of residuals, which are empirically inestimable without outcome data, and therefore generally unknown at the design stage. are known factors at the design stage, whereas the correlation ρ is an estimable factor that is also essentially known to the investigator. Half of the n units are assigned to the treatment group and the other half to the control group, where the allocation used is generated by one of the three randomization schemes being compared. Given the covariates and allocation variable, the outcome is simulated from the following model with a fixed additive treatment effect τ , where g(x, β) represents the function of response surface that is unknown to the investigator but estimable to some extent, and the residual follows a normal distribution ε ∼ N (0, σ 2 ε ), also unknown to the investigator. We adopt two different functions for the response surface: g(x, β) ∈ {x β, exp(x) β} with β ∈ 1 d , (1 d/2 , 2 × 1 d/2 ) . The residual variance σ 2 ε takes values from {0.5, 1}, and we set τ = 1. All factors with their levels and descriptions are summarized in Table 1.
We compare PCA rerandomization (PCAReR) with three other randomization schemes: (a) complete (pure) randomization (CR) with the constraint (2.1), (b) original rerandomization (ReR), and (c) ridge rerandomization (RidgeReR). For fair comparisons, we set the same acceptance probability for all rerandomization-based methods, i.e., p a k = p a λ = p a = 0.05, corresponding to PCAReR, RidgeReR and ReR. For PCAReR, we set γ k = 0.95 in (3.2) to specify the number of top principal components to be selected. The optimal ridge coefficient λ is determined by the procedure given in Branson and Shao (2021).
All four randomization methods are evaluated using three criteria: the final covariate balance, the true estimation precision of an additive treatment effect, and computational time. Because all approaches produce balanced covariates on average, we adopt the average variance of the mean differencex T,j −x C,j across all covariates j = 1, . . . , d, denoted byσ 2 , to evaluate the empirical balance. For the treatment effect, we choose the mean squared error (MSE) ofτ as the evaluation metric. To normalize these metrics, we compute the reduction percents ofσ 2 and MSE for each rerandomization scheme relative to CR, which are denoted by rσ2 and r MSE respectively. We also record the computational time for generating a feasible

Comparisons of Rerandomization Schemes
Following Rubin (1979) and Gutman and Rubin (2013), we begin with three separate ANOVAs based on the seven-factor design to identify the most influential factors in terms of three major evaluation metrics. In the ANOVA, we consider the main effects of the factors as well as all of their interactions. The relative importance of factors and their interactions are evaluated by their F -ratios or, equivalently, the ANOVA mean squares. Table 2 presents the ANOVA results of rσ2 for known factors, where we find that the covariate dimension, d, and correlation, ρ, strongly influence the covariate balance metric, and the first three factors (d, scheme and ρ) account for around 82% of the total sum of squares. We only report the top 11 factorial effects because the rest have mean squares much smaller than the mean squares for the top 11. Furthermore, the within-configuration mean square indicates small variability of the metric due to random sampling in the data generation. The ANOVA results of r MSE are reported in Table 3, where we only display the top 20 influential factorial factors based on the F -ratios. We observe that the covariate dimension, d, the correlation coefficient, ρ, and the type of response surface, g(x, β), are the most important factors in addition to the scheme, and the residuals also have relatively small mean square. Additionally, the first four factors explain around 72% of the total sum of squares. From Table 4, we can identify, the scheme, d and n as the most influential factors for the computational time, and about 68% of the total sum of squares can be explained by the first five factorial effects. After identifying the crucial factors, we take the average of each metric over the levels of all the remaining factors to provide a comprehensive characterization for the effects of the selected factors.
The first panel of Table 5 compares different rerandomization schemes in terms of rσ2 with respect to ρ and d. To indicate the degree of dimension reduction by PCA, we also note the number of selected top components in parentheses. In general, although PCAReR uses a smaller number of principal components, it can result in more balanced covariates than ReR and the improvement is more obvious in the cases with large ρ and d. The main reason is that the variance shrinkage coefficient v a k of PCAReR is typically smaller than the counterpart v a of ReR especially for large ρ and d, as revealed in Figure 1. Compared with RidgeReR, which takes all principal components into account and puts more emphasis on the top ones, PCAReR only uses the most variable subspace to perform the rerandomization, and thus its performance is inferior to a certain extent. The difference in rσ2 between RidgeReR and PCAReR is small when k is close to d, for example, when k = 9.9 or 43.2. x β, because these approaches only balance the first moments of covariates. Additionally, all three rerandomization schemes perform better for small d given the same ρ and g(x, β), because it is more difficult to simultaneously balance numerous covariates.
As shown by Theorem 4 in this paper, and Theorem 4.3 in Branson and Shao (2021), the estimation precision ofτ for PCAReR and RidgeReR involves the transformed coefficientβ = V β. Therefore, it is expected a superior performance of PCAReR relative to RidgeReR, when the values ofβ corresponding to the top components are smaller than those of the tail components among the selected k principal components by PCA, such asβ = (1, 1 + 2, . . . , k i=1 i, 0 d−k ) . We demonstrate this phenomenon through a simulation by setting the coefficient as β = Vβ under a linear response surface g(x, β) = x β with sample size n = 200 and the residual variance σ 2 ε = 1. The factors d and ρ are kept at the same values as given in Table 1. The last panel of Table 5 presents the percent reduction r MSE .
We still observe that PCAReR works better than ReR, because PCAReR tends to yield more balanced covariates in the selected subspace. This example shows that PCAReR may outperform RidgeReR if the subspace spanned by the tail components among the k selected ones is associated with larger values of regression coefficients. ReR much longer than other cases with n d, to generate an allocation. This is due to the long rejection period for a given threshold a, because it is more difficult to balance a large number of covariates. Note that a is determined under a multivariate normal assumption that only holds asymptotically, and the assumption may not be satisfied when d is close to n, so the given threshold level a may be too restrictive for ReR. This result also reflects the practical advantage of using PCA to accelerate the search for a feasible allocation.
In Table 6, ReR displays some unusual results when d > n, i.e., (n, d) = (100, 180). In such a case, Mahalanobis distance of ReR reduces to a constant, where X X − refers to the pseudo-inverse of X X and U ∈ R n×n is composed of the left singular vectors of X with X X X − X = UU = I n . We further find that the threshold a of ReR is larger than n − 1 for (n, d) = (100, 180), so that ReR is reduced to CR in such situations, which explains the computational times reported in Table 6.

Guideline for Rerandomization
Based on the simulation results, we provide a guideline for the investigator to select a proper scheme under different settings of factors. Tables 2-4 show that d and ρ are the most influential factors in terms of the F -ratios. Given the known factors d and n and estimable factor ρ, the investigator may choose a suitable rerandomization scheme as follows. When both d and ρ are small, all of the three rerandomization schemes show similar performance in terms of rσ2 and r MSE . When either d or ρ is large, PCAReR and RidgeReR should be considered because they leverage the importance of principal components to improve covariate balance and thus estimation precision. Note that β and g(x, β) are unknown to the investigator during the design stage, and Table 5 implies that both methods could yield better performance than ReR. We hence recommend using PCAReR because the fast implementation and sufficient dimension reduction of PCAReR allows the investigator to make prompt attempts on expanding the dimension of the covariate matrix by incorporating different combinations of nonlinear features (e.g., second moments of covariates), and may boost its performance using nonlinear response surfaces.

Real Application
For illustration, we compare different rerandomization methods using a real dataset. We use the first part of the data obtained from the Infant Health and Development Program (IHDP) (Hill, 2011;Louizos et al., 2017). This program aimed at improving cognitive developtment for the low-birth-weight and premature infants by providing high-quality child care and home visits from trained specialists for the infants in the treatment group. This intervention successfully promoted cognitive test scores for the treated children compared with the control group. The dataset consists of 747 participants with 25 covariates and the continuous test scores. Six covariates are continuous which are standardized, and the others are binary. To mimic the real process, we use all observed data to fit a sparse linear model via LASSO using 25 main effect covariates and all 25 2 = 300 two-way interactions among variables. The LASSO penalty is chosen to be 0.055 from cross-validation, and 35 covariates are finally selected. The fitted model is used to generate responses given the allocation during randomization. We construct the covariate matrix X ∈ R 747×325 by all instances, and plan to assign an approximately equal number of units to the treatment (373) and control (374) groups. Different randomization approaches are then applied to generate 1000 independent allocations, from which we calculate various empirical metrics.
We first draw the box plot of the mean differencex T,j −x C,j for continuous features (X 1 , . . . , X 6 ) and first six binary variables (X 7 , . . . , X 12 ) in Figure 2(a); the other binary covariates demonstrate similar balance properties as the first six ones. We observe that PCAReR using 81 principal components achieves more balanced covariates than CR and its performance is comparable to ReR. We further present the density plot of the treatment effect estimatorτ with respect to 1000 replications for ReR, RidgeReR and PCAReR in Figure 2(b), where the distribution ofτ under PCAReR is more concentrated than that under ReR. Table 7 reports the overall reduction percentages for covariate mean differences and the MSE ofτ with respect to CR, where PCAReR yields performance between ReR and RidgeReR, but PCAReR runs 159 times faster than RidgeReR, which takes around 7.9 hours for a total of 1000 allocations, which could be an issue if inferences were to be based on randomization tests.

Discussion
We propose a PCA rerandomization scheme to allocate participants in experiments where there are high-dimensional or high-collinearity covariates. Compared with the classical rerandomization (Morgan and Rubin, 2012), the major difference is that we use the top principal components rather than the original covariates to calculate Mahalanobis distance. We show that not only does PCA rerandomization share most theoretical characteristics with the sim-ple rerandomization, but it also demonstrates empirical advantages with high-dimensional or highly correlated covariates in terms of covariate balance, the precision of treatment effect estimation as well as computational time.
Analogous to rerandomization, we can extend PCA rerandomization to other types of experiments. A line of future work is to develop the asymptotic property of the treatment effect estimator under PCA rerandomization as done in Li et al. (2018) for rerandomization.
Another path worth exploring is to apply PCA rerandomization to 2 K factorial designs following Branson et al. (2016).

A Appendix
A.1 Proof of Theorem 2 Proof. From (2.2), we havē Furthermore, it is easy to see that k . This concludes that M k |X ∼ χ 2 k according to the property of a multivariate normal distribution.
Note that we have Let a d = b denote two variables a and b having the same distribution. First, one can obtain that E(η j |X, M k ≤ a k ) = 0 through the symmetry of a normal distribution, i.e., −η j d = η j ∼ N (0, C n σ 2 j ), ∀j ∈ [k] (Branson and Shao, 2021). Specifically, which leads to E(η j |X, M k ≤ a k ) = 0. Now, we focus on the covariance, cov(η i η j |X, M k ≤ a k ) with i = j and i, j ∈ [k]. Similar to (A.4), we only need to flip the sign of η j to show On the other hand, η 2 j /(C n σ 2 j ) are independent and identically follows χ 2 1 for j ∈ [k].
Combining the above results, the variance of where the last equation follows the exchangeability of i.i.d. {η 2 j /(C n σ 2 j )} j∈ [k] . Because ing the proof of Theorem 3.1 in Morgan and Rubin (2012). Therefore, the variance is It is evident that for j = k + 1, . . . , d, we have Finally, we show cov(η i η j |X, M k ≤ a k ) = 0 when i ∈ [k] and j = k + 1, . . . , d as follows, Considering all the aforementioned results of the variance and covariance, we have The theorem can be proved after plugging the above equation into (A.3).

A.3 Proof of Corollary 1
Proof. According to the definition of PRV, we can obtain the expression of v a k,j as the ratio of the jth diagonal entry of the covariance matrix for PCAReR and Σ; that is, Here, we show that v a k ,j ∈ (0, 1).
correspond to the first k and last d − k components. Let α j = (0 j−1 , 1, 0 d−j ) be a d-dimensional vector of 0's except for the jth element taking a value of 1. We have that Since v a k = P (χ 2 k+2 ≤ a k )/P (χ 2 k ≤ a k ) ∈ (0, 1) for any a k > 0. We can conclude that A.4 Proof of Theorem 4 Proof. Defineε T = 2W ε/n andε C = 2(1 − W) ε/n, where ε = (ε 1 , . . . , ε n ) . According to (3.4),τ can be written asτ Leveraging the orthogonality between the first and last terms, we have that Furthermore, the conditional normal assumption onτ andx T −x C leads to the conditional independence betweenx T −x C andε T −ε C , because these two terms are uncorrelated.
Therefore, M k is also conditionally independent ofε T −ε C , and we have that Combining (A.5) and (A.6), it can be shown that where the non-negativity arises from the positive semi-definiteness of the matrix. When β ∈ C, since v a k ∈ (0, 1) and D k is positive definite, one can conclude that var(τ |X) > var(τ |X, M k ≤ a k ). X 1 X 2 X 3 X 4 X 5 X 6 X 7 X 8 X 9 X 10 X 11 X 12    Table 2: ANOVA results of the 11 most influential factors for rσ2, which is the reduction percent of the average empirical variance ofx T,j −x C,j across all covariates (j = 1, . . . , d) relative to complete randomization. DF is the degree of freedom, MS represents mean square, and the last row Residual refers to the within-configuration mean square. Table 3: ANOVA results of the 20 most influential factors based on the ANOVA F -ratio for Table 4: ANOVA results of known or estimable factors for the computational time of each rerandomization scheme in seconds. DF is the degree of freedom, MS represents mean square, and the last row Residual refers to the within-configuration mean square. across all covariates (j = 1, . . . , d) and mean squared error (MSE) ofτ relative to complete randomization, respectively. We report the number k of selected top principal components for PCA rerandomization in the parentheses.