{'fraction_non_alphanumeric': 0.04734160551912435, 'fraction_numerical': 0.011750079227712635, 'mean_word_length': 4.50261569416499, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 2, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Fairness Transferability Subject to Bounded Distribution Shift

Given an algorithmic predictor that is"fair"on some source distribution, will it still be fair on an unknown target distribution that differs from the source within some bound? In this paper, we study the transferability of statistical group fairness for machine learning predictors (i.e., classifiers or regressors) subject to bounded distribution shifts. Such shifts may be introduced by initial training data uncertainties, user adaptation to a deployed predictor, dynamic environments, or the use of pre-trained models in new settings. Herein, we develop a bound that characterizes such transferability, flagging potentially inappropriate deployments of machine learning for socially consequential tasks. We first develop a framework for bounding violations of statistical fairness subject to distribution shift, formulating a generic upper bound for transferred fairness violations as our primary result. We then develop bounds for specific worked examples, focusing on two commonly used fairness definitions (i.e., demographic parity and equalized odds) and two classes of distribution shift (i.e., covariate shift and label shift). Finally, we compare our theoretical bounds to deterministic models of distribution shift and against real-world data, finding that we are able to estimate fairness violation bounds in practice, even when simplifying assumptions are only approximately satisfied.


Introduction
Distribution shift is a common, real-world phenomenon that affects machine learning deployments when the target distribution of examples (features and labels) ultimately encountered by a datadriven policy diverges from the source distribution it was trained for. For socially consequential decisions guided by machine learning, such shifts in the underlying distribution can invalidate fairness guarantees and cause harm by exacerbating social disparities. Unfortunately, distribution shift can be technically difficult or impossible to model at training time (e.g., when depending on complex social dynamics or unrealized world events). Nonetheless, we still wish to certify the robustness of fairness metrics for a policy on possible target distributions. has typically considered static distributions or prescribed models of distribution shift. Our work is the first to systematically bound quantifiable violations of statistical group fairness while remaining agnostic to (1) the mechanisms responsible for distribution shift, (2) how group-specific distribution shifts are quantified, and (3) the specific statistical definition of group fairness applied.  Figure 1: In Section 7, we evaluate our bounds against historical, temporal distribution shifts in demographics and income recorded by the US Census Bureau [13]. The above figure depicts changes to income-prediction accuracy and demographic parity violation when a classifier initially trained on US state-specific demographic data for 2014 is reused on 2018 data, thus exemplifying the negative potential effects of distribution shift.
Our primary result is a bound on a policy's potential "violation of statistical group fairness"-defined in terms of the differences in policy outcomes between groups-when applied to a target distribution shifted relative to the source distribution within known constraints. Such settings naturally arise whenever training data represents a random sample of a target population with different statistics or a sample from dynamic environments, when a policy is reused on a new distribution without retraining, or whenever policy deployment itself induces a distribution shift. As an example of this last case, strategic individuals seeking loans might change their features or abstain from future application (thus shifting the distribution of examples) in response to policies trained on historical data [18, 38,43]. Beyond policy selection, exogenous pressure such as economic trends and noise may also drive distribution shift in this example.
In Figure 1, we show how a real-world distribution shift in demographic and income data for US states between 2014 and 2018 may increase fairness violations while decreasing accuracy for a hypothetical classifier trained on the 2014 distribution. In such settings, it is useful to quantify how fairness guarantees transfer across distributions shifted within some bound, thus allowing the deployment of unfair machine learning policies to be avoided.

Related Work
Our work considers a setting similar to recent studies of domain adaptation, which have largely focused on characterizing the effects of distribution shift on prediction performance rather than fairness. Our work also builds on efforts in algorithmic fairness, especially dynamical treatments of distribution shift in response to deployed machine learning policies [23, 11,30]. We reference specific prior work in these domains in Appendix B, and here discuss existing work that focuses on how certain measures of fairness are affected when policies are subject to specific distribution shift.
Fairness subject to Distribution Shift: A number of recent studies have considered specific examples of fairness transferability subject to distribution shift [34,9,36,31,21]. In particular, Schumann et al. [34] examine equality of opportunity and equalized odds as definitions of group fairness subject to distribution shifts quantified by an H-divergence function; Coston et al. [9] consider demographic parity subject to a covariate shift assumption while group identification remains unavailable to the classifier; Singh et al. [36] focus on common group fairness definitions for binary classifiers subject to a class of distribution shift that generalizes covariate shift and label shift by preserving some conditional probability between variables; and Rezaei et al. [31] similarly consider common binary classification fairness definitions such as equalized odds subject to covariate shift. While we address similar settings to these works as special cases of our bound, we propose a unifying formulation for a broader class of statistical group fairness definitions and distribution shifts. In doing so, we recognize that particular settings recommend themselves to more natural measures of distribution shift, providing examples in Section 4.1, Section 4.2, and Section 5).
Another thread in existing literature is the development of robust models with the goal of guaranteeing fairness on a modelled target distribution (e.g., [1,32,26,3,21]) , for example, by assuming covariate shift and the availability of some unlabelled target data [9,36,31]. In particular, Singh et al. [36] focus on learning stable models that will preserve prediction accuracy and fairness, utilizing a causal graph to describe anticipated distribution shifts. Rezaei et al.
[31] takes a robust optimization approach, and Coston et al. [9] develops prevalence-constrained and target-fair covariate shift method for getting the robust model. In contrast, our goal is to quantify fairness violations after an adversarial distribution shift for any given policy, including those not trained with robustness in mind.

Our Contributions
Our primary contribution is formulating a general, worst-case upper bound for a given policy's violation of statistical group fairness subject to group-dependent distribution shifts within presupposed bounds (i.e., Equation (9)). Bounding violations of fairness subject to distribution shift allows us to recognize and avoid potentially inappropriate deployments of machine learning when the potential disparities of a prospective policy eclipse a given threshold within bounded distribution shifts of the training distribution.
We first characterize the space of statistical group fairness definitions and possible distribution shifts by appeal to premetric functions (Definition 2.1). After formulating the worst-case upper bound, we explore common sets of simplifying assumptions for this bound as special cases, yielding tractable calculations for several familiar combinations of fairness definitions and subcases of distribution shift (Theorem 4.1, Theorem 5.2) with readily interpretable results. Finally, we compare our theoretical bounds to prescribed models of distribution shift in Section 6 and to real-world data in Section 7. The details for reproducing our experimental results can be found at https://github.com/UCSC-REAL/Fairness_Transferability.

Formulation
The appendices include a table of notation (Appendix A) and all proofs (Appendix F).

Algorithmic Prediction
We consider two distributions, S (source) and T (target), each defined as a probability distribution for examples, where each example defines values for three random variables: X, a feature (e.g., x) with arbitrary domain X ; Y , a label (e.g., y) with arbitrary domain Y; and G, a group (e.g., g or h) with finite, countable domain G. The predictor's policy π, intended for S but used on T , defines a fourth variable for each example: viz.,Ŷ , a predicted label (e.g.,ŷ) with domainŶ = Y.
Using P(·) to denote the space of probability distributions over some domain, we denote the space of distributions over examples as D := P(X × Y × G), such that S, T ∈ D. It will also be useful for us to notate the space of distributions over example outcomes associated with a given policy as O := P(X × Y ×Ŷ) and the space of distributions over of group-specific examples as G := P(X × Y).
Without loss of generality, we allow the prediction policy π to be stochastic, such that, for any combination (x, g), the predictor effectively samplesŶ from a corresponding probability distribution π(x, g). Stochastic classifiers arise in various constrained optimization problems and proven useful for making problems with custom losses or fairness constraints tractable [10,17,29,40].
We denote the space of nondeterministic policies as Π := (X × G → P(Ŷ)) (e.g., π ∈ Π) and utilize the natural transformations that relate the spaces of distributions D, policies Π, and outcomes O: We abuse the Pr notation for both probability density and probability mass functions as appropriate.

Statistical Group-Fairness
We next define a broad class of disparity functions ∆ : Π × D → R representing how "unfair" a given policy is for a given distribution (e.g., writing ∆ (π, T )), noting that this notion of fairness is limited to capturing statistical discrepancies of outcomes between groups. Definition 2.1. We define a premetric 3 Ψ on the space of distributions p with respect to q by the properties Ψ(p q) ≥ 0 and Ψ(p p) = 0 for all p, q, and refer to the value of Ψ as a "shift". Definition 2.2. We define a statistical group disparity ∆ for policy π and distribution T in terms of the symmetrized shifts between group-specific outcome distributions. We measure shifts between outcome distributions with a given premetric Ψ : In Definition 2.2, Ψ quantifies the specific statistical differences in outcomes between groups that are "unfair", where a value of 0 implies perfect fairness. In this work, we assume that Ψ is the same for all g, h and that ∆ is insensitive to relative group size Pr(G). Examples Familiar applications of Definition 2.2 include demographic parity (DP) and equalized odds (EO). A policy satisfying DP, in expectation, assigns a given binary classification y ∈ {0, 1} to the same fraction of examples in each group. We may measure the violation of DP as The associated premetric To satisfy EO, for binary Y = {0, 1}, π must maintain group-invariant true positive and false positive classification rates. We may measure the violation of EO as The associated premetric is Note that the restriction of EO to the (Y = 1) case is known as Equal Opportunity (EOp).
We remark that Definition 2.2 provides a unifying representation for a wide array of statistical group "unfairness" definitions and may be used with inequality constraints. That is, we may recover many working definitions of fairness that effectively specify a maximum value of disparity: Definition 2.3. A policy π is -fair with respect to ∆ on distribution T iff ∆ (π, T ) ≤ .

Vector-Bounded Distribution Shift
Suppose, after developing policy π for distribution S, we realize some new distribution T on which the policy is actually operating. This realization may be the consequence of sampling errors during the learning process, strategic feedback to our policy, random processes, or the reuse of our policy on a new distribution for which retraining is impractical. Our goal is to bound ∆ (π, T ) given knowledge of ∆ (π, S) and some notion of how much T possibly differs from S. Definition 2.4. K(p q) is a divergence if and only if for all p and q, K(p q) ≥ 0 and K(p q) = 0 ⇐⇒ q = p. Definition 2.5. Define the group-vectorized shift D, as S mutates into T , as where e g represents a unit vector indexed by g, and each D g : G 2 → R is a divergence (Definition 2.4). Note that each D g also defines a premetric (but not necessarily a divergence) on D. Assumption 2.6. Let there exist some vector B 0 bounding D(T S) B, where and denote element-wise inequalities.
In Assumption 2.6, B limits the possible distribution shift as S mutates into T , without requiring us to specify a model for how distributions evolve. When modelling distribution shift requires complex dynamics (e.g., when agents learn and respond to classifier policy), we reduce a potentially difficult dynamical problem to a more tractable, adversarial problem to achieve a bound. Lemma 2.7. For all π, ∆ , and D, when B = 0, ∆ (π, S) = ∆ (π, T ).
Lemma 2.7 indicates that, for a fixed policy π, a change in disparity requires a measurable shift in distributions from S to T , confirming intuition. Restricted Distribution Shift Common assumptions that restrict the set of distribution shifts include covariate shift and label shift. For covariate shift, the distribution of labels conditioned on features is preserved across distributions for all groups, while for label shift, the distributions of features conditioned on labels is preserved across distributions for all groups.
Label shift implies Pr In Section 4, we explore a deterministic model of a population's response to classification as an example of covariate shift. We do the same in Section 5 for label shift.

General Bounds
We first define a primary bound in Definition 3.1 before considering simplifying special cases.
Given an element-wise bound B on the vector-valued shift D(T S) (Assumption 2.6) we may bound the disparity ∆ of policy π on any realizable target distribution T by its supremum value.
In general, our strategy is to exploit the mathematical structure of the setting encoded by ∆ (i.e., Ψ) and D to obtain an upper bound for v defined in Equation (8). We first explore general cases of simplifying assumptions before presenting worked special examples for frequently encountered settings. Finally, we compare the resulting theoretical bounds to numerical results and simulations.

Lipshitz Conditions
The value of v defines a scalar field in B and therefore a conservative vector field F = ∇ B v. For any curve in D from S to T , bounds of the form F L for some constant L along the curve imply a Lipshitz bound on ∆ . We visualize a bound in Figure 2 for all possible curves in the region D(T S) B.
Succinctly, if we are guaranteed that disparity can never increase faster than a certain rate in some measure of distribution shift, then, given a maximum distribution shift, this rate bounds the maximum possible disparity. The utility of Theorem 3.2 arises when a Lipshitz condition L is known, but direct computation of v is difficult. We provide an example of a Lipshitz bound in Section 5. Theorem 3.4. Suppose, in the region D(T S) B, that w is subadditive in its last argument. That is, w(..., a) + w(..., c) ≥ w(..., a + c) for a, c 0 and a + c B. If w is also locally differentiable, then a first-order approximation of w(..., b) evaluated at 0, i.e.,

Subadditivity Conditions
provides an upper bound for v(..., Theorem 3.4 notes that "diminishing returns" in the change of ∆ as the difference of T with respect to S is increased implies a bound on ∆ in terms of its local sensitivity to D at S (i.e., using a firstorder Taylor approximation). Note that, if w is concave in the bounded region, it is also subadditive in the bounded region, but the converse is not true, nor does the converse imply Lipshitzness.

Geometric Structure
It may happen that Ψ : O 2 → R and each D g : G 2 → R share structure that permits a geometric interpretation of distribution shift. While the utility of this observation depends on the specific properties of Ψ and D, we demonstrate a worked example building on Section 4.2 in Appendix D, in which we allow ourselves to select a suitable D for ease of interpretation. We proceed to consider worked examples that adopt common assumptions limiting the form of distribution shift and apply common definitions of statistical group fairness.

Covariate Shift
We now present our fairness transferability results subject to covariate shift for both demongraphic parity (Section 4.1) and equalized opportunity (Section 4.2) as fairness criteria.

Demographic Parity
The simplest way to work with Equation (8) is to bound the supremum v. We first consider demographic parity (Equation (3)) for Y = {0, 1} and G = {g, h}, subject to covariate shift (Equation (6)). We find that the form of ∆ DP subject to covariate shift recommends itself to a natural choice of vector divergence, D. First, define a re-weighting coefficient ω g (T , S, x) := Pr T (X=x|G=g) Pr S (X=x|G=g) . Theorem 4.1. For demographic parity between two groups under covariate shift (denoting, for each g, β g := Pr π,S (Ŷ =1 | G=g)), We notice that Var S [ω g (T , S, x)] recommends itself as a suitable divergence D g from S to T . Using basis vectors e g , for this example, we could define D( Comparing the inequality in Theorem 4.1 and the consequent of Equation (10), we can interpret Pr π,S (Ŷ =1) in Theorem 4.1 as an upper bound for the average value of ∇ b v(∆ DP , D, π, T , b) along any curve from S to T . Interpreting this result, the closer Pr(Ŷ =1) is to 0.5 for any group, the more potentially sensitive the fairness of the policy is to distribution shifts for that group. We can further generalize the results to multi-class and multi-group setting: We remark that in general, binary classification bounds may frequently be generalized to multi-class bounds by redefining fairness violations as a sum of binary-class fairness violations (i.e., same-class vs. different-class labels) and summing the bounds on each.

Equal Opportunity
Consider an example using the (Y =1)-conditioned case of Equalized Odds-termed Equal Opportunity (EOp). Denoting, for each group g, the true positive rate β + g := Pr π,T (Ŷ =1 | Y =1, G=g) as an implicit function of π and T , we define disparity for EOp as ∆ EOp (π, T ) := g,h∈G |β + g − β + h |. We may bound the realized value of ∆ EOp (π, T ) by bounding β + g for each group: Theorem 4.3. Subject to covariate shift and any given D, B, assume extremal values for β + g , i.e., In Appendix D, we bound the extremal values of β + g by geometrically interpreting this quantity as an inner product on an appropriate vector space, utilizing the freedom to select an appropriate D.

Label Shift
Under label shift (Pr S (X|Y ) = Pr T (X|Y )), violations of EO and EOp are invariant, because the independence ofŶ and Y given X implies Pr π,T (Ŷ |Y ) = Pr π,S (Ŷ |Y ). We therefore focus on the violation of demographic parity (DP) (Equation (3)) subject to the label shift condition, treating a binary classification task over two groups for simplicity.
In this setting, we choose to measure group-specific distribution shifts from S to T by the change in proportion of ground-truth positive labels, which we refer to as the group qualification rate Specifically, for true positive rates β + g and false positive rates β − g : Because β + g and β − g are invariant under label shift given a constant policy π, we elide their explicit dependence on the underlying distribution. Theorem 5.2. For DP under the bounded label-shift assumption ∀g, Intuitively, the change in ∆ DP subject to label shift depends on |β + g − β − g |, the marginal change in acceptance rates as agents change their qualifications Y . We measure the distribution shift as agents change their qualifications by |Q g (S) − Q g (T )|. When β + g is close to β − g , the policy looks like a random classifier, and a label shift has limited effect on statistical group disparity. When |β + g − β − g | is large, indicating high classifier accuracy, the effect on supremal disparity is larger. Our bound thus exposes a direct trade-off between accuracy and fairness transferability guarantees.

Comparisons to Synthetic Distribution Shifts (Demographic Parity)
To further interpret our results, in this section, we consider specific and popular agent models to characterize distribution shift and instantiate our bounds for particular forms of D, B, and ∆ .

Covariate Shift via Strategic Response
Let us consider a specific example of covariate shift (Equation (6)) caused by a deterministic, groupindependent model of strategic response in which agents react to a binary classification policy π characterized by group-specific feature thresholds: For simplicity, we assume the feature domain X = [0, 1]. In response to threshold τ g , agents in each group g may modify their feature x to x by incurring a cost c g (x, x ) ≥ 0. Similar to [18], we define the utility u g for agents in group g to be Contrary to the standard strategic classification setting, we do not assume that feature updates represent false reports, but that such updates may correspond to actual changes underlying the true qualification Y of each agent. This assumption has been made in a recent line of research in incentivizing improvement from human agents subject to such classification [5].
Next, we assume all agents are rational utility maximizers (Equation (24)). For a given threshold τ g and manipulation budget m g , the best response of an agent with original feature x is To make the problem tractable, we make additional assumptions about the agents' best responses.
Under Assumption 6.2, only those agents with features x ∈ [τ g − m g , τ g ) will attempt to change their feature. We also assume that feature updates are non-deterministic, such that agents with features closer to the decision boundary τ g have a greater chance of updating their feature and each updated feature x is sampled from a uniform distribution depending on τ g , m g , and x: Assumption 6.3. For agents who attempt to update their features, the probability of a successful feature update is Pr(X = X ) = 1 − |x−τg| mg . Assumption 6.4. An agent's updated feature x , given original feature x, manipulation budget m g , and classification boundary τ g , is sampled as With the above setting, we can specify the reweighting coefficient ω g (x) for our setting (Equation (102) in Appendix F.1 and get the following bound for the strategic response setting 5 : Proposition 6.5. For our assumed setting of strategic response involving DP for two groups {g, h}, Theorem 4.1 implies The above result shows that two factors lead to a smaller difference between the source and target fairness violations: a less stochastic classifier (when the threshold τ g is far away from 0.5) and a smaller manipulation budget m g (diminishing agents' ability to adapt their feature). In this case, These factors lead to less potential manipulation and result in a tighter upper bound for the fairness violation on T .

Label Shift via Replicator Dynamics
We now evaluate our theoretical bound for demographic parity subject to label shift (Theorem 5.2) on the replicator dynamics model of Raab and Liu [30]. Briefly, replicator dynamics assumes that the proportion of agents in a population choosing one strategy over another grows in proportion to the ratio of average utilities realized by the two strategies. The cited model additionally assumes X = R, Y = {0, 1}, and a monotonicity condition for S given by d dx Label shift under the discrete-time (t) replicator dynamics may be expressed in terms of group qualification rates Q g := Pr t (Y =1 | G=g) and agent utilities (i.e., group-and feature-independent values U y,ŷ ) such that, in each group, the popularity and average utility associated with a label determines its frequency at the next time step t+1.  Denote the fractions of group-conditioned, feature-independent outcomes with the expression ρ y,ŷ g := Pr t (Ŷ =ŷ, Y =y | G=g) and abbreviate the fraction-weighted utility as u y,ŷ g (t) := U y,ŷ · ρ y,ŷ g . We may then represent the replicator dynamics as To apply Theorem 5.2, we also observe that |β where β + g and β − g represent the true positive rate and false positive rate for group g, respectively, and we use the change in qualification rate as our measurement of label shift, i.e., B g = |Q g [t + 1] − Q g [t]|. When demographic parity is perfectly satisfied, we note that the acceptance rate (ρ 1,1 g + ρ 0,1 g ) is group-independent. Theorem 6.6. For DP subject to label replicator dynamics, In Figure 3(b), we graphically represent all possible states of an initially fair system (thus determining β and ρ as a result of the monotonicity condition) by the tuple of qualification rates for each group. With the dynamics prescribed by Equation (27), we depict the rate of change of disparity given a fixed, locally DP-fair policy, and compare this to the theoretical bound when Interpreting our results, we note that the bound lacks information about the relative directions of the change in acceptance rates for each group, and thus over-approximates possible fairness violations when group acceptance rates shift the same direction. When group acceptance rates move in opposing directions, however, the bound gives excellent agreement with the modelled replicator dynamics.

Comparisons to Real-World Distribution Shifts
We now compare our special-case theoretical bounds (i.e., label/covariate shift) to real-world distribution shifts and hypothetical classifiers. We use American Community Survey (ACS) data provided by the US Census Bureau [16]. We adopt the sampling and pre-processing approaches following the Folktables package provided by Ding et al. [13] 6 to obtain 1,599,229 data points. The data is partitioned by (1) all fifty US states and (2) years from 2014 to 2018. We use 10 features covering the demographic information used in the UCI Adult dataset [4], including age, occupation, education, etc., as X for our model, select sex as binary protected group, i.e., G ∈ {g = female, h = male}.
We set the label Y to whether an individual's annual income is greater than $50K.
To apply our label-shift or covariate-shift bounds, we first need to verify whether the two datasets satisfy either of these assumptions. We adopted a conditional independence test [22], which takes data from source and target domains as input and returns a divergence score for each covariate and label variable, reflecting to what extent the variable is shifted between distributions. We find that the likelihood that the covariates shift across US states is approximately two orders of magnitude higher than for labels. More specifically, there are 4 covariates, including class of worker (probabilistic divergence score of 2.67e-2), hours worker per week (3.56e-2), sex (3.56e-2) and race (2.55e-1), that are more likely to be shifted than the label variable (1.29e-4). For temporal shifts within states, we find that the label variable is more likely to be shifted (0.1) than all the other covariates (which are below 0.01), approximately two orders of magnitude in favor of label shift over covariate shift. We therefore compare the disparities of hypothetical policies on these distributions to bounds generated from the corresponding, approximately satisfied assumptions.
On this data, we train a set of group-dependent, linear threshold classifiers Pr π(x,g) (Ŷ =1) = 1[σ(w · x) > τ g ], for a range of thresholds τ g and τ h for each source distribution. Here, σ(·) is the logistic function and w denotes a weight vector. We then consider two types of real-world distribution shift: (1) geographic, in which a model trained for one state is evaluated on other US state in the same year, and (2) temporal, in which a model trained for 2014 is evaluated on the same state in 2018.  We graphically compare the theoretical bounds of Theorem 4.1 and Theorem 5.2 for the increased violation of DP subject to covariate and label shift, respectively, to the simulated violations for our model and data in Figure 4. We provide additional examples and an evaluation of bounds for EO subject to covariate shift (noting that label shift preserves EO in theory) in Appendix E.2. Despite the fact that geographic or temporal distribution shifts only approximately satisfy the assumptions of covariate or label shift, these comparisons demonstrate that our theoretical bounds are not vacuous, approximately bounding the change of fairness violation across real-world domain shifts. For geographic shifts, the covariate shift EO bounds (Appendix E.2) correctly overestimate disparity and tighten near accurate policies, while our DP bounds are useful only for a subset of policy thresholds (Figure 4(b)). add specific pointer. e.g, 4.a, that one is 4.b For temporal shift, the label shift bound for DP correctly overestimates the real change of DP violations but still remains at the same order of magnitude (Figure 4(c) and 4(d)).

Conclusion and Discussion
In this paper, we have developed a unifying framework for bounding the violation of statistical group fairness guarantees when the underlying distribution shifts within presupposed bounds. We hope that this work can generate meaningful discussion regarding the viability of fairness guarantees subject to distribution shift, the bounds of adversarial attacks against algorithmic fairness, and evaluations of robustness with respect to algorithmic fairness. We believe that, just as published empirical measurements are of limited use without reported uncertainties, fairness guarantees must be accompanied by bounds on their robustness to distribution shift.
Future work remains to apply our framework for to problem of fairness transferability in settings with more complicated distribution shift dynamics. For example, compound distribution shifts [33], which compose covariate shifts and label shifts, cannot be treated by composing the theoretical bounds developed herein without additional information regarding intermediate distributions. Another potential future direction is to develop reasonable bounds on anticipated distribution shift from models of human behavior and exogenous pressures.

A Notation
Symbol Usage X A random variable representing an example's features. X The domain of features X. Y A random variable representing an example's ground truth label. Y The domain of labels Y . Y A random variable representing the predicted label for an example. Y The domain of predicted labelsŶ (distinguished semantically from Y). G A random variable representing an example's group membership. G The domain for group membership G. π A learned (non-deterministic) policy for predictingŶ from X and G.

Pr
A sample probability (density) according to a referenced distribution. P The space of probability distributions over a given domain. The source distribution in D.

T
The target distribution in D to which π is now applied.   [2,27], or in conjunction with the dynamic response of a population to classification [25]. We are interested in a similar setting and concern, but address the transferability of fairness guarantees, rather than accuracy. In considering covariate shift and label shift as special cases in this paper, our work may be paired with studies that address the transferability of prediction accuracy under such assumptions [35,37,42].
Algorithmic Fairness: Many formulations of fairness have been proposed for the analysis of machine learning policies. When it is appropriate to ignore the specific social and dynamical context of a deployed policy, the statistical regularity of policy outcomes may be considered across individual examples [14] and across groups [41,15,8,19,6]. In our paper, we focus on such statistical definitions of fairness between groups, and develop bounds for demographic parity [6] and equalized odds [19] as specific examples.
Dynamic Modeling: When the dynamical context of a deployed policy must be accounted for, such as when the policy influences control over the future trajectories of a distribution of features and labels, we benefit from modelling how populations respond to classification. Among this line of work, [23] initiate the discussion of the long-term effect of imposing static fairness constraints on a dynamic social system, highlighting the importance of measurement and temporal modeling in the evaluation of fairness criteria. However, developing such models remains a challenging problem [11,36,31,12,43,39,24,7,20,28,30]. In particular, [11] discuss causal directed acyclic graphs (DAGs) as a unifying framework on fairness in dynamical systems. In this work, rather than relying precise models of distribution shift to quantify the transferability of fairness guarantees in dynamical contexts, we assume a bound on the difference between source and target distributions. We thus develop bounds on realized statistical group disparity while remaining agnostic to the specific dynamics of the system.

C Additional Figures
x ω g (x) 0 τ g − B g τ g τ g + B g 1 1 2 Figure 5: Distribution of the reweighting coefficient wg(x) for the setting of Covariate shift via Strategic Response.

D A Geometric Interpretation
In this extension of Section 4.2, we fulfill the promise of Section 3.3 and consider a case in which shared structure of between Ψ : O 2 → R and each D g : G 2 → R permits a geometric interpretation of distribution shift for Equal Opportunity EOp, building on Theorem 4.3. We continue to defer rigorous proof to Appendix F.
We first recall the definition of the true positive rate of policy π, for each group, on distribution T .
The true positive rate may be expressed as a ratio of inner products defined over the space of square-integrable L 2 functions on X . 7 where we use the shorthands and assume that s g (x) > 0 for all x and g.
We observe that the only degree of freedom in β + g as T varies subject to covariate shift is r g : by the covariate assumption, s g is fixed; t meanwhile remains independent of T for fixed policy π, since π is independent of Y conditioned on X and G.
Selection of D We now select each D g to be the standard metric for the inner product defined by Equation (31), where, for each group, distributions in G are mapped to the corresponding vector r g : In this geometric picture, D(T S) B implies that all possible values for r g [T ] lie within a ball of radius B g centered at r g [S]. By the normalization condition of a probablity (density) function, denoting s −1 g (x) := (s g (x)) −1 , the vector r g [T ] must also lie on the hyperplane Recalling Equation (30), the group-specific true positive rate β + g [T ] for policy π is given by a ratio of the projected distances of r g along the t g and 1 vectors. Let us therefore denote the projection of r g [T ] onto the (1, t g )-plane as r ⊥ g [T ]. We may then consider the possible values of r ⊥ g [T ] as projections from the intersection of the r g [S]-centered hypersphere of radius B g and the hyperplane of normalized distributions (Equation (37)). Using ∠(·, ·) to denote the angle between vectors and denoting φ g := ∠(r g , t g ), θ g := ∠(r g , 1), φ g := ∠(r ⊥ g , t g ), and θ g := ∠(r ⊥ g , 1), we appeal to the geometric relationship a, b = cos (∠(a, b)) a b to write From these observations, we need only bound the ratio between cos(φ g ) and cos(θ g ) to bound β + g . Relating these angles in the (1, t g )-plane by φ g = ξ g − θ g where ξ g := ∠(t g , 1), we arrive at the following theorem: Theorem D.1. The true positive rate β + g is bounded over the domain of covariate shift D cov [B], which we define by the bound D(T S) B, and the invariance of Pr(Y =1 | X=x, G=g) for all x, g, as with upper (φ u g ) and lower (φ l g ) bounds for φ g represented as We obtain a final bound on ∆ EOp by substituting Equation (39) into Equation (17). We visualize the geometric bound on β + g (Theorem D.1) in Figure 6. In Appendix E.1, we apply this bound to real-world credit score data assuming the model of strategic manipulation given in Section 6.1. Although the result is not an easily interpretted formula, it provides a demonstration of geometric reasoning applied to statistical fairness guarantees.
Finally, we note that, in addition to the constraints considered above, each vector r g is subject to the positivity condition, ∀x ∈ X , r g (x) ≥ 0. The bound developed in this section, however, does not benefit from this additional constraint; we leave this to potential future work.
Bg (c) Bound -Modelled Shift Figure 9: A policy satisfying DP is subject to distribution shift prescribed by replicator dynamics (Section 6.2). Realized disparity increases (blue) are compared to the theoretical bound (Theorem 5.2, gradated), which is tight when group have dissimilar qualification rates.

E.2 Comparisons to Real-World Data
We provide additional graphics comparing bounds on demographic parity or equal opportunity to real-world distribution shifts. Figure 10 compares the covariate shift bound of Theorem 4.1 to the violation of demographic parity for hypothetical policies trained on one US state and deployed in another. Figure
Proof. By the definitions of group-vectorized shift (Definition 2.5) and divergence (Definition 2.4) together with the bounded distribution shift assumption (Assumption 2.6), we note and Combining these implications and invoking the independence ofŶ ∼ π and Y conditioned on X and G (Equation (1)), it follows that Consulting the definition of disparity (Definition 2.2), it follows that ∆ (π, S) and ∆ (π, T ) are equal when B = 0.

Proof of Theorem 3.2:
Statement: If there exists an L such that ∇ b v(∆ , D, π, S, b) L, everywhere along some curve from 0 to B, then Proof. We reiterate that v(∆ , D, π, S, b) defines a scalar field over the non-negative cone b ∈ (R + ∪ 0) |G| . Treating v as a scalar potential, we may define the conservative vector field F: provides an upper bound for v(..., B): v(∆ , D, π, S, B) ≤ ∆ (π, S) + L · B (52) Then, invoking the definition of the derivative as a Weierstrass limit from elementary calculus, as well as Lemma 2.7, and by repeatedly appealing to the assumed subadditivity condition within our domain, we find Therefore, we obtain v(π, S, B) ≤ ∆ (π, S) + B · L (56) Lemma F.1. For each group g ∈ G, under covariate shift, Then, adopting the shorthand ω g (x) = ω g (T , S, x), we have: Proof.
Proof. We again adopt the shorthand ω g (x) = ω g (T , S, x). We first generalize Lemma F.1. For each group g ∈ G, under covariate shift, for all y ∈ Y, ≤ ∆ DP (π, S) + y∈Y g,h∈G β g,y (1 − β g,y ) Var ≤ ∆ DP (π, S) + y∈Y g,h∈G β g,y (1 − β g,y )B g (76) = ∆ DP (π, S) + y∈Y g,h∈G β g,y (1 − β g,y )B g and For our assumed setting, τg−x mg , x ∈ [τ g − m g , τ g ) 1 mg (−x + τ g + 2m g ), x ∈ [τ g , τ g + m g ) 1, x ∈ [τ g + m g , 1] (102) Proof for Lemma F.3: Proof. We discuss the target distribution by cases: • For the target distribution between [0, τ g − M g ]: since we assume the agents are rational, under assumption 6.2, agents with feature that is smaller than [0, τ g − M g ] will not perform any kinds of adaptations, and no other agents will adapt their features to this range of features either, so the distribution between [0, τ g − M g ] will remain the same as before.
• For target distribution between [τ g − M g , τ g ], it can be directly calculated from assumption 6.3.
• For distribution between [τ g , τ g + M g ], consider a particular feature x ∈ [τ g , τ g + M g ], under Assumption 6.4, we know its new distribution becomes: • For the target distribution between [τ g + M g , 1]: under assumption 6.2 and 6.4, we know that no agents will change their feature to this feature region. So the distribution between [τ g + M g , 1] remains the same as the source distribution. Thus, the new feature distribution of x

1,
x ∈ [0, τ g − M g ) and x ∈ [τ g + M g , 1] τg−x Mg , x ∈ [τ g − M g , τ g ) 1 Mg (−x + τ g + 2M g ), x ∈ [τ g , τ g + M g ) 0, otherwise Proof of Proposition 6.5: Statement: For our assumed setting of strategic response involving DP for two groups {g, h}, Theorem 4.1 implies Proof. According to Lemma F.3, we can compute the variance of w g (x): Var(w g (x)) = E w g (x) − E[w g (x)] 2 = 2 3 M g . Then by plugging it to the general bound for Theorem 4.1 gives us the result.