{'fraction_non_alphanumeric': 0.028175195295356798, 'fraction_numerical': 0.008601773018520144, 'mean_word_length': 5.129101667563206, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The relation between alignment covariance and background-averaged epistasis

Epistasis, or the context-dependence of the effects of mutations, limits our ability to predict the functional impact of combinations of mutations, and ultimately our ability to predict evolutionary trajectories. Information about the context-dependence of mutations can essentially be obtained in two ways: First, by experimental measurement the functional effects of combinations of mutations and calculating the epistatic contributions directly, and second, by statistical analysis of the frequencies and co-occurrences of protein residues in a multiple sequence alignment of protein homologs. In this manuscript, we derive the mathematical relationship between epistasis calculated on the basis of functional measurements, and the covariance calculated from a multiple sequence alignment. There is no one-to-one mapping between covariance and epistatic terms: covariance implies epistasis, but epistasis does not necessarily lead to covariance, indicating that covariance in itself is not the directly relevant quantity for functional prediction. Having calculated epistatic contributions from the alignment, we can directly obtain a functional prediction from the alignment statistics by applying a Walsh-Hadamard transform, fully analogous to the transformation that reconstructs functional data from measured epistatic contributions. This embedding into the Hadamard framework is directly relevant for solidifying our theoretical understanding of statistical methods that predict function and three-dimensional structure from natural alignments.


Introduction
Some time ago the traditional scope of epistasis has been extended from interactions between pairs of mutations, to interactions between triple, quadruple, and larger sets of mutations (see e.g. [1]). An effective way to characterize this so-called higher-order epistasis is by applying a Walsh-Hadamard transform to experimentally obtained phenotypic data [2][3][4]. In this framework, the epistatic contributionsω for all combinations of mutations are obtained by performing a linear transformation denoted asω = V Hȳ [4].
Hereȳ is a vector of the phenotypic data for a complete combinatorial mutant dataset, i.e. measured phenotypes for all combinations of amino acid mutations between two initial sequences. H is the Walsh-Hadamard matrix [5], and V is a weight matrix that provides pre-factors for the terms. Conversely, since this is a one-to-one mapping, if we know all epistatic contributions between mutations, we are able to make reconstruct all phenotypes by applying the the inverse transform (ȳ = H −1 V −1ω ). This particular of epistasis was termed 'background-averaged epistasis', indicating that, for example, a first-order contribution is the effect of a mutation at one position, averaged over all combinations of mutations at the remaining positions. This distinguishes the description from the more traditional, or local, epistasis, where all effects are calculated with respect to a chosen reference sequence.
A complementary approach to investigate mutational context-dependence is by statistical analysis of the occurrences and co-occurrences of amino acids in multiple sequence alignments of protein homologs.
The reasoning behind this is that if some residue is beneficial globally across the homologs, it will occur in a multiple sequence alignment with a frequency higher than the background expectation. Similarly, pairs of residues that are beneficial, or deleterious, should be over-or under-represented in a representative alignment. Methods have been developed to extract various types of information, from cooperative interactions between amino acids [6], to three-dimensional structure [7,8], and protein-protein interactions [9]. Additionally, global probability models have been applied to the prediction of phenotypes in mutant libraries [10].
In the current manuscript we directly calculate epistatic terms from an alignment of functional sequences and compare this to covariance terms based on amino acid co-occurrences. To do this, we initially make the assumption that the phenotype is categorical, i.e. assumes the values 0 and 1 for non-functional and functional sequences, which allows us to directly compare epistasis and covariance terms. We argue that the categorical assumption will not affect the applicability of the theoretical framework, as sequences in natural alignments can be reasonably expected to be highly functional. We use experimental data from [11], where a combinatorially complete dataset consisting of 2 13 mutants in the Entacmaea quadricolor fluorescent protein was phenotypically assayed for color and brightness, to show that the assumption is justified. Having distilled epistatic contributions from an alignment, we can now make functional predictions fully analogous to the procedure in the background-averaged epistasis framework, by inversion of the transformationŷ aln = H −1 V −1ωaln , whereŷ aln designates that functional prediction.

Results
For a certain combinatorial sequence space, if we were able to make an alignment that contained all functional sequences in that space, we would write for the frequency f i of the amino acid designated by where N func g i =1 is the number of sequences in the alignment with amino acid '1' at position g i , and N func tot represents the total number of functional sequences. We can write for the first-order background-averaged where N tot is the total size of the sequence space and y ... denote the phenotypes associated with the subscripted genotypes. The signs in front of the phenotypes correspond to the signs of the row in the Hadamard matrix that calculates the first-order background-averaged epistatic contribution for position i (see [4]). If we now make the assumption that all phenotypes are either functional, '1', or nonfunctional, '0', we can write for the first-order background-averaged epistatic terms [4] where we obtain a direct link to the amino acid frequencies f i in the alignment. In what follows we will designate such epistatic terms obtained from the alignment by 'alignment epistais',ω aln .
# mutants in alignment R epistatic terms and aligment terms 2 Figure 2: R 2 between 1st and 2nd order background-averaged epistatic versus alignment terms. The backgroundaveraged terms were obtained based on the functional values for the complete dataset [11], according to the transformω = V HĒ [4]. Shown here is the goodness-of-fit (R 2 ) between first-and second-order terms fromω versus ω aln as a function of an increasing number of functional sequences in the alignment that is used to calculate the alignment epistasisω aln . When all functional mutants in the combinatorial space are included (2984 out of 8192 total mutants) R 2 reaches 0.86.

Now we can find the second-order terms in a similar fashion
allowing us to express the second-order background-averaged epistatic terms in terms of frequencies This clarifies the similarity to the covariance of frequencies in the alignment defined by since ω aln ij and cov ij share the same leading term f ij . To be more precise, the relation between ω aln ij and cov ij cannot be written down as a closed form expression. We can see from a simple simulation based on random values for f i , f j , and compatible choices for f ij that covariance implies epistasis, but that epistasis does not imply covariance (Fig 1). There is a general relationship between the two quantities with an R 2 of 0.50.
Clearly, some initial idealizations were made to arrive at this result. First, no realistic alignment will contain all functional sequences. However, since the relevant terms are frequencies, they should be robust if the alignment is a representative subset of the functional sequences. Indeed, Fig. 2, where we calculated the goodness-of-fit between epistatic terms based on experimental data in [11] and alignment epistasis, illustrates that the two rapidly converge for alignments of limited depth. The second assumption, that of binary phenotypes, may be quite reasonable for sequences that are found in nature, because we generally observe the outcomes of evolutionary optimization instead of intermediates. However, even for the mutational dataset in [11] where we do observed a range of phenotypic values, the similarity between alignment epistasis and background-averaged epistasis from the functional assay is surprisingly good (when we include all functional mutants, the value for R 2 for first and second-order terms combined is 0.86).
In fact, we can derive an expression to calculate for every order of alignment epistatic terms,ω aln .
In the ideal, but unrealistic, case of the alignment containing all functional sequences, this constitutes an invertible decomposition. In the realistic case where the alignment only contains a fraction of all functional sequences, one will have to determine how many orders of alignment epistasis can be meaningfully calculated before noise takes over.
The current demonstration becomes more straightforward if we reparametrize the genotype as σ i = −1/1 instead of g i = 0/1 (in ref [4] we showed that this reparametrization naturally leads to a backgroundaveraged description). Now we can write, in analogy to the frequency f i in equation 1, for the average value of a column in the alignment Comparison to equation 3 identifies φ i with 2f i − 1 and Now it is straightforward to show that Computationally, the φ ij...yz are simply obtained by taking the mean value of the element-wise product of entries in columns ij...yz. The obtained vectorω aln now can be used in exactly the same way as the regular background-averaged epistasis vectorω to calculate the phenotypic values for all the mutants in the combinatorial spaceŷ Doing this for the data in [11], we see that reasonable Spearman correlations are reached even for very limited alignment depths; in this case containing 200-300 of more than 8000 sequences (Fig. 3).

Discussion
The prediction of the functional effects of multiple mutations has been the focus of several recent studies, either based on high-throughput experimental functional characterization (e.g. [12]), or based on amino acid frequency statistics in natural sequence alignments (e.g. [10]). The latter approach has originated from the 3D structure prediction community using global probability models for the sequences [8]. In the current work we derived the link between amino acid covariance and background-averaged epistasis [4] from first principles. We showed that covariance implies epistasis, but that the converse is not true.
We subsequently showed that the obtained terms can be used in a straightforward way, using an inverse Hadamard transform, to make a functional prediction for all mutants in the combinatorial space. Making the identification between experimentally obtained epistasis and and alignment statistics should help us solidify the theoretical basis for covariance methods that predict function and 3D structure from natural alignments. That said, completing the theoretical description, and ensuring its direct applicability to natural libraries -where each position has 20 amino acids choices instead of 2-will require an explicit extension of the Hadamard framework to a cardinality larger than 2, which, as of yet, has not been made.