{'fraction_non_alphanumeric': 0.029227480465870558, 'fraction_numerical': 0.004312251216275984, 'mean_word_length': 4.136879969708444, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Entropy and actions of sofic groups

. In recent years there has been a great deal of progress in the study of actions of countable groups. In particular, the concept of the entropy of an action has been extended to all soﬁc groups following the seminal work of Lewis Bowen. This survey is an invitation to these new developments. It includes a new proof of the analogue of Kolmogorov’s theorem for soﬁc groups, namely that isomorphic Bernoulli shifts have the same base entropy.


1.
Introduction. Ten years after Claude Shannon introduced entropy as a fundamental concept in information theory in 1948, Andrey N. Kolmogorov used Shannon's concept to define the entropy of a measure preserving transformation in ergodic theory. His first application of this was to settle the isomorphism problem for Bernoulli shifts when he showed that isomorphic Bernoulli shifts must have the same base entropy. In the ensuing years entropy turned out to be a very basic invariant and about ten years after Kolmogorov's work Donald Ornstein succeeded in proving the converse of Kolmogorov's theorem, namely that any two Bernoulli shifts with the same base entropy are isomorphic. In the course of his proof Ornstein introduced powerful new methods that led to many results far beyond his isomorphism theorem. He was able to show that there exists a Bernoulli flow. This is an ergodic action, T s , of R on a probability space that is measure preserving and such that for any fixed s T s is isomorphic to a Bernoulli shift. Thus his "Bernoulli theory" was extended beyond the integers Z to the continuous group R. His methods were also able to show that many of the classical examples in ergodic theory such as toral automorphisms and hyperbolic flows, exemplified by the geodesic flow on surfaces of negative curvature, are isomorphic to Bernoulli systems. For a nice survey of this see the historically oriented article by Ornstein [14].
In quite another direction entropy was introduced into topological dynamics with the definition of topological entropy by Roy Adler, Alan Konheim and Harry McAndrew in [1]. Here too topological entropy turned out to be a very basic invariant with many applications. Topological entropy is related to measure entropy by the variational principle which asserts that for a continuous action on a compact space the topological entropy equals the supremum of the measure entropy taken over all the invariant probability measures.
Subsequent developments included extending the application of entropy to actions of more general groups beyond the one parameter groups of Z and R. These developments focused on the class of amenable groups. These are the groups where there is a natural extension of the ergodic theorems which lie at the foundation of ergodic theory. I will define these more precisely later on, for the moment it suffices to say that they include all abelian, nilpotent and solvable groups. All of this work culminated in a sense with our paper [15] in which both Kolmogorov's theorem and Ornstein's isomorphism theorem and its ramifications were extended to a large class of locally compact, unimodular amenable groups that includes all countable amenable groups.
When we tried to go beyond this class to the most basic non-amenable group, the free group on two generators, we were stymied by a simple example that showed that the Bernoulli 4-shift was a two point factor of the Bernoulli 2-shift. This phenomenon completely contradicts the intuition built up from the study of actions of amenable groups. That is where matters stood until a few years ago when Lewis Bowen showed that one can develop an entropy theory for a much larger class of groups, namely the sofic groups. This class was first introduced by Misha Gromov in [5] and named in [21]. Once again I will defer a precise definition for later and content myself for the moment by saying that this class includes both all amenable groups and all residually finite groups, such as the free groups.
This seminal work of Bowen has led to many new results and it is the purpose of this survey to a give an introduction to a part of this work. I will also discuss briefly some extensions of entropy from the setting of measure preserving actions to the setting of topological dynamics. Following this introduction, in the second section I will define the classes of groups that we will be discussing. The third section will include a new proof of the analogue of Kolmogorov's theorem for sofic groups and a discussion of systems with completely positive entropy. The fourth will be devoted to the isomorphism theorems and the final section will briefly survey the entropy-like invariants for continuous actions. I should reiterate that I have only given a taste of these new developments, much more has been done and the interested reader should have no difficulty in finding many more papers in this line of research.
2. Sofic groups and their actions. Let's begin by recalling the first ergodic theorem, that of von Neumann, which dealt with sums of the form where (X, B, µ) is a probability space, T is an invertible measurable transformation that preserves the measure µ and f is a function in L 2 . Actually von Neumann dealt with flows but throughout this paper we will restrict attention to countable groups. He proved that these averages converge in norm to the projection of f onto the space of invariant functions. His proof was based on the spectral theorem for unitary operators, but a short while afterwards F. Riesz gave a proof that was based on the fact that the set {1, 2, ..., n} is approximately invariant. His proof carries over immediately to any group G which has a sequence of finite sets F n with the property that for any g ∈ G lim n→∞ |gF n F n | |F n | = 0 These groups are the amenable groups, and if such a group acts by measure preserving transformations T g on a probability space (X, B, µ) then where P is the projection on the space of invariant functions and the convergence is in the norm of L 2 . This observation motivated the extensive work done in generalizing classical ergodic theory to actions of amenable groups. A sequence of finite sets satisfying (1) is called a Følner sequence after E. Følner who first gave this characterization of amenable groups. The original definition was that a group is amenable if there is an invariant mean on the space of bounded functions on the group. The fact that the free group on two or more generators has no such invariant mean lies at the basis of the Hausdorff-Banach-Tarski paradox and motivated von Neumann to introduce this class of groups.
It is easy to see that commutative groups are amenable and furthermore one can show that if H is a normal subgroup of G and both H and G/H are amenable then G is amenable. It follows that all solvable groups are amenable. In addition, the increasing union of amenable groups is amenable and so all locally finite groups are amenable. One more general result is that subgroups of amenable groups are amenable. The groups obtained in this way are called the elementary amenable groups and they are far from exhausting the class.
Returning to actions, the simplest examples of measure preserving actions of arbitrary groups G are the Bernoulli actions which we proceed to define. Begin with a finite or countable set A and form the space X = A G and give it the product topology where the topology on A is discrete. When A is a finite set this space is compact. There are two actions of G on this space, the right shift: R g x(h) = x(hg), and the left shift: L g x(h) = x(g −1 h). The inverse in the definition of the left action is to make this a homomorphism of G to the group of measure preserving transformations where the group operation is composition. To get an invariant probability measure for these actions we fix a probability measure π on A and form the product measure on X taking for each coordinate the probability space (A, π). The coordinate functions taking a point x ∈ X to x(g) become independent identically distributed random variables and thus this product measure is easily seen to be invariant under both the right and the left shift actions. When G = Z these x(n) are the familiar independent identically distributed random variables of the classical laws of large numbers whence the name Bernoulli shift for the actions we have just defined.
Next we define the class of sofic groups that was introduced by M. Gromov. For any integer d let Sym(d) denote the symmetric group on d points. A sofic approximation of G is a sequence of maps σ n : and for all g = 1 we have This means that 'G almost acts on {1, . . . , d n }, almost freely'. The group G is sofic, if it admits a sofic approximation. If G is amenable then it is easy to see that any Følner sequence gives rise to a sofic approximation and thus every amenable group is sofic. Recall that a group is residually finite if there is a sequence of finite groups L n and homomorphisms θ n from G onto L n such that the intersection of the kernels of θ n is trivial. These finite groups L n also give rise to a sofic approximation of G and thus all residually finite groups are also sofic. As is well known free groups are residually finite and hence sofic. In addition any finitely generated linear group is residually finite, in fact there is no group that is known to be non-sofic.
Here is another, more geometric, way of viewing a sofic approximation. Suppose that G is finitely generated and fix S, a symmetric set of generators for G. The Cayley graph of G with this set of generators S is a graph, C(G, S), whose vertex set consists of the elements of G and for each g ∈ G and s ∈ S there is an edge between g and sg. We will also view this as a colored directed graph with the colors being elements of S and a directed edge colored s from g to sg. Thus the Cayley graph of the free group on two generators is the four regular tree with vertices labeled by the elements of F 2 . As a colored labeled graph each edge (g, h) becomes a pair of directed edges with the labeling being given by s = hg −1 . Denote by B k the ball of radius k centered at the identity in the Cayley graph of G. A sequence of finite colored directed graphs G n with vertex sets V n will be called a sofic approximation of C(G, S) if the coloring of the G n 's is given by S and for all k there are subsetŝ V n,k of V n whose relative density in V n tends to one such that for each v ∈V n,k the neighborhood of radius k is isomorphic to C(G, S) restricted to B k . For F 2 any sequence of finite four regular graphs with girth tending to infinity gives rise to such a sofic graph approximation. For Følner sets F n the setsF n,k are what is called the k-interior and the fact that their relative density tends to one is exactly the property encoded in (1).
3. Entropy for sofic groups and Kolmogorov's theorem. If (X, B, µ, T g ) is a measure preserving action of a group G then a measurable partition P = {P a : a ∈ A} is called a generating partition, or simply a generator, if the smallest G-invariant σ-algebra that contains the sets of P coincides with B modulo µ-null sets. For the Bernoulli actions that we described above, the partition of X defined by the values of x(e), where e is the identity element of the group, is a generating partition. Whenever an action admits a generating partition P, if A labels the elements of the partition, an isomorphic representation of the system is given by the space A G with the shift action but the product measure replaced by some shift invariant measure on A G . Explicitly if l : X → A is the map that sends a point to the label of the set of the partition that contains it, then the map L : intertwines the action T g with the right shift. The measure L • µ is thus invariant under the right shift and the fact that P is a generator means that L is µ almost surely one to one and thus defines an isomorphism.
In his first ground breaking paper [2] Lewis Bowen defined what he called the finvariant for an action of a free group that has a finite generating partition in terms of this partition. He then went on to show that his f-invariant has two important properties. The first is that if Q is another generating partition then the f-invariant computed from Q equals the f-invariant computed from P. Thus the f-invariant is an isomorphism invariant. The second fact was that for a Bernoulli shift the f-invariant simply equals the Shannon entropy of the defining probability measure π. As a consequence he obtained the analogue of Kolmogorov's theorem for the free groups.
The definition of the f-invariant depended heavily on the fact that he was dealing with the free group. In an even more remarkable paper, a short time later, in [3], he defined an entropy-like invariant for actions of any sofic group that satisfies the same two properties and was thus able to extend Kolmogorov's theorem to all sofic groups. Once again the entropy was defined in terms of a generating partition. The definition of Bowen was based on the existence of a finite generator. In contrast, for amenable groups one can define the entropy of an action as the supremum of the entropy of the actions defined by finite partitions. This naive definition cannot be used in the case of the free group as we shall explain in the next section. However, David Kerr in [8] did succeed in defining an entropy-like invariant based on finite partitions that do not generate and he showed that in the presence of a finite generator his definition coincides with Bowen's.
I shall not give a precise formulation of these definitions here but just give some rough idea of what Bowen did. For simplicity suppose that G is finitely generated with a symmetric generating set S that also contains e the identity element of the group and denote by S n the ball of radius n, i.e. S n . This is just for convenience of notation. Fix a sofic approximation and consider a system (X, B, µ, T g ) with a finite generator P indexed by A. Finally fix a k and an and observe that for n sufficiently large for most i ∈ {1, 2, . . . , d n } in the sofic approximation σ(S k )(i) looks just like S k and so for any A-coloring of {1, 2, . . . , d n } it makes sense to define the empirical distribution defined by this coloring as a probability measure on A S k . You then count the number of colorings for which this empirical distribution is within of the distribution defined by µ on h∈S k T h (P) and look at the maximal exponential growth rate of these colorings. Then one takes an appropriate limit of these quantities as and 1/k tend to zero. Bowen went on to show that if Q is a different generating partition then these sofic entropies are equal hence it is an isomorphism invariant. For a Bernoulli shift Bowen showed that the sofic entropy equals the Shannon entropy of π thus proving Kolmogorov's theorem.
I have recently developed, in collaboration with Miklos Abert, a different approach which uses measure approximations rather than combinatorial counting. Our approach led to a short proof of Kolmogorov's theorem for sofic groups and it is that proof that I will now sketch. The idea goes back to Rokhlin's work on generators for Z actions (see [16]) and consists in defining the Rokhlin entropy, RE(X, B, µ, T g ), of an action that has at least one generating partition P as RE(X, B, µ, T g ) = inf{H(P)} where the infimum is taken over all generating partitions P and H(P) is the Shannon entropy of the probability distribution: {µ(P ); P ∈ P}. This definition is clearly an isomorphism invariant and it remains only to prove that for a Bernoulli action defined by (A, π) that this infimum is attained at the canonical generating partition. For simplicity I will also assume that we are dealing only with finite A and finite partitions.
We denote by P the partition defined by the values of x(0) and suppose that Q is another finite generating partition. We have to show that H(Q) ≥ H(π). We shall denote the shift on X = A by T g . Fix some small > 0. The fact that Q is a generator means that for some k we can find a coarsening of g∈S k T g Q, denoted byP such that a∈A |µ(P a ) − µ(P a )| < .

BENJAMIN WEISS
If B is the set of labels of Q this coarsening is given by a function Since P is also a generator we can find an N and a coarsening of h∈S N T h P, denoted byQ such that This coarsening is given by a function The δ is chosen to be sufficiently small so that when we use the function ψ to define a coarsening of g∈S k T gQ to get another partition denoted byP, we will also satisfy a∈A |µ(P a ) − µ(P a )| < . and hence a∈A |µ(P a ) − µ(P a )| < 2 . (2) We remark that the partitionP is a coarsening of g∈S N +k T g P and this last inequality compares a complicated coarsening of g∈S N +k T g P with P itself and depends only on the distribution of the µ measure of the elements of these partitions. We now use the fact that the group is sofic to find a finite set V and permutations {σ(g) : g ∈ S N +k } of V with the property that for a set V 0 ⊂ V of relative density at least 1 − for all v ∈ V 0 the set {σ(g)(v) : g ∈ S N +k } is in a natural one to one correspondence with S N +k . Now we define independent random variables For all v ∈ V 0 the inequality (2) implies This is because the µ measure was defined by having the coordinate functions being independent. The point is that we can compare the average entropy of the X v 's, namely H(X v : v ∈ V )/|V | with the average entropy of the Z v 's. By a standard inequality (3) implies that as tends to zero this difference will tend to zero. On the other hand, the Z v 's are functions of the Y v 's and hence cannot have entropy greater than the entropy of the Y v 's and in turn for all v ∈ V 0 their entropy is close (as a function of ) to the entropy of µ(Q b ). This concludes the proof the invariant that we have defined when applied to a Bernoulli shift yields the entropy of the base distribution and thus we have: Theorem 3.1 (Bowen). Let G be a sofic group and let π 1 and π 2 be finite probability spaces. If the Bernoulli actions π G 1 and π G 2 are isomorphic, then H(π 1 ) = H(π 2 ). This proof can be extended to countably infinite partitions with finite entropy by a careful approximation with finite partitions, the details will be given in a paper now in preparation with Miklos Abert. For a general measure preserving action that has a finite generator one can define RE in the same way and it is clearly an isomorphism invariant. Even for sofic groups it is difficult to prove interesting theorems using this invariant. Perhaps the main obstacle is that in order to use the finite models {V, σ} that the definition of soficity yields to study measure preserving actions one needs measures on P V that are good local approximations of the measure µ.
In the classical theory entropy is also used to define an important class of actions -those that have completely positive entropy. For Z actions instead of talking about systems and partitions an equivalent formulation is in terms of stationary stochastic processes which are sequences of random variables, {X n : n ∈ Z} taking values in some set A whose joint distributions are invariant under the shifting of all the indices by a fixed amount. Such a process is called a Kolmogorov process (or simply K-process) if any event that is measurable with respect to the remote past has probability either zero or one, in other words the process satisfies the classical zero-one law. More formally the intersection ∩ i<−n F i is trivial, where F n is the smallest σ-field with respect to which all {X i : i ≤ −n} are measurable. After the introduction of entropy it was shown by V. A. Rokhlin and Y. Sinai in [17] that this property is equivalent to the property that any nontrivial process {Y n }, with values in B, that is a factor of {Y n } has positive entropy. Here {Y n } is a factor of {X n } means that there is a measurable mapping θ : A Z → B Z that takes {X n } to {Y n } and is equivariant with respect to the shift. There is a third classical way of characterizing these K-processes and that is via a strong mixing condition which asserts that any fixed event is asymptotically independent from F n as n tends to infinity. While the notion of completely positive entropy can be defined as soon as there is a notion of entropy, for a general group there is no notion of a remote past. Nonetheless, in [18] Dan Rudolph and I were able to show that for amenable groups completely positive entropy was equivalent to a uniform mixing condition. It is easy to see that Bernoulli actions have completely positive entropy but that class is much more general. For sofic groups, Kerr in [9] succeeded in showing that Bernoulli shifts have completely positive sofic entropy.

4.
Ornstein's theorem for countable groups. Several years before Ornstein established the converse of Kolmogorov's theorem for Z, namely that Bernoulli shifts with the same base entropy are isomorphic, Y. Sinai showed that they are weakly isomorphic. In general two systems are said to be weakly isomorphic if there are factor maps going from (X, B, µ, T g ) to (X , B , µ , T g ) and vice versa, where a factor map θ : X → X is a measurable map that takes µ to µ and satisfies θT g = T g θ for all g ∈ G. If such a θ exists one calls the second system a factor of the first while the first is an extension of the second. For amenable groups the entropy of a factor is at most equal to the entropy of its extension and so weakly isomorphic systems must have the same entropy. Examples show that even for Z there are weakly isomorphic systems that are not isomorphic. In light of Bowen's result the example that I mentioned in the introduction shows that for the free group the situation is even more dramatic, namely there are weakly isomorphic Bernoulli shifts that are not isomorphic.
The example is simple enough. Let G = F 2 be the free group on two generators with generating set S = {a, a −1 , b, b −1 }. Consider the Bernoulli shift X = {0, 1} G with π 0 = π 1 = 1/2 and define a map θ : X → Y with Y = {00, 01, 10, 11} G by setting θ(x)(g) = (x(g) + x(ag))(x(g) + x(bg)) with the sum being taken modulo 2. It is easy to see that this map is equivariant with respect to the right shifts and that the y(g) are independent and have a uniform distribution on the four possible values and thus the base entropy is log 4. There is clearly an equivariant map in the other direction and so the 2-shift and 4-shift over F 2 are weakly isomorphic but not isomorphic. In [4] Bowen showed that one can use this example to build up such weak isomorphisms between any pair of Bernoulli shifts over any group G that contains F 2 as a subgroup.
As far as extending Ornstein's theorem itself to more general groups the first step was an observation of A. Stepin [19] who pointed out that if G is any group that contains an infinite cyclic subgroup H then one can apply the codings given by Ornstein's theorem coset by coset to establish the analogue of Ornstein's theorem for such groups. The next step was the extension to arbitrary amenable groups in [15]. Finally, Bowen extended Ornstein's theorem to all groups assuming that the basic distribution had at least three elements with positive probability, and to a large class of groups even without this minor restriction. His proof was completely independent of his sofic entropy methods and relied on more refined results in the theory of Z actions. Here is a quick sketch of his proof.
An elementary lemma (already used by M. Keane and M. Smorodinsky in [7]) shows that it suffices to consider the case where the two distributions π and π have equal entropy and in addition π(0) = π (0) > 0. One then considers the Bernoulli shift Z = {0, 1} G with base distribution equal to {π(0), 1 − π(0)} as a common factor of the two Bernoulli shifts. Now one borrows an idea from the theory of orbit equivalence to define an ergodic invertible transformation R on Z that is defined by some function ρ : Z → G by the formula τ (z) = R ρ(z) (z) where R g is the shift on Z.
At this point one applies the "relative Bernoulli theory" that was developed by Jean-Paul Thouvenot [20]. After proving the isomorphism theorem for Bernoulli shifts Ornstein gave several abstract conditions which are necessary and sufficient for an ergodic transformation to be isomorphic to a Bernoulli shift. These conditions can be verified in many concrete situations. Thouvenot's contribution was to give analogous conditions for an extension of a system (Z, τ ) to be isomorphic to the direct product of (Z, τ ) with a Bernoulli shift. Bowen used this theorem of Thouvenot to define the desired isomorphism between the given Bernoulli shifts over G defined by π and π . 5. Topological entropy and mean dimension. As I mentioned in the introduction, following the introduction of entropy into the theory of measurable dynamics an analogous notion was introduced into topological dynamics, which is the study of continuous actions on compact spaces. The original definition proceeded as follows. If X is a compact space and U is an open cover of X then cov(U) is defined to be the least cardinality of a finite subset of U that covers X. If U i is a finite family of covers then i {U i } is the open cover whose elements are intersections ∩ i U i where the U i ∈ U i range over the elements of the covers U i . One then shows that if T is a homeomorphism of X the limit as n tends to infinity of the expression cov( n i=1 {T i (U)})/n exists and the supremum of this limit as U ranges over all open covers of X is defined to be the topological entropy of the system (X, T ).
The connection with the measure theoretic entropy is given by the variational principle which asserts that topological entropy of (X, T ) equals the supremum of the measure entropy of the system (X, T, µ) where µ ranges over all T -invariant Borel measures. A similar definition can be given for any amenable group with Følner sets replacing the interval [1, 2, ...n]. There are other ways of defining the topological entropy for the most studied case when the compact space is metric. Following the work of Bowen on sofic entropy David Kerr and Hanfeng Li in [10] defined topological entropy for actions of sofic groups and gave also a variational principle.
In case the topological entropy of a system is infinite not much information about the system is encoded by that fact. A more refined invariant for these systems was introduced by M. Gromov in [6]. If K is a compact infinite metric space and X = K Z and T is the shift then it is easy to see that the topological entropy is infinite whereas Gromov's invariant of this system turns out to be exactly the topological dimension of the base space K. In [13] this invariant was called mean dimension and explored with alternative definitions. In addition it was shown there how to extend the theory to actions of amenable groups. For some deeper results in this direction see [12]. In [11] Hanfeng Li succeeded in extending the theory of mean dimension to all sofic groups.