{'fraction_non_alphanumeric': 0.041741757945286784, 'fraction_numerical': 0.010640479145308369, 'mean_word_length': 3.968952758861065, 'pattern_counts': {'":': 0, '<': 11, '<?xml version=': 0, '>': 13, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 13, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Nonequilibrium Entropy in an Extended State Space

This chapter deals with our recent attempt to extend the notion of equilibrium (EQ) entropy to nonequilibrium (NEQ) systems so that it can also capture memory effects. This is done by enlarging the equilibrium state space by introducing internal variables. These variables capture the irreversibility due to internal processes. By a proper choice of the enlarged state space, the entropy becomes a state function, which shares many properties of the EQ entropy, except for a nonzero irreversible entropy generation. We give both a thermodynamic and statistical extension of the entropy and prove their equivalence in all cases by taking an appropriate state space. This provides a general nonnegative statistical expression of the entropy for any situation. We use the statistical formulation to prove the second law. We give several examples to determine the required internal variables, which we then apply to several cases of interest to calculate the entropy generation. We also provide a possible explanation for why the entropy in the classical continuum 1-d Tonks gas can become negative by considering a lattice model for which the entropy is always nonnegative.


A. Entropy as a Primitive Concept
What distinguishes a thermodynamic system Σ, the focus of our study here, from a mechanical system is the concepts of the entropy S and the temperature T , both of which are new concepts in thermodynamics without any mechanical analogs. To be useful, S and T must uniquely refer to the thermodynamic state, simply called the state and denoted by M here, of Σ. Being functions of the state of Σ, they must be interrelated in some fashion so only one of them can be treated as a primitive concept, which we take to be the entropy to describe it. Although S plays important roles in diverse fields ranging from classical thermodynamics of Clausius , quantum mechanics and uncertainty [23][24][25], black holes [26], coding and computation [27,28], to information technology [29,30], it does not seem to have a standard definition in all cases, even though it is well defined under equilibrium (EQ) conditions as extensively discussed in the literature; see, for example [31,32]. Therefore, in this chapter, we are interested in understanding this concept in nonequilibrium (NEQ) statistical thermodynamics of irreversible processes (we will use NEQT to denote NEQ thermodynamics in this chapter). This will require an extension of the classical concept of entropy from EQ states to NEQ states where irreversible entropy will be generated.
There are two distinct approaches to understand the physics of S. One is the original approach of Clausius [1] in classical thermodynamics, where it appears as a primitive concept, and determines the Gibbs fundamental relation from which follows the entire thermodynamics. The other approach is the statistical one introduced by Boltzmann [2] and Gibbs [3]. The statistical extension requires dealing with the set of microstates {m k } , k = 1, 2, · · · , which we take to be countable. What we mean by a microstate is simply this: a microstate m is nothing but an instantaneous state of Σ. If we consider an instantaneous ensemble [3] of Σ by making a large number of its replicas at any instant t, all prepared under identical conditions, then {m k } is the set formed from distinct microstates at that time. Each microstate is specified by the instantaneous values of the state variables in the corresponding replica. How often a particular microstate m k appears among the replicas determines its probability p k . The connection with classical thermodynamics is through the ensemble average Z of some quantity Z, which is also written simply as Z: where Z k is value Z takes on m k . Thus, the statistical thermodynamics requires a probabilistic approach and deals directly with the set {m k , p k }. In contrast, the classical thermodynamics has no association with microstates and/or their probabilities. The system Σ is usually embedded in a medium Σ as shown in Fig. 1. Sometime, it can also be driven by inserting it between two non-interacting media such as a rod between two heat sources. Even though Σ is macroscopic in size, it is extremely small compared to the medium Σ so it does not affect the state of Σ. We will always assume that Σ is in EQ so there cannot be any irreversibility in it. Any irreversibility will be associated with Σ. The collection Σ 0 = Σ ∪ Σ forms an isolated system, which we assume to be stationary. We use a suffix 0 to denote all quantities pertaining to Σ 0 , a tilde ( ) for all quantities pertaining to Σ, and no suffix for all quantities pertaining to Σ even if it is isolated.
We will use the term "body" to refer to any of Σ, Σ, and Σ 0 and use Σ b to denote it. However, to avoid notational complication, we will use the notation suitable for Σ for Σ b if no confusion would arise in the context. The states for Σ b are determined by the set X of extensive observables (the energy E, the volume V , the number of particles N , etc.) specifying it. Thus, we need X 0 , X and X, respectively, for Σ, Σ, and Σ 0 for their description.
Let Γ(X) be the microstate space containing W (X) distinct microstates for Σ b . While a temporal evolution  1: (a) An isolated nonequilibrium system Σ0 with internally generated diZ driving it towards equilibrium, during which its SI-fields T (t), P (t), · · · , A(t) continue to change to their equilibrium values; diZ k denote the microanalog of diZ. The sign of diZ is determined by the second law. (b) A nonequilibrium systen Σ in a surrounding medium Σ, both forming an isolated system Σ0. The macrostates of the medium and the system are characterized by their fields T0, P0, ..., A0 = 0 and T (t), P (t), ..., A(t), respectively, which are different when the two are out of equilibrium. Exchange quantities (deZ) carry a suffix "e" and irreversibly generated quantities (diZ) within the system by a suffix "i" by extending the Prigogine notation. Their sum deZ + diZ is denoted by dZ, which is a system-intrinsic quantity (see text).
for Σ b is not our primary interest in this chapter, we still need to remember the importance of temporal evolution in any thermodynamic study. We will say that two microstates belonging to the microstate subspace Γ(X) are "connected" if one evolves from the other after some time τ c < ∞. Before this time, they will be treated as "disconnected." Let τ c,max denote the maximum τ c over all pairs of microstates. The space Γ(X) is simply connected for all times longer than τ c,max in that each microstate can evolve into another microstate ∈ Γ(X) in due time. For t < τ c,max , the space Γ(X) will consist of disjoint components, an issue that neither Boltzmann nor Gibbs has considered to the best of our knowledge. But the issue, which we consider later in Sect. IV B, becomes important in considering nonequilibrium states, especially in glasses.

B. EQ Entropies
We first discuss the EQ statistical entropy in the literature. Boltzmann assumes equal probability of various microstates in the simply connected set Γ(X), in which τ eq = τ eq (X) represents the equilibration time for Σ b . Under the equiprobable assumption, Boltzmann identifies the entropy in terms of W (X) [15,33]: we will set the Boltzmann constant to be unity so that the entropy will always be a pure number. The idea behind the above formula, which we will call the Boltzmann's postulate, implicitly appears for the first time in a paper [2] by Boltzmann, and then appears more or less in the above form later in his lectures [34] where he introduces the combinatorial approach for the first time to statistical mechanics. The formula itself does not appear but is implied when he takes the logarithm of the number of combinations [34,35], an approach that we will adopt later in Sect. IV. (There is another formulation for entropy given by Boltzmann [2,34], which is also known as the Boltzmann entropy [36] that we will discuss later and that has a restricted validity; see Eq. (81).) Gibbs, also using the probabilistic approach, gives the following formula for the entropy in a canonical ensemble [3,15] S (c) where p (c) k is the canonical ensemble probability of m k of energy E k , and the sum is over all microstates (with other elements in X besides E held fixed). The Gibbsian approach assumes an ensemble at a given instant, while the Boltzmann approach considers the evolution of a particular system in time; see for example a recent review [37]. In equilibrium, both entropy expressions yield the same result. In quantum mechanics, this entropy is given by the von Neumann entropy formulation [23,24] in terms of the density matrix ρ: S vN = −T r(ρ ln ρ).
The entropy formulation in the information theory [29,30] has a form that appears to be similar in form to the above Gibbs entropy even though the temperature has no significance in the information theory. There is also another statistical formulation of entropy, heavily used in the literature, in terms of the phase space distribution function f (x, t), which follows from Boltzmann's celebrated H-theorem: here, x denotes a point in the phase space. This quantity is not only not dimensionless but also not the correct formulation in general [31,32]. We now turn to Clausius' thermodynamic entropy S. It is oblivious to the set {m k , p k } and deals only with the observables of the body X = (E, V, N, · · · ). In equilibrium, S = S(X) is a state function of X. This functional dependence results in the Gibbs fundamental relation For a lattice model, S is non-negative in accordance with the Boltzmann definition of S (X), but is known to become negative for a continuum model such as for an ideal gas. The latter observation implies that such continuum models are not realistic as they violate Nernst's postulate (the third law) and require quantum mechanics to ensure non-negativity of the entropy [15]. Even the change ∆S, the heat capacity, etc. do not satisfy thermodynamic consequences of Nernst's postulate. By invoking Nernst's postulate, which according to Planck states that the equilibrium entropy vanishes at absolute zero, one can determine the equilibrium entropy everywhere uniquely. The consensus is that in EQ, the thermodynamic entropy is not different from the above statistical entropies due to Boltzmann and Gibbs, the exceptions being the negative classical entropies. However, there is at present no consensus when the system is out of equilibrium. There is also some doubt whether the nonequilibrium thermodynamic entropy has any meaning in classical thermodynamics. We will follow Clausius and take the view here that the thermodynamic entropy is a well-defined notion even for an irreversible process going on in a body (see [31,32] for supporting arguments) for which Clausius [1, p. 214] writes in terms of the exchange heat d e Q = T 0 d e S with the medium at temperature T 0 ; see Fig. 1. The question that arises is whether the statistical definitions above can be applied to a body out of equilibrium. We find the answer to be affirmative. The next question that arises is the following: Do they always give the same results? We will show that under certain conditions, they give the same results. This is important as the literature is not very clear on this issue [38][39][40][41]. For an isolated system, we are not concerned with any thermostat or external working source. As a consequence, E 0 ,V 0 ,N 0 , etc. in X 0 must remain constant even if the system is out of equilibrium. While this will simplify our discussion to some extent for Σ b = Σ 0 , it will also create a problem discussed in the following Remark 1 For an isolated system, all the observables in X 0 are fixed so if the entropy is a function of X 0 only, it cannot change [31,32,42,43] even if the system is out of EQ.
Thus, we need additional independent variables to ensure the law of increase of entropy for a NEQ isolated system. There must be internal irreversible processes as discussed in the next section, where we discuss the concept of a nonequilibrium state, state variables and state functions. We show that the situation requires a NEQ state M to be discussed in an extended state space S Z formed by the union Z = X∪ξ, where ξ is a set of internal variables. In an appropriately chosen S Z , the entropy of M becomes a unique state function of Z. Such a state M ieq is identified as in internal EQ (IEQ) in S Z , and shares many property of an EQ state, except that it has irreversible entropy generation. In Sect. III, we consider various situations to identify the required number of internal variables to guide us to identify S Z . In Sect. IV, we introduce the statistical entropy formulation denoted by S, and show its equivalence with thermodynamic nonequilibrium entropy S when the latter is a state function. In Sect. V, we discuss the extended state space needed for a state not in IEQ and show how it can be converted to M ieq by adding more internal variables to enlarge the state space to S Z ′ ⊃ S Z . In Sect. VI, we give a simple calculation and a brief introduction to chemical reaction model to change p k 's. In Sect. VII, we give many applications of using S Z and calculate the irreversible entropy generation in M ieq . In Sect. V, we consider a 1-d lattice model appropriate for Tonks gas in continuum so that the statistical lattice entropy can be calculated rigorously. We take the continuum limit and compare the resulting entropy with the continuum entropy of the Tonks gas and obtain an interesting result. In Sect. IX, we revisit Jaynes and obtain his bound. A brief summary and discussion is presented in the final section.

A. Second Law
Following deGroot and Prigogine, we write during any infinitesimal physical process in Σ b , where d e S is the entropy exchange with the medium and d i S is the irreversible entropy generated within Σ b ; see Fig. 1.
The second law states that d i S satisfies the inequality the equality occurs for a reversible process. For an isolated system, d e S ≡ 0. Hence, dS 0 = d i S in any arbitrary process and satisfies We thus see that the second law statement in EQ. (8) is the most general one and applies to any body during any physical process. The law refers to the thermodynamic entropy.
As the thermodynamic entropy is not measurable except when the process is reversible, the second law remains useless as a computational tool. In particular, it says nothing about the rate at which the irreversible entropy increases. Therefore, it is useful to obtain a computational formulation of the NEQ entropy. Several examples in Sect. VII show how irreversible entropy generation can be measured by going to the enlarged state space; see, for example, the discussion after Eq. (66b). In addition, the statistical entropy also proves useful to obtain a computational scheme to determine the entropy by using the microstate probabilities derived in Eq. (46).

B. Concept of a Nonequilibrium State
For a body in equilibrium, the entropy can be expressed as a function of its observables (variables that can be controlled by an observer), as is easily seen form the Gibbs fundamental relation in Eq. (5). The thermodynamic state M eq in EQ remains the same unless it is disturbed. Therefore, we can uniquely identify M eq by its observable X. Accordingly, its equilibrium entropy S(X) is also expressed as a function of X, i.e., S(X) is a state function, and X is the set of state variables. We denote the state space associated with X by S X .
The above conclusion is most certainly not valid for a body out of equilibrium. Let us consider an isolated body Σ b = Σ 0 , and use the suffix 0 for the moment for clarity. If it is not in equilibrium, its state M 0 (t) will continuously change, which is reflected in its entropy increase in time; this requires expressing S as S(X 0 , t) with an explicit time-dependence, since X 0 = constant for Σ 0 . The change in the entropy and the state must come from the variations of additional variables, distinct from the observables, that keep changing with time until Σ 0 comes to EQ as explained elsewhere [42][43][44]. These are known as the internal variables (sometimes also called hidden variables); see Sect. III for how to identify them in some simple situations. We should emphasize that the concept of internal variables and their usefulness in NEQT has a long history. We refer the reader to an excellent exposition of this topic in the monograph by Maugin [45,see Ch. 4]. These variables cannot be controlled by the observer. Once the body has come to equilibrium, the entropy has no explicit time-dependence and becomes a state function. In this state, the entropy has its maximum possible value for given X 0 .

C. Internal Equilibrium States
We turn to a body again. We assume that there is a set ξ of additional variables, known as the internal variables (sometimes also called hidden variables); see Sect. III for how to identify them in some simple situations. We will refer to the variables in X and ξ as (nonequilibrium) state variables (see below for justification) and denote them collectively as Z in the following, with S Z for the enlarged state space. From Theorem 4 presented elsewhere [43], it follows that with a proper choice of the number of internal variables, the entropy can be written as S(Z(t)) as a state function with no explicit t-dependence so it becomes unique. The situation is now almost identical to that of a body in equilibrium: The entropy is a function of with no explicit time-dependence; here, we have introduced W(t) and w(t) obtained by taking out E(t) from Z(t) and X(t), respectively, so that W(t) = (w(t), ξ(t)). (We will see below that W determines the work done by the body.) As S is a state function, we can identify Z(t) as the set of NEQ state variables. Thus, M(t) can also be uniquely specified by Z(t). This allows us to extend Eq. (5) to dS = (∂S/∂Z) · dZ (10) in which the partial derivatives are related to the fields of the body: These fields will change in time unless the body has reached equilibrium. In conventional notation, where P is the pressure and A the affinity corresponding to ξ; the missing terms · · · represent terms from the rest of X besides E and V . In EQ, F w takes the EQ value F w0 associated with Σ. We can invert Eq. (10) to express E in terms of S and W: where F w = −∂E/∂W is identified as a force F w , and W as a "work variable." We will call F w the generalized force as not all components of dW represent displacement in space. This allows us to identify the generalized work as the work done by the body; here f w = −∂E/∂w. As f w · dw = P dV + · · · in terms of P , · · · of the body, it must not be confused with the work d W = P 0 d V + · · · done on Σ b = Σ by the medium Σ. Note that the EQ value A 0 = 0 of the affinity represents the affinity of Σ so A 0 · dξ ≡ 0 contributes nothing to the work done on Σ . We use d e W = −d W to denote this external or exchange work done by Σ against Σ. Their difference is the irreversible work d i W [10,11,43] in Σ so that This partition is similar to the one in Eq. (7) for the entropy, and has a similar explanation: d e W is the exchange work done by Σ on Σ, and d i W is the irreversible or internal work generated within Σ. In contrast, dW is the net work done by Σ. We wish to remind the reader that the concepts of two different kinds of NEQ work in classical thermodynamics have been well known but not with the present interpretation; see for example, [10,Sec. 3.3]. Thus, care must be exercised in distinguishing the two works in a NEQ process, which is not always the case [46]. The above partition first proposed in [43] formalizes this distinction as a natural extension of Eq. (7). Indeed, the above partitioning can be done for any extensive quantity Z pertaining to the body as shown in Fig. 1: having the conventional interpretation: d e Z is the part exchanged with the medium and d i Z is the irreversible part generated within the system. Their sum makes up the net change dZ in the quantity. It is well known that d i E = 0 and d i V = 0 for the simple reason that no internal process can change E and V . However, d i N = 0 when there is chemical reaction within the system. The expression for d e W in the general case is obtained by replacing f w by its EQ value f w0 and A by its EQ value A 0 = 0 associated with Σ, and replacing dw =d e w+d i w by its exchange contribution d e w. Thus, as noted above. This finally determines the irreversible work which also establishes the fact that internal variables only contribute to internal processes; they are never involved in any exchange processes. The inequality is a consequence of the second law as is easily proven [47]; we refer the reader to this for further details. The exchange heat d e Q = T 0 d e S is used in the traditional way of writing the first law in terms of exchange quantities with Σ. Using the definition of dW , we have from Eq. (12) where we have introduced the generalized heat which is partitioned similar to dW above as Again, the interpretation of d i Q is similar: it is the irreversible or internal heat generated within Σ. Thus, dQ is the net heat. Comparing the two formulations of dE above, we conclude that It should be clear from the above that the use of quantities pertaining to the body alone, which we call systemintrinsic (SI-) quantities, captures the entire irreversibility in the NEQ body. As an example, Eq. (18b), which originated from the Gibbs fundamental relation, is in terms of SI-quantities dQ and dW . Thus, it also represents a new version of the first law but in terms of the SI-quantities and fully captures the irreversibility in the body. In contrast, the exchange quantities are determined by the medium, which we call medium-intrinsic (MI-) quantities, that are oblivious to what is going on within the system. Thus, the first law in Eq. (18a), although applicable to any process, cannot provide any information of the irreversibility in the body. This shows the usefulness of Eq. (18b), and the use of SI-quantities.
We now have a complete NEQ thermodynamics in terms of SI-quantities, which contain all the irreversible components as seen above. We have identified this thermodynamics by MNEQT, with M standing for the use of MI-quantities [48]. To verify that we have captured entire irreversibility, we determine d i Q = dQ − d e Q, which turns out to be For the simple case of W = (V, ξ), we have dW = P dV + Adξ. The exchange work is d e W = P 0 dV so from which we also obtain by using Eq. (21) the following expression for the irreversible entropy for this simple case. The first two contributions are due to exchanges with Σ, and the last term is from the internal variable; it should be replaced by A · dξ in the general case. As we see from Eq. (17), there is an additional contribution, not seen above, from f w · d i w. Each contribution in d i S must be nonnegative in accordance with the second law. As Z(t) changes in time, M(t) changes, but at each instant the NEQ entropy as a state function, has a maximum possible value for given Z(t) even though M(t) = M eq . We have identified this particular state as an internal equilibrium state (IEQ) [42][43][44] and express it as M ieq . In S Z , M ieq is uniquely described. There are many states that are not in IEQ in that they are not unique in S Z . We will denote such states by M nieq (non-IEQ state), and use an arbitrary state M arb to denote any state by not specifying the state space S.
We clarify this point. If we do not use ξ for a NEQ M, it is not unique in S X . Then its entropy cannot be a state function in S X , and must be expressed as S(X, t). This explains the importance of ξ; it allows us to deal with a state function entropy S(Z) by choosing an appropriate number of internal variables. Throughout this chapter, we will mostly deal with IEQ states. But our discussion will cover all states (M arb ) in any S.
As a state function, S(Z) shares many of the properties of EQ entropy S(X) [43]: 1. Maximum: S(Z) is the maximum possible value of the NEQ entropy in S Z for a given Z.

2.
No memory -Its value also does not depend on how the system arrives in M ieq ≡ M(Z), i.e., whether it arrives there from another IEQ state M ieq or a non-IEQ state M nieq . Thus, it has no memory of the earlier state.
There are states (M nieq ) in S Z for which the entropy is not a state function. They possess memory of the initial states in S Z with the entropy S(Z, t) retaining an explicit time-dependence. In this case, we need to enlarge the state space to S Z ′ by including additional internal variables as shown elsewhere [43] and latter in In S Z ′ , the derivatives in Eq. (11b) with Z replaced by Z ′ can again be identified as field variables like, temperature, pressure, etc. We will explain in Sect. V A how to determine the correct NEQ state space based on the way experiments are performed.
It may appear to a reader that the concept of entropy being a state function is very restrictive. This is not the case as this concept, although not recognized by several workers, is implicit in the literature where the relationship of the thermodynamic entropy with state variables is investigated. To appreciate this, we observe that the entropy of a body in internal equilibrium [42,43] is given by the Boltzmann formula S(Z(t)) = ln W (Z(t)), (24) in terms of the number of microstates corresponding to Z(t). In classical nonequilibrium thermodynamics [12], the entropy is always taken to be a state function. In the Edwards approach [49] for granular materials, all microstates are equally probable as is required for the above Boltzmann formula. Bouchbinder and Langer [50] assume that the nonequilibrium entropy is given by Eq. (24). Lebowitz [41] also takes the above formulation for his definition of the nonequilibrium entropy. As a matter of fact, we are not aware of any work dealing with entropy computation that does not assume the nonequilibrium entropy to be a state function. This does not, of course, mean that all states of a system are internal equilibrium states. For states that are not in internal equilibrium, the entropy is not a state function so that it will have an explicit time dependence. But this can be avoided by further enlarging the space of internal variables that results in Z ′ discussed above. The choice of how many internal variables are needed will depend on experimental time scales and has been answered in generality earlier in [47], and is briefly summarized in Sect. V.

III. INTERNAL VARIABLES
It should again be stated that in order to capture a NEQ process, internal variables are usually necessary; see Remark 1. While a point in S X represents M eq , we need to use the enlarged state space S Z in which a point represents M. As internal variables are not required in EQ, they must no longer be independent of the observables in S X . Consequently, their affinities (see Eq. (11b) for A) vanish in EQ. It is common to define the internal variables so their EQ values vanish, but it is not necessary. We now discuss various scenarios where they are required for a proper NEQ thermodynamic consideration.

A. A Two-level System
Consider a NEQ body Σ b = Σ 0 of N particles such as Ising spins, each of which can be in two levels, forming an isolated system Σ 0 of volume V . Let ρ l and e l (V ), l = 1, 2 denote the probabilities and energies of the two levels of a particle in a NEQ state so that ρ 1 , ρ 2 keep changing. We have assumed that e l (V ) depends on the observable V only, which happens to be constant for Σ b = Σ 0 . We have e = ρ 1 e 1 (V ) + ρ 2 e 2 (V ) for the average energy per particle, which is also a constant. We have as a consequence of ρ 1 + ρ 2 = 1. Using de = 0, we also get dρ 1 + dρ 2 e 2 /e 1 = 0, which, for e 1 = e 2 , is inconsistent with the first equation (unless dρ 1 = 0 = dρ 2 , which corresponds to EQ). Thus, e l (V ) cannot be treated as constant in evaluating de. In other words, there must be an extra dependence in e l so that e 1 dρ 1 + dρ 2 e 2 + ρ 1 de 1 + ρ 2 de 2 = 0, and the inconsistency is removed. This extra dependence must be due to independent internal variables that are not controlled from the outside so they continue to relax in Σ b as it approaches EQ. Let us imagine that there is a single internal variable ξ so that we can express e l as e l (V, ξ) in which ξ continues to change as the system comes to equilibrium. The above equation then relates dρ 1 and dξ; they both vanish simultaneously as EQ is reached. We also see that without any ξ, the isolated system cannot equilibrate in accordance with Remark 1.

B. A Many-level System
The above discussion is easily extended to a Σ with many energy levels of a particle with the same conclusion that at least a single internal variable is required FIG. 2: A composite system Σ consisting of two identical subsystems Σ1 at temperature T1 and Σ2 at temperature T2. It will be seen later in Sec. VII A that the thermodynamic temperature of Σ can be defined as T given by Eq. (64a). The irreversibility in Σ requires one internal variable ξ given in Eq. (25).
to express e l = e l (V, ξ) for each level l. We can also visualize the above system in terms of microstates. A microstate m k refers to a particular distribution of the N particles in any of the levels with energy E k = l N l e l , where N l is the number of particles in the lth level, and is obviously a function of N, V, ξ so we will express it as E k (N, V, ξ). This makes the average energy of the system also a function of N, V, ξ, which we express as E(N, V, ξ).

C. Disparate Degrees of freedom
In classical statistical mechanics, the kinetic and potential energies K and U , respectively, are functions of independent variables. Only their sum K +U = E can be controlled from the outside, but not individually. Thus, one of them can be treated as an internal variable. In a NEQ states, each term can have its own temperature. Only in EQ, do they have the same temperature.
This has an important consequence for glasses [51,52], where the translational degrees of freedom come to EQ with the heat bath at T 0 faster than the configurational degrees of freedom, which have a different temperature than T 0 . The disparity cannot be controlled by the observer so it plays the role of an internal variable.
Consider a collection of semiflexible polymers in a solution on a lattice. The interaction energy E consists of several additive terms as discussed in [43,Eq. (40)]: the interaction energy E ps between the polymer and the solvent, the interaction energy E ss between the solvent, the interaction energy E pp between polymers. Only the total E can be controlled from the outside so the remaining terms determine several internal variables.

D. Nonuniformity
In the examples above, the internal variables are not due to any spatial inhomogeneity. An EQ system is uniform. Thus, the presence of ξ suggests some sort of nonuniformity in the system. To appreciate its physics, we consider a slightly different situation below as possible examples of nonuniformity. We will consider a system Σ for simplicity.
(a) We consider as a simple NEQ example a composite isolated system Σ, see Fig. 2, consisting of two subsystems Σ 1 and Σ 2 of identical volumes and numbers of particles but at different temperatures T 1 and T 2 at any time t < τ eq before EQ is reached at t = τ eq so the subsystems have different time-dependent energies E 1 and E 2 , respectively. We assume a diathermal wall separating Σ 1 and Σ 2 . Treating each subsystem in EQ at each t, we write their entropies as S 1 (E 1 , V /2, N/2) and S 2 (E 2 , V /2, N/2), which we simply show as S 1 (E 1 ) and S 2 (E 2 ) as we will not let their volumes and particles numbers change. The entropy S of Σ is a function of E 1 and E 2 . Obviously, Σ is in a NEQ state at each t < τ eq . From E 1 and E 2 , we form two independent combinations so that we can express the entropy as S(E, ξ) for Σ treated as a black box Σ B ; we do not need to know about its interior (its inhomogeneity) anymore. Here, ξ plays the role of an internal variable, which continues to relax towards zero as Σ approaches EQ. For given E and ξ, S(E, ξ) has the maximum possible values since both S 1 and S 2 have their maximum value. As we will see below, this is the idea behind the concept of internal equilibrium in which S(E, ξ) is a state function of state variables and continues to increase as ξ decreases and vanishes in EQ. In this state, S(E, ξ = 0) has the maximum possible value for fixed E so it becomes a state function. This case and its various extensions are investigated in MNEQT in Sect. VII A.
(b) We can easily extend the model to include four identical subsystems of fixed and identical volumes and numbers of particles, but of different energies E 1 , E 2 , E 3 , and E 4 . Instead of using these 4 independent variables, we can use the following four independent combinations to express the entropy of Σ as S(E, ξ, ξ ′ , ξ ′′ ). The pattern of extension for this simple case of energy inhomogeneity. is evident.
(c) We make the model a bit more interesting by allowing the volumes V 1 and V 2 to also vary as Σ equilibrates. Apart from the internal variable ξ, we require another internal variable ξ ′ to form two independent combinations so that we can use S(E, V, ξ, ξ ′ ) for the entropy of Σ in terms of the entropies of Σ 1 and Σ 2 .
(d) In the above examples, we have assumed the subsystems to be in EQ. We now consider when the subsystems are in IEQ. We consider the simple case of two subsystems Σ 1 and Σ 2 of identical volumes and numbers of particles. Each subsystem is in different IEQ states described by E 1 , ξ 1 and E 2 , ξ 2 . We now construct four independent combinations which can be used to express the entropy of Σ as S(E, ξ, ξ ′ , ξ ′′ ). (e) The example in (a) can be easily extended to the case of expansion and contraction by replacing E, E 1 , and E 2 by N, N L , and N R to describe the diffusion of particles from the left (L) region to the right (R) region containing N L and N R particles, respectively. The role of β and E, etc. are played by βµ and N , etc.
It should be clear from the above examples that the choice of the number n * of internal variables is determined uniquely by a particular modeling of Σ b . The number is fixed by matching the number of independent variables needed for the subsystems to be exactly the number of variables needed for Σ b . This uniqueness ensures that the entropy of Σ b is uniquely determined by the sum of the entropies of the subsystems. Ensuring subsystems to be in EQ so that their entropies are known, the uniqueness implies that the entropy of Σ b is also known as a function of all the variables of subsystems.

IV. GENERAL FORMULATION OF THE STA-TISTICAL ENTROPY
In this section, we will use S for the statistical entropy to distinguish it from the thermodynamic entropy S. We provide a very general formulation of S for a general body Σ b , which will be shown to be identical to the thermodynamic entropy S by appealing to the third law. As a consequence, this will also demonstrate that the entropy in general is a statistical average. We consider a state M(t) ≡ M(Z(t)) of Σ b at a given instant t. In the following, we suppress t unless necessary. The state M(Z) refers to the sets of microstates m = {m k } and their probabilities p = {p k }. The microstates are determined by the Hamiltonian of the body, whose value will determine their energies E k for a given set W(t). Thus, m k is specified by (E k (t), W(t)), and may not uniquely specify M(t) unless we are in the appropriate state space where they become uniquely specified. In the following, we will not require M(t) to be unique in the state space S Z . We will denote Z(t) by Z so that we can separate out the explicit variation due to t, and simply use M in the following.
For the computation of combinatorics, the probabilities are handled in the following abstract way. We consider a large number N = CW (Z) of independent replicas or samples of Σ b , with C some large constant integer and W (Z) the number of distinct microstates m k . The samples should be thought of as identically prepared experimental samples [37].

A. Simply
Connected Sample Space

An Isolated Body
We assume that Γ(Z) ⊃ Γ(X) is simply connected in this section. Let N k (t) denote the number of kth samples (samples in the m k -microstate) so that The above sample space is a generalization of the ensemble introduced by Gibbs, except that the latter is restricted to an equilibrium body in S X , whereas our sample space refers to the body in S Z in any arbitrary state for which p k may be time-dependent. The ensemble average of some quantity Z over these samples is given by where Z k is the value of Z in m k . This definition is identical to the Gibbs ensemble average in Eq. (1). The samples are, by definition, independent of each other so that there are no correlations among them. Because of this, we can treat the samples to be the outcomes of some random variable, the state M. This independence property of the outcomes is crucial in the following, and does not imply that they are equiprobable. The number of ways W to arrange the N samples into W (Z) distinct microstates is Taking its natural log to obtain an additive quantity per sample, and in accordance with Boltzmann's postulate in Eq. (2), and using Stirling's approximation, we see easily that the statistical entropy S, which we hope to identify later with the entropy S(Z) of Σ b , can be written as the average of the negative of what Gibbs [3] calls the index of probability: where we have also shown an explicit time-dependence, which is distinct from the implicit time-dependence in Z. The explicit time-dependence in S(Z, t) merely reflects the fact that it is not a state function and that M is not uniquely specified in S Z . The above derivation is based on fundamental principles and the Boltzmann hypothesis and does not require the body to be in equilibrium; therefore, it is always applicable. To the best of our knowledge, even though such an expression has been extensively used in the literature, it has been used without any derivation; one simply appeals to this form by appealing to the information entropy [30,71]; however, see Sect. X.Thus, Eq. (34) is a generalization of Eq. (3) to the general case, and thus justifies it for an arbitrary M.

Remark 2
The statistical entropy S appears as an instantaneous ensemble average, see Eq.
(1) or (30). This average should be contrasted with a temporal average in which a quantity ϕ is considered as the average over a long period τ 0 of time where ϕ(t) is the value of ϕ at time t [15]. For M eq , both definitions give the same result provided ergodicity holds. The physics of this average is that ϕ(t) at t represents one of the microstate of M eq . As M eq is invariant in time, these microstates belong to M eq , and the time average is the same as the ensemble average if ergodicity holds. However, for a NEQ state M(t), which continuously changes with time, the temporal average is not physically meaningful as the microstate probabilities at time t corresponds to M(t) and not to M(t = 0) in that the probabilities and Z are different in the two states.
Only the ensemble average makes any sense at any time as was first pointed out in [53] because p k 's correspond to M(t = 0). Because of this, we only consider ensemble averages in this chapter.
Because of its similarity in form with S (c) G in Eq. (3), we will refer to S(Z, t) simply as the Gibbs or the statistical entropy from now on. The distinction between S and S should be emphasized. The latter appears in the Gibbs fundamental relation that relates the energy change dE with the entropy change dS as is well known in classical thermodynamics, see Eqs. (12) and (18b) in S Z . The concept of microstates is irrelevant for this relation, which is purely thermodynamic. On the other hand, S is solely determined by {m k }, and is a statistical quantity which may not always satisfy Eq. (12) in S Z .
To identify S(Z, t) with the NEQ thermodynamic entropy S requires the following additional steps: (1) It is necessary to establish that S(Z, t) satisfies Eq. (9).
(2) For body in canonical equilibrium, it is necessary to establish that S(t) is identical to the equilibrium thermodynamic entropy given by S G [3,15].
(3) It is necessary to show that S(Z) is identical to S(Z) for M ieq .
(4) It is necessary to show that S(Z, t) is identical to S(Z, t) for M nieq .
There are several proofs available in the literature [6,7,36,37,53] for (1). Here, we give a simple proof of it in Sect. IV C. We will prove (2) and (3) in Sect. V B, where we follow closely their justification given earlier [31,32]. We prove (4) in Sect. V C.
A word of caution must be offered. If S is not a state function, it cannot be measured or computed. Thus, while S can be computed in principle in all cases, there is no way to compare its value with S in all cases. Thus, no comment can be made about their relationship in general. We merely conjecture with respect to (4) that as the two entropies are the same when the thermodynamic entropy is a state function, it is no different from its statistical analog even when it is not a state function by following, in principle, the procedure described in Sect. V C.
This allows us to identify S as the statistical entropy formulation of the thermodynamic entropy S in all cases; see Proposition 8. Indeed, we use S, which is defined for any arbitrary state, to define the thermodynamic entropy S in all cases. From now on, we will not make any distinction between them. We summarize this conclusion by the following Remark 3 Because of this equivalence, we will no longer make any distinction between the statistical entropy and the thermodynamic entropy and will use the standard notation S for both of them, unless clarity is needed.
The maximum possible value of S(Z, t) for given Z occurs when m k are equally probable (ep): In this case, the explicit time dependence in S(t) will disappear and we have which is identical in form to the Boltzmann entropy in Eq. (2) for an isolated body in equilibrium, except that the current formulation has been extended to an isolated body out of equilibrium; see also Eq. (24). The only requirement is that all microstates in m 0 ≡ m 0 (Z) are equally probable. The statistical entropy in this case becomes a state function, just as the classical entropy is for M ieq that was treated in Sect. II C. As the equivalence covers M eq , this proves (3). The proof for (2) is deferred to Sect. (V B), where we discuss p k .
The simplest way to understand the physical meaning of Eq. (36) is as follows: Consider Z ∈ S Z at some time t. As S(Z, t) may not be a unique function of Z, we look at all possible entropy functions for this Z. These entropies correspond to all possible sets of {p k (t)} for a fixed Z, and define different possible states M(Z) . We pick that particular M(Z) ∈ M(Z) among these that has the maximum possible value of the entropy, which we denote by S(Z) or S(Z(t)) without any explicit tdependence. This entropy is a state function S(Z) in S Z . For a macroscopic body, this occurs when the corresponding microstate probabilities for M(Z) are given by p ep k above.

Remark 4
We emphasize that Z = (E, W) so p ep k is determined by the average energy E and not by the microstate energy E k as derived later in Sect. (V B). The equiprobability assumption in Eq. (35) basically replaces the actual probability distribution in Eq. (46) by a flat distribution of height 1/W (Z) and width W (Z), a common practice in statistical mechanics [15]. Despite this modification, the entropy has the same value S(Z) for a macroscopic body, for which β and F w are given by Eq. (11a).
Applying the above formulation to a state characterized by a given X in S X and consisting of microstates {m k }, forming the set m ≡ m(X), with probabilities {p k (t) > 0}, we find that is the entropy of this state, where W (X) is the number of distinct microstates m k . It should be obvious that Again, under the equiprobable assumption Γ(X) denoting the space spanned by microstates {m k }, the above entropy takes its maximum possible value; here, which is identical in value to the Boltzmann entropy in Eq. (2) for an isolated body in equilibrium. The maximum value occurs at t = τ eq . It is evident that We will refer to S(Z) in terms of microstate number W (Z) in Eq. (36) as the time-dependent Boltzmann formulation of the entropy or simply the Boltzmann entropy [41], whereas S(X) in Eq. (38) represents the equilibrium (Boltzmann) entropy. It is evident that the Gibbs formulation in Eqs. (34) and (37) supersedes the Boltzmann formulation in Eqs. (36) and (38), respectively, as the former contains the latter as a special limit. However, it should be also noted that there are competing views on which entropy is more general [40,41]. We believe that the above derivation, being general, makes the Gibbs formulation more fundamental. The continuity of S(Z, t) follows directly from the continuity of p k (t). The existence of the statistical entropy S(Z, t) follows from the observation that it is bounded above by ln W (Z) and bounded below by 0, see Eq. (36).
It should be stressed that W is not the number of microstates of the N replicas; the latter is given by [W (Z)] N . Thus, the entropy in Eq. (32) should not be confused with the Boltzmann entropy, which would be given by N ln W (Z). It should be mentioned at this point that Boltzmann uses the combinatorial argument to obtain the entropy of a gas, see Eq. (81), resulting in an expression similar to that of the Gibbs entropy in Eq. (3) except that the probabilities appearing in his formulation represents the probability of various discrete states of a particle, and should not be confused with the microstate probabilities used here; see Sect. IX. The approach of Boltzmann is limited to that of an ideal gas only and is not general as it neglects the correlations present due to the interactions between particles [36,41]. On the other hand, our approach is valid for any body with any arbitrary interactions between particles as all microstates in the collection are independent.

System in a Medium and Quasi-independence
The above formulation of S(Z, t) can be applied to Σ, Σ, and Σ 0 . We assume that Σ, and Σ are quasiindependent so that S 0 (t) can be expressed as a sum of entropies S(t) and S(t) of Σ, and Σ, respectively: The two statistical entropies are given by an identical formulation respectively. Here, m k with probability p k denotes a microstate of Σ and m k with probability p k that of the medium. In the derivation of the above additivity, see [43], we have neither assumed the medium nor the system to be in internal equilibrium; only quasi-independence is assumed. The above formulation of the additivity of statistical entropies will not remain valid if the two are not quasi-independent. From this, we also conclude that the additivity will also not be true of the thermodynamic entropies.

B. Disjoint Sample Space (Component Confinement)
The consideration of dynamics resulting in the simple connectivity of the sample (or phase) space has played a pivotal role in developing the kinetic theory of gases [34,41], where the interest is at high temperatures [15,37,53,54]. As dynamics is very fast here, it is well known that the ensemble entropy agrees with its temporal formulation. However, at low temperatures, where dynamics becomes sluggish as in a glass [51,52,55,56], the body can be confined into disjoint components.
Sample (or phase) space confinement at a phase transition such as a liquid-gas transition is well known in equilibrium statistical mechanics [15,37,53]. It also occurs when the body undergoes symmetry breaking such as during magnetic transitions, crystallizations, etc. But confinement can also occur under nonequilibrium conditions, when the observational time scale τ obs becomes shorter than the equilibration time τ eq [51,52,55,56], such as for glasses, whose behavior and properties have been extensively studied. In the following, we will focus on Σ b .
The issue has been recently considered by us [37], where only energy as an observable was considered. The discussion is easily extended to the present case when confinement occurs for whatever reasons into one of the thermodynamically significant number of disjoint components Γ λ , λ = 1, 2, 3 · · · , C, each component corresponding to the same set Z. Such a situation arises, for example, in Ising magnets at the ferromagnetic transition., where the system is either confined to Γ + with positive magnetization or Γ − with negative magnetization. Even a weak external magnetic field |H| → 0, that we can control as an observer, will allow the system to make a choice between the two parts of Γ. It just happens that in this case C = 2 and is thermodynamically insignificant.
The situation with glasses or other amorphous materials is very different [55], In the first place, Γ is a union of thermodynamically significant number C ∼ e cN , c > 0, disjoint components. In the second place, there is no analog of a symmetry breaking field. Therefore, there is no way to prepare a sample in a given component Γ λ . Thus, the samples will be found in all different components. Taking into consideration disjointness of the components generalizes the number of configurations in Eq. (31) to where N k λ denotes the number of sample in the microstate m k λ in the λ-th component. In terms of p k λ = N k λ (t)/N , this combination immediately leads to for the statistical entropy of the system and has already been used earlier [37, see Sec. 4.3.3] by us. From what has been said above, this statistical entropy is also the thermodynamic entropy of a nonequilibrium state under component confinement for which the entropy is a state function of Z. Therefore, as before, we take S to be the general expression of the nonequilibrium thermodynamic entropy and use S in place of S. Introducing it is easy to see [37] that Here, the entropy of the component Γ λ in terms of the reduced microstate probability p k λ ≡ p k λ /p λ is so that the first contribution is its average over all components. The second term is given by and represents the component entropy. It is this entropy that determines the residual entropy [53] in disordered systems for Σ b = Σ.

C. A Proof of The Second Law
The second law has been proven so far under different assumptions [6, 7, 36, 37, 53, among others]. Here, we provide a simple proof of it based on the postulate of the flat distribution; see Remark 4. The current proof is an extension of the proof given earlier, see [37,Theorem 4]. We consider an isolated system Σ b = Σ 0 for which the second law is expressed by Eq. (9). As the law requires considering the instantaneous entropy as a function of time, we need to focus on the sample space at each instant to determine its entropy S as a function of time. At each instance, it is an ensemble average over the instantaneous sample space Γ(t) formed by the instantaneous set m(t) of available microstates, see Eq. (34). We will use the flat distributions for the microstates at each instance, see Remark 4, so that the entropy is given by Eq. (36).
To prove the second law, we proceed in steps by considering a sequence of sample spaces belonging to Γ(t) as follows [37,53]. At a given instant, Σ b happens to be in some microstate. We start at t = t 1 = 0, at which time Σ happens to be in a microstate, which we label m 1 . It forms a sample space Γ 1 containing m 1 with probability p (1) 1 = 1, with the superscript denoting the sample space. We have S (1) = 0. At some t = t 2 , the sample space is enlarged from Γ 1 to Γ 2 , which contains m 1 and m 2 , with probabilities p 2 . Using the flat distribution, the entropy is now S 2 = ln 2. We just follow the system in a sequence of time so that at t = t n , we have a sample space Γ n with m 1 , m 2 , · · · , m n so that S n = ln n. Continuing this until all microstates in Γ have appeared, we have S max = ln W .
Thus, we have proven that the entropy continues to increase until it reaches its maximum in accordance with the second law in Eq. (9).

A. Choice of Z for M ieq
We first discuss how to choose a particular state space for a unique description of M depending on the experimental setup. To understand the procedure for this, we begin by considering a set ξ n of internal variables (ξ 1 , ξ 2 , · · · , ξ n ) and Z n . = X ∪ ξ n to form a sequence of state spaces S (n) Z . In general, one may need many internal variables, with the value of n increasing as M is more and more out of EQ [47] relative to M eq . We will take n * to be the maximum n in this study as discussed in Sect. III, even though n << n * needed for S (n) Z will usually be a small number in most cases. The two most important but distinct time scales are τ obs , the time to make observations, and τ eq , the equilibration time for a state M to turn into M eq . For τ obs < τ eq , the system will be in a NEQ state. Let τ i denote the relaxation time of ξ i needed to come to its equilibrium value so that its affinity A i → 0 [9,12,18,47,51,52]. For convenience, we order ξ i so that we assume distinct τ i 's for simplicity without affecting our conclusions. For τ 1 < τ obs , all internal variables have equilibrated so they play no role in equilibration except thermodynamic forces T −T 0 , P −P 0 , etc. associated with X that still drive the system towards EQ. We choose n satisfying τ n > τ obs > τ n+1 so that all of ξ 1 , ξ 2 , · · · , ξ n have not equilibrated (their affinities are nonzero). They play an important role in the NEQT, while ξ n+1 , ξ n+2 , · · · need not be considered as they have all equilibrated. This specify M uniquely in S (n) Z , which was earlier identified as being in IEQ.
Note that NEQ states with τ n+1 > τ obs > τ n+2 are not uniquely identifiable in S Z . Thus, given τ obs , we look for the window τ n > τ obs > τ n+1 to choose the particular value of n. This then determines S (n) Z in which the states are in IEQ. From now onward, we assume that n has been found and S (n) Z has been identified. We now suppress n and simply use S Z below.

B. Microstate probabilities for M ieq
As M ieq is unique in S Z , we need to identify the unique set {p k }. As we keep W fixed in M ieq as the parameter, then F wk , the value F w takes on m k , are fluctuating forces in S Z that satisfy a sum rule, just as the microstate energies E k : We need to maximize the entropy S(Z) at fixed E and F w by varying p k without changing {m k } , i.e. without changing E k and F wk to identify the particular set {p k } to achieve it. Using the Lagrange multiplier technique, it is easy to show that the condition for this in terms of three Lagrange multipliers with obvious definitions is from which follows the statistical entropy S = −(λ 1 + λ 2 E + λ 3 · F w ); we have reverted back to the original symbol for the statistical entropy here. It is now easy to identify λ 2 = −β, λ 3 βW by comparing dS with dS in Eq. (10) by varying E and W so we finally have where λ 1 = β G with G(t) is a normalization constant and defines a NEQ partition function It is easy to verify that so that if we neglect the fluctuations E k − E and F wk − F w , then p k reduces to the flat distribution p k = 1/W (E, W) in Remark 4, which can be identified as the microstate probability in the NEQ microcanonical ensemble. It should be remarked that the Lagrange multipliers in p k are determined by comparing dS with dS in the Gibbs fundamental relation, a thermodynamic relation. This then proves that S is the same as the thermodynamic entropy S up to a constant [32], which can be fixed by appeals to the third law. We do not consider here the issue of a residual entropy, which is discussed elsewhere [47,53] and can be done based on the discussion in Sect. IV B. The p k above clearly shows the effect of irreversibility and is very different from its equilibrium analog p eq k in S X : obtained by replacing W by w, F wk by f w0k , and β by β 0 . The fluctuating E k , f wk satisfy The observation time τ obs is determined by the way T and W are changed during a process. Thus, during each change, τ obs must be compared with the time needed for Σ to come to the next IEQ state, and for the microstate probabilities to be given by Eq. (46) with the new values of T and W.

C. M nieq in SZ
We now focus on a non-unique state M nieq in S Z . This will be needed if τ obs is reduced to make the process faster so that instead of falling in the window (τ n , τ n+1 ), it now falls in a higher window such as (τ n+1 , τ n+2 ). As said above, M can now be treated as a unique state in a larger state space S Z ′ ⊃ S Z . Let ξ ′ (t) denote the set of additional internal variables needed over S Z so that The entropy S(Z ′ (t)) = S(Z(t), t) for M ieq (t) in S Z ′ satisfies the Gibbs fundamental relation where W is the work variable in S Z . Expressing the last term as we obtain the following generalization of the Gibbs fundamental relation for M nieq (t) in S Z : where ∂S ∂t In S Z ′ , we can identify the temperature T as the thermodynamic temperature in S Z ′ using the standard definition. But, it is clear from the above discussion that ∂S(Z ′ (t))/∂E in S Z ′ has the same value as ∂S(Z(t), t)/∂E in S Z . Therefore, we are now set to identify T as a thermodynamic temperature for any arbitrary state M.

Remark 5
The temperature T in S Z and S Z ′ are the same Definition 6 As the presence of ∂S/∂t above in S Z is due to "hidden" internal variables in ξ ′ , we will call it the hidden entropy generation rate, and the hidden entropy generation. It results in a hidden irreversible work in S Z due to the hidden internal variable with affinity A ′ .
Remark 7 A state M nieq (t) with S(Z(t), t) can be converted to M ieq (t) with a state function S(Z ′ (t)) in an appropriately chosen state space S Z ′ ⊃ S Z by finding the appropriate window in which τ obs lies. The needed additional internal variable ξ ′ determines the hidden entropy generation rate ∂S/∂t in Eq. (51) due to the non-IEQ nature of M nieq (t) in S Z , and ensures validity of the Gibbs relation in Eq. (50) for it, thereby providing not only a new interpretation of the temporal variation of the entropy due to hidden variables but also extends the MNEQT to M nieq (t) in S Z .
The above discussion strongly points towards the possible Proposition 8 The MNEQT provides a very general framework to study any M nieq (t) in S Z , since it can be converted into a M ieq (t) in an appropriately chosen state space S Z ′ , with d i S hid (t) originating from hidden internal variable ξ ′ .

Remark 9
In a process P resulting in M nieq (t) in S Z , it is natural to assume that the terminal states in P are M ieq so the affinity corresponding to ξ ′ must vanish in them.
Remark 10 By replacing Z by X, and Z ′ by Z, we can also express the Gibbs fundamental relation for any NEQ state in S X as by treating M as M ieq in S Z . In a NEQ process P between two EQ states but resulting in M nieq (t) between them in S Z , the affinity corresponding to ξ must vanish in the terminal EQ states of P.
Eq. (54) proves extremely useful to describe M in S X as it may not be easy to identify ξ in all cases.

Remark 11
The explicit time dependence in the entropy for M neq in S X or M ieq (t) in S Z is solely due to the internal variables, which do not affect dQ = T dS, with T defined as the inverse of ∂S/∂E at fixed w, t or W, t in the two state spaces, respectively.

VI. A MODEL ENTROPY CALCULATION
We consider a gas of non-interacting identical structureless particles with no spin, each of mass m, in a fixed region confined by impenetrable walls (infinite potential well). Initially, the gas is in a NEQ state, and is isolated in that region. In time, the gas will equilibrate and the microstate probabilities change in a way that the entropy increases. We wish to understand how the increase happens.
A. 1-dimensional ideal Gas: In order to be able to carry out an exact calculation, we consider the gas in a 1-dimensional box of initial size L in . As there are no interactions between the particles, the wavefunction Ψ for the gas is a product of individual particle wavefunctions ψ. Thus, we can focus on a single particle to study the nonequilibrium behavior of the gas [57][58][59][60][61]. The simple model of a particle in a box has been extensively studied in the literature but with a very different emphasis [62][63][64]. The particle only has nondegenerate eigenstates whose energies are determined by L, and a quantum number k. We use the energy scale ε 1 = π 2 2 /2mL 2 to measure the energy of the eigenstate, and α = L/L in so that the corresponding eigenfunction is given by The pressure generated by the eigenstate on the walls is given by [23] P k (L) ≡ −∂ε k /∂L = 2ε k (L)/L.
In terms of the eigenstate probability p k (t), the average energy and pressure are given by (58b) The single particle entropy follows from Eq. (34) by using the single particle probability p k (t). The time dependence in ε(t) or P (t) is due to the time dependence in p k and ε k (L). Even for an isolated system, for which ε remains constant, p k cannot remain constant as follows directly from the second law [31] and creates a conceptual problem because the eigenstates are mutually orthogonal and there can be no transitions among them to allow for a change in p k . As the gas is isolated, its energy, volume and the number of particles remain constant. As it is originally not in equilibrium, it will eventually reach equilibrium in which its entropy must increase. This requires the introduction of some internal variables even in this system whose variation will give rise to entropy generation by causing internal variations d i p k (t) in p k (t). Here, we will assume a single internal variable ξ(t). What is relevant is that the variation in ξ(t) is accompanied by changes dp k (t) occurring within the isolated system.

B. Chemical Reaction Approach
A way to change p k in an isolated system is to require the presence of some stochastic interactions, whose presence allows for transitions among eigenstates [37]. As these transitions are happening within the system, we can treat them as "chemical reactions" between different eigenstates [4,12,18] by treating each eigenstate k as a chemical species. During the transition, these species undergo chemical reactions to allow for the changes in their probabilities.
We follow this analogy further and extend the traditional approach [4,12,18] to the present case. For the sake of simplicity, our discussion will be limited to the ideal gas in a box; the extension to any general system is trivial. Therefore, we will use microstates {m k } instead of eigenstates in the following to keep the discussion general. Let there be N k (t) particles in m k at some instant t so that at all times, and p k (t) = N k (t)/N . We will consider the general case that also includes the case in which final microstates refer to a box size L ′ different from its initial value L. Let us use A k to denote the reactants (initial microstates) and A ′ k to denote the products (final microstates). For the sake of simplicity of argument, we will assume that transitions between microstates is described by a single chemical reaction, which is expressed in stoichiometry form as Let N k and N ′ k denote the population of A k and A ′ k , respectively, so that N = k N k = k N ′ k . Accordingly, p k (t) = N k (t)/N for the reactant and p k (t + dt) = N ′ k (t)/N for the product. The single reaction is described by a single extent of reaction ξ and we have It is easy to see that the coefficients satisfy an important relation which reflects the fact that the change |dN | in the reactant microstates is the same as in the product microstates. The affinity in terms of the chemical potentials µ is given by and will vanish only in "equilibrium," i.e. only when p k ' s attain their equilibrium values. Otherwise, A(t) will remain non-zero. It acts as the thermodynamic force in driving the chemical reaction [4,12,18]. But we must wait long enough for the reaction to come to completion, which happens when A(t) and dξ/dt both vanish. The extent of reaction ξ is an example of an internal variable. There may be other internal variables depending on the initial NEQ state as discussed in Sect. III.

VII. SIMPLE APPLICATIONS
A. Composite Σ with Temperature Inhomogeneity.
Here, we will show by examples that the thermodynamic temperature T of Σ allows us to treat it as a "black box" Σ B without knowing its detailed internal structure such as its composition in terms of two subsystems Σ 1 and Σ 2 . Alternatively, we can treat Σ as a combination Σ C of Σ 1 at T 1 and Σ 2 at at T 2 < T 1 , and obtain same thermodynamics. Thus, both approaches are equivalent, which justifies the usefulness of T as thermodynamically appropriate global temperature.
In the following, we will consider various cases that can be obtained as special cases of the following general situation: Σ 1 in thermal contact with a heat medium Σ h1 at temperature T 01 , and Σ 2 in thermal contact with another heat medium Σ h2 at temperature T 02 , with the two media having no mutual interaction.
We will consider the two realizations for Σ: Σ B and Σ C to compare their predictions. As discussed for the case (b) in Sect. III, Σ 1 and Σ 2 are always taken to be in EQ, but Σ in IEQ. The entropies in the two realizations are S B (t) = S(E(t), ξ(t)); S C = S 1 (E 1 (t)) + S 2 (E 2 (t)), and have the same value; recall that E(t) = E 1 (t)+E 2 (t), and ξ(t) = E 1 (t) − E 2 (t) for Σ(t); see Eq. (25). For clarity, we will often use the argument t to emphasize the variations in time t in this section. In general, the irreversible entropy generation is given by where dS should be replaced by dS B or dS C as the case may be: where we are using the inverse temperatures for various bodies. Let d e Q l (t), l = 1, 2 be the energy or heat transferred to Σ l (t) from Σ (l) h , and dE in (t) = d e Q in (t) the energy or heat transferred from Σ 1 (t) to Σ 2 (t). We have, using δ 1 = −1 and δ 2 = +1, We see that dE(t) is unaffected by the internal energy transfer dE in (t), while is affected by the heat exchange disparity d e Q 1 (t) − d e Q 2 (t) along with dE in (t).
We finally have We now consider various cases to make an important point.

Isolated Σ
We first consider the realizations Σ B . Using dE(t) = dE 1 (t) + dE 2 (t), dξ(t) = dE 1 (t) − dE 2 (t), see Eqs. (25), and (61) for dS B (t) above, we obtain This identifies T (t) in terms of T 1 (t) and T 2 (t). As EQ is attained, T (t) → T 0 , the EQ temperature between Σ 1 and Σ 2 , and A(t) → A 0 = 0 as expected. In the following, we will use A ′ (t) for β(t)A(t) for simplicity. In terms of β and A ′ , we also have We now justify that in this simple example, where we have also used Eq. (21). It should be emphasized that the existence of d i S(t) ≥ 0 due to ξ in M ieq is consistent with M ieq as a NEQ state, even though its entropy is a state function in the extended state space. We now consider Σ C , which is also very instructive to understand the origin of d i S(t) in a different way. Considering internal energy or heat transfer dE in (t) = d e Q in (t) between Σ 1 (t) and Σ 2 (t) at some instant t, we have due to this transfer. This results in Thus, the physical origin of d i S(t) is the internal entropy change of the subsystems, and shows how d i S(t) can be measured by measuring the EQ temperatures T 1 and T 2 and dE in (t) between them. It is this internal energy flow that gives rise to ξ in this case.

Σ Interacting with Σ h
To further appreciate the physical significance of the NEQ T (t) of the above composite system Σ(t), we allow it to interact with a heat medium Σ h at its EQ temperature T 0 . For this, we take Σ h1 and Σ h2 at the same common temperature T 0 = T 01 = T 02 above so that we can treat them as a single medium Σ h with heat exchange d e Q(t). We thus obtain from Eq. (63) We will consider two different kinds of interaction below: (i) We first consider Σ B (t) in M ieq at T (t) so we use dS B (t) above. We thus have which is consistent with the general identity where d i W is replaced using Eq. (65). The above identity is derived for a single (composite) system at temperature T (t). This confirms that the composite Σ C here can be treated as a noncomposite Σ B at T (t). To be convinced that the above d i S(t) includes the internally generated irreversibility in Eq. (65) due to heat transfer between Σ 1 (t) and Σ 2 (t), we only have to set d e S(t) = 0 to ensure the isolation of Σ. We reproduce Eq. (65) as The remaining source of irreversibility T (t)d i S Q (t) given by the first term above is due to external heat exchange between Σ and Σ h as expected.
(ii) We take treat Σ(t) as Σ C (t) in contact with Σ h . We deal directly with the two heat exchanges d e Q l (t), l = 1, 2 to Σ l (t) from Σ h , and the internal energy transfer dE in (t). Using dE l (t) from Eq. (62a) in dS C given in Eq. (63), we find that Using Eq. (64b) to express β l , we can rewrite the above equation as where we have used the identity and have found using its general definition dξ(t) = dE 1 (t) − dE 2 (t). We thus see that d i S(t) obtained by both realizations are the same as they must. However, the realization Σ C (t) allows us to also identify dξ. Each exchange generates irreversible entropy following Eq. (68a). Using d e Q(t) = d e Q 1 (t) + d e Q 2 (t) in dQ(t) = T (t)dS(t) to determine d i Q(t), we find the generalization of Eq. (67): (71) It is easy to see that the last term above gives nothing but the sum of the irreversible entropy due to external exchange of heat by Σ 1 (t) and Σ 2 (t) with Σ h : where is the external entropy exchange of Σ l (t) with Σ h . Thus, whether we treat Σ as a system at temperature T (t) or a collection of Σ 1 (t) and Σ 2 (t) at temperatures T 1 (t) and T 2 (t), respectively, we obtain the same irreversibility. In other words, T (t) is a sensible thermodynamic temperature even in the presence of inhomogeneity.
B. Σ Interacting with Σ h1 and Σ h2 We now consider our composite Σ in thermal contact with two distinct and mutually noninteracting stochastic media Σ h1 and Σ h2 at temperatures T 01 and T 02 . We will again discuss the two different realizations as above.
(i) We first consider Σ B (t) at temperature T (t), which interacts with the two Σ h 's, and use the general result in Eq. (63). A simple calculation using dS B generalizes Eq. (67) and yields since this reduces to that result when we set β 01 = β 02 = β 0 . As above, d i Q(t) = d i W (t) = A(t)dξ(t); see Eq. (65), which gives rise to the last term above. Thus, setting d e Q l (t) = 0, l = 1, 2 to make Σ isolated, we retrieve d i S(t) in Eq. (65) as expected. The first sum above gives the external entropy exchanges with the two heat media as above.
(ii) We now consider Σ C , and allow Σ h1 to directly interact with Σ 1 (t) at temperature T 1 (t) and Σ h2 to directly interact with Σ 2 (t) at temperature T 2 (t). Using dS C generalizes Eq. (67) and yields (74b) Again using Eq. (64b) to express β l , we can rewrite the above d i S(t) as the d i S(t) in Eq. (74a) for Σ B , and also find that dξ is given by Eq. (70).
It should be emphasized that the determination of d i S(t) in Eqs. (74a-74b) is valid for all cases of Σ interacting with Σ h1 and Σ h2 as we have not imposed any conditions on T 1 (t) and T 2 (t) with respect to T 01 and T 02 , respectively. Thus it is very general. The derivation also applies to the NEQ stationary state, which happens when T 1 (t) → T 01 and T 2 (t) → T 02 . For the stationary case, using Eq. (74b), we have where all quantities on the right have their steady state values. Thus, d i S st is only determined by the stationary value of the internal energy exchange dE in . The reader can easily verify that d i S(t) in Eqs. (74a) also reduces to the above result in the stationary limit. From the above examples, we see that we can consider Σ in any of the two realization Σ B and Σ C as we obtain the same thermodynamics in that d i S(t) is identical. We emphasize this important observation by summarizing it in the following conclusion.

Conclusion 12
If we consider Σ(t) as a single system Σ B with an uniform temperature T (t) and with an internal variable ξ(t), we do not need to consider the energy transfer dE in (t) explicitly to obtain d i S(t). If we consider Σ(t) as a composite system Σ C formed of Σ 1 (t) and Σ 2 (t) at their specific temperatures, then we specifically need to consider the energy transfer dE in (t) to obtain d i S(t) but no internal variable.
This conclusion emphasizes the most important fact of the MNEQT that the homogeneous thermodynamic temperature T (t) of Σ B can also describe an inhomogeneous system Σ C . This observation justifies using the thermodynamic temperature T (t) for treating Σ(t) as a single system Σ B , a black box, without any need to consider the internal energy transfers.
The above discussion can be easily extended to also include inhomogeneities such as two different work media Σ (1) w and Σ (2) w corresponding to different pressures P 01 and P 02 . We will not do that here.

C. Σ Interacting with Σw and Σ h
In this case, Σ is specified by two observables E and V so to describe any inhomogeneity will require considering at least two subsystems Σ 1 and Σ 2 specified by E 1 , V 1 and E 2, V 2 , respectively. From these four observables, we construct the following four combinations to express the entropy in terms of FIG. 3: The calculated equilibrium (continuous) and nonequilibrium (broken) entropies per particle for an ideal gas in a box as a function of the expansion box length L. The nonequilibrium state is the result of a sudden expansion from the initial state corresponding to L = 1 and T0 = 4. The energy of the gas remains constant in the sudden expansion. As expected, the nonequilibrium entropy lies below the equilibrium entropy. In time, the former will increase to the latter as the gas equilibrates.
Note that we have assumed that Σ 1 and Σ 2 are in EQ (no internal variables for them). We now follow the procedure carried out in Sect. VII A to identify thermodynamic temperature T , pressure P , and affinities: (76) All these quantities are SI-quantities and have the same values regardless of whether Σ is isolated or interacting. A more complicated inhomogeneities will require more internal variables.

Remark 13
We now make an important remark about Eq. (23) that contains only a single internal variable. From what is said above, it must include at least two internal variables if Σ contains inhomogeneity in both E and V . If it contains inhomogeneity in only one variable, then and only then we will have at least one internal variable. Thus, either we will have ξ E or ξ V as the case may be.

D. Free (Sudden) Expansion of the Box
We consider the 1-d ideal gas considered in Sect. VI. The box expands as a function of time, which need not be quasi-static (extremely slow) so there is no reason to assume that the gas remains in equilibrium after expansion. The entropy of the gas per particle can be obtained by calculating S(L, t) = − k p k (t) ln p k (t) for the particle under consideration so that the irreversible entropy change d i S(t) will never be negative. The discussion about the chemical reaction in Sect. VI shows that the change d i p k (t) is caused by the transitions between different eigenstates.
We prepare the gas in equilibrium at some initial temperature T 0in in a box of length L in , which we take to be L in = 1. This is obtained by keeping the box in a medium of temperature T 0in . The corresponding microstate probabilities follow the Boltzmann law (β in ≡ 1/T 0in ): ) denotes the equilibrium partition function; compare with Eq. (48). The initial energy per particle ε in is obtained by replacing p k (β, L) by p eq k (β 0 , L in ) in Eq. (58); the corresponding pressure is P in = 2ε in /L in . The equilibrium entropy can be obtained by using the single particle probability p eq k (β 0 , L in ) in Eq. (34). The initial temperature T 0in is taken to be T 0in = 4, see Fig. 3, so that the initial energy ε in ≈ 2.786.
We now consider NEQ states. For this, we isolate the box from its medium and consider its free expansion as it expands suddenly from L in to a new size L > L in . Because of its isolation, its energy remains ε in during this expansion. As the expansion is sudden, the initial eigenfunctions ψ lin (x) for L in have no time to change, but are no longer the eigenfunctions of the new size L; the latter are given by ψ k (x) in Eq. (56) for L. However, ψ lin (x) can be expanded in terms of ψ k (x) as a sum over k. The corresponding expansion coefficients b kl are easily seen to be [62] b kl (L, L in ) = 2lα 3/2 (−1) l π(k 2 − α 2 l 2 ) sin( kπ α ).
Using b kl , we can determine the probability p k (β 0in , L, L in ) for the kth microstate in the new box.
We have checked that the new probabilities add to 1 and that the (average) energy after the free expansion is equal to ε in to within our computational accuracy. Thus, ∆ε = 0 in the sudden expansion. This is consistent with the fact that the gas does no external work and that no external heat is exchanged.
Despite this, the free expansion is spontaneous once the confining walls have moved. Therefore, the (thermodynamic) entropy of the gas must increase in this process in accordance with the second law. We use p k (β 0in , L, L in ) to evaluate the nonequilibrium statistical entropy, which is shown by the dashed curve in Fig.  3. The significance of this curve is as follows: Choose a particular value L in this graph. Then, the NEQ entropy for this L is given by numerically evaluating the sum in Eq. (34): S(β in , L, L in ) = − k p k (β in , L, L in ) ln p k (β in , L, L in ). This is the entropy after the sudden expansion from the initial state at L in = 1 and follows from the above quantum superposition principle. Evidently, this entropy is higher than the initial equilibrium entropy S eq (β in , L in ). It is also obvious that this entropy has a memory of the initial state at L in = 1 and T 0in = 4. Therefore, it does not represent the equilibrium entropy. If we now wait at the new value of L, the isolated gas in the new box will relax to approach its equilibrium state in which its nonequilibrium entropy will gradually increase until it becomes equal to its value on the upper curve.

VIII. 1-D TONKS GAS: A SIMPLE CONTINUUM MODEL
A careful reader would have realized by this time that the proposed entropy form in Eq. (34) is not at all the same as the standard classical formulation of entropy, such as for the ideal gas, which can be negative at low temperatures or at high pressures. The issue has been discussed elsewhere [65] but with a very different perspective. Here, we visit the same issue that allows us to investigate if and how the entropy in continuum models is related to the proposed entropy in this work. For this, we turn to a very simple continuum model in classical statistical mechanics: the Tonks gas [66,67], which is an athermal model and contains the ideal gas as a limiting case when the rod length l vanishes. We will simplify the discussion by considering the Tonks gas in one dimension. The gas consists of r impenetrable rods, each of length l lying along a line of length L. We will assume r to be fixed, but allow l and L to change with the state of the system, such as its pressure.. The configurational entropy per rod determined by the configurational partition function is found to be [67] where v is the "volume" available per rod L/r. Even though the above result is derived for an equilibrium Tonks gas, it is easy to see that the same result also applies for the gas in internal equilibrium. The only difference is that the parameters in the model are also functions of internal variables now. The entropy vanishes when v = l + 1/e and becomes negative for all v < l + 1/e. Indeed, it diverges to −∞ in the incompressible limit v = l. This is contrary to the Boltzmann approach in which the entropy is determined by the number of microstates (cf. Eq. (2)) or the Gibbs approach (cf. Eq. (34)) and can never be negative. Can we reconcile the contradiction between the continuum entropy and the current statistical formulation?
We now demonstrate that the above entropy for the Tonks gas is derivable from the current statistical approach under some approximation, to be noted below, by first considering a lattice model for the Tonks gas and then taking its continuum limit. It is in the lattice model can we determine the number of microstates. In a continuum, this number is always unbounded (see below also). For this we consider a 1-d lattice Λ f with N f sites; the lattice spacing, the distance between two consecutive sites, is given by δ. We take N f >> 1 so that L f = (N f − 1)δ ≈ N f δ is the length of the the lattice Λ f . We randomly select r sites out of N f . The number of ways, which then represents the number of configurational microstates, is given by After the choice is made, we replace each selected site by λ + 1 consecutive sites, each site representing an atoms in a rod, to give rise to a rod of length l ≡ λδ. It is clear that δ also changes with the state of the system. The number of sites in the resulting lattice Λ is so that the length of Λ is given by L = (N − 1)δ ≈ N δ since N >> 1. We introduce various densities ϕ f = r/N f , ρ f = r/N f δ ≈ r/L f and ρ = r/N δ ≈ r/L. A simple calculation shows that S = ln W c is given by This result can also be obtained by taking the athermal entropy for a polydisperse polymer solution a Bethe lattice [68] by setting the coordination number q to be q = 2. We now take the continuum limit δ → 0 for fixed ρ f and ρ, that is fixed L f and ρL, respectively. In this Use of these limits in S yields The continuum limit of the entropy from the Boltzmann approach has resulted in a diverging entropy regardless of the value of ρ f [65], a well known result. By introducing an arbitrary constant a with the dimension of length, we can rewrite S as S/r = − ln(e/ρ f a) + ln(δ/a), in which the first term remains finite in the continuum limit, and the second term contains the divergence. The diverging part, although explicitly independent of ρ f , still depends on the state of the gas through δ, and cannot be treated as a constant unless we assume δ to be independent of the state of the gas. It is a common practice to approximate the lattice spacing δ as a constant. In that case, the diverging term represents a constant that can be subtracted from S/r. Recognizing that 1/ρ f = v − l, we see that the first term in Eq. (80) is nothing but the entropy of the Tonks gas in Eq. (77) for the arbitrary constant a = 1. However, this equivalence only occurs in the state independent constant-δ approximation.
As the second term above has been discarded, the continuum entropy s c also has no simple relationship with the number (≥ 1) of microstates in the continuum limit, which means that the continuum entropy cannot be identified as the Boltzmann entropy in Eq. (38). To see this more clearly, let us focus on the centers of mass of each rod, which represent one of the r sites that were selected in Λ f . Each of the k sites x k , k = 1, 2, · · · , r, is free to move over L f . The adimensional volume |Γ f |, also called the probability and denoted by Z by Boltzmann, [35,41] of the corresponding phase space Γ f is L r f /a r . However, contrary to the conventional wisdom [41], ln |Γ f | does not yield s c . The correct expression is given by the Gibbsmodified adimensional volume |Γ f | /r!, i.e.
The presence of r! is required to restrict the volume due to indistinguishability of the rodsà la Gibbs. For large r, this quantity correctly gives the entropy s c . However, this quantity is not only not an integer, it also cannot be always larger than or equal to unity, as noted above.

IX. JAYNES REVISITED
Boltzmann [2] provides the following alternative expression of the entropy [2,69] in terms of a single particle probability p (1) i for the particle to be in the ith state: not to be confused with that in Eq. (2). Boltzmann is only interested in the maximum entropy, which occurs when all states are equally probable. In this case,

S
B, max = N ln w where w is the number of possible states of a single particle in the gas. In general, particles are not independent due to interactions and number of possible states W < w N . Accordingly, maximum Gibbs entropy S max per particle is less than the corresponding equiprobable Boltzmann entropy S The equality occurs only if there are no interactions between the particles, as we have asserted above.

X. SUMMARY AND DISCUSSION
Recognizing that there does not exists a thermodynamic definition of the classical Clausius entropy for a NEQ state M of a body Σ b , we have proposed a way to identify it by choosing a large enough state space S Z ′ in which the entropy becomes a state function S(Z ′ ) so that M becomes uniquely defined in this space. A unique state in an extended state space has been called an IEQ state M ieq . However, M ieq is no longer unique, which we denote by M nieq , in a smaller subspace S Z ⊂ S Z ′ , where its entropy S(Z, t) is also not a state function. This is discussed in Sect. V C. The Gibbs fundamental relation in Eq. (49a) for S(Z ′ ) reduces to an effective Gibbs fundamental relation in Eq. (50), in which the partial time derivative ∂S/∂t is found to be related to the hidden entropy generation d i S hid (t) ≡ (∂S/∂t)dt due to the hidden internal variable ξ ′ . This entropy generation is in addition to d i S in S Z . The sum of the two is the net entropy generation in S Z ′ . Such an identification is the reason for Proposition 8, according to which one can always identify a state space S Z ′ in which an arbitrary M becomes M ieq . The rational for this is the hierarchy of equilibration times τ i of internal variables and its interplay with τ obs that has been recently discussed by us [47], and which is briefly reviewed in Sect. V A. Thus, one can, in principle, identify S Z ′ in which any M can be uniquely specified. This then leads to a state function S(Z ′ ). However, for the purpose of computation, one needs to choose a small enough subspace S Z ⊂ S Z ′ in which the same M will become nonunique M nieq , with an entropy S(Z, t).
Unfortunately, having a Gibbs fundamental relation does not allow for a way to experimentally "measure" S(Z ′ ) or S(Z, t). This can only be done for M eq . To overcome this limitation, we propose a statistical definition S of the entropy. By giving a first principles statistical formulation of a NEQ S for Σ b in terms of microstate probabilities, we have attempted to fill in the gap. We use a formal approach (frequentist interpretation of probability) by extending the EQ ensemble of Gibbs in S X to a NEQ ensemble, which is nothing but a large number N of samples of the thermodynamic system under consideration in the enlarged state space. We refer to the ensemble as a sample space. The formal approach enables us to evaluate the combinatorics for a given set of microstate probabilities. The resulting statistical entropy is independent of the number of samples and depends only on the probabilities as is seen from Eqs. (34) and (42). Thus, the use of a large number of samples is merely a formality and is not required in practice as we can use Eq. (46) for any computation. The resulting statistical entropy S is shown to be identical to S(Z ′ ) for M ieq or S(Z, t) for M nieq , which establishes their equivalence for any arbitrary state, EQ or not, and generalizes our previous result [31,32] that only shows their equivalence (S(X) = S(X)) for M eq or (S(Z) = S(Z)) for M ieq . Our demonstration goes beyond the standard practice to use the classical nonequilibrium thermodynamics [12] or its variant to calculate NEQ entropy. In this approach, one treats the entropy at the local level as a state function without any internal variables. In contrast, our approach allows us to theoretically "measure" the classical entropy S(Z ′ ) for M ieq or S(Z, t) for M nieq by knowing the probabilities p k , and use Eq. (34) to evaluate S = S.
The choice of the appropriate state space S Z for M to become M ieq is determined by the observation time τ obs as discussed in Sect. V A. We have given several examples to show how to identify the internal variables in a unique manner for some Σ b . Once S Z has been identified, we can identify all the fields that have thermodynamic significance. Among these is the temperature T , which plays the role of a global temperature over Σ b . In addition, the internal variable ξ captures the entire internally produced irreversibility so we can treat Σ b as a black box Σ B as discussed in Sect. VII so that we do not need to know anything about its internal structure. We have shown that we obtain the same description by not using ξ and treating Σ b as a composite box Σ C requiring the knowledge of its internal structure. Both descriptions are, therefore, equivalent but we believe the extended state space description to be less tedious and more revealing.
Some readers may think that our statistical formulation is no different than that used in the information theory [29,30]. We disagree. We refer the reader to an excellent overview [70] on this topic. As pointed out by Jaynes [71], information entropy can be related to our EQ statistical mechanical entropy. Our concern is with NEQ S and the information entropy. Here, we limit the discussion to the following relevant issues that highlight their differences. For one, there is no concept of the temperature, and internal variable ξ in the latter entropy. Because of this, our approach allows us to consider multilevel descriptions so that we can consider several different entropies S(Z n , t), S(Z n ), n ≥ 1, S(X, t) and S(X) satisfying the inequalities in Eq. (39). The information theory can only deal with two levels of entropies. There is also no concept of relaxation and dynamics and the concept of τ obs so there is also no concept of component confinement in time to explain a residual entropy in the latter. It is also not clear if there is an analog of d e p k and d i p k and of the second law.
For an isolated system in internal equilibrium (p k = p for ∀m k ), just a single sample will suffice to determine the entropy as samples are unbiased. The entropy in this case is no different than the "entropy" − ln p of a single sample [37,41]: where W represents W (Z) or W (X).
Changes in microstate probabilities result in changes in the entropy. There are two ways probabilities can change within an isolated system, both of them being irreversible in nature. One cause of changes is due to the quantum nature as seen in the sudden expansion of the box. Here, the parameter λ (= L) changes non-adiabatically and creates irreversibility. The resulting irreversible change in the entropy for the 1-d gas has been calculated and shown by the lower curve in Fig. 3. The other cause of probability changes is due to the "chemical reaction" going on among the microstates that brings about equilibration in the system. The corresponding irreversible rise in the entropy for the gas is shown by the difference between the two curves in Fig. 3. The interaction of a body with its medium can also result in the changes in microstate probabilities, and has been considered elsewhere [72].
By considering a lattice model of the Tonks gas, for which the entropy remains nonnegative, we show how its continuum limit results in a negative entropy. This strongly suggests that negative entropies arise from a continuum model such as S f in Eq. (4), which in some cases can diverge to −∞, even though the statistical entropy remains finite and positive. We should also point out that the definition of S f in Eq. (4) makes no sense as f has the dimension of inverse volume in the phase space so ln f has no meaning. We suggest that our statistical Gibbs formulation can be applied to any nonequilibrium state under any condition.