# Information-Theoretic Limits of Control

## Abstract

Fundamental limits on the controllability of physical systems are discussed in the light of information theory. It is shown that the second law of thermodynamics, when generalized to include information, sets absolute limits to the minimum amount of dissipation required by open-loop control. In addition, an information-theoretic analysis of closed-loop control shows feedback control to be essentially a zero sum game: each bit of information gathered directly from a dynamical systems by a control device can serve to decrease the entropy of that system by at most one bit additional to the reduction of entropy attainable without such information (open-loop control). Consequences for the control of discrete binary systems and chaotic systems are discussed.

Information and uncertainty represent complementary aspects of control. Open-loop control methods attempt to reduce our uncertainty about system variables such as position or velocity, thereby increasing our information about the actual values of those variables. Closed-loop methods obtain information about system variables, and use that information to decrease our uncertainty about the values of those variables. Although the literature in control theory implicitly recognizes the importance of information in the control process, information is rarely regarded as the central quantity of interest . In this Letter we address explicitely the role of information and uncertainty in control processes by presenting a novel formalism for analyzing these quantities using techniques of statistical mechanics and information theory. Specifically, based on a recent proposal by Lloyd and Slotine , we formulate a general model of control and investigate it using entropy-like quantities. This allows us to make mathematically precise each part of the intuitive statement that in a control process, information must constantly be acquired, processed and used to constrain or maintain the trajectory of a system. Along this line, we prove several limiting results relating the ability of a control device to reduce the entropy of an arbitrary system in the cases where (i) such a controller acts independently of the state of the system (open-loop control), and (ii) the control action is influenced by some information gathered from the system (closed-loop control). The results are applied both to the stochastic example of coupled Markovian processes and to the deterministic example of chaotic maps. These results not only combine concepts of dynamical entropy and information in a unified picture, but also prove to be fundamental in that they represent the ultimate physical limitations faced by any control systems.

The basic framework of our present study is the following. We assign to the physical plant $`𝒳`$ we want to control a random variable $`X`$ representing its state vector (of arbitrary dimension) and whose value $`x`$ is drawn according to a probability distribution $`p(x)`$. Physically, this probabilistic or ensemble picture may account for interactions with an unknown environment, noisy inputs, or unmodelled dynamics; it can also be related to a deterministic sensitivity to some parameters which make the system effectively stochastic. The recourse to a statistical approach then allows the treatment of both the unexpectedness of the control conditions and the dynamical stochastic features as two faces of a single notion: uncertainty.

As it is well known, a suitable measure quantifying uncertainty is entropy . For a classical system with a discrete set of states with probability mass function $`p(x)`$, it is expressed as

$$H(X)\underset{x}{}p(x)\mathrm{log}p(x),$$
(1)   
(all logarithms are assumed to the base $`2`$ and the entropy is measured in bits). Other similar expressions also exist for continuous state systems (fine-grained entropy), quantum systems (von Neumann entropy), and coarse-grained systems obtained by discretization of continuous densities in the phase space by means of a finite partition. In all cases, entropy offers a precise measure of disorderliness or missing information by characterizing the minimum amount of resources (bits) required to encode unambiguously the ensemble describing the system . As for the time evolution of these entropies, we know that the fine-grained (or von Neumann) entropy remains constant under volume-preserving (unitary) evolution, a property closely related to a corollary of Landauer’s principle  which asserts that only one-to-one mappings of states, i.e., reversible transformation preserving information are exempt of dissipation. Coarse-grained entropies, on the other hand, usually increase in time even in the absence of noise. This is due to the finite nature of the partition used in the coarse-graining which, in effect, blurs the divergence of sufficiently close trajectories, thereby inducing a “randomization” of the motion. For many systems, the typical average rate of this increase is given by a dynamical invariant known as the Kolmogorov-Sinai entropy .

In this context, we now address the problem of how a control device can be used to reduce the entropy of a system or to immunize it from sources of entropy, in particular those associated with noise, motion instabilities, incomplete specification of states, and initial conditions. Although the problem of controlling a system requires more than limiting its entropy, the ability to limit entropy is a prerequisite to control. Indeed, the fact that a control process is able to localize a system in definite stable states or trajectories simply means that the system can be constrained to evolve into states of low entropy starting from states of high entropy.

To illustrate, in its most simple way, how the entropy of a system can be affected by external systems, let us consider a basic model consisting of our system $`𝒳`$ coupled to an environment $``$. For simplicity, and without loss of generality, we assume that the states of $`𝒳`$ form a discrete set. The initial state is again distributed according to $`p(x)`$, and the effect of the environment is taken into account by introducing a perturbed conditional distribution $`p(x^{}|e)`$, where $`x^{}`$ is a value of the state later in time and $`e`$, a particular realization of the stochastic perturbation appearing with probability $`p(e)`$. For each value $`e`$, we assume that $`𝒳`$ undergoes a unique evolution, referred here to as a subdynamics, taken to be entropy conserving in analog to the Hamiltonian time evolution for a continuous physical system:

$$H(X^{}|e)\underset{x^{}}{}p(x^{}|e)\mathrm{log}p(x^{}|e)=H(X).$$
(2)   
After the time transition $`XX^{}`$, the distribution $`p(x^{})`$ is obtained by tracing out the variables of the environment, and is used to calculate the change of the entropy $`H(X^{})=H(X)+\mathrm{\Delta }H`$. From the concavity property of entropy, it can be easily shown that $`\mathrm{\Delta }H0`$, with equality if and only if (iff) the state $``$ is perfectly specified, i.e., if a value $`e`$ appears with probability one. In practice, however, the environment degrees of freedom are uncontrollable and the uncertainty associated with the environment coupling can be suppressed by “updating” somehow our knowledge of $`X`$ after the evolution. One direct way to reveal that state is to imagine a measurement apparatus $`𝒜`$ coupled to $`𝒳`$ in such a way that the dynamics of the composed system $`𝒳+`$ is left unaffected. For this measurement scheme, the outcome of some discrete random variable $`A`$ of the apparatus is described by a conditional probability matrix $`p(a|x^{})`$ and the marginal $`p(a)`$ from which we can derive $`H(X^{}|A)H(X^{})`$ with equality iff $`A`$ is independent of $`X`$ . In this last inequality we have used $`H(X^{}|A)_aH(X^{}|a)p(a)`$, and $`H(X^{}|a)`$ given similarly as in Eq.(2).

Now, upon the application of the measurement, one can define the reduction of entropy of the system conditionally on the outcome of $`A`$ by $`\mathrm{\Delta }H_AH(X^{}|A)H(X)`$, which, obviously, satisfies $`\mathrm{\Delta }H_A\mathrm{\Delta }H`$, and $`H(A)\mathrm{\Delta }H\mathrm{\Delta }H_A`$. In other words, the decrease in the entropy of $`𝒳`$ conditioned on the state of $`𝒜`$ is conpensated for by the increase in entropy of $`𝒜`$. This latter quantity represents information that $`𝒜`$ posses about $`𝒳`$. Accordingly, the entropy of $`X`$ given $`A`$ plus the entropy of $`A`$ is nondecreasing, which is an expression of the second law of thermodynamics as applied to interacting systems . In a similar line of reasoning, Schack and Caves , showed that some classical and quantum systems can be termed “chaotic” because of their exponential sensitivity to perturbation, by which they mean that the minimal information $`H(A)`$ needed to keep $`\mathrm{\Delta }H_A`$ below a tolerable level grows exponentially in time in comparison to the entropy reduction $`\mathrm{\Delta }H\mathrm{\Delta }H_A`$.

It must be stressed that the reduction of entropy of $`𝒳`$ discussed so far is conditional on the outcome of $`A`$. By assumption, $`𝒳`$ is not affected by $`𝒜`$; as a result, according to an observer who does not know this outcome, the entropy of $`𝒳`$ is unchanged. In order to reduce entropy for all observers unconditioned on the state of any external systems, a direct dynamical action on $`𝒳`$ must be established externally by a controller $`𝒞`$ whose influence on the system is represented by a set of control actions $`x\stackrel{c}{}x^{}`$ triggered by the controller’s state $`c`$. Mathematically, these actions can be modelled by a probability transition matrix $`p(x^{}|x,c)`$ giving the probability that the system in state $`x`$ goes to state $`x^{}`$ given that the controller is in state $`c`$. The specific form of this actuation matrix will in general depend on the subdynamics envisaged in the control process: some of the actions, for example, may correspond to control strategies forcing several initial conditions to a common stable state, in which case the corresponding subdynamics is entropy decreasing. Others can model uncontrolled transitions perturbed by external or internal noise leading to “fuzzy” actuation rules which increase the entropy of the system. Hence, the systems $`𝒳`$ and $`𝒞`$ need not in general model a closed system; $`𝒳`$, as we already noted, can also be affected by external systems (e.g., environment) on which one has usually no control. However, formally speaking, one can always embed any open-system evolution in a higher dimensional closed system whose dynamics mimics a Hamiltonian system. This can be done by supplementing an open system with a set of ancillary variables acting as an environment $``$ in order to construct a global volume-preserving transition matrix such that, when the ancillary variables are traced out, the reduced transition matrix reproduces the dynamics of the system $`𝒳+𝒞`$.

Note that these ancillary variables thus introduced need not have any physical significance: they are only there for the purpose of simplifying the analysis of the evolution of the system. In particular, no control can be achieved through the choice of $``$. Within our model, the control of the system $`𝒳`$ can only be assured by the choice of the control $`C`$ whereby we can force an ensemble of transitions leading the system to a net entropy change $`\mathrm{\Delta }H`$. Since the overall dynamics of the system, controller and environment is Hamiltonian, Landauer’s principle immediately implies that if the controller is initially uncorrelated with the system (open-loop control), a decrease in entropy $`\mathrm{\Delta }H`$ for the system must be compensated for by an increase in entropy of at least $`\mathrm{\Delta }H`$ for the controller and the environment . Furthermore, using again the concavity property of $`H`$, it can be shown that the maximum decrease of entropy achieved by a particular subdynamics of control variable $`\widehat{c}`$ is always optimal in the sense that no probabilistic mixture of the control parameter can improve upon that decrease. Explicitly, we have the following theorem (we omit the proof which follows simply from the concavity property.)

Theorem 1.—For open-loop control, the maximum value of $`\mathrm{\Delta }H`$ can always be attained for a pure choice of the control variable, i.e., with $`p(\widehat{c})=1`$ and $`p(c)=0`$ for all $`c\widehat{c}`$, where $`\widehat{c}`$ is the value of the controller leading to $`\mathrm{max}\mathrm{\Delta }H`$. Any mixture of the control variables either achieves the maximum or yields a smaller value.

From the standpoint of the controller, one major drawback of acting independently of the state of the system is that often no information other than that available from the state of $`𝒳`$ itself can provide a reasonable way to determine which subdynamics are optimal or even accessible given the initial state. For this reason, open-loop control strategies implemented independently of the state of the system or solely on its statistics usually fail to operate efficiently in the presence of noise because of their inability to react or be adjusted in time. In order to account for all the possible behaviors of a stochastic dynamical system, we have to use the information contained in its evolution by considering a closed-loop control scheme in which the state of the controller is allowed to be correlated to the initial state of $`𝒳`$. This correlation can be thought as a measurement process described earlier that enables $`𝒞`$ to gather an amount of information given formally in Shannon’s information theory  by the mutual information $`I(X;C)=H(X)+H(C)H(X,C),`$where $`H(X,C)=_{x,c}p(x,c)\mathrm{log}p(x,c)`$ is the joint entropy of $`𝒳`$ and $`𝒞`$. Having defined these quantities, we are now in position to state our main result which is that the maximum improvement that closed-loop can give over open-loop control is limited by the information obtained by the controller. More formally, we have

Theorem 2.—The amount of entropy $`\mathrm{\Delta }H_{\text{closed}}`$ that can be extracted from any dynamical system by a closed-loop controller satisfies

$$\mathrm{\Delta }H_{\text{closed}}\mathrm{\Delta }H_{\text{open}}+I(X;C),$$
(3)   
where $`\mathrm{\Delta }H_{\text{open}}`$ is the maximum entropy decrease that can be obtained by open-loop control and $`I(X;C)`$ is the mutual information gathered by the controller upon observation of the system state.

Proof.—We construct a closed system by supplementing an ancilla $``$ to our previous system $`𝒳+𝒞`$. Moreover, let $`𝒞`$ and $``$ be collectively denoted by $``$ with state variable $`B`$. Since the complete system $`𝒳+`$ is closed, its entropy has to be conserved, and thus $`H(X,B)=H(X^{},B^{})`$. Defining the entropy changes of $`𝒳`$ and $``$ by $`\mathrm{\Delta }H=H(X)H(X^{})`$ and $`\mathrm{\Delta }H_B=H(B^{})H(B)`$ respectively, and by using the definition of the mutual information, this condition of entropy conservation can also be rewritten in the form $`\mathrm{\Delta }H=\mathrm{\Delta }H_BI(X^{};B^{})+I(X;B)`$ . Now, define $`\mathrm{\Delta }H_{\text{open}}`$ as the maximum amount of entropy decrease of $`𝒳`$ obtained in the open-loop case where $`I(X;C)=I(X;B)=0`$ (by construction of $``$, $`I(X;E)=0`$.) From the conservation condition, we hence obtain $`\mathrm{max}\mathrm{\Delta }H=\mathrm{\Delta }H_{\text{open}}+I(X;B)`$, which is the desired upper bound for a feedback controller.

To illustrate the above results, suppose that we control a system in a mixture of the states $`\{0,1\}`$ using a controller restricted to use the following two actions

$$\{\begin{array}{c}c=0:xx^{}=x\hfill \\ c=1:xx^{}=\mathrm{not}x\hfill \end{array}$$
(4)   
(in other words, the controller and the system behave like a so-called ‘controlled-not’ gate). Since these actuation rules simply permute the state of $`𝒳`$, $`H(X^{})H(X)`$ with equality if we use a pure control strategy or if $`H(X)=H_{\text{max}}=1`$ bit, in agreement with our first theorem. Thus, $`\mathrm{\Delta }H_{\text{open}}=0`$. However, by knowing the actual value of $`x`$ ($`H(X)`$ bit of information) we can choose $`C`$ to obtain $`\mathrm{\Delta }H=H(X)`$, therefore achieving Eq.(3) with equality. Evidently, as implied by this equation, information is required here as a result of the non-dissipative nature of the actuations and would not be needed if we were allowed to use dissipative (volume contracting) subdynamics. Alternatively, no open-loop controlled situation is possible if we confine the controller to use entropy-increasing actuations as, for instance, in the control of nonlinear systems using chaotic dynamics.

To demonstrate this last statement, let us consider the feedback control scheme proposed by Ott, Grebogi and Yorke (OGY)  as applied to the logistic map

$$x_{n+1}=rx_n(1x_n),x[0,1],$$
(5)   
(the extension to more general systems naturally follows). The OGY method, specifically, consists of applying to Eq.(5) small perturbations $`rr+\delta r_n`$ according to $`\delta r_n=\gamma (x_nx^{})`$, whenever $`x_n`$ falls into a region $`D`$ in the vicinity of the target point $`x^{}`$. The gain $`\gamma >0`$ is chosen so as to ensure stability . For the purpose of chaotic control, all the accessible control actions determined by the values of $`\delta r_n`$, and thereby by the coordinates $`x_nD`$, can be constrained to be entropy-increasing for a proper choice of $`D`$, meaning that the Lyapunov exponent $`\lambda (r)`$ associated with any actuation indexed by $`r`$ is such that $`\lambda (r)>0`$ . Physically, this implies that almost any initial uniform distribution for $`X`$ covering an interval of size $`\epsilon `$ “expands” by a factor $`2^{\lambda (r)}`$ on average after one iteration of the map with parameter $`r`$ . Now, for an open-loop controller, it can be readily be shown in that case that no control of the state $`x`$ is possible; without knowing the position $`x_n`$, a controller merely acts as a perturbation to the system, and the optimal control strategy then consists of using the smallest Lyapunov exponent available so as to achieve $`\mathrm{\Delta }H_{\text{open}}=\lambda _{\mathrm{min}}<0`$. Following theorem 2, it is thus necessary, in order to achieve a controlled situation $`\mathrm{\Delta }H>0`$, to have $`I(X;C)\lambda _{\mathrm{min}}`$ using a measurement channel characterized by an information capacity  of at least $`\lambda _{\mathrm{min}}`$ bit per use.

In the controlled regime ($`n\mathrm{}`$), this means specifically that if we want to localize the trajectory generated by Eq.(5) uniformly within an interval of size $`\epsilon `$ using a set of chaotic actuations, we need to measure $`x`$ within an interval no larger than $`\epsilon 2^{\lambda _{\mathrm{min}}}`$. To understand this, note that an optimal measurement of $`I(X;C)=\mathrm{log}a`$ bits consists, for a uniform distribution $`p(x)`$ of size $`\epsilon `$, in partitioning the interval $`\epsilon `$ into $`a`$ subintervals of size $`\epsilon /a`$. The controller under the partition then applies the same actuation $`r^{(i)}`$ for all the coordinates of the initial density lying in each of the subintervals $`i`$, therefore stretching them by a factor $`2^{\lambda (r^{(i)})}`$. In the optimal case, all the subintervals are directed toward $`x^{}`$ using $`\lambda _{\mathrm{min}}`$ and the corresponding entropy change is thus

$$\mathrm{\Delta }H_{\text{closed}}=\mathrm{log}\epsilon \mathrm{log}2^{\lambda _{\mathrm{min}}}\epsilon /a=\lambda _{\mathrm{min}}+\mathrm{log}a,$$
(6)   
which is consistent with Eq.(3) and yields the aforementioned value of $`a`$ for $`\mathrm{\Delta }H=0`$. Clearly, this value constitutes a lower bound for the OGY scheme since not all the subintervals are controlled with the same parameter $`r`$, a fact that we observed in numerical simulations .

In summary, we have introduced a formalism for studying control problems in which control units are analyzed as informational mechanisms. In this respect, a feedback controller functions analogously to a Maxwell’s demon , getting information about a system and using that information to decrease the system’s entropy. Our main result showed that the amount of entropy that can be extracted from a dynamical system by a controller is upper bounded by the sum of the decrease of entropy achievable in open-loop control and the mutual information between the dynamical system and the controller instaured during an initial interaction. This upper bound sets a fundamental limit on the performance of any controllers whose designs are based on the possibilities to accede low entropy states and was proven without any reference to a specific control system. Hence, its practical implications can be investigated for the control of linear, nonlinear and complex systems (discrete or continuous), as well as for the control of quantum systems for which our results also apply. For this latter topic, our probabilistic approach seems particularly suitable for the study of quantum controllers.

The authors would like to thank J.-J.E. Slotine for helpful discussions. This work has been partially supported by NSERC (Canada), and by a grant from the d’Arbeloff Laboratory for Information Systems and Technology, MIT.