Centralized active tracking of a Markov chain with unknown dynamics
  In this paper, selection of an active sensor subset for tracking a discrete
time, finite state Markov chain having an unknown transition probability matrix
(TPM) is considered. A total of N sensors are available for making observations
of the Markov chain, out of which a subset of sensors are activated each time
in order to perform reliable estimation of the process. The trade-off is
between activating more sensors to gather more observations for the remote
estimation, and restricting sensor usage in order to save energy and bandwidth
consumption. The problem is formulated as a constrained minimization problem,
where the objective is the long-run averaged mean-squared error (MSE) in
estimation, and the constraint is on sensor activation rate. A Lagrangian
relaxation of the problem is solved by an artful blending of two tools: Gibbs
sampling for MSE minimization and an on-line version of expectation
maximization (EM) to estimate the unknown TPM. Finally, the Lagrange multiplier
is updated using slower timescale stochastic approximation in order to satisfy
the sensor activation rate constraint. The on-line EM algorithm, though adapted
from literature, can estimate vector-valued parameters even under time-varying
dimension of the sensor observations. Numerical results demonstrate
approximately 1 dB better error performance than uniform sensor sampling and
comparable error performance (within 2 dB bound) against complete sensor
observation. This makes the proposed algorithm amenable to practical
implementation.
