TITLE: Understanding invariant and stationary distributions for Markov chains
QUESTION [0 upvotes]: I have 3 little questions regardings invariant and stationary probability distributions.

Let $E=\{a,b\}$ be a state space and $\textbf{Q}$ a transition matrix $$\textbf{Q}:=\begin{pmatrix}1-p&p \\ q &1-q\end{pmatrix}$$
  with $0<p,q \leq 1$ and $p+q<2$.
Then a measure $\mu$ is invariant if $\mu \textbf{Q}=\mu$. Now this means the following must hold $$\begin{cases}\mu(a)+\mu(b)=1\\ p\cdot\mu(a)-q\cdot\mu(b)=0\end{cases}$$


Now I don't understand why $p\cdot\mu(a)-q\cdot\mu(b)=0$ must hold? 

(If I have $3$ states would it be $p\cdot\mu(a)-q\cdot\mu(b) -0\cdot\mu(c)=0$?)

The invariant distribution is 
  $$\mu(a)=\frac{q}{p+q}\quad \mu(b)=\frac{p}{p+q}$$
  We have that
  $$\lim\limits_{n\to \infty}\textbf{Q}^n=\frac{1}{p+q}\begin{pmatrix}q&p \\ q &p\end{pmatrix}$$
  So $\lim\limits_{n\to \infty}\textbf{Q}^n=(\mu,\mu)^T$


So the transition matrix is converging to the invariant probability distribution. Is the invariant probability always the stationary distribution (and vice versa)? It confuses me that we have invariant/stationary distributions defined as vectors, but now the stationary distribution is the matrix?
Does the limit matrix $\lim\limits_{n\to \infty}\textbf{Q}^n$ always have the same row? Meaning the first row of the limit matrix is also the second etc. (Here it is $\mu$ and again $\mu$)

REPLY [1 votes]: Let $P(i\mid j)$ denote the conditional probability that the next state of the system will be $i$ given that it is in  state $j$. With this the state transition matrix is
$$\begin{bmatrix}
P(a\mid a)& P(b\mid b)\\
P(a\mid b)& P(b\mid b)
\end{bmatrix}=\begin{bmatrix}
1-p& p\\
q& 1-q
\end{bmatrix}.$$
Given the state transition matrix and assuming that the probabilities that the system is in state $a$ or $b$ are $\mu(a)$ and $\mu(b)$, respectively, one can calculate the probability that the system willy in state $a$ is
$$\mu'(a)=P(a\mid a)\mu(a)+p(a\mid b)\mu(b)$$
The same way 
$$\mu'(b)=P(b\mid a)\mu(a)+P(b\mid b)\mu(b).$$
If $\mu'(a)=\mu(a)$ and $\mu'(b)=\mu(b)$ then the these probabilities are the stationary state probabilities.
So, if one wants to calculate the stationary probabilities then the one has the following three equations for $\mu(a)$ and $$\mu(b)$::
$$\mu(a)=(1-p)\mu(a)+q\mu(b)$$
$$\mu(b)=p\mu(a)+(1-q)\mu(b).$$
$$\mu(a)+\mu(b)=1.$$
Too many equations... But if the stationary probabilities exist then it is enough to take only two of the equations above:
$$\mu(a)=(1-p)\mu(a)+q\mu(b)$$
$$\mu(a)+\mu(b)=1.\tag 1$$
From the first equation you get
$$0=q\mu(b)-p\mu(a).$$
This answers your first question.

The $n^{th}$ power of the state transition matrix is called the $n$ step transition matrix because it shows the state transition probabilities (in one) if we let the chain step ahead $n$ times.
If $p,q>0$ then the limit of the $n^{th}$ power of the state transition matrix tends to a matrix whose both rows equal the solutions (vector) of $(1)$. If you apply this limiting matrix to the vector of the stationary distributions then you get them back:
$$\frac1{p+q}[q \ p]
\frac1{p+q}\begin{bmatrix}
q&p\\
q& p
\end{bmatrix}=\frac1{(p+q)^2}  
[q^2+pq\ \ qp+p^2]
=\frac1{p+q}[ q \ p].$$
I expect that this latter relationship between the limit of the $n$-step transition matrix and the stationary solution enlightens you.
{"set_name": "stack_exchange", "score": 0, "question_id": 2113485}