5 Continuous Markov chains 5.2 The rate matrix 5.4 Detailed balance

5.3 Asymptotic and invariant distributions

Definition 5.3.1.

A homogeneous cts MC has an asymptotic distribution $\pi$ if

\pi(t)\rightarrow\pi

whatever the initial distribution $\pi(0)$ .

Definition 5.3.2.

A homogeneous cts MC has an invariant distribution $\pi$ if

\pi(0)=\pi\ \Rightarrow\pi(t)=\pi\

for all $t>0$ .

Lemma 5.3.3.

An invariant distribution satisfies $\pi Q=0$ .

Proof.

Recall that $\pi(t)=\pi(0)P(t)$ . If $\pi$ is invariant, then $\pi=\pi P(t)$ . Differentiating with respect to $t$ gives $0=\pi P^{\prime}(t)$ and the result follows by setting $t=0$ . ∎

Lemma 5.3.4.

If $X(t)$ has an asymptotic distribution $\pi$ then $\pi$ is also its unique invariant distribution.

Proof.

Let $t\rightarrow\infty$ in $\pi(t+h)=\pi(t)P(h)$ so that $\pi=\pi P(h)$ . Using this, if $\pi(0)=\pi$ we get $\pi(h)=\pi$ and this is true for all $h>0$ so $\pi$ is an invariant distribution. If $\pi^{*}$ is also invariant then set $\pi(0)=\pi^{*}$ so $\pi(t)=\pi^{*}$ for all $t>0$ . Let $t\rightarrow\infty$ to see that $\pi^{*}=\pi$ . ∎

Remark.

This means that if an asymptotic distribution exists, then we can find it by solving for the invariant distribution.

Theorem 5.3.5.

If a continous time Markov process with a finite number of states is irreducible (i.e. for all pairs of states $i, j$ , $\mathrm{P}(X_{t}=j|X_{0}=i)>0$ for some (any) $t>0$ ) then it has an asymptotic distribution.

Example 5.3.6.

The general two state chain has

Q=\left(\begin{array}[]{cc}-\lambda&\lambda\\ \mu&-\mu\end{array}\right).

Provided neither $\lambda$ nor $\mu$ are zero the chain is irreducible (aperiodic always) and the asymptotic distribution is same as the invariant distribution $\pi$ satisfying:

-\lambda\pi_{1}+\mu\pi_{2}=0

so that

\pi\propto(\mu,\lambda)\Rightarrow\pi=\left(\frac{\mu}{\lambda+\mu},\frac{% \lambda}{\lambda+\mu}\right).