Hitting Times and Interlacing Eigenvalues: A Stochastic Approach Using Intertwinings

Abstract

We develop a systematic matrix-analytic approach, based on intertwinings of Markov semigroups, for proving theorems about hitting-time distributions for finite-state Markov chains—an approach that (sometimes) deepens understanding of the theorems by providing corresponding sample-path-by-sample-path stochastic constructions. We employ our approach to give new proofs and constructions for two theorems due to Mark Brown, theorems giving two quite different representations of hitting-time distributions for finite-state Markov chains started in stationarity. The proof, and corresponding construction, for one of the two theorems elucidates an intriguing connection between hitting-time distributions and the interlacing eigenvalues theorem for bordered symmetric matrices.

Appendix: \(P^*\) When \(P\) is a Star Chain

In Remark 3.6, it is claimed that if the given chain $P$ is a star chain, then the star chain of Lemma 3.4 is simply obtained by collapsing all leaves with the same one-step transition probability to state $0$ into a single leaf. More precisely, we establish the following:

Proposition 5.1

Let $P$ be the transition matrix of an ergodic star chain with hub at $0$. If for each $\gamma _i$ in the reduced set of eigenvalues of $P_0$, we define

$$\begin{aligned} m(i):=\{j\in [n]:\eta _j=\gamma _i\}, \end{aligned}$$

then $P^*(0,i) = \sum _{j\in m(i)} P(0,j)$.

Proof

Define $H:=\text{ diag}(\eta _1,\ldots ,\eta _n)$ and

$$\begin{aligned} x&:= (P(0,1),\ldots ,P(0,n)), \\ y&:= (1-\eta _1,\ldots ,1-\eta _n), \end{aligned}$$

so that

$$\begin{aligned} P = \left( \begin{array}{c|c} P(0,0)&x \\ \hline y^T&H \end{array} \right). \end{aligned}$$

By the standard formula for the determinant of a partitioned matrix (e.g., [16, Sect. 0.8.5]), if $t$ is not in the spectrum $\{\eta _1, \dots , \eta _n\}$ of $H$, then we find

$$\begin{aligned} \det (tI-P) = [t-P(0,0)-x(tI-H)^{-1}y^T] \det (tI-H) \end{aligned}$$

(5.1)

for the characteristic polynomial of $P$. Analogously, define $\Gamma :=\text{ diag}(\gamma _1,\ldots ,\gamma _r)$ and

$$\begin{aligned} x^*&:= (P^*(0,1),\ldots ,P^*(0,r)), \\ y^*&:= (1-\gamma _1, \ldots ,1-\gamma _r); \end{aligned}$$

if $t$ is not in the spectrum $\{\gamma _1, \dots , \gamma _r\}$ of $\Gamma$, then we find

$$\begin{aligned} \det (tI-P^*)=[t-P^*(0,0)-x^*(tI-\Gamma )^{-1}{y^{*T}}] \det (tI-\Gamma ) \end{aligned}$$

(5.2)

for the characteristic polynomial of $P^*$.

Note that

$$\begin{aligned} P(0,0)&= \text{ tr}P -\text{ tr}H = \sum _{i = 0}^n \theta _i - \sum _{i = 1}^n \eta _i \\ \nonumber&= \sum _{i = 0}^r \lambda _i - \sum _{i = 1}^r \gamma _i = \text{ tr}P^*-\text{ tr}\Gamma = P^*(0,0), \end{aligned}$$

(5.3)

where the third equality is a result of the eigenvalue reduction procedure discussed in Sect. 3.1 and the fourth equality is from Lemma 2.6 in Ref.  [21]. Similarly, for all $t\notin \{\eta _1,\ldots ,\eta _n\}$, we have

$$\begin{aligned} \frac{\det (tI-P)}{\det (tI-H)} = \frac{\det (tI-P^*)}{\det (tI-\Gamma )}. \end{aligned}$$

(5.4)

Therefore, for all $t\notin \{\eta _1,\ldots ,\eta _n\}$, we have

$$\begin{aligned} \sum _{i=1}^n P(0,i) \frac{1-\eta _i}{t-\eta _i} = \sum _{i=1}^r P^*(0,i) \frac{1-\gamma _i}{t-\gamma _i}, \end{aligned}$$

(5.5)

because using definitions of $H, x, y, \Gamma , x^*, y^*$ and equations (5.1)–(5.4) we find

$$\begin{aligned}&\sum _{i=1}^n P(0,i) \frac{1-\eta _i}{t-\eta _i} =x(tI-H)^{-1}y^T = t - P(0, 0) - \frac{\det (tI-P)}{\det (tI-H)} \\&\quad = t - P^*(0, 0) - \frac{\det (tI-P^*)}{\det (tI-\Gamma )} = x^*(tI-\Gamma )^{-1}{y^*}^T= \sum _{i=1}^r P^*(0,i) \frac{1-\gamma _i}{t-\gamma _i}. \end{aligned}$$

Rewrite (5.5) as

$$\begin{aligned} \sum _{i=1}^r P^*(0,i) \frac{1-\gamma _i}{t-\gamma _i}=\sum _{i=1}^r\left(\sum _{j\in m(i)} P(0,j)\right) \frac{1-\gamma _i}{t-\gamma _i}. \end{aligned}$$

Since $\gamma _1, \dots , \gamma _r$ are distinct, it follow easily that $P^*(0,i)=\sum _{j\in m(i)} P(0,j)$ for $i = 1, \dots , r$, as desired. $\square$

Let $\pi$ be the stationary distribution for $P$. Using the formula for $P^*(0, i)$ provided by Proposition 5.1, it is a simple matter to check that the probability mass function $\pi ^*$ defined by $\pi ^*(0) := \pi (0)$ and $\pi ^*(i)=\sum _{j\in m(i)} \pi (j)$ for $i \ne 0$ satisfies the detailed balance condition and is therefore the stationary distribution for $P^*$; indeed, using the reversibility of $P$ with respect to $\pi$, we have

$$\begin{aligned} \pi ^*(0) P^*(0, i)&= \pi (0) \sum _{j \in m(i)} P(0, j) = \sum _{j \in m(i)} \pi (j) P(j, 0) \\&= \sum _{j \in m(i)} \pi (j) (1 - \gamma _i) = \pi ^*(i) P^*(i, 0). \end{aligned}$$