Frequentist Properties of Bayesian Multiplicity Control for Multiple Testing of Normal Means

Abstract

We consider the standard problem of multiple testing of normal means, obtaining Bayesian multiplicity control by assuming that the prior inclusion probability (the assumed equal prior probability that each mean is nonzero) is unknown and assigned a prior distribution. The asymptotic frequentist behavior of the Bayesian procedure is studied, as the number of tests grows. Studied quantities include the false positive probability, which is shown to go to zero asymptotically. The asymptotics of a Bayesian decision-theoretic approach are also presented.

Appendix

Fact 6.1 (Normal tail probability).

Let Φ(t) be the cumulative distribution function of a standard normal random variable. Then as $t \rightarrow \infty$,

$$ \frac{t\frac{1}{\sqrt{2\pi}} e^{-t^{2}/2} }{t^{2}+1}\leq\ 1-{\Phi}(t)\leq \frac{\frac{1}{\sqrt{2\pi}} e^{-t^{2}/2} }{t} , $$

$$ 1-{\Phi}(t) =\frac{\frac{1}{\sqrt{2\pi}} e^{-t^{2}/2}}{t}(1+o_{p}(1)) . $$

The proof can be found in Durrett (2010).

Fact 6.2 (Weak law for triangular arrays).

For each m, let X_m,k, 1 ≤ k ≤ m be independent random variables. Let β_m > 0 with $\beta _{m} \rightarrow \infty$ and let $\bar {X}_{m,k}=X_{m,k}1_{\{|X_{m,k}| \leq \beta _{m} \}}$. Suppose that as $m\rightarrow \infty$

$$\sum \limits^{m}_{k=1} P(|X_{m,k}|>\beta_{m}) \rightarrow 0 \quad \text{and} \quad \frac{1}{{\beta_{m}^{2}}}\sum \limits^{m}_{k=1} E\bar{X}_{m,k}^{2} \rightarrow 0 .$$

Defining S_m = X_m,1 + ... + X_m,m and $\alpha _{m}=\sum \limits ^{m}_{k=1} E\bar {X}_{m,k}$, it follows that

$$ \frac{(S_{m}-\alpha_{m})}{\beta_{m}} \rightarrow 0 \quad \text{in probability} . $$

See Durrett (2010) for a proof.

Lemma 6.3.

If $Y_{j}=\frac {1}{\sqrt {1+\tau ^{2}}}\exp \{\frac {\tau ^{2}}{2(1+\tau ^{2})}{X_{j}^{2}}\}$ and τ² > 1/3, then

$$ m^{-2\tau^{2} /(1+\tau^{2})}\sum\limits_{j=1}^{m} {Y_{j}^{2}} \rightarrow 0 \quad \text{ when } m\rightarrow \infty . $$

Proof.

First we need the following order estimate before applying Fact 6.2: If $\lim _{m\rightarrow \infty } a_{m} \in (0,\infty ]$, $\lim _{m\rightarrow \infty }b_{m}= \infty$, then

$$ {\int}^{b_{m}}_{0} e^{\frac{a_{m} x^{2}}{2}}dx = O\bigg(\frac{e^{a_{m}{b_{m}^{2}}/2}}{a_{m}b_{m}}\bigg) . $$

(6.1)

To show (6.1) is true, notice that ${\int \limits }^{b_{m}}_{0} e^{\frac {a_{m}x^{2}}{2}}dx = \frac {1}{\sqrt {a_{m}}}{\int \limits }_{0}^{\sqrt {a_{m}}b_{m}} e^{\frac {x^{2}}{2}}dx$ and by L’Hôpital’s rule:

$$\begin{array}{llll} &\lim\limits_{m\rightarrow \infty} \frac{\frac{1}{\sqrt{a_{m}}}{\int}^{\sqrt{a_{m}}b_{m}}_{0} e^{\frac{x^{2}}{2}}dx} {e^{\frac{a_{m}{b_{m}^{2}}}{2}}/(a_{m} b_{m})} = \lim\limits_{m\rightarrow \infty} \frac{{\int}^{\sqrt{a_{m}}b_{m}}_{0} e^{\frac{x^{2}}{2}}dx} {e^{\frac{a_{m}{b_{m}^{2}}}{2}}/(\sqrt{a_{m}} b_{m})}\\ &= \lim\limits_{v\rightarrow \infty}\frac{{{\int}^{v}_{0}} e^{\frac{x^{2}}{2}}dx}{e^{\frac{v^{2}}{2}}/v} \quad \text{ where } v = \sqrt{a_{m}}b_{m}\\ &= \lim\limits_{v\rightarrow \infty} \frac{\frac{d}{dv}{{\int}^{v}_{0}} e^{x^{2}/2}dx}{\frac{d}{dv}\frac{e^{v^{2}/2}}{v}} = \lim\limits_{v \rightarrow \infty} \frac{1}{1-\frac{1}{v^{2}}} =1 . \end{array}$$

Take $X_{m,j}= \exp \{\frac {\tau ^{2}}{1+\tau ^{2}} {x_{j}^{2}}\}$ and $\beta _{m} = m^{2\tau ^{2}/(1+\tau ^{2})}$ in Fact 6.2. Both assumptions in Fact 6.2 hold since

$$\sum \limits_{j=1}^{m} P(|X_{m,j}|>\beta_{m}) = O\bigg(\frac{1}{\sqrt{log m}}\bigg) \rightarrow 0 \qquad \text{ by Fact } 6.1 ,$$

and

$$\begin{array}{llll} \frac{1}{{\beta_{m}^{2}}} \sum \limits^{m}_{j=1} E\bar{X}_{m,j}^{2} &=\frac{m}{m^{4\tau^{2}/(1+\tau^{2})}} \int\limits_{|x|\leq \sqrt{ 2 \log m} }\frac{1}{\sqrt{2\pi}}e^{(\frac{2\tau^{2}}{1+\tau^{2}} -1) \frac{x^{2}}{2}}dx\\ &= \left\{\begin{array}{llll} O\big(m^{-\frac{2\tau^{2}}{1+\tau^{2}}} \frac{1}{\sqrt{log m }}\big) &\text{ if } \tau^{2}>1, \text{ by } (6.1) \\ O\big(m^{\frac{1-3\tau^{2}}{1+\tau^{2}}}\big) &\text{ else,} \end{array}\right. \\&= o(1) \qquad \text{ if } \tau^{2}>1/3 . \end{array} $$

The limit is

$$ \begin{array}{llll} \frac{1}{\beta_{m}} \sum \limits^{m}_{j=1} E\bar{X}_{m,j} &= m^{-2\tau^{2}/(1+\tau^{2})} \frac{m}{\sqrt{2\pi}}\int\limits_{|x|<\sqrt{2 \log m}} \exp\left\{ \frac{x^{2}}{2}(\frac{2\tau^{2}}{1+\tau^{2}}-1) \right\} \\ &= m^{(1-\tau^{2})/(1+\tau^{2})} O\bigg(m^{(\tau^{2}-1)/(1+\tau^{2})} \frac{1}{\sqrt{log m}} \bigg) \text{ by } (6.1)\\ &= O\bigg(\frac{1}{\sqrt{log m}}\bigg) \rightarrow 0 . \end{array} $$

□

Fact 6.4 (Marcinkiewicz and Zygmund).

Let X₁,X₂,... be i.i.d. with EX₁ = 0 and $E|X_{1}|^{p} < \infty$ where 1 < p < 2. If S_m = X₁ + ... + X_m then

$$\frac{S_{m}}{m^{1/p}} \rightarrow 0 \text{ a.s. } $$

The proof can be found in Durrett (2010).

Lemma 6.5.

If $Y_{j}=\frac {1}{\sqrt {1+\tau ^{2}}}\exp \{\frac {\tau ^{2}}{2(1+\tau ^{2})}{X_{j}^{2}}\}$ and k = O(m^r), with $r\in [0, \frac {1}{1+\max \limits \{1,\tau ^{2}\}})$, then

$$ \bigg(\frac{1}{m} \sum\limits_{j=1}^{m} Y_{j} \bigg)^{k} = 1+o(1) . $$

Proof.

If τ² < 1, then E[Y_j] = 1 and Y_j has finite variance, so

$$ \bigg(\frac{1}{m} \sum\limits_{j=1}^{m} Y_{j} \bigg)^{k} = (1+O(m^{-1/2}))^{k} = 1+ O(m^{(r-1/2)}) $$

and the result holds.

If τ² ≥ 1, set X_i = Y_i − 1 and p = 1 + τ^− 2 − 𝜖. Fact 6.4 then implies that

$$\frac{{{\sum}_{1}^{m}} Y_{j} - m} {m^{1/p}} =\bigg(\frac{{{\sum}_{1}^{m}} Y_{j}} {m^{1/p}} - m^{1-1/p} \bigg)=o(1) ,$$

so that

$$ \frac{{{\sum}^{m}_{1}} Y_{j}}{m} -1 = \frac{1}{m^{1-1/p}}\bigg(\frac{\sum Y_{j}}{m^{1/p}} -m^{1-1/p} \bigg) = o\bigg(\frac{1}{m^{1-1/p}}\bigg) . $$

(6.2)

Thus

$$ \left( \frac{{{\sum}^{m}_{1}} Y_{j}}{m} \right)^{k} = (1+o(m^{-(1-1/p)}))^{k} = (1+o(m^{(r-1+1/p)})) . $$

(6.3)

Computation shows that r − 1 + 1/p < 0 for small enough 𝜖, proving the result. □

Proof Proof of Lemma 3.1.

Let $Y_{j} = \frac {1}{\sqrt {1+\tau ^{2}}} e^{{X_{j}^{2}}\tau ^{2}/2(1+\tau ^{2})}$ and expand $(\frac {1}{m}{{\sum }_{1}^{m}} Y_{j})^{k}$:

$$ \begin{array}{llll} \bigg[\frac{{{\sum}_{1}^{m}} Y_{j}}{m} \bigg]^{k} &= \sum \limits_{k_{1}+...+k_{m} = k} \binom{k}{k_{1},k_{2},...,k_{m}}\prod\limits_{1\leq j \leq m} \frac{Y_{j}^{k_{j}}}{m^{k_{j}}} \\ &= \underbrace{\sum \limits_{\begin{array}{llll}k_{1}+...+k_{m} = k \\ max|k_.|=1 \end{array}} \frac{k!}{m^{k}} \prod\limits_{1\leq j \leq m} Y_{j}^{k_{j}}}_{I} + \underbrace{\sum \limits_{\begin{array}{llll}k_{1}+...+k_{m} = k \\ max|k_.|\geq 2 \end{array}} \frac{k!}{k_{1}!...k_{m}!} \prod\limits_{1\leq j \leq m} \frac{Y_{j}^{k_{j}}}{m^{k_{j}}}}_{II} . \end{array} $$

(6.4)

Notice that

$$ \begin{array}{llll} I &= \frac{(m-1)...(m-k+1)}{m^{k-1}}\frac{1}{\binom{m}{k}} \sum \limits_{\begin{array}{llll}k_{1}+...+k_{m} = k \\ max|k_.|=1 \end{array}} \prod\limits_{1\leq j \leq m} \exp \left\{ \frac{\tau^{2} k_{j}}{2(1+\tau^{2})} {X_{j}^{2}}\right\}\\ &= \frac{(m-1)...(m-k+1)}{m^{k-1}} \frac{1}{\binom{m}{k}} \sum\limits_{|\boldsymbol{\gamma}| =k} B_{\boldsymbol{\gamma}0}\\ &= \bigg[\frac{1}{\binom{m}{k}}\sum\limits_{|\boldsymbol{\gamma}| =k} B_{\boldsymbol{\gamma}0}(\boldsymbol{X})\bigg](1+O(k^{2}/m))= \bigg[\frac{1}{\binom{m}{k}}\sum\limits_{|\boldsymbol{\gamma}| =k} B_{\boldsymbol{\gamma}0}(\boldsymbol{X})\bigg](1+o(1)) , \end{array} $$

where the order of the O term can be bounded by a constant independent of k, as established by a detailed Stirling’s approximation argument.

$$ \begin{array}{llll} II &= \frac{1}{m^{k}}\sum \limits_{\begin{array}{llll}k_{1}+...+k_{m} = k \\ max|k_.|\geq 2 \end{array}} \frac{k!}{k_{1}!...k_{m}!} \prod\limits_{1\leq j \leq m} Y_{j}^{k_{j}} \\ &\leq \frac{k(k-1)}{m^{k}}\bigg({\sum\limits_{1}^{m}} {Y_{i}^{2}}\bigg) \sum\limits_{\tilde{k}_{1}+...+\tilde{k}_{m}=k-2} \frac{(k-2)!}{\tilde{k}_{1}!...\tilde{k}_{m}!} Y_{1}^{\tilde{k}_{1}}...Y_{m}^{\tilde{k}_{m}} \\ &= \bigg(\frac{k(k-1)}{m^{2/(1+\tau^{2})}}\bigg) \bigg(\frac{{{\sum}_{1}^{m}} {Y_{i}^{2}}}{m^{2\tau^{2}/(1+\tau^{2})}}\bigg) \bigg(\frac{{{\sum}_{1}^{m}} Y_{i}}{m}\bigg)^{k-2} . \end{array} $$

To conclude that $II\rightarrow 0$ when $m\rightarrow \infty$, note that $\frac {k(k-1)}{m^{2/(1+\tau ^{2})}}$ converges to 0 because of the k = O(m^r) assumption;

By Lemma 6.3, when τ² > 1/3, $\frac {{{\sum }_{1}^{m}} {Y_{i}^{2}}}{m^{2\tau ^{2}/(1+\tau ^{2})}}=o_{p}(1)$;

By Lemma 6.5, $\big (\frac {{{\sum }_{1}^{m}} Y_{i}}{m}\big )^{k-2}= 1+o_{p}(1)$.

Hence, the right hand side of Eq. 6.4 converges to $1/\binom {m}{k}{\sum }_{|\boldsymbol {\gamma }| =k}B_{\boldsymbol {\gamma }0}(1+o(1))$. By Lemma 6.5, the left hand side goes to 1. Hence,

$$ \lim \limits_{m\rightarrow \infty} \frac{1}{\binom{m}{k} }\sum\limits_{|\boldsymbol{\gamma}| =k}B_{\boldsymbol{\gamma} 0} = 1 . $$

□

Lemma 6.6.

$$\frac{1}{\binom{m}{k}} \sum \limits_{|\boldsymbol{\gamma}| =k} B_{\boldsymbol{\gamma}0}(x) \text{ is a U-statistic} . $$

Proof.

By definition,

$$\begin{array}{llll} B_{\boldsymbol{\gamma}0}(x_{1},x_{2},...,x_{m}) = \frac{1}{(1+\tau^{2})^{\frac{|\boldsymbol{\gamma}| }{2}}} \exp \left\{ \frac{\tau^{2}}{2(1+\tau^{2})} \sum \limits_{i:\gamma_{i}\neq 0 } {x_{i}^{2}} \right\} \end{array}$$

$$h^{k}(x_{1},x_{2},...,x_{k})= \frac{1}{(1+\tau^{2})^{\frac{k}{2}}} \exp \left\{ \frac{\tau^{2}}{2(1+\tau^{2})} \sum \limits_{i=1}^{k} {x_{i}^{2}} \right\} . $$

Notice that an iteration over models is the same as an iteration through inputs:

$$\sum \limits_{\gamma:|\boldsymbol{\gamma}| =k} B_{\boldsymbol{\gamma}0}(x) = \sum \limits_{{C^{m}_{k}}} h^{k}(x_{i_{1}},x_{i_{2}},...,x_{i_{k}}) .$$

Therefore,

$$\frac{1}{\binom{m}{k}} \sum \limits_{|\boldsymbol{\gamma}| =k} B_{\boldsymbol{\gamma}0}(x) \text{ is a U-statistics with kernel } h^{k} .$$

□

Definition 6.7.

For c ∈{0,1,...,k},

$$\begin{array}{llll} {h^{k}_{c}}(x_{1},...,x_{c})&=\mathbbm{E}(h^{k}(x_{1},x_{2},...,x_{c},X_{c+1},...,X_{k}))\\ ({\sigma_{c}^{k}})^{2} &= Var({h^{k}_{c}}(X_{1},...,X_{c})) . \end{array}$$

Note that ${h^{k}_{k}} = h^{k}$, $({\sigma ^{k}_{k}})^{2} =Var(h^{k})={\sigma _{k}^{2}}$.

Fact 6.8.

Let U be the U-statistics with kernel h^k. If ${\sigma _{k}^{2}}<\infty$, then

$$\sqrt{m}\big[U-\mathbbm{E}(h^{k}(X_{1},...,X_{k}))\big] \rightarrow N(0,k^{2}({\sigma_{1}^{k}})^{2}) .$$

The asymptotic theory of U-statistics can be found in Ferguson (2003).

Theorem 6.9.

If τ² < 1, and k_max = o(m), then under the null model,

$$\sqrt{m} \bigg[ \frac{1}{\binom{m}{k}}\sum\limits_{|\boldsymbol{\gamma}| =k} B_{\boldsymbol{\gamma}0}(x)- 1\bigg] \rightarrow N\bigg(0,k^{2}\bigg[\frac{1}{\sqrt{1-\tau^{4}}}-1\bigg]\bigg) .$$

Proof.

By Lemma 6.6, $1/\binom {m}{k}{\sum }_{|\boldsymbol {\gamma }| =k}B_{\boldsymbol {\gamma }0}$ is a U-statistics, then evaluate the expectation and variance in Definition 6.7, one can get under the null model:

$$ \begin{array}{llll} \mathbbm{E}(B_{\boldsymbol{\gamma}0}(X))&= \mathbbm{E}({h^{k}_{1}}(X)) = 1 ,\\ \mathbbm{E}(B_{\boldsymbol{\gamma}0}(X)^{2}) &= \left\{\begin{array}{llll} \frac{1}{[(1+\tau^{2})(1-\tau^{2})]^{\frac{|\boldsymbol{\gamma}| }{2}}} &\text{ if } \tau^{2}<1 , \\ \infty &\text{ else, } \end{array}\right.\\ ({\sigma^{k}_{1}})^{2} &= \left\{\begin{array}{llll} \frac{1}{\sqrt{1-\tau^{4}}}-1 &\text{ if } \tau^{2}<1 ,\\ \infty &\text{ else, } \end{array}\right.\\ ({\sigma^{k}_{k}})^{2} &= \left\{\begin{array}{llll} \frac{1}{[(1+\tau^{2})(1-\tau^{2})]^{\frac{k}{2}}}-1 &\text{ if } \tau^{2}<1 , \\ \infty &\text{ else. } \end{array}\right. \end{array} $$

then apply Fact 6.8. □