Efficient likelihood-based inference for the generalized Pareto distribution

Abstract

It is well known that inference for the generalized Pareto distribution (GPD) is a difficult problem since the GPD violates the classical regularity conditions in the maximum likelihood method. For parameter estimation, most existing methods perform satisfactorily only in the limited range of parameters. Furthermore, the interval estimation and hypothesis tests have not been studied well in the literature. In this article, we develop a novel framework for inference for the GPD, which works successfully for all values of shape parameter k. Specifically, we propose a new method of parameter estimation and derive some asymptotic properties. Based on the asymptotic properties, we then develop new confidence intervals and hypothesis tests for the GPD. The numerical results are provided to show that the proposed inferential procedures perform well for all choices of k.

Appendices

Propositions for derivatives of the likelihood function of k

Proposition 8

For $k \in {\mathbb {R}}$ and any given $\varvec{s}_n^{(j)}$, where j, $1\le j\le n$, is fixed, the derivative $l'(k ; \varvec{s}_n^{(j)})=(\partial /\partial k)l(k ; \varvec{s}_n^{(j)})$ is given by

$$\begin{aligned}&l'(k ; \varvec{s}_n^{(j)}) \nonumber \\&\quad = {\left\{ \begin{array}{ll} \displaystyle n!\,\int _{\chi _k}\left( -\frac{n}{k}-\frac{\sum _{i=1}^n \log \left( 1-u s_i\right) }{k^2}\right) \frac{1}{\left| k \right| }\left( \frac{u}{k}\right) ^{n-1}\prod _{i=1}^n \left( 1- u s_i\right) ^{1/k-1}\,du, &{} k \ne 0, \\ \displaystyle \left\{ 1-\frac{\left( n+1\right) \sum _{i=1}^n s_i^2}{2\left( \sum _{i=1}^n s_i\right) ^2}\right\} \frac{\left( n!\right) ^2}{\left( \sum _{i=1}^n s_i\right) ^n}, &{} k=0, \\ \end{array}\right. }\nonumber \\&\qquad s_1\le \cdots \le s_{j-1} \le 1 \le s_{j+1} \le \cdots \le s_n, {\text{ and }} s_j=1. \end{aligned}$$

(5)

Proposition 9

For $k \in {\mathbb {R}}$ and any given $\varvec{s}_n^{(j)}$, where j, $1\le j\le n$, is fixed, the second derivative $l''(k ; \varvec{s}_n^{(j)})=(\partial ^2/\partial k^2)l(k ; \varvec{s}_n^{(j)})$ is given by

$$\begin{aligned}&l''(k ; \varvec{s}_n^{(j)}) \\&\quad = {\left\{ \begin{array}{ll} \displaystyle n!\,\int _{\chi _k}\left( \frac{n(n+1)}{k^2}+\frac{2(n+1)\sum _{i=1}^n \log \left( 1-u s_i\right) }{k^3}+\frac{\left\{ \sum _{i=1}^n\log \left( 1-u s_i\right) \right\} ^2}{k^4}\right) &{} \\ \displaystyle \times \frac{1}{\left| k \right| }\left( \frac{u}{k}\right) ^{n-1}\prod _{i=1}^n \left( 1- u s_i\right) ^{1/k-1}\,du, &{} k \ne 0, \\ \displaystyle \left\{ 1 -\left( n+1\right) \frac{\sum _{i=1}^n s_i^2}{\left( \sum _{i=1}^n s_i\right) ^2} +\frac{\left( n+2\right) \left( n+3\right) }{4}\frac{\left( \sum _{i=1}^n s_i^2\right) ^2}{\left( \sum _{i=1}^n s_i\right) ^4} -\frac{2\left( n+2\right) }{3}\frac{\sum _{i=1}^n s_i^3}{\left( \sum _{i=1}^n s_i\right) ^3} \right\} &{} \\ \displaystyle \times \frac{n!\,\left( n+1\right) !}{\left( \sum _{i=1}^n s_i\right) ^n} , &{} k=0, \\ \end{array}\right. }\\&s_1\le \cdots \le s_{j-1} \le 1 \le s_{j+1} \le \cdots \le s_n, {\text{ and }} s_j=1. \end{aligned}$$

Proposition 10

For $k \in {\mathbb {R}}$ and any given $\varvec{s}_n^{(j)}$, where j, $1\le j\le n$, is fixed, the third derivative $l'''(k ; \varvec{s}_n^{(j)})=(\partial ^3/\partial k^3)l(k ; \varvec{s}_n^{(j)})$ is given by

$$\begin{aligned}&l'''(k ; \varvec{s}_n^{(j)}) \nonumber \\&\quad = n!\,\int _{\chi _k} \left( -\frac{n\left( n+1\right) \left( n+2\right) }{k^3} -\frac{3\left( n+1\right) \left( n+2\right) \sum _{i=1}^n \log \left( 1-u s_i\right) }{k^4}\right. \nonumber \\&\qquad \left. -\frac{3\left( n+2\right) \left\{ \sum _{i=1}^n \log \left( 1-u s_i\right) \right\} ^2}{k^5} -\frac{\left\{ \sum _{i=1}^n \log \left( 1-u s_i\right) \right\} ^3}{k^6} \right) \nonumber \\&\qquad \times \frac{1}{\left| k \right| }\left( \frac{u}{k}\right) ^{n-1}\prod _{i=1}^n \left( 1- u s_i\right) ^{1/k-1}\,du, \quad {\text{ for }} k \ne 0, \end{aligned}$$

(6)

and

$$\begin{aligned}&l'''(0 ; \varvec{s}_n^{(j)}) \nonumber \\&\quad = \left\{ 1 -\frac{3\left( n+1\right) }{2}\frac{\sum _{i=1}^n s_i^2}{\left( \sum _{i=1}^n s_i\right) ^2} -\frac{\left( n+3\right) \left( n+4\right) \left( n+5\right) }{8}\frac{\left( \sum _{i=1}^n s_i^2\right) ^3}{\left( \sum _{i=1}^n s_i\right) ^6}\right. \nonumber \\&\qquad -2\left( n+2\right) \frac{\sum _{i=1}^n s_i^3}{\left( \sum _{i=1}^n s_i\right) ^3} +\left( n+3\right) \left( n+4\right) \frac{\left( \sum _{i=1}^n s_i^2\right) \left( \sum _{i=1}^n s_i^3\right) }{\left( \sum _{i=1}^n s_i\right) ^5}\nonumber \\&\qquad \left. +\frac{3\left( n+2\right) \left( n+3\right) }{4}\frac{\left( \sum _{i=1}^n s_i^2\right) ^2}{\left( \sum _{i=1}^n s_i\right) ^4} -\frac{3\left( n+3\right) }{2}\frac{\sum _{i=1}^n s_i^4}{\left( \sum _{i=1}^n s_i\right) ^4} \right\} \frac{n!\,\left( n+2\right) !}{\left( \sum _{i=1}^n s_i\right) ^n}, \end{aligned}$$

(7)

where $s_1\le \cdots \le s_{j-1} \le 1 \le s_{j+1} \le \cdots \le s_n$, and $s_j=1$.

Proofs

1.1 Proof of Proposition 1

Denote the cdf and pdf of the GPD with $\sigma =1$, $F(\cdot ;\, k, 1)$ and $f(\cdot ;\, k, 1),$ by $G(\cdot \,;k)$ and $g(\cdot \,;\beta )$, respectively, for simplicity. Suppose $Z_{i}$, $i=1, \ldots ,n$, are n independent random variables from such a standard GPD with shape parameter k. For $i=1, \ldots ,n$, let $Z_{i:n}$ be the i-th order statistic among $Z_{1}, \ldots ,Z_{n}$.

First, we assume that $k\ne 0$. Define $\varvec{i_n^{(j)}}=\left\{ i|i=1, \ldots ,j-1, j+1, \ldots , n\right\}$. For a fixed positive integer value j, and for any $n-1$ real values $s_1\le \cdots \le s_{j-1} \le 1 \le s_{j+1} \le \cdots \le s_n$, we consider

$$\begin{aligned}&\Pr \left( S_{i:n}^{(j)} \le s_i,\, i \in \varvec{i_n^{(j)}} \right) \nonumber \\&\quad =\Pr \left( \frac{Z_{i:n}}{Z_{j:n}} \le s_i,\, i\in \varvec{i_n^{(j)}}\right) \nonumber \\&\quad =\int _{{\mathscr {X}}_{k, 1}}\Pr \left( Z_{i:n} \le u s_i,\, i\in \varvec{i_n^{(j)}} \big | Z_{j:n}=u\right) h_j\left( u;\, k\right) \, du \nonumber \\&\quad =\int _{{\mathscr {X}}_{k, 1}}(j-1)!\prod _{i=1}^{j-1}\frac{G(u s_i\,;k)}{G(u\,;k)}\times (n-j)!\prod _{i=j+1}^{n}\frac{G(u s_i\,;k)}{1-G(u\,;k)} \nonumber \\&\qquad \times \frac{n!}{(j-1)!(n-j)!}{\left\{ G(u\,;k)\right\} }^{j-1}{\left\{ 1-G(u\,;k)\right\} }^{n-j}g(u\,;k)\, du \nonumber \\&\quad =\int _{{\mathscr {X}}_{k, 1}} n!\,g(u\,;k)\prod _{i \in \varvec{i_n^{(j)}}}G(u s_i\,;k)\, du, \end{aligned}$$

(8)

where $h_j\left( \cdot ;\, k\right)$ is the pdf of $Z_{j:n}$.

We note that the integrand in Eq. (8) has its partial derivative with respect to $s_i$, $i \in \varvec{i_n^{(j)}}$, as $n!g(u\,;k)\prod _{i \in \varvec{i_n^{(j)}}}u\,g(u s_i\,;k)$, and further that

$$\begin{aligned}&(j-1)!\prod _{i=1}^{j-1}\frac{u\,g(u s_i\,;k)}{G(u\,;k)}\times (n-j)!\prod _{i=j+1}^{n}\frac{u\,g(u s_i\,;k)}{1-G(u\,;k)}, \end{aligned}$$

(9)

is bounded above. From the boundedness of (9), we have

$$\begin{aligned} n!g(u\,;k)\prod _{i \in \varvec{i_n^{(j)}}}u\,g(u s_i\,;k)\le C_0\, h_j\left( u;\, k\right) , \end{aligned}$$

where $C_0$ is a positive constant, and

$$\begin{aligned} \int _{{\mathscr {X}}_{k, 1}}n!g(u\,;k)\prod _{i \in \varvec{i_n^{(j)}}}u\,g(u s_i\,;k)\, du \le \int _{{\mathscr {X}}_{k, 1}}C_0\, h_j\left( u;\, k\right) \,du =C_0<\infty . \end{aligned}$$

Then, upon using Part (ii) of Theorem 16.8 of Billingsley (1994), we can interchange the derivatives and the integration in (8), so that the partial derivative of it with respect to $s_i, i \in \varvec{i_n^{(j)}}$, as

$$\begin{aligned} n!\,\int _{{\mathscr {X}}_{k, 1}} g(u\,;k)\prod _{i \in \varvec{i_n^{(j)}}}u\,g(u s_i\,;k)\, du. \end{aligned}$$

The result for $k=0$ can be obtained by letting $k \rightarrow 0$ in the result for $k\ne 0$. After some simple algebra, the proof of Proposition 1 gets completed. $\square$

1.2 Proof of Theorem 3

First, we shall show that the likelihood equation has at least one solution. Given $\varvec{s}_n^{(j)}$, the derivative of the likelihood function in (5) for $k \ne 0$ can be rewritten as

$$\begin{aligned} l'(k ; \varvec{s}_n^{(j)})= & {} n!\,\int _{\chi _k} \eta \left( k, u\right) \Lambda \left( k, u\right) \,du, \end{aligned}$$

where $\eta \left( k, u\right) =-\frac{n}{k}-\frac{\sum _{i=1}^n \log \left( 1-u s_i\right) }{k^2}$, $\Lambda \left( k, u\right) =\frac{1}{\left| k \right| }\left( \frac{u}{k}\right) ^{n-1}\prod _{i=1}^n \left( 1- u s_i\right) ^{1/k-1}$ and $s_j=1$. It follows from the facts that $\eta \left( k, u\right) =-\frac{1}{k}\left( n+\frac{\sum _{i=1}^n \log \left( 1-u s_i\right) }{k}\right) >0$ for sufficiently small $k\in {\mathbb {R}}$, and $\eta \left( k, u\right) <0$ for sufficiently large $k\in {\mathbb {R}}$, and $\Lambda \left( k, u\right) >0$ for every $k\in {\mathbb {R}}$, for every $u\in \chi _k$, there exist real values $\delta _1$ and $\delta _2$ such that $l'(k ; \varvec{s}_n^{(j)})>0$ for every $k<\delta _1$, and $l'(k ; \varvec{s}_n^{(j)})<0$ for every $k>\delta _2$, respectively. In addition, we see from Proposition 8 that $l'(k ; \varvec{s}_n^{(j)})$ is continuous with respect to $k\in {\mathbb {R}}$. Thus, $l'(k ; \varvec{s}_n^{(j)})=0$ has at least one solution.

Next, we shall show that the number of solutions is exactly one. Let $k^*$ be one of the solutions of $l'(k ; \varvec{s}_n^{(j)})=0$. We see that $\eta \left( k^*, u\right)$ is strictly increasing in u and takes on values over $(-\infty , -n/k^*)$ for $k^*<0$. Thus, there exists a unique value of u such that $\eta \left( k^*, u\right) =0$, which we denote by $u_0$. We also see that $\eta \left( k^*, u\right) <0$ for $u<u_0$ and $\eta \left( k^*, u\right) >0$ for $u>u_0$.

We have, for $k^*<0$ and sufficiently small $\Delta k>0$ such that $k^* + \Delta k<0$,

$$\begin{aligned} l'(k^*+\Delta k ; \varvec{s}_n^{(j)})= & {} n!\,\int _{-\infty }^0\eta \left( k^*, u\right) \frac{\eta \left( k^*+\Delta k, u\right) }{\eta \left( k^*, u\right) }\Lambda \left( k^*+\Delta k, u\right) \,du\nonumber \\< & {} n!\,\int _{-\infty }^{u_0-}\eta \left( k^*, u\right) \left( 1+\frac{\Delta k}{k^*}\right) ^{-2}\Lambda \left( k^*+\Delta k, u\right) \,du\nonumber \\&+n!\,\int _{u_0}^0\eta \left( k^*, u\right) \left( 1+\frac{\Delta k}{k^*}\right) ^{-2}\Lambda \left( k^*+\Delta k, u\right) \,du\nonumber \\= & {} \left( 1+\frac{\Delta k}{k^*}\right) ^{-2}n!\,\int _{-\infty }^0\eta \left( k^*, u\right) \Lambda \left( k^*+\Delta k, u\right) \,du, \end{aligned}$$

(10)

where $u_0-=\lim _{u\uparrow u_0}u$, and the inequality follows from the facts that

$$\begin{aligned} \frac{\eta \left( k^*+\Delta k, u\right) }{\eta \left( k^*, u\right) }= & {} \left( 1+\frac{\Delta k}{k^*}\right) ^{-2}\left\{ 1+\frac{\Delta k}{k^*+\frac{1}{n}\sum _{i=1}^n \log \left( 1-u s_i\right) }\right\} \end{aligned}$$

is greater than $\left( 1+\Delta k/k^*\right) ^{-2}$ for $u\in (-\infty , u_0)$, is less than $\left( 1+\Delta k/k^*\right) ^{-2}$ for $u\in (u_0, 0)$, and $\eta \left( k^*+\Delta k, u_0\right) <0 =\eta \left( k^*, u_0\right) \left( 1+\Delta k/k^*\right) ^{-2}$.

We further note that

$$\begin{aligned} \frac{\Lambda \left( k^*+\Delta k, u\right) }{\Lambda \left( k^*, u\right) }= & {} \left( 1+\frac{\Delta k}{k^*}\right) ^n \prod _{i=1}^n\left( 1-u\,s_i\right) ^{\frac{1}{k^*+\Delta k}-\frac{1}{k^*}} \end{aligned}$$

is strictly increasing in u and takes on value over $\left( 0, (1+\Delta k/k^*)^n\right)$. Then, it follows from (10) and by the mean value theorem that

$$\begin{aligned} l'(k^*+\Delta k ; \varvec{s}_n^{(j)})< & {} \left( 1+\frac{\Delta k}{k^*}\right) ^{-2}M\, n!\,\int _{-\infty }^0\eta \left( k^*, u\right) \Lambda \left( k^*, u\right) \,du \\= & {} \left( 1+\frac{\Delta k}{k^*}\right) ^{-2}M\, n!\,l'(k^* ; \varvec{s}_n^{(j)})=0, \end{aligned}$$

where $M=\Lambda \left( k^*+\Delta k, u'\right) /\Lambda \left( k^*, u'\right) \in \left( 0, (1+\Delta k/k^*)^n\right)$, for $u'\in (-\infty , 0)$.

We can also obtain the same results for $k^*\ge 0$. The proofs are very similar to the proof for $k^*< 0$ (for $k^*= 0$, by using the fact that $l'(0 ; \varvec{s}_n^{(j)})=\lim _{k\rightarrow 0}l'(k ; \varvec{s}_n^{(j)})$ and Lebesgue’s dominated convergence theorem) and are therefore omitted here. The fact that $l'(k^*+\Delta k ; \varvec{s}_n^{(j)})<0$ for every $k^* \ne 0$ clearly implies that $l'(k ; \varvec{s}_n^{(j)})$ changes sign only once with respect to k.

From the above arguments, $l'(k ; \varvec{s}_n^{(j)})=0$ always has a unique solution with respect to k, and the proof of Theorem 3 thus gets completed.$\square$

1.3 Proof of Lemma 1

Let $\varvec{S}_{n,1}^{(j)}=\left( S_{1:n}^{(j)}, \ldots ,S_{j-1:n}^{(j)}\right)$ and $\varvec{S}_{n,2}^{(j)}=\left( S_{j+1:n}^{(j)}, \ldots ,S_{n:n}^{(j)}\right)$. Then, by Theorem 2 of Iliopoulos and Balakrishnan (2009), conditional on $Z_{j:n}=u\in \lambda _{k}$, where $Z_{j:n}=X_{j:n}/\sigma _0$ and $\lambda _{k}=\{u:0< u< \infty , {\text{ if }} k < 0,$ or $0< u< 1/k, {\text{ if }} k>0\}$ as defined in the proof of Proposition 1, we see that $\varvec{S}_{n,1}^{(j)}$ are distributed exactly as order statistics from a sample of size $j-1$ from the distribution with density $\psi _1(s;\, k_0, u)=u\, g(u\,s;\, k_0)/G(u;\, k_0)$, $0\le s\le 1$, and $\varvec{S}_{n,2}^{(j)}$ are distributed exactly as order statistics from a sample of size $n-j$ from the distribution with density $\psi _2(s;\, k_0, u)=u\, g(u\,s;\, k_0)/(1-G(u;\, k_0))$, $s\ge 1$, where $g(\cdot ;\, k)=f(\cdot ;\, k, 1)$ and $G(\cdot ;\, k)=F(\cdot ;\, k, 1)$. We also have $\varvec{S}_{n,1}^{(j)}$ and $\varvec{S}_{n,2}^{(j)}$ to be conditionally independent. Hence, under the condition that $Z_{j:n}=u \in \lambda _{k}$, we have the joint density function of $\varvec{S}_{n,1}^{(j)}$ and $\varvec{S}_{n,2}^{(j)}$ to be

$$\begin{aligned} \left( j-1\right) !\prod _{i=1}^{j-1}\psi _1(s_i;\, k_0, u)\times \left( n-j\right) !\prod _{i=j+1}^{n}\psi _2(s_i;\, k_0, u), \end{aligned}$$

(11)

denoted by $l_u\left( k_0;\,\varvec{s}_{n}^{(j)}\right)$, where $\varvec{s}_n^{(j)}=(s_1, \ldots , s_{j-1}, s_{j+1}, \ldots , s_n)$, for $0\le s_1\le \cdots \le s_{j-1} \le 1 \le s_{j+1}\le \cdots \le s_{n}$. Equation (11) implies that $\varvec{S}_{1*}^{(j)}=(S_1^{(j)}, \ldots ,S_{j-1}^{(j)})$, which are the corresponding random variables to $\varvec{S}_{n,1}^{(j)}=(S_{1:n}^{(j)}, \ldots ,S_{j-1:n}^{(j)})$, are i.i.d. distributed with the conditional density function $\psi _1$, and $\varvec{S}_{2*}^{(j)}=(S_{j+1}^{(j)}, \ldots ,S_{n}^{(j)})$, which are the corresponding random variables to $\varvec{S}_{n,2}^{(j)}=(S_{j+1:n}^{(j)}, \ldots ,S_{n:n}^{(j)})$, are i.i.d. with the conditional density function $\psi _2$, given $Z_{j:n}=u$.

Let $Z'_{j:n}=X'_{j:n}/\sigma$, where $X'_{j:n}$ is the jth-order statistic from the GPD with parameters $k\ne k_0$ and $\sigma \ne \sigma _0$. Then, for any fixed $u\in \lambda _{k}$ and $u'\in \lambda _{k}$, and any $k\ne k_0$, conditional on $Z_{j:n}=u$ and $Z'_{j:n}=u'$, it follows that

$$\begin{aligned} \frac{1}{n-1}\log \frac{l_{u'}\left( k;\,\varvec{S}_{n}^{(j)}\right) }{l_u\left( k_0;\,\varvec{S}_{n}^{(j)}\right) }= & {} \frac{1}{n-1}\log \frac{l_{u'}\left( k;\,\varvec{S}_{n*}^{(j)}\right) }{l_u\left( k_0;\,\varvec{S}_{n*}^{(j)}\right) }\nonumber \\= & {} \frac{j-1}{n-1}\frac{1}{j-1}\sum _{i=1}^{j-1}\log \frac{\psi _1(S_{i}^{(j)};\, k, u')}{\psi _1(S_{i}^{(j)};\, k_0, u)}\nonumber \\&+\frac{n-j}{n-1}\frac{1}{n-j}\sum _{i=j+1}^{n}\log \frac{\psi _2(S_{i}^{(j)};\, k, u')}{\psi _2(S_{i}^{(j)};\, k_0, u)}, \end{aligned}$$

(12)

where $\varvec{S}_{n*}^{(j)}=\,(\varvec{S}_{1*}^{(j)}, \varvec{S}_{2*}^{(j)})=\,(S_{1}^{(j)}, \ldots , S_{j-1}^{(j)}, S_{j+1}^{(j)},\ldots , S_{n}^{(j)})$. By the weak law of large numbers, (12) converges in probability to

$$\begin{aligned} p\, E_1\left[ \log \frac{\psi _1(S_1;\, k, u')}{\psi _1(S_1;\, k_0, u)}\right] +\left( 1-p\right) \, E_2\left[ \log \frac{\psi _2(S_2;\, k, u')}{\psi _2(S_2;\, k_0, u)}\right] , \end{aligned}$$

where $S_1$ and $S_2$ are random variables which are distributed with the conditional density functions $\psi _1(x;\, k_0, u)$ and $\psi _2(x;\, k_0, u)$, given $Z_{j:n}=u$, respectively, and $p=\lim _{n\rightarrow \infty } j/n$ ($0\le p\le 1$). $E_1$ and $E_2$ denote the conditional expectations with respect to $\psi _1$ and $\psi _2$, given $Z_{j:n}=u$, respectively. By Jensen’s inequality, we have

$$\begin{aligned}&q\, E_1\left[ \log \frac{\psi _1(S_1;\, k, u')}{\psi _1(S_1;\, k_0, u)}\right] +\left( 1-q\right) \, E_2\left[ \log \frac{\psi _2(S_2;\, k, u')}{\psi _2(S_2;\, k_0, u)}\right] \\&\quad <\log E_1\left[ \frac{\psi _1(S_1;\, k, u')}{\psi _1(S_1;\, k_0, u)}\right] +\log E_2\left[ \frac{\psi _2(S_2;\, k, u')}{\psi _2(S_2;\, k_0, u)}\right] \\&\quad =\log \int _0^1 \psi _1(t;\, k, u')\,dt + \log \int _1^\infty \psi _2(t;\, k, u')\,dt=0. \end{aligned}$$

Hence, for any fixed u, $u'\in \lambda _{k}$, we have

$$\begin{aligned} \lim _{n\rightarrow \infty } P\left( \frac{1}{n-1}\log \frac{l_{u'}\left( k;\,\varvec{S}_{n}^{(j)}\right) }{l_u\left( k_0;\,\varvec{S}_{n}^{(j)}\right) }<0\,\Big |\, Z_{j:n}=u,\, Z'_{j:n}=u'\right) =1, \end{aligned}$$

or

$$\begin{aligned} \lim _{n\rightarrow \infty } P\left( l_{u'}\left( k;\,\varvec{S}_{n}^{(j)}\right) <l_u\left( k_0;\,\varvec{S}_{n}^{(j)}\right) \,|\, Z_{j:n}=u,\, Z'_{j:n}=u'\right) =1. \end{aligned}$$

Now, the density of $Z_{j:n}$, with $k_0 \in {\mathbb {R}}$, is given by

$$\begin{aligned} h_j\left( u;\, k_0\right)= & {} \frac{n!}{\left( j-1\right) !\left( n-j\right) !}\left\{ G\left( u;\, k_0\right) \right\} ^{j-1}g\left( u;\, k_0\right) \left\{ 1-G\left( u;\, k_0\right) \right\} ^{n-j}, \end{aligned}$$

and thus we see that

$$\begin{aligned} \int _{\lambda _{k}} \int _{\lambda _{k_0}}&P\left( l_{u'}\left( k;\,\varvec{S}_{n}^{(j)}\right) <l_u\left( k_0;\,\varvec{S}_{n}^{(j)}\right) \,|\, Z_{j:n}=u,\, Z'_{j:n}=u'\right) h_j\left( u;\, k_0\right) h_j\left( u';\, k\right) \, du\,du'\\&\le \int _{\lambda _{k_0}} h_j\left( u;\, k_0\right) \, du \int _{\lambda _{k}} h_j\left( u';\, k\right) \, du'=1, \end{aligned}$$

since $P\left( l_{u'}\left( k;\,\varvec{S}_{n}^{(j)}\right) <l_u\left( k_0;\,\varvec{S}_{n}^{(j)}\right) \,|\, Z_{j:n}=u,\, Z'_{j:n}=u'\right)$ is bounded by 1. Then, by applying the dominated convergence theorem, we have

$$\begin{aligned} \lim _{n\rightarrow \infty }&P\left( l\left( k;\,\varvec{S}_{n}^{(j)}\right)<l\left( k_0;\,\varvec{S}_{n}^{(j)}\right) \right) \\&=\int _{\lambda _{k}} \int _{\lambda _{k_0}} \lim _{n\rightarrow \infty } P\left( l_{u'}\left( k;\,\varvec{S}_{n}^{(j)}\right) \right. \\&\left. <l_u\left( k_0;\,\varvec{S}_{n}^{(j)}\right) \,|\, Z_{j:n}=u,\, Z'_{j:n}=u'\right) h_j\left( u;\, k_0\right) h_j\left( u';\, k\right) \, du\,du' =1, \end{aligned}$$

which completes the proof of Lemma 1.$\square$

1.4 Proof of Theorem 5

Here, we shall use the shorthand notation $L\left( k;\, \varvec{S}_{n}^{(j)} \right)$ for the log-likelihood function based on $\varvec{S}_{n}^{(j)}$, $\log l\left( k;\, \varvec{S}_{n}^{(j)} \right)$, and $L'\left( k;\, \varvec{S}_{n}^{(j)} \right)$ and $L''\left( k;\, \varvec{S}_{n}^{(j)} \right)$ for its derivatives with respect to k.

First, we assume that $k\ne 0$ and $k^*\ne 0$. We then have

$$\begin{aligned}&\frac{1}{n}L'\left( k;\, \varvec{S}_{n}^{(j)} \right) {\mathop {\rightarrow }\limits ^{{\mathscr {P}}}} \frac{1}{n} \frac{ \int _{\lambda _k} \left( -\frac{n}{k}-\frac{\sum _{i=1}^n \log \left( 1-k\, u\, S_{i:n}^{(j)}\right) }{k^2}\right) \psi _{n,j}(\varvec{S}_{n}^{(j)}, k, u)\,\delta \left( u-v\right) \, du }{ \int _{\lambda _k} \psi _{n,j}(\varvec{S}_{n}^{(j)}, k, u)\,\delta \left( u-v\right) \, du }\nonumber \\&\quad = \frac{1}{n} \sum _{i=1}^n \left( -\frac{1}{k}-\frac{\log \left( 1-k\, v\, S_{i}^{(j)}\right) }{k^2}\right) \nonumber \\&\qquad {\mathop {\rightarrow }\limits ^{{\mathscr {P}}}} p\,\frac{1}{j-1} \sum _{i=1}^{j-1} \left( -\frac{1}{k}-\frac{\log \left( 1-k\, v\, S_{i}^{(j)}\right) }{k^2}\right) +\frac{1}{n}C_1\left( k, v\right) \nonumber \\&\qquad +\left( 1-p\right) \,\frac{1}{n-j} \sum _{i=j+1}^{n} \left( -\frac{1}{k}-\frac{\log \left( 1-k\, v\, S_{i}^{(j)}\right) }{k^2}\right) , \quad {\text{ as }} n\rightarrow \infty , \end{aligned}$$

(13)

where $S_{j:n}^{(j)}=S_{j}^{(j)}=1$, $C_1\left( k, v\right) =-\frac{1}{k}-\frac{\log \left( 1-k\, v\right) }{k^2}$, $\delta (\cdot )$ is the Dirac delta function, $\psi _{n,j}(\varvec{S}_{n}^{(j)}, k, u)=(j-1)!\prod _{i=1}^{j-1}\psi _1(S_{i:n};\, k, u)\times (n-j)!\prod _{i=j+1}^{n}\psi _2(S_{i:n};\, k, u)$, and $\psi _1$, $\psi _2$, $h_j$, $\varvec{S}_{1*}^{(j)}=(S_1^{(j)}, \ldots ,S_{j-1}^{(j)})$ and $\varvec{S}_{2*}^{(j)}=(S_{j+1}^{(j)}, \ldots ,S_{n}^{(j)})$ are all as defined in the proof of Lemma 1.

As with the likelihood function under regularity conditions (see Lemma 5.3 of Lehmann and Casella 1998), we obtain

$$\begin{aligned} E\left( L'\left( k;\, \varvec{S}_{n}^{(j)} \right) \right)= & {} \frac{\partial }{\partial k}\, 1 =0, \end{aligned}$$

(14)

$$\begin{aligned} Var\left( L'\left( k;\, \varvec{S}_{n}^{(j)} \right) \right)= & {} E\left( L'\left( k;\, \varvec{S}_{n}^{(j)} \right) ^2\right) =E\left( -\frac{\partial ^2}{\partial k^2}L\left( k;\, \varvec{S}_{n}^{(j)} \right) \right) +\frac{\partial ^2}{\partial k^2}\,1\nonumber \\= & {} -E\left( L''\left( k;\, \varvec{S}_{n}^{(j)} \right) \right) . \end{aligned}$$

(15)

Hence, it follows, from (13)–(15), with the use of central limit theorem, that

$$\begin{aligned} \frac{L'\left( k;\, \varvec{S}_{n}^{(j)} \right) }{\sqrt{I_{j, n}\left( k\right) }} {\mathop {\longrightarrow }\limits ^{d}} N\left( 0, 1\right) . \end{aligned}$$

We can obtain the same results when $k=0$ or $k^*=0$, by noting that $L'(0 ; \varvec{S}_n^{(j)})=\lim _{k\rightarrow 0}L'(k ; \varvec{S}_n^{(j)})$ and $L''(0 ; \varvec{S}_n^{(j)})=\lim _{k\rightarrow 0}L''(k ; \varvec{S}_n^{(j)})$, and by Lebesgue’s dominated convergence theorem. These details are therefore omitted for the sake of brevity.$\square$

1.5 Proof of Theorem 6

Here, we shall use the shorthand notation $L\left( k;\, \varvec{S}_{n}^{(j)} \right)$ for the log-likelihood function based on $\varvec{S}_{n}^{(j)}$, $\log l\left( k;\, \varvec{S}_{n}^{(j)} \right)$, and $L'\left( k;\, \varvec{S}_{n}^{(j)} \right)$, $L''\left( k;\, \varvec{S}_{n}^{(j)} \right)$ and $L'''\left( k;\, \varvec{S}_{n}^{(j)} \right)$ for its derivatives with respect to k. By a Taylor expansion of $L'\left( {\hat{k}};\, \varvec{S}_{n}^{(j)} \right)$ around k, we obtain

$$\begin{aligned} {\hat{k}}-k= & {} \frac{L'\left( k;\, \varvec{S}_{n}^{(j)} \right) }{-L''\left( k;\, \varvec{S}_{n}^{(j)} \right) -\frac{{\hat{k}}-k}{2}L'''\left( k^*;\, \varvec{S}_{n}^{(j)} \right) }, \end{aligned}$$

(16)

where $k^*$ lies between k and ${\hat{k}}$.

By Theorem 1, we can take any $j\in \left\{ 1,\ldots ,n\right\}$ to treat ${\hat{k}}$ that is the MLE based on $\varvec{S}_n^{(j)}$, without loss of generality. Here, we take j such as $Z_{j:n} {\mathop {\rightarrow }\limits ^{{\mathscr {P}}}} v\in \lambda _k$ as $n\rightarrow \infty$, where ${\mathop {\rightarrow }\limits ^{{\mathscr {P}}}}$ denotes convergence in probability, and $\lambda _{k}=\{u:0< u< \infty , {\text{ if }} k < 0,$ or $0< u< 1/k, {\text{ if }} k>0\}$ as defined in the proof of Proposition 1, and let $p=\lim _{n\rightarrow \infty } j/n$.

From here, we shall show the following facts:

$$\begin{aligned}&\frac{L'\left( k;\, \varvec{S}_{n}^{(j)} \right) }{\sqrt{I_{j, n}\left( k\right) }} {\mathop {\rightarrow }\limits ^{d}} N\left( 0, 1\right) , \end{aligned}$$

(17)

$$\begin{aligned}&-L''\left( k;\, \varvec{S}_{n}^{(j)} \right) {\mathop {\rightarrow }\limits ^{{\mathscr {P}}}} I_{j, n}\left( k\right) , \end{aligned}$$

(18)

$$\begin{aligned}&\frac{1}{n}L'''\left( k^*;\, \varvec{S}_{n}^{(j)}\right) {\text{ is }} {\text{ bounded }} {\text{ in }} {\text{ probability }}, \end{aligned}$$

(19)

as $n \rightarrow \infty$.

We first note that (17) holds from Theorem 6. So, we shall show now (18), for which we assume that $k\ne 0$ and $k^*\ne 0$.

As in (13), we have

$$\begin{aligned}&-\frac{1}{n}L''\left( k;\, \varvec{S}_{n}^{(j)} \right) -\frac{1}{n}I_{j, n}\left( k\right) \\&\quad {\mathop {\rightarrow }\limits ^{{\mathscr {P}}}} p\frac{1}{j-1}\sum _{i=1}^{j-1}\left\{ -\frac{1}{k^2}-\frac{2\, \log \left( 1-k\, v\, S_{i}^{(j)}\right) }{k^3}\right\} \\&\qquad +\left( 1-p\right) \frac{1}{n-j}\sum _{i=j+1}^{n}\left\{ -\frac{1}{k^2}-\frac{2\, \log \left( 1-k\, v\, S_{i}^{(j)}\right) }{k^3}\right\} \\&\qquad -pE\left( -\frac{1}{k^2}-\frac{2\, \log \left( 1-k\, v\, S_{1}^{(j)}\right) }{k^3}\right) -\left( 1-p\right) E\left( -\frac{1}{k^2}-\frac{2\, \log \left( 1-k\, v\, S_{j+1}^{(j)}\right) }{k^3}\right) \\&\quad {\mathop {\rightarrow }\limits ^{{\mathscr {P}}}} 0. \end{aligned}$$

The last convergence follows by weak law of large numbers.

Next, (19) holds since

$$\begin{aligned}&\left| \frac{1}{n}L'''\left( k^*;\, \varvec{S}_{n}^{(j)} \right) \right| \nonumber \\&\quad =\left| \frac{2}{n}\left\{ \frac{l'(k^* ; \varvec{S}_n^{(j)})}{l(k^* ; \varvec{S}_n^{(j)})}\right\} ^3 -\frac{3}{n}\frac{l'(k^* ; \varvec{S}_n^{(j)})l''(k^* ; \varvec{S}_n^{(j)})}{\left\{ l(k^* ; \varvec{S}_n^{(j)})\right\} ^2} +\frac{1}{n}\frac{l'''(k^* ; \varvec{S}_n^{(j)})}{l(k^* ; \varvec{S}_n^{(j)})}\right| \nonumber \\&\quad {\mathop {\rightarrow }\limits ^{{\mathscr {P}}}} \left| p\frac{1}{j-1}\sum _{i=1}^{j-1}\left\{ -\frac{2}{{k^{*}}^3}-\frac{6\, \log \left( 1-k^*\, v\, S_{i}^{(j)}\right) }{{k^{*}}^4}\right\} \right. +\frac{1}{n}C_2\left( k^{*}, v\right) \nonumber \\&\qquad \left. +\left( 1-p\right) \frac{1}{n-j}\sum _{i=j+1}^{n}\left\{ -\frac{2}{{k^{*}}^3}-\frac{6\, \log \left( 1-k^*\, v\, S_{i}^{(j)}\right) }{{k^{*}}^4}\right\} \right| \nonumber \\&\quad {\mathop {\rightarrow }\limits ^{{\mathscr {P}}}} \left| pE_1\left( -\frac{2}{{k^{*}}^3}-\frac{6\, \log \left( 1-k^*\, v\, S_{1}^{(j)}\right) }{{k^{*}}^4}\right) \right. +\frac{1}{n}C_2\left( k^{*}, v\right) \nonumber \\&\qquad \left. +\left( 1-p\right) E_2\left( -\frac{2}{{k^{*}}^3}-\frac{6\, \log \left( 1-k^*\, v\, S_{j+1}^{(j)}\right) }{{k^{*}}^4}\right) \right| \nonumber \\&\quad \le \frac{2}{\left| {k^{*}}^3\right| } +\frac{6p}{{k^{*}}^4}E_1\left( \left| \log \left( 1-k^*\, v\, S_{1}^{(j)}\right) \right| \right) +\frac{6\left( 1-p\right) }{{k^{*}}^4}E_2\left( \left| \log \left( 1-k^*\, v\, S_{j+1}^{(j)}\right) \right| \right) \nonumber \\&\qquad +\frac{1}{n}\left| C_2\left( k^{*}, v\right) \right| \nonumber \\&\quad \le \frac{2}{\left| {k^{*}}^3\right| } +\frac{6\,v\,p}{\left| {k^{*}}^3\right| }E_1\left( S_{1}^{(j)}\right) +\frac{6\,v\,\left( 1-p\right) }{\left| {k^{*}}^3\right| }E_2\left( S_{j+1}^{(j)}\right) +\frac{1}{n}\left| C_2\left( k^{*}, v\right) \right| \nonumber \\&\quad = \frac{2}{\left| {k^{*}}^3\right| } +\frac{6\,v\,p}{\left| {k^{*}}^3\right| \left( 1+k^* \right) } \left[ \frac{1}{1-\left( 1-v\,k^* \right) ^{-1/k^*}}+\frac{1}{v} \right] +\frac{6\,v\,\left( 1-p\right) }{\left| {k^{*}}^3\right| \left( 1+k^* \right) } \left( 1+\frac{1}{v}\right) \nonumber \\&\qquad +\frac{1}{n}\left| C_2\left( k^{*}, v\right) \right| \nonumber \\&\quad <\infty , \end{aligned}$$

(20)

where $C_2\left( k^{*}, v\right) =-\frac{2}{{k^{*}}^3}-\frac{6\, \log \left( 1-k^*\, v\right) }{{k^{*}}^4}$, and $E_1$ and $E_2$ are conditional expectations with respect to $\psi _1$ and $\psi _2$, given $Z_{j:n}=v$, respectively, as defined in the proof of Lemma 1.

The interchangeability of integrations, differentiations and limits in the proofs of (17), (18) and (19) can be justified by Lebesgue’s dominated convergence theorem. These proofs are quite similar to the proof of interchangeability of differentiations and integration in Proposition 1 and are therefore omitted. We can further obtain the same results when $k=0$ or $k^*=0$, by noting that $L''(0 ; \varvec{S}_n^{(j)})=\lim _{k\rightarrow 0}L''(k ; \varvec{S}_n^{(j)})$ and $L'''(0 ; \varvec{S}_n^{(j)})=\lim _{k^*\rightarrow 0}L'''(k^* ; \varvec{S}_n^{(j)})$ and by the use of Lebesgue’s dominated convergence theorem. These proofs are not presented here for the sake of brevity. Thus, the proof of Theorem 6 gets completed.$\square$