Affiliation weighted networks with a differentially private degree sequence

Abstract

Affiliation network is one kind of two-mode social network with two different sets of nodes (namely, a set of actors and a set of social events) and edges representing the affiliation of the actors with the social events. The asymptotic theorem of a differentially private estimator of the parameter in the private $p_{0}$ model has been established. However, the $p_{0}$ model only focuses on binary edges for one-mode network. In many case, the connections in many affiliation networks (two-mode) could be weighted, taking a set of finite discrete values. In this paper, we derive the consistency and asymptotic normality of the moment estimators of parameters in affiliation finite discrete weighted networks with a differentially private degree sequence. Simulation studies and a real data example demonstrate our theoretical results.

Appendix

In this appendix, we will present the proofs of Theorems 1 and 2. We start with some preliminaries. For a vector ${\mathbf {x}}=(x_1,\dots ,x_n)^{T}\in \mathbb {R}^n$, denote the $\ell _{\infty }$ norm of ${\mathbf {x}}$ by $\Vert {\mathbf {x}}\Vert _{\infty }=\max _{1\le i\le n}\mid x_i\mid$. For an $n\times n$ matrix $J=(J_{i,j}), \Vert J\Vert _{\infty }$ denotes the matrix norm induced by the $\Vert \cdot \Vert _{\infty }$-norm on vectors in $\mathbb {R}^n$:

$$\begin{aligned} \Vert J\Vert _{\infty }=\max _{{\mathbf {x}}\ne 0}\frac{\Vert J{\mathbf {x}}\Vert _{\infty }}{\Vert {\mathbf {x}}\Vert _{\infty }}=\max _{1\le i\le n}\sum _{j=1}^{n}\mid J_{i,j}\mid . \end{aligned}$$

Let D be an open convex subset of $\mathbb {R}^n$. We say an $n\times n$ function matrix $F({\mathbf {x}})$ whose elements $F_{ij}({\mathbf {x}})$ are functions on vectors ${\mathbf {x}}$, is Lipschitz continuous on D if there exists a real number $\lambda$ such that for any ${\mathbf {v}}\in R^n$ and any $\mathbf {x,y}\in D,$

$$\begin{aligned} \parallel F({\mathbf {x}})({\mathbf {v}})-F({\mathbf {y}})({\mathbf {v}})\parallel _{\infty }\le \lambda \parallel \mathbf {x-y}\parallel _{\infty }\parallel {\mathbf {v}}\parallel _{\infty }, \end{aligned}$$

where $\lambda$ may depend on n but independent of ${\mathbf {x}}$ and ${\mathbf {y}}$. For fixed $n,\lambda$ is a constant.

We present Lemma 1–3 in Yan et al. (2016) stated as three lemmas here, which will be used in the proofs.

Lemma 2

If $V\in {{\mathcal {L}}}_{mn}(q,Q)$ with $Q/q=o(n),$ then for large enough n,

$$\begin{aligned} \Vert V^{-1}-S \Vert \le \frac{c_1Q^2}{q^3mn}, \end{aligned}$$

where $c_1$ is a constant that dose not depend on M, m and n, and $\Vert A \Vert := \max _{i,j}|a_{i,j}|$ for a general matrix $A=(a_{i,j})$.

Note that if Q and q are bounded constants, then the upper bound of the above approximation error is on the order of $(mn)^{-1},$ indicating that S is a high-accuracy approximation to $V^{-1}.$ Further, based on the above proposition, we immediately have the following lemma.

Lemma 3

If $V\in {{\mathcal {L}}}_{mn}(q,Q)$ with $Q/q=o(n),$ then for a vector ${\mathbf {x}}\in R^{m+n-1},$

$$\begin{aligned} \parallel V^{-1}{\mathbf {x}}\parallel _{\infty }\le \frac{2c_1Q^2}{q^3mn}+\frac{\mid x_{m+n}\mid }{v_{m+n,m+n}}+\max _{i=1,\dots ,m+n-1}\frac{\mid x_{i}\mid }{v_{i,i}}, \end{aligned}$$

where $x_{m+n}:=\sum _{i=1}^{m}x_{i}-\sum _{i=m+1}^{m+n-1}x_{i}.$

Lemma 4

Define a system of equations:

$$\begin{aligned} \begin{array}{rcl} F_i({\varvec{\theta }})&{}=&{}d_i-\sum _{k=1}^{n}f(\alpha _i+\beta _k),i=1,\dots ,m,\\ F_{m+j}({\varvec{\theta }})&{}=&{}b_j-\sum _{k=1}^{m}f(\alpha _k+\beta _j),j=1,\dots ,n-1,\\ F({\varvec{\theta }})&{}=&{}(F_1({\varvec{\theta }}),\dots ,F_m({\varvec{\theta }}),F_{m+1} ({\varvec{\theta }}),\dots ,F_{m+n-1}({\varvec{\theta }}))^T, \end{array} \end{aligned}$$

where $f(\cdot )$ is a continuous function with the third derivative. Let $D\subset \mathbb {R}^{m+n-1}$ be a convex set and assume for any $\mathbf {x,y,v}\in D$, we have

$$\begin{aligned}&\parallel [F^{'}({\mathbf {x}})-F^{'}({\mathbf {y}})]{\mathbf {v}}\parallel _{\infty }\le K_1\parallel \mathbf {x-y}\parallel _{\infty }\parallel {\mathbf {v}}\parallel _{\infty },\\&\quad \max _{i=1,\dots ,m+n-1}\parallel F^{'}_i({\mathbf {x}})-F^{'}_i({\mathbf {y}})\parallel _{\infty }\le K_2\parallel \mathbf {x-y}\parallel _{\infty }, \end{aligned}$$

where $F^{'}({\varvec{\theta }})$ is the Jacobin matrix of F on ${\varvec{\theta }}$ and $F^{'}_{i}({\varvec{\theta }})$ is the gradient function of $F_i$ on ${\varvec{\theta }}.$ Consider ${\varvec{\theta }}^{(0)}\in D$ with $\Omega ({\varvec{\theta }}^{(0)},2\xi )\subset D$ where $\xi =\parallel [F^{'}({\varvec{\theta }}^{(0)})]^{-1}F({\varvec{\theta }}^{(0)})\parallel _{\infty }$ for any ${\varvec{\theta }} \in \Omega ({\varvec{\theta }}^{(0)},2\xi ),$ we assume

$$\begin{aligned} F^{'}({{\varvec{\theta }}})\in {{\mathcal {L}}}_{mn}(q,Q)~~~~or~~-F^{'}({{\varvec{\theta }}})\in {{\mathcal {L}}}_{mn}(q,Q). \end{aligned}$$

For $k=1,2,\dots ,$ define the Newton iterates ${\varvec{\theta }}^{(k+1)}={\varvec{\theta }}^{(k)}-[F^{'}({\varvec{\theta }}^{(k)})]^{-1}F({\varvec{\theta }}^{(k)}).$ Let

$$\begin{aligned} \rho =\frac{c_1(m+n-1)Q^2K_1}{2q^3mn}+\frac{K_2}{mq}. \end{aligned}$$

If $\xi \rho <1/2,$ then ${\varvec{\theta }}^{(k)} \in \Omega ({\varvec{\theta }}^{(0)},2r),k=1,2,\dots ,$ are well defined and satisfy

$$\begin{aligned} \parallel {\varvec{\theta }}^{(k+1)}-{\varvec{\theta }}^{(0)}\parallel _{\infty }\le \xi /(1-\rho \xi ). \end{aligned}$$

Further, $\lim _{k\rightarrow \infty }{\varvec{\theta }}^{(k)}$ exists and the limiting point is precisely the solution of $F({\varvec{\theta }})=0$ in the rage of ${\varvec{\theta }} \in \Omega ({\varvec{\theta }}^{(0)},2\xi ).$

1.1 Appendix A: Proof for Theorem 1

We define a system of functions:

$$\begin{aligned} \begin{array}{rcl} F_{i}({\varvec{\theta }})&{}=&{} {\tilde{d}}_i-\sum _{j=1}^{n}\frac{\sum _{k=0}^{r-1}ke^{k(\alpha _{i} + \beta _{j})}}{\sum _{k=0}^{r-1}e^{k(\alpha _{i} + \beta _{j})}} ,\quad i=1,\dots ,m,\\ F_{m+j}({\varvec{\theta }})&{}=&{} {\tilde{b}}_j-\sum _{i=1}^{m}\frac{\sum _{k=0}^{r-1}ke^{k(\alpha _{i} + \beta _{j})}}{\sum _{k=0}^{r-1}e^{k(\alpha _{i} + \beta _{j})}},\quad j=1,\dots ,n-1\\ F({\varvec{\theta }})&{}=&{} (F_{1}({{\varvec{\theta }}}),\dots ,F_{m+n-1}({{\varvec{\theta }}}))^T. \end{array} \end{aligned}$$

Note that the solution to the equation $F({\varvec{\theta }})=0$ is precisely the private-parameter estimators. Then the Jacobin matrix $F^{'}({\varvec{\theta }})$ of $F({\varvec{\theta }})$ can be calculated as follows. For $i=1,\dots ,m,$

$$\begin{aligned}&\frac{\partial {F_i}}{\partial {{\alpha _l}}}=0,l=1,\dots ,m,l\ne i; \frac{\partial {F_i}}{\partial {{\alpha _i}}}=-\sum ^{n}_{j=1}\frac{\sum _{k=0}^{r-1}\sum _{l=k+1}^{r-1}(k-l)^{2}e^{(k+l)(\alpha _{i} + \beta _{j})}}{(\sum _{k=0}^{r-1}e^{k(\alpha _{i} + \beta _{j})})^2}, \\&\frac{\partial {F_i}}{\partial { {\beta _j}}}=-\frac{\sum _{k=0}^{r-1}\sum _{l=k+1}^{r-1}(k-l)^{2}e^{(k+l)(\alpha _{i} + \beta _{j})}}{(\sum _{k=0}^{r-1}e^{k(\alpha _{i} + \beta _{j})})^2},\quad j=1,\dots ,n-1 \end{aligned}$$

and for $j=1,\dots ,n-1$

$$\begin{aligned}&\frac{\partial {F_{m+j}}}{\partial { {\alpha _l}}}=-\frac{\sum _{k=0}^{r-1}\sum _{l=k+1}^{r-1}(k-l)^{2}e^{(k+l)(\alpha _{i} + \beta _{j})}}{(\sum _{k=0}^{r-1}e^{k(\alpha _{i} + \beta _{j})})^2},\quad l=1,\dots ,m,\\&\frac{\partial {F_{m+j}}}{\partial {{\beta _j}}}=-\sum ^{m}_{i=1}\frac{\sum _{k=0}^{r-1}\sum _{l=k+1}^{r-1}(k-l)^{2}e^{(k+l)(\alpha _{i} + \beta _{j})}}{(\sum _{k=0}^{r-1}e^{k(\alpha _{i} + \beta _{j})})^2};\frac{\partial {F_{m+j}}}{\partial {{\beta _k}}}=0,\\&\quad k=1,\dots ,n-1,k\ne j. \end{aligned}$$

Since $2\Vert \varvec{\theta }\Vert _{\infty } \ge \alpha _{i} + \beta _{j}$, we have:

$$\begin{aligned} e^{2k(\alpha _{i} + \beta _{j})} \le {\left\{ \begin{array}{ll} e^{\alpha _{i} + \beta _{j}+2\Vert \varvec{\theta }\Vert _{\infty }}&{}\quad when ~~~ k =0,\\ e^{(k+(k-1))(\alpha _{i} + \beta _{j})+2\Vert \varvec{\theta }\Vert _{\infty }}&{}\quad when ~~~ k = 1,\ldots ,r-2, \\ e^{[(r-1)+(r-2)](\alpha _{i} + \beta _{j})+2\Vert \varvec{\theta }\Vert _{\infty }}&{}\quad when ~~~ k = r-1. \end{array}\right. } \end{aligned}$$

(A1)

Therefore, we have

$$\begin{aligned} \begin{aligned} e^{0\times (\alpha _{i} + \beta _{j})}&\le \sum _{k=0}^{0}\sum _{l=1}^{r-1}e^{(k+l)(\alpha _{i} + \beta _{j})}e^{2\Vert \varvec{\theta }\Vert _{\infty }} \\ \sum _{k=1}^{r-2}e^{2k(\alpha _{i} + \beta _{j})}&\le \sum _{k=1}^{r-1}\sum _{l=0}^{k-1}e^{(k+l)(\alpha _{i} + \beta _{j})}e^{2\Vert \varvec{\theta }\Vert _{\infty }} \\ e^{2(r-1)(\alpha _{i} + \beta _{j})}&\le \sum _{k=r-2}^{r-2}\sum _{l=k+1}^{r-1}e^{(k+l)(\alpha _{i} + \beta _{j})}e^{2\Vert \varvec{\theta }\Vert _{\infty }} \end{aligned} \end{aligned}$$

Combing the above three inequalities, it yields such that

$$\begin{aligned} \begin{aligned} \sum _{k=0}^{r-1}e^{2k(\alpha _{i} + \beta _{j})}&\le \sum _{k=0}^{0}\sum _{l=1}^{r-1}e^{(k+l)(\alpha _{i} + \beta _{j})}e^{2\Vert \varvec{\theta }\Vert _{\infty }} + \sum _{k=1}^{r-1}\sum _{l=0}^{k-1}e^{(k+l)(\alpha _{i} + \beta _{j})}e^{2\Vert \varvec{\theta }\Vert _{\infty }} \\&\quad + \sum _{k=r-2}^{r-2}\sum _{l=k+1}^{r-1}e^{(k+l)(\alpha _{i} + \beta _{j})}e^{2\Vert \varvec{\theta }\Vert _{\infty }}\\&\le \sum _{0\le k \ne l\le r-1} e^{(k+l)(\alpha _{i} + \beta _{j})}e^{2\Vert \varvec{\theta }\Vert _{\infty }} \end{aligned} \end{aligned}$$

Here, the notation “$\sum _{0\le k \ne l\le r-1}$” is a shorthand for the double summation “$\sum _{k=0}^{r-1}\sum _{l=0, l\ne k}^{r-1}$”. Therefore,

$$\begin{aligned} \begin{aligned}&\frac{\frac{1}{2}\sum _{k\ne l}(k-l)^{2}e^{(k+l)(\alpha _{i} + \beta _{j})}}{(\sum _{k=0}^{r-1}e^{k(\alpha _{i} + \beta _{j})})^2} \ge \frac{\frac{1}{2}\sum _{k\ne l}e^{(k+l)(\alpha _{i} + \beta _{j})}}{(\sum _{k=0}^{r-1}e^{k(\alpha _{i} + \beta _{j})})^2}\\&= \frac{\frac{1}{2}\sum _{k\ne l}e^{(k+l)(\alpha _{i} + \beta _{j})}}{\sum _{k \ne l}e^{(k+l)(\alpha _{i} + \beta _{j})}+\sum _{k=0}^{r-1}e^{2k(\alpha _{i} + \beta _{j})}} \ge \frac{\sum _{k\ne l}e^{(k+l)(\alpha _{i} + \beta _{j})}}{2(1+e^{\Vert \varvec{\theta }\Vert _{\infty }})\sum _{k\ne l}e^{(k+l)(\alpha _{i} + \beta _{j})}}\\&=\frac{1}{2(1+e^{\Vert \varvec{\theta }\Vert _{\infty }})} \end{aligned} \end{aligned}$$

On the other hand, it is easy to verify that

$$\begin{aligned} \frac{\frac{1}{2}\sum _{k \ne l}(k-l)^{2}e^{(k+l)(\alpha _{i} + \beta _{j})} }{{(\sum _{k=0}^{r-1}e^{k(\alpha _{i} + \beta _{j})})^2}} \le \frac{1}{2}\max _{k \ne l}(k-l)^{2} \le \frac{(r-1)^{2}}{2} \end{aligned}$$

Consequently, when $\varvec{\theta } \in \Omega (\varvec{\theta }^{*},2\xi )$, for any $i \ne j$ , we have

$$\begin{aligned} \frac{1}{2(1+e^{2(\Vert \varvec{\theta }\Vert _{\infty }+2\xi )})} \le - F_{i,j}^{'} \le \frac{(r-1)^{2}}{2} \end{aligned}$$

According to definition of $L_{mn}(q,Q)$, we have that $-F^{'}_{i,j} \in L_{mn}(q,Q)$, where

$$\begin{aligned} m = \frac{1}{2(1+e^{2(\Vert \varvec{\theta }\Vert _{\infty }+2\xi )})},\quad M = \frac{(r-1)^{2}}{2} \end{aligned}$$

The constants $K_{1}, K_{2}$ and r in the upper bounds of Lemma 4 are given below.

Lemma 5

Take $D = R^{m+n-1}$ and $\varvec{\theta }^{0} = \varvec{\theta }^{*}$ in Lemma 4. Assume

$$\begin{aligned} \max \{\max _{i=1,\dots ,m}|{\tilde{d}}_i-\mathbb {E}(d_i)|,\max _{j=1,\dots ,n}|{\tilde{b}}_j-\mathbb {E}(b_j)|\}= O_p( \sqrt{n\log n } + \kappa \sqrt{\log n} ). \end{aligned}$$

(A2)

Then we can choose the constants K1, K2 and r in Lemma 4 as

$$\begin{aligned} K_1=n,K_2=n/2,\xi \le \frac{(\log n)^{1/2}}{n^{1/2}}\left( c_{11}e^{6\parallel {\varvec{\theta }}^{*}\parallel _{\infty }}+c_{12} e^{2\parallel {\varvec{\theta }}^{*}\parallel _{\infty }}\right) , \end{aligned}$$

where $c_{11},c_{12}$ are constants.

Proof

For fixed m, n, we first derive $K_1$ and $K_2$ in the inequalities of Lemma 4. Let ${\mathbf {x}},{\mathbf {y}}\in R^{m+n-1}$ and

$$\begin{aligned} {F_i}^{'}({\varvec{\theta }})=({F^{'}_{i,1}}({\varvec{\theta }}),\dots ,{F^{'}_{i,m+n-1}}({\varvec{\theta }}):= (\frac{\partial {F_i}}{\partial {\alpha _1}},\dots , \frac{\partial {F_i}}{\partial {\alpha _m}},\frac{\partial {F_i}}{\partial {\beta _1}},\dots , \frac{\partial {F_i}}{\partial {\beta _{n-1}}}). \end{aligned}$$

Then, for $i=1,\dots ,m$, we have

$$\begin{aligned} \frac{\partial ^2{F_i}}{\partial {{\alpha _l}}\partial { {\alpha _s}}}&=0,\quad s\ne l ,\\ \frac{\partial ^2{F_i}}{\partial {{\alpha _i}}^2}&= - \sum ^{n}_{j=1}\frac{1}{2}\frac{\sum _{k=0}^{r-1}\sum _{l=0}^{r-1}\sum _{s=0}^{r-1}(k-l)^{2}(k+l-2s)e^{(k+l+s)(\alpha _{i}+\beta _{j})}}{\left( \sum _{k=0}^{r-1}e^{k(\alpha _{i} + \beta _{j})}\right) ^3},\\ \frac{\partial ^2{F_i}}{\partial { {\alpha _i}}\partial { {\beta _s}}}&=-\frac{1}{2}\frac{\sum _{k=0}^{r-1}\sum _{l=0}^{r-1}\sum _{s=0}^{r-1}(k-l)^{2}(k+l-2s)e^{(k+l+s)(\alpha _{i}+\beta _{j})}}{\left( \sum _{k=0}^{r-1}e^{k(\alpha _{i} + \beta _{j})}\right) ^3},\\&\quad s=1,\dots ,n-1;\\ \frac{\partial ^2{F_{i}}}{\partial {{\beta _l}}\partial { {\beta _s}}}&=0,s\ne l ,\\ \frac{\partial ^2{F_i}}{\partial {{\beta _j}}^2}&=-\frac{1}{2}\frac{\sum _{k=0}^{r-1}\sum _{l=0}^{r-1}\sum _{s=0}^{r-1}(k-l)^{2}(k+l-2s)e^{(k+l+s)(\alpha _{i}+\beta _{j})}}{\left( \sum _{k=0}^{r-1}e^{k(\alpha _{i} + \beta _{j})}\right) ^3},\\&\quad j=1,\dots ,n-1. \end{aligned}$$

Since $k+l-2s \le 2(r-1)$ and

$$\begin{aligned} \sum _{k=0}^{r-1}\sum _{k=0}^{r-1}\sum _{k=0}^{r-1}e^{(k+l+s)(\alpha _{i}+\beta _{j})} = \left( \sum _{k=0}^{r-1}e^{k(\alpha _{i}+\beta _{j})}\right) ^{3} \end{aligned}$$

We have

$$\begin{aligned} \begin{aligned}&\frac{1}{2}\frac{\sum _{k=0}^{r-1}\sum _{l=0}^{r-1}\sum _{s=0}^{r-1}(k-l)^{2}(k+l-2s)e^{(k+l+s)(\alpha _{i}+\beta _{j})}}{\left( \sum _{k=0}^{r-1}e^{k(\alpha _{i} + \beta _{j})}\right) ^3}\\&\quad \le \frac{(r-1)^{3}\sum _{k=0}^{r-1}\sum _{l=0}^{r-1}\sum _{s=0}^{r-1}e^{(k+l+s)(\alpha _{i}+\beta _{j})}}{\left( \sum _{k=0}^{r-1}e^{k(\alpha _{i} + \beta _{j})}\right) ^3}\\&\quad =(r-1)^{3} \end{aligned} \end{aligned}$$

(A3)

By the mean value theorem for vector-valued functions (Lang 1993, p.341), we have

$$\begin{aligned} F^{'}_i({\mathbf {x}})-F^{'}_i({\mathbf {y}})=J^{(i)}({\mathbf {x}}-{\mathbf {y}}), \end{aligned}$$

where

$$\begin{aligned} J^{(i)}_{s,l}=\int ^1_0\frac{\partial F^{'}_{i,s}}{\partial {{\varvec{\theta }}}_l}(t{\mathbf {x}}+(1-t){\mathbf {y}})dt,~~s,l=1,\dots ,m+n-1. \end{aligned}$$

Therefore,

$$\begin{aligned} \max _s\sum _{l}^{m+n-1}\mid J^{(i)}\mid \le 2(n-1)(r-1)^{3}~~, ~~ \sum ^{}_{s,l}\mid J^{(i)}_{(s,l)}\mid \le 4(n-1)(r-1)^{3} \end{aligned}$$

Similarly, for $i=m+1,\dots ,m+n-1,$ we also have $F^{'}_i({\mathbf {x}})-F^{'}_i({\mathbf {y}})=J^{(i)}({\mathbf {x}}-{\mathbf {y}})$ and $\sum ^{}_{s,l}\mid J^{(i)}_{(s,l)}\mid \le 4(m-1)(r-1)^{3}.$

Consequently,

$$\begin{aligned} \parallel F^{'}_i({\mathbf {x}})-F^{'}_i({\mathbf {y}})\parallel _{\infty }\le & {} \parallel J^{(i)}\parallel _{\infty }\parallel \mathbf {x-y}\parallel _{\infty }\le 2(n-1)(r-1)^{3}\parallel \mathbf {x-y}\parallel _{\infty },\\&i=1,\dots ,m+n-1, \end{aligned}$$

and for $\forall ~{\mathbf {v}}\in R^{m+n-1}$,

$$\begin{aligned} \begin{array}{rcl} \parallel [F^{'}_i({\mathbf {x}})-F^{'}_i({\mathbf {y}})]{\mathbf {v}}\parallel _{\infty }&{}=&{}\displaystyle \max _{i}\mid \sum _{j=1}^{m+n-1}(F_{i,j}^{'}({\mathbf {x}})-F_{i,j}^{'}({\mathbf {y}}))v_j\mid \\ &{}=&{}\displaystyle \max _{i}\mid (\mathbf {x-y})J^{(i)}{\mathbf {v}}\mid \\ &{}\le &{}\parallel \mathbf {x-y}\parallel _{\infty }\parallel {\mathbf {v}}\parallel _{\infty }\sum ^{}_{k,j}\mid J^{(i)}_{(s,l)}\mid \\ &{}\le &{} 4(n-1)(r-1)^{3} \parallel \mathbf {x-y}\parallel _{\infty }\parallel {\mathbf {v}}\parallel _{\infty } \end{array} \end{aligned}$$

so we choose $K_1=4(n-1)(r-1)^{3}$ and $K_2=2(n-1)(r-1)^{3}$ in the inequalities of Lemma 4.

It’s obvious that $-F^{'}({ {\varvec{\theta }}}^{*})\in \mathcal{L}_{mn}(q_{*},Q_{*})$ where,

$$\begin{aligned} q_{*}=\frac{1}{2(1+e^{2\parallel {{\varvec{\theta }}^{*}}\parallel _{\infty }})},\quad Q_{*}=\frac{(r-1)^{2}}{2}. \end{aligned}$$

Note that

$$\begin{aligned} F( {\varvec{\theta }}^*)=(d_1-\mathbb {E}(d_1),\dots ,d_m-\mathbb {E}(d_m),b_1-\mathbb {E}(b_1),\dots ,b_{n-1}-\mathbb {E}(b_{n-1})). \end{aligned}$$

By Lemmas 2 and 3, we have

$$\begin{aligned} \xi =\parallel [F^{'}( {\varvec{\theta }}^*)]^{-1}F( {\varvec{\theta }}^*)\parallel _{\infty }&\le \frac{2c_1(m+n-1)Q_{*}^2\parallel F( {\varvec{\theta }}^*)\parallel _{\infty }}{q_*^3mn}\\&\quad +\displaystyle \max _{i=1,\dots ,m+n-1}\frac{\mid F_{i}( {\varvec{\theta }}^*)\mid }{v_{i,i}}+\frac{\mid F_{m+n}( {\varvec{\theta }}^*)\mid }{v_{m+n,m+n}}\\&\le \frac{(\log n)^{1/2}}{n^{1/2}}\left( c_{11}e^{6\parallel {\varvec{\theta }}^{*}\parallel _{\infty }}+c_{12} e^{2\parallel {\varvec{\theta }}^{*}\parallel _{\infty }}\right) \end{aligned}$$

where $c_{11},c_{12}$ are constants. $\square$

We present several results that we will use in the following lemmas. Recall that a random variable X is sub-exponential with parameter $\kappa > 0$ (Vershynin R 2012) if

$$\begin{aligned}{}[\mathbb {E}|X|^p]^{1/p} \le \kappa p \quad \text { for all } p \ge 1. \end{aligned}$$

(A4)

and sub-exponential random variables satisfy the concentration inequality.

Theorem 3

(Corollary 5.17 in Vershynin R (2012)). Let $X_{1},\ldots ,X_{n}$ be independent centered variables, and suppose each $X_{i}$ is sub-exponential with parameter $\kappa$. Then for every $\epsilon >0$,

$$\begin{aligned} P\left( |\frac{1}{n}\sum _{i}^{n}|\geqslant \epsilon \right) \leqslant \exp [-\gamma n\cdot \min \left( \frac{\epsilon ^{2}}{\gamma ^{2}},\frac{\epsilon }{\kappa }\right) ], \end{aligned}$$

where $\gamma >0$ is an absolute constant.

Note that if X is a sub-exponential random variable with parameter X, then the centered random variable $X-\mathbb {E}[X]$ is also sub-exponential with parameter $2 \kappa$. This follows from the triangle inequality applied to the p-norm, followed by Jensen’s inequality for $p \ge 1$:

$$\begin{aligned} \begin{aligned} \big [\mathbb {E}\big |X-\mathbb {E}[X]\big |^p\big ]^{1/p}&\le [\mathbb {E}|X|^p]^{1/p} + \big |\mathbb {E}[X]\big | \le 2[\mathbb {E}|X|^p]^{1/p}. \end{aligned} \end{aligned}$$

Lemma 6

Let X be a discrete Laplace random variable with the probability distribution

$$\begin{aligned} \mathbb {P}(X=x)= \frac{1-\lambda }{1+\lambda } \lambda ^{|x|},~~x=0, \pm 1, \ldots , \lambda \in (0,1). \end{aligned}$$

Then X is sub-exponential with parameter $2( \log \frac{1}{\lambda } )^{-1}$.

Proof

Note that

$$\begin{aligned} \mathbb {E}|X|^p= & {} \frac{ 2(1-\lambda )}{1+\lambda } \sum _{x=0}^\infty \lambda ^x x^p \le \frac{ 2(1-\lambda )}{1+\lambda } \int _0^\infty t^p e^{-t\log \frac{1}{\lambda }} dt \\\le & {} \frac{ 2(1-\lambda )}{1+\lambda }(\frac{1}{ \log \frac{1}{\lambda } })^{p+1} \Gamma (p). \end{aligned}$$

It follows that

$$\begin{aligned}{}[\mathbb {E}|X|^p]^{1/p}< 2^{1/p} (\frac{1}{ \log \frac{1}{\lambda } })^{1+1/p} p < 2p \frac{1}{ \log \frac{1}{\lambda } } \end{aligned}$$

$\square$

The following lemma assures that condition (A5) hold with a large probability.

Lemma 7

Let $\kappa _{mn}=2(r-1)(-\log \lambda _{mn})^{-1}=4(r-1)^{2}/\epsilon _{mn}$, where $\lambda _{mn} \in (0,1)$. We have

$$\begin{aligned} \max \left\{ \max _{i=1,\dots ,m}|{\tilde{d}}_i-\mathbb {E}(d_i)|,\max _{j=1,\dots ,n}|{\tilde{b}}_j-\mathbb {E}(b_j)|\right\} = O_p\left( \sqrt{n\log n } + \kappa _{mn}\sqrt{\log n}\right) .\nonumber \\ \end{aligned}$$

(A5)

Proof

By Hoeffding’s inequality (Hoeffding 1963), and $m<n$ we have

$$\begin{aligned} \begin{aligned}&P\left( |d_i-\mathbb {E}(d_i)|\ge (r-1)\sqrt{n\log {n}}\right) \\&\le 2\exp {\left\{ -\frac{2(r-1)^{2}(n-1)\log (n-1)}{(m-1)(r-1)^{2}}\right\} }=\frac{2}{n^{2n/m}} \le \frac{2}{n^2} \end{aligned} \end{aligned}$$

Therefore,

$$\begin{aligned} \begin{array}{rcl} P\left( \displaystyle \max _{i}|d_i-\mathbb {E}(d_i)|\ge (r-1)\sqrt{n\log {n}}\right) &{}\le &{}P\left( \displaystyle \bigcup _{i}|d_i-\mathbb {E}(d_i)|\ge (r-1)\sqrt{n\log {n}}\right) \\ &{}\le &{}\displaystyle \sum _{i=1}^{m}P\left( |d_i-\mathbb {E}(d_i)|\ge (r-1)\sqrt{n\log {n}}\right) \\ &{}\le &{}m\times \frac{2}{n^2}\le \frac{2}{n}. \end{array}\end{aligned}$$

Similarly, we have

$$\begin{aligned} P\left( \displaystyle \max _{j}|b_j-\mathbb {E}(b_j)|\ge (r-1)\sqrt{n\log {n}}\right) \le \frac{2}{n}. \end{aligned}$$

Consequently,

$$\begin{aligned} \begin{array}{rcl} &{}~&{}P\left( \displaystyle \max \left\{ \max _{i=1,\dots ,m}|d_i-\mathbb {E}(d_i)|,\max _{j=1,\dots ,n}|b_j-\mathbb {E}(b_j)|\right\} \ge (r-1)\sqrt{n\log {n}}\right) \\ &{}\le &{}P\left( \displaystyle \max _{i}|d_i-\mathbb {E}(d_i)|\ge (r-1)\sqrt{n\log {n}}\right) +P\left( \displaystyle \max _{j}|b_j-\mathbb {E}(b_j)|\ge (r-1)\sqrt{n\log {n}}\right) \\ &{}\le &{}\frac{4}{n}. \end{array} \end{aligned}$$

Then, we have

$$\begin{aligned} \max \left\{ \max _{i=1,\dots ,m}|{\tilde{d}}_i-\mathbb {E}(d_i)|,\max _{j=1,\dots ,n}|{\tilde{b}}_j-\mathbb {E}(b_j)|\right\} \le (r-1)\sqrt{n\log (n)}. \end{aligned}$$

Note that $\{e_i^+\}_{i=1}^n$ and $\{ e_i^- \}_{i=1}^n$ are independently discrete Laplace random variables and sub-exponential with the same parameter $\kappa _{mn}$ by Lemma 6. By the concentration inequality in Theorem 3, we have

$$\begin{aligned} \mathbb {P}\left( \max _{i=1, \ldots , m} |e_i^+| \ge 2\kappa _{mn} \sqrt{\frac{\log n}{\gamma }}\right) \le \sum _i \mathbb {P}\left( |e_i^+| \ge 2\kappa _{mn} \sqrt{\frac{\log n}{\gamma }}\right) \le n\times e^{-2\log n} = \frac{1}{n}\nonumber \\ \end{aligned}$$

(A6)

and

$$\begin{aligned} \mathbb {P}\left( |\sum _{i=1}^m e_i^+| \ge 2\kappa _{mn} \sqrt{\frac{n\log n}{\gamma }}\right) \le 2 \exp \left( - \frac{\gamma }{n} \times \frac{n\log n}{\gamma }\right) = \frac{2}{n}, \end{aligned}$$

(A7)

where $\gamma$ is an absolute constant appearing in the concentration inequality. So, with probability at least $1-4n/(n-1)^2-2/n$, we have

$$\begin{aligned} \max _{i=1, \ldots , m} |{\tilde{d}}_{i} - \mathbb {E}(d_i) | \le \max _i|d_i- \mathbb {E}(d_i) | + \max _i |e_i^+| \le (r-1)\sqrt{n\log n } + 2\kappa _{mn} \sqrt{\frac{\log n}{\gamma }} . \end{aligned}$$

Similarly, with probability at least $1-4n/(n-1)^2-2/n$, we have

$$\begin{aligned} \max _{i=1, \ldots , n} |{\tilde{b}}_i- \mathbb {E}(b_i) | \le (r-1)\sqrt{n\log n } + 2\kappa _{mn} \sqrt{\frac{\log n}{\gamma }} . \end{aligned}$$

(A8)

Let A and B be the events:

$$\begin{aligned} \begin{array}{rcl} A &{} = &{} \left\{ \max _{i=1, \ldots , m} |{\tilde{d}}_i - \mathbb {E}(d_i) | \le (r-1)\sqrt{n\log n } + 2\kappa _{mn} \sqrt{\frac{\log n}{\gamma }} \right\} , \\ B &{} = &{} \left\{ \max _{i=1, \ldots , n} |{\tilde{b}}_i - \mathbb {E}(b_i) | \le (r-1)\sqrt{n\log n } + 2\kappa _{mn} \sqrt{\frac{\log n}{\gamma }}\right\} . \end{array} \end{aligned}$$

Consequently, as n goes to infinity, we have

$$\begin{aligned} \mathbb {P}(A\bigcap B) \ge 1 - \mathbb {P}(A^c) - \mathbb {P}(B^c) \ge 1- 8n/(n-1)^2-4/n \rightarrow 1. \end{aligned}$$

This completes the proof. $\square$

It can be easily checked that $-F'(\theta )\in \mathcal {L}_n(m, M)$, (Zhang et al. 2016) where $M=(r-1)^2/2$ and $m= 1/2( 1+ e^{ 2\Vert \theta \Vert _\infty })^2$. We are now ready to present the proof of Theorem 1.

Proof of Theorem 1

Assume that condition (A5) holds. Recall that the Newton’s iterates in Lemma 4, ${\varvec{\theta }}^{(k+1)}={\varvec{\theta }}^{(k)}-[F^{'}( {\varvec{\theta }}^{(k)})]^{-1}F( {\varvec{\theta }}^{(k)})$ with ${\varvec{\theta }}^{(0)}= {\varvec{\theta }}^{*}.$ If ${\varvec{\theta }}\in \Omega ( {\varvec{\theta }}^{*},2\xi )$, then $-F^{'}({ {\varvec{\theta }}}^{*})\in {{\mathcal {L}}}_{mn}(q,Q)$ with

$$\begin{aligned} q=\frac{1}{2(1+e^{2(\parallel {{\varvec{\theta }}^{*}}\parallel _{\infty }+2\xi )})},\quad Q=\frac{(r-1)^{2}}{2}. \end{aligned}$$

To apply Lemma 4, we need to calculate r and $\rho r$ in this theorem. Let

$$\begin{aligned} {\tilde{F}}_{m+n}(\theta ^*)= \sum _{i=1}^{m}F_{i}(\theta ^*)-\sum _{i=1}^{n-1}F_{n+i}(\theta ^*)={\tilde{b}}_n-\sum _{i=1}^{n-1}\frac{\displaystyle \sum \nolimits ^{q-1}_{k=0}k e^{k({\alpha }_{i}+{\beta }_{n})}}{\displaystyle \sum \nolimits ^{q-1}_{k=0}e^{k({\alpha }_{i}+{\beta }_{n})}}+\sum _{i=1}^{m}e_{i}^{+} - \sum _{i=1}^{n-1}e_{i}^{-}. \end{aligned}$$

By Lemma 7, we have

$$\begin{aligned} |{\tilde{F}}_{m+n}(\theta ^*)| = O_p( (1+\kappa _{mn} )\sqrt{n\log n} ). \end{aligned}$$

By Lemma 2,

$$\begin{aligned} \xi&= \parallel [F^{'} \theta ^*]^{-1}F( \theta ^*)\parallel _{\infty }\nonumber \\&\leqslant (m+n-1)\Vert V^{-1} - S \Vert \Vert F(\theta ^*) \Vert _\infty + \max _{i=1,\dots ,m+n-1}\frac{\mid F_{i}( \theta ^*)\mid }{v_{i,i}} + \frac{\mid F_{m+n}(\theta ^*)\mid }{v_{m+n,m+n}}\nonumber \\&\leqslant \frac{2c_{1}(r-1)^{2}( 1 + e^{2(\Vert \theta ^*\Vert _\infty ) } )^6}{mn} \left( (r-1)\sqrt{(n-1)\log (n-1)}+\kappa _{mn}\sqrt{\log n } \right) \nonumber \\&\quad +\frac{2(1+e^{2\theta ^*\parallel _{\infty }})}{n-1} \left( (1+\kappa _{mn})\sqrt{n\log n}+(r-1)\sqrt{(n-1)\log (n-1)}+\kappa _{mn}\sqrt{\log n} \right) \nonumber \\&= O_p \left( \frac{(\log n)^{1/2}}{n^{1/2}}(1+\kappa _{mn}) e^{6\parallel \theta ^{*}\parallel _{\infty }} \right) . \end{aligned}$$

(A9)

By Lemma (5), we have

$$\begin{aligned} \rho = \frac{ c_1(2n-1)Q^24(n-1)(r-1)^3}{2q^3n^2} + \frac{ 2(n-1) }{ q(n-1)(r-1)^3 } = O( e^{6\Vert \theta ^*\Vert _\infty } ) \end{aligned}$$

By Lemma (5) and condition (A5), for sufficient small $\xi$,

$$\begin{aligned} \begin{array}{rcl} \rho \xi &{}\le &{}\left[ \frac{ c_1(2n-1)Q^24(n-1)(r-1)^3}{2q^3n^2} + \frac{ 2(n-1) }{ q(n-1)(r-1)^3 }\right] \times O_p \left( \frac{(\log n)^{1/2}}{n^{1/2}}(1+\kappa _{mn}) e^{6\parallel \theta ^{*}\parallel _{\infty }} \right) \\ &{}\le &{}O_p \left( \frac{(\log n)^{1/2}}{n^{1/2}}(1+\kappa _{mn}) e^{12\parallel \theta ^{*}\parallel _{\infty }} \right) \end{array} \end{aligned}$$

Note that if $(1+\kappa ) e^{12\Vert \theta ^*\Vert _\infty } = o( (n/\log n)^{1/2} )$, then $\xi =o(1)$, and $\rho \xi \rightarrow 0$ as $n\rightarrow \infty$. Consequently, there exists N, when $n\ge N$, $\rho \xi <\frac{1}{2}$, by Lemma 4, $\lim _{n\rightarrow \infty }{\widehat{{\varvec{\theta }}}}^{(n)}$ exists. Denote the limit as $\widehat{\varvec{\theta }}$, then it satisfies

$$\begin{aligned} \parallel {\widehat{{\varvec{\theta }}}}- {\varvec{\theta }}^*\parallel _{\infty }\le 2r=O( \frac{(\log n)^{1/2}}{n^{1/2}}(1+\kappa _{mn}) e^{12\parallel \theta ^{*}\parallel _{\infty }})=o(1) \end{aligned}$$

By Lemma 7, condition (A5) holds with probability one, thus the above inequality also holds with probability one. The uniqueness of the parameter estimator comes from Proposition 5 in Yan et al. (2016) of Sect. 5. $\square$

1.2 Appendix B: Proof for Theorem 2

We first present one proposition. Since $d_i=\sum ^{n}_{j= 1}a_{i,j}$ and $b_j=\sum ^{m}_{ j= 1}a_{i,j}$ are sums of n and m independent random variables, by the central limit theorem for the bounded case in Loeve (1977), p. 289, we know that ${v_{i,i}}^{-1/2}(d_i-\mathbb {E}(d_i))$ and ${v_{m+j,m+j}}^{-1/2}(b_j-\mathbb {E}(b_j))$ are asymptotically standard normal if $v_{i,i}$, $v_{m+j,m+j}$ diverges. Note that

$$\begin{aligned}&\frac{m}{2(1+e^{2\parallel {{\varvec{\theta }}^{*}}\parallel _{\infty }})}\le v_{i,i}\le \frac{m(r-1)^{2}}{2}.\\&\quad \frac{n}{2(1+e^{2\parallel {{\varvec{\theta }}^{*}}\parallel _{\infty }})}\le v_{m+j,m+j}\le \frac{n(r-1)^{2}}{2}. \end{aligned}$$

Following Yan (2020), we have the following proposition.

Proposition 1

Let $\kappa _{mn}=2(r-1)(-\log \lambda _{mn})^{-1}$, where $\lambda _{mn}=\exp (-\epsilon _{mn}/2(r-1))$.

• (i) If $\kappa _{mn} (\log n)^{1/2} e^{2\Vert \theta ^*\Vert _\infty }=o(1)$ and $e^{\Vert \theta ^*\Vert _\infty }=o( n^{1/2} )$, then for any fixed $k \ge 1$, as $n\rightarrow \infty$, the vector consisting of the first k elements of $S({\tilde{g}} - \mathbb {E}g )$ is asymptotically multivariate normal with mean zero and covariance matrix given by the upper left $k \times k$ block of S.

• (ii) Let

  $$\begin{aligned} s_{mn}^2=\mathrm {Var}\left( \sum _{i=1}^m e_i^+ - \sum _{i=1}^{n-1} e_i^-\right) = (m+n-1)\frac{ 2\lambda _{mn}}{ (1-\lambda _{mn})^2}. \end{aligned}$$

  Assume that $s_{mn}/v_{m+n,m+n}^{1/2} \rightarrow c$ for some constant c. For any fixed $k \ge 1$, the vector consisting of the first k elements of $S({\tilde{g}} - \mathbb {E}g )$ is asymptotically k-dimensional multivariate normal distribution with mean ${\mathbf {0}}$ and covariance matrix

  $$\begin{aligned} \mathrm {diag}\left( \frac{1}{v_{1,1}}, \ldots , \frac{1}{v_{k,k}}\right) + \left( \frac{1}{v_{m+n,m+n}} + \frac{s_{mn}^2}{v_{m+n,m+n}^2}\right) {\mathbf {1}}_k {\mathbf {1}}_k^\top , \end{aligned}$$

  where ${\mathbf {1}}_k$ is a k-dimensional column vector with all entries 1.

To complete the proof of Theorem 2, we need two lemmas as follows.

Lemma 8

Let $R=V^{-1}-S$ and $U=Cov[R\{{\mathbf {g}}-\mathbb {E}{\mathbf {g}}\}]$. Then

$$\begin{aligned} \Vert U \Vert \le \Vert V^{-1}-S\Vert +\frac{6(q-1)^2(1+e^{2\parallel {{\varvec{\theta }}^{*}}\parallel _{\infty }})^{2}}{4mn}. \end{aligned}$$

Proof

Note that

$$\begin{aligned} U=RVR^T=(V^{-1}-S)V(V^{-1}-S)^T=(V^{-1}-S)-S(I-VS), \end{aligned}$$

where I is a $(m+n-1)\times (m+n-1)$ identity matrix, and by Yan et al. (2016), we have

$$\begin{aligned} \mid \{S(I-VS)\}_{i,j}\mid =\mid w_{i,j}\mid \le \frac{6(r-1)^2(1+e^{2\parallel {{\varvec{\theta }}^{*}}\parallel _{\infty }})^{2}}{4mn}. \end{aligned}$$

Thus,

$$\begin{aligned} \parallel U \parallel \le \parallel V^{-1}-S\parallel +\parallel \{S(I_{m+n-1}-VS)\}\parallel \le \parallel V^{-1}-S\parallel +\frac{6(r-1)^2(1+e^{2\parallel {{\varvec{\theta }}^{*}}\parallel _{\infty }})^{2}}{4mn}. \end{aligned}$$

$\square$

Lemma 9

Let $\kappa _{mn}=2(r-1)(-\log \lambda _{mn})^{-1}=4(r-1)^2\epsilon _{mn}^{-1}$. If $(1+\kappa _{mn})^2 e^{ 18\Vert \theta ^*\Vert _\infty } = o( (n/\log n)^{1/2} )$, then for any i,

$$\begin{aligned} {\widehat{\theta }}_i- \theta _i^* = [V^{-1} ({\tilde{g}} - \mathbb {E}g ) ]_i + o_p( n^{-1/2} ). \end{aligned}$$

(B1)

Proof

The proof is very similar to the proof of Lemma 9 in Yan et al. (2016). It only requires verification of the fact that all the steps hold by replacing g with ${\tilde{g}}$. $\square$

Proof

By Lemma 9 and noting that $V^{-1}=S+R$, we have

$$\begin{aligned} ({\widehat{\theta }}-\theta )_i= [S({\tilde{g}} - \mathbb {E}g) ]_i+ [R \{ {\tilde{g}} - \mathbb {E}g \}]_i + o_p( n^{-1/2} ). \end{aligned}$$

By (A6) and $\Vert {\tilde{g}} - g \Vert _\infty = O_p( \kappa _{mn} \sqrt{\log n})$, we have

$$\begin{aligned}{}[R ( {\tilde{g}} - g )]_i = O_p\left( n \frac{ M^2}{m^3n^2} \kappa _{mn} \sqrt{\log n}\right) = O_p\left( \frac{ \kappa _{mn} (\log n)^{1/2}e^{6\Vert \theta ^*\Vert _\infty } }{n}\right) , \end{aligned}$$

where

$$\begin{aligned} m=\frac{1}{2(1+e^{2\Vert \theta ^*\Vert _\infty })^2}, ~~M=\frac{(r-1)^2}{2}. \end{aligned}$$

If $\kappa _{mn} e^{6\Vert \theta ^*\Vert _\infty } = o( (n/\log n)^{1/2})$, then $[R \{ {\widetilde{g}} - g \}]_i=o_p(n^{-1/2})$. Combing Lemma 8, it yields

$$\begin{aligned}{}[R ( {\tilde{g}} - \mathbb {E}g )]_i= [R({\tilde{g}} - g)]_i + [R(g - \mathbb {E}g)]_i = o_p(n^{-1/2}). \end{aligned}$$

Consequently,

$$\begin{aligned} ({\widehat{\theta }}-\theta )_i = [S({\tilde{g}} - \mathbb {E}g) ]_i + o_p( n^{-1/2} ). \end{aligned}$$

Theorem 2 immediately follows from Proposition 1. $\square$