Distributed Noise-Shaping Quantization: I. Beta Duals of Finite Frames and Near-Optimal Quantization of Random Measurements

Abstract

This paper introduces a new algorithm for the so-called “analysis problem” in quantization of finite frame representations that provides a near-optimal solution in the case of random measurements. The main contributions include the development of a general quantization framework called distributed noise shaping and, in particular, beta duals of frames, as well as the performance analysis of beta duals in both deterministic and probabilistic settings. It is shown that for random frames, using beta duals results in near-optimally accurate reconstructions with respect to both the frame redundancy and the number of levels at which the frame coefficients are quantized. More specifically, for any frame E of m vectors in $\mathbb {R}^k$ except possibly from a subset of Gaussian measure exponentially small in m and for any number $L \ge 2$ of quantization levels per measurement to be used to encode the unit ball in $\mathbb {R}^k$, there is an algorithmic quantization scheme and a dual frame together that guarantee a reconstruction error of at most $\sqrt{k}L^{-(1-\eta )m/k}$, where $\eta$ can be arbitrarily small for sufficiently large problems. Additional features of the proposed algorithm include low computational cost and parallel implementability.

A Appendix

Lemma A.1

Let $\xi \sim \mathcal {N}(0,I_l)$. For any $0 < \varepsilon \le 1$, we have

$$\begin{aligned} \mathbb {P}\left( \Vert \xi \Vert _2 \le \varepsilon \sqrt{l} \right) \le \varepsilon ^{l} \mathrm{e}^{(1-\varepsilon ^2)l/2}. \end{aligned}$$

Proof

For any $t \ge 0$, we have

$$\begin{aligned} \mathbb {P}\left( \Vert \xi \Vert ^2_2 \le \varepsilon ^2 l \right)\le & {} \int _{\mathbb {R}^l} \hbox {e}^{(\varepsilon ^2 l - \Vert x\Vert _2^2)t/2} \frac{\hbox {e}^{-\Vert x\Vert _2^2/2}}{(2\pi )^{l/2}} \,\mathrm {d}x \\= & {} \hbox {e}^{\varepsilon ^2 t l/2} \int _{\mathbb {R}^l} \frac{\hbox {e}^{-(1+t)\Vert x\Vert _2^2/2}}{(2\pi )^{l/2}} \,\mathrm {d}x = \left( \frac{\hbox {e}^{\varepsilon ^2 t}}{1+t} \right) ^{l/2}. \end{aligned}$$

Choosing $t = \varepsilon ^{-2}-1$ yields the desired bound. $\square$

Proof of Theorem 4.3

For an arbitrary $\tau > 1$, let $\mathcal {E}_1$ be the event $\{ \Vert \Omega \Vert _{2\rightarrow 2} \le 2\tau \sqrt{l}\}$. Proposition 4.1 with $m=l$, $E=\Omega$, $t=2(\tau -1)\sqrt{l}$ implies

$$\begin{aligned} \mathbb {P}(\mathcal {E}_1^c) \le \hbox {e}^{-2(\tau -1)^2 l}. \end{aligned}$$

Next, consider a $\rho$-net Q of the unit sphere of $\mathbb {R}^k$ with $|Q| \le 2k(1+2/\rho )^{k-1}$ (see [27, Proposition 2.1]), where we set $\rho = \varepsilon /(4\tau )$. Let $\mathcal {E}_2$ be the event $\{ \Vert \Omega z\Vert _2 \ge \varepsilon \sqrt{l},\ \forall z \in Q\}$. For each fixed $z\in \mathbb {R}^k$ with unit norm, $\Omega z$ has entries that are i.i.d. $\mathcal {N}(0,1)$. By Lemma A.1, we have

$$\begin{aligned} \mathbb {P}(\mathcal {E}_2^c) \le |Q| \varepsilon ^{l} \hbox {e}^{(1-\varepsilon ^2)l/2} \le 2k(\varepsilon + 8 \tau )^{k-1} \varepsilon ^{l-k+1} \hbox {e}^{(1-\varepsilon ^2)l/2}. \end{aligned}$$

Suppose the event $\mathcal {E}_1 \cap \mathcal {E}_2$ occurs. For any unit norm $x \in \mathbb {R}^k$, there exists a $z \in Q$ such that $\Vert x-z\Vert _2 \le \rho$. Then $\Vert \Omega (x-z)\Vert _2 \le 2\tau \rho \sqrt{l} = \varepsilon \sqrt{l}/2$ and $\Vert \Omega z\Vert _2 \ge \varepsilon \sqrt{l}$, so that

$$\begin{aligned} \Vert \Omega x\Vert _2 \ge \Vert \Omega z\Vert _2 - \Vert \Omega (x-z)\Vert _2 \ge \varepsilon \sqrt{l}/2, \end{aligned}$$

hence $\sigma _{\mathrm{min}}(\Omega ) \ge \varepsilon \sqrt{l}/2$. It follows that $\mathcal {F}:=\left\{ \sigma _{\mathrm{min}}(\Omega ) \le \varepsilon \sqrt{l}/2 \right\} \subset \mathcal {E}_1^c \cup \mathcal {E}_2^c$, and therefore

$$\begin{aligned} \mathbb {P}\left( \mathcal {F}\right) \le \hbox {e}^{-2(\tau -1)^2 l} + 2k(\varepsilon + 8 \tau )^{k-1} \varepsilon ^{l-k+1} \hbox {e}^{l/2}. \end{aligned}$$

We still have the freedom to choose $\tau > 1$ as a function of $\varepsilon$, l, and k. For simplicity, we choose $\tau = 1+\sqrt{\log \varepsilon ^{-1}}$, so that $\hbox {e}^{-2(\tau -1)^2 l} = \varepsilon ^{2l}$. Noting that $1 + k(1+8\tau )^{k-1} < (2+8\tau )^k$, we obtain

$$\begin{aligned} \mathbb {P}\left( \mathcal {F}\right)< \varepsilon ^{l-k+1} \left( 1 + 2k(1 + 8\tau )^{k-1} \hbox {e}^{l/2}\right) < 2 \left( 10+8\sqrt{\log \varepsilon ^{-1}}\right) ^k \hbox {e}^{l/2} \varepsilon ^{l-k+1}. \end{aligned}$$