Skip to main content
SpringerLink
Log in

Embedding-Based Representation of Signal Geometry

  • Chapter
  • First Online:
Excursions in Harmonic Analysis, Volume 5

Part of the book series: Applied and Numerical Harmonic Analysis ((ANHA))

  • 833 Accesses

Abstract

Low-dimensional embeddings have emerged as a key component in modern signal processing theory and practice. In particular, embeddings transform signals in a way that preserves their geometric relationship but makes processing more convenient. The literature has, for the most part, focused on lowering the dimensionality of the signal space while preserving distances between signals. However, there has also been work exploring the effects of quantization, as well as on transforming geometric quantities, such as distances and inner products, to metrics easier to compute on modern computers, such as the Hamming distance.Embeddings are particularly suited for modern signal processing applications, in which the fidelity of information represented by the signals is of interest, instead of the fidelity of the signal itself. Most typically, this information is encoded in the relationship of the signal to other signals and templates, as encapsulated in the geometry of the signal space. Thus, embeddings are very good tools to capture the geometry, while reducing the processing burden.In this chapter, we provide a concise overview of the area, including foundational results and recent developments. Our goal is to expose the field to a wider community, to provide, as much as possible, a unifying view of the literature, and to demonstrate the usefulness and applicability of the results.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Technically, we could incorporate g(⋅ ) into \(d_{\mathscr{S}}(\cdot,\cdot )\) and remove it from this definition. However, we choose to make it explicit here and consider it a distortion to be explicitly analyzed. In an abuse of nomenclature, we generally refer to d(⋅ , ⋅ ) as distance, even if in some cases it is not strictly a distance metric but might be an inner product, or another geometric quantity of interest.

References

  1. D. Achlioptas, Database-friendly random projections: Johnson-Lindenstrauss with binary coins. J. Comput. Syst. Sci. 66, 671–687 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  2. D. Achlioptas, F. Mcsherry, B. Schölkopf, Sampling techniques for kernel methods, in Advances in Neural Information Processing Systems (2002), pp. 335–342

    Google Scholar 

  3. A. Ai, A. Lapanowski, Y. Plan, R. Vershynin, One-bit compressed sensing with non-gaussian measurements. Linear Algebra Appl. 441, 222–239 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  4. N. Ailon, B. Chazelle, Approximate nearest neighbors and the fast Johnson-Lindenstrauss transform, in Proceedings of the Thirty-Eighth Annual ACM Symposium on Theory of Computing (2006), pp. 557–563

    Google Scholar 

  5. A. Andoni, P. Indyk, Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions. Commun. ACM 51(1), 117–122 (2008). DOI:10.1145/1327452.1327494

    Article  Google Scholar 

  6. A. Andoni, M. Deza, A. Gupta, P. Indyk, S. Raskhodnikova, Lower bounds for embedding edit distance into normed spaces, in Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms (2003), pp. 523–526

    Google Scholar 

  7. A.S. Bandeira, D.G. Mixon, B. Recht, Compressive classification and the rare eclipse problem (2014), arXiv preprint arXiv:1404.3203

    Google Scholar 

  8. Z. Bar-Yossef, T. Jayram, R. Krauthgamer, R. Kumar, Approximating edit distance efficiently, in Proceedings of the 45th Annual IEEE Symposium on Foundations of Computer Science, 2004 (IEEE, Los Alamitos, 2004), pp. 550–559

    Google Scholar 

  9. R. Baraniuk, M. Wakin, Random projections of smooth manifolds. Found. Comput. Math. 9(1), 51–77 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  10. R. Baraniuk, M. Davenport, R. DeVore, M. Wakin, A simple proof of the restricted isometry property for random matrices. Const. Approx. 28(3), 253–263 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  11. R. Baraniuk, V. Cevher, M. Duarte, C. Hegde, Model-based compressive sensing. IEEE Trans. Inf. Theory 56(4), 1982–2001 (2010)

    Article  MathSciNet  Google Scholar 

  12. T. Blumensath, M. Davies, Sampling theorems for signals from the union of finite-dimensional linear subspaces. IEEE Trans. Inf. Theory 55(4), 1872–1882 (2009)

    Article  MathSciNet  Google Scholar 

  13. P.T. Boufounos, Universal rate-efficient scalar quantization. IEEE Trans. Inf. Theory 58(3), 1861–1872 (2012). DOI:10.1109/TIT.2011.2173899

    Article  MathSciNet  Google Scholar 

  14. P.T. Boufounos, Angle-preserving quantized phase embeddings, in Proceedings of SPIE Wavelets and Sparsity XV, San Diego, CA (2013)

    Google Scholar 

  15. P.T. Boufounos, On embedding the angles between signals, in Signal Processing with Adaptive Sparse Structured Representations, Lausanne, Switzerland (2013)

    Google Scholar 

  16. P.T. Boufounos, Sparse signal reconstruction from phase-only measurements, in Proceedings of International Conference on Sampling Theory and Applications, Bremen, Germany (2013)

    Google Scholar 

  17. P.T. Boufounos, H. Mansour, Universal embeddings for kernel machine classification, in Proceedings of Sampling Theory and Applications, Washington, DC (2015)

    Google Scholar 

  18. P.T. Boufounos, S. Rane, Secure binary embeddings for privacy preserving nearest neighbors, in Proceedings of the IEEE Workshop on Information Forensics and Security, Foz do Iguau, Brazil (2011). DOI:10.1109/WIFS.2011.6123149

    Google Scholar 

  19. P.T. Boufounos, S. Rane, Efficient coding of signal distances using universal quantized embeddings, in Proceedings of Data Compression Conference, Snowbird, UT (2013)

    Google Scholar 

  20. P.T. Boufounos, S. Rane, H. Mansour, Representation and coding of signal geometry (2015), arXiv preprint arXiv:1512.07636

    Google Scholar 

  21. J. Bourgain, S. Dirksen, J. Nelson, Toward a unified theory of sparse dimensionality reduction in euclidean space. Geom. Funct. Anal. 25(4), 1009–1088 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  22. B. Brinkman, M. Charikar, On the impossibility of dimension reduction in l 1. J. ACM 52(5), 766–788 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  23. E. Candès, The restricted isometry property and its implications for compressed sensing. C. R. Acad. Sci. I 346(9–10), 589–592 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  24. E.J. Candes, J.K. Romberg, T. Tao, Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 59(8), 1207–1223 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  25. S. Dasgupta, A. Gupta, An elementary proof of a theorem of Johnson and Lindenstrauss. Random Struct. Algoritm. 22(1), 60–65 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  26. M. Datar, N. Immorlica, P. Indyk, V.S. Mirrokni, Locality-sensitive hashing scheme based on p-stable distributions, in Proceedings of the Twentieth Annual Symposium on Computational Geometry (ACM, New York, 2004), pp. 253–262

    Book  Google Scholar 

  27. M.A. Davenport, P.T. Boufounos, M.B. Wakin, R.G. Baraniuk, Signal processing with compressive measurements. IEEE J. Sel. Top. Sign. Proces. 4(2), 445–460 (2010). DOI:10.1109/JSTSP.2009.2039178. http://dx.doi.org/10.1109/JSTSP.2009.2039178

    Article  Google Scholar 

  28. S. Dirksen, Dimensionality reduction with subgaussian matrices: a unified theory. Found. Comput. Math. 16(5), 1367–1396

    Google Scholar 

  29. Y. Eldar, M. Mishali, Robust recovery of signals from a structured union of subspaces. IEEE Trans. Inf. Theory 55(11), 5302–5316 (2009)

    Article  MathSciNet  Google Scholar 

  30. J. Haupt, R. Nowak, A generalized restricted isometry property. Tech. rep., University of Wisconsin-Madison (2007)

    Google Scholar 

  31. C. Hegde, A. Sankaranarayanan, W. Yin, R. Baraniuk, NuMax: a convex approach for learning near-isometric linear embeddings. IEEE Trans. Signal Process. 63(22), 6109–6121 (2015). DOI:10.1109/TSP.2015.2452228

    Article  MathSciNet  Google Scholar 

  32. P. Indyk, Stable distributions, pseudorandom generators, embeddings, and data stream computation. J. ACM 53(3), 307–323 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  33. P. Indyk, R. Motwani, Approximate nearest neighbors: towards removing the curse of dimensionality, in ACM Symposium on Theory of computing (1998), pp. 604–613

    Google Scholar 

  34. L. Jacques, A quantized Johnson-Lindenstrauss lemma: the finding of buffon’s needle. IEEE Trans. Inf. Theory 61(9), 5012–5027 (2015). DOI:10.1109/TIT.2015.2453355

    Article  MathSciNet  MATH  Google Scholar 

  35. L. Jacques, D.K. Hammond, J.M. Fadili, Dequantizing compressed sensing: when oversampling and non-gaussian constraints combine. IEEE Trans. Inf. Theory 57(1), 559–571 (2011)

    Article  MathSciNet  Google Scholar 

  36. L. Jacques, D.K. Hammond, J.M. Fadili, Stabilizing nonuniformly quantized compressed sensing with scalar companders. IEEE Trans. Inf. Theory 59(12), 7969–7984 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  37. L. Jacques, J.N. Laska, P.T. Boufounos, R.G. Baraniuk, Robust 1-bit compressive sensing via binary stable embeddings of sparse vectors. IEEE Trans. Inf. Theory 59(4) (2013). DOI:10.1109/TIT.2012.2234823. http://dx.doi.org/10.1109/TIT.2012.2234823

  38. T. Jayram, D.P. Woodruff, Optimal bounds for Johnson-Lindenstrauss transforms and streaming problems with subconstant error. ACM Trans. Algoritm. 9(3), 26 (2013)

    Google Scholar 

  39. A. Jimenez, B. Raj, J. Portelo, I. Trancoso, Secure modular hashing, in IEEE International Workshop on Information Forensics and Security (IEEE, Piscataway, 2015), pp. 1–6

    Google Scholar 

  40. W. Johnson, J. Lindenstrauss, Extensions of Lipschitz mappings into a Hilbert space. Contemp. Math. 26, 189–206 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  41. F. Krahmer, R. Ward, New and improved Johnson-Lindenstrauss embeddings via the restricted isometry property. SIAM J. Math. Anal. 43(3), 1269–1281 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  42. V.I. Levenshtein, Binary codes capable of correcting deletions, insertions, and reversals. Sov. Phys. Dokl. 10(8), 707–710 (1966)

    MathSciNet  MATH  Google Scholar 

  43. M. Li, S. Rane, P.T. Boufounos, Quantized embeddings of scale-invariant image features for mobile augmented reality, in Processing of the IEEE International Workshop on Multimedia Signal Processing, Banff, Canada (2012)

    Google Scholar 

  44. R. Ostrovsky, Y. Rabani, Low distortion embeddings for edit distance. J. ACM 54(5), 23 (2007)

    Google Scholar 

  45. D. Otero, G.R. Arce, Generalized restricted isometry property for alpha-stable random projections, in IEEE International Conference on Acoustics, Speech and Signal Processing (2011), pp. 3676–3679

    Google Scholar 

  46. S. Oymak, B. Recht, Near-optimal bounds for binary embeddings of arbitrary sets (2015), arXiv preprint arXiv:1512.04433

    Google Scholar 

  47. Y. Plan, R. Vershynin, One-bit compressed sensing by linear programming. Commun. Pure Appl. Math. 66(8), 1275–1297 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  48. Y. Plan, R. Vershynin, Robust 1-bit compressed sensing and sparse logistic regression: a convex programming approach. IEEE Trans. Inf. Theory 59(1), 482–494 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  49. Y. Plan, R. Vershynin, Dimension reduction by random hyperplane tessellations. Discret. Comput. Geom. 51(2), 438–461 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  50. G. Puy, M. Davies, R. Gribonval, Recipes for stable linear embeddings from Hilbert spaces to \(\mathbb{R}^{m}\) (2015), arXiv preprint arXiv:1509.06947

    Google Scholar 

  51. A. Rahimi, B. Recht, Random features for large-scale kernel machines, in Advances in Neural Information Processing Systems (2007), pp. 1177–1184

    Google Scholar 

  52. S. Rane, P.T. Boufounos, Privacy-preserving nearest neighbor methods: comparing signals without revealing them, in IEEE Signal Processing Magazine (2013). DOI:10.1109/MSP.2012.2230221, http://dx.doi.org/10.1109/MSP.2012.2230221

    Google Scholar 

  53. S. Rane, P.T. Boufounos, A. Vetro, Quantized embeddings: an efficient and universal nearest neighbor method for cloud-based image retrieval, in Proceedings of SPIE Applications of Digital Image Processing XXXVI, San Diego, CA (2013)

    Google Scholar 

  54. A. Sadeghian, B. Bah, V. Cevher, Energy-aware adaptive bi-Lipschitz embeddings, in Proceedings of International Conference on Sampling Theory and Applications, Bremen, Germany (2013)

    Google Scholar 

  55. C. Strecha, A. Bronstein, M. Bronstein, P. Fua, LDAHash: improved matching with smaller descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 34(1), 66–78 (2012). DOI:10.1109/TPAMI.2011.103

    Article  Google Scholar 

  56. Y. Weiss, A. Torralba, R. Fergus, Spectral hashing, in Advances in Neural Information Processing Systems 21 (MIT, London, 2009), pp. 1753–1760

    Google Scholar 

  57. X. Yi, C. Caramanis, E. Price, Binary embedding: fundamental limits and fast algorithm, in Proceedings of the 32nd International Conference on Machine Learning, Lille, France, vol. 37 (2015), pp. 2162–2170

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Mitsubishi Electric Research Laboratories, 201 Broadway, Cambridge, MA, 02139, USA

    Petros T. Boufounos & Hassan Mansour

  2. Palo Alto Research Center, 3333 Coyote Hill Road, Palo Alto, CA, 94304, USA

    Shantanu Rane

Authors
  1. Petros T. Boufounos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shantanu Rane
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hassan Mansour
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Petros T. Boufounos .

Editor information

Editors and Affiliations

  1. Norbert Wiener Center, University of Maryland, College Park, Maryland, USA

    Radu Balan

  2. Norbert Wiener Center, University of Maryland, College Park, Maryland, USA

    John J. Benedetto

  3. Norbert Wiener Center, University of Maryland, College Park, Maryland, USA

    Wojciech Czaja

  4. Norbert Wiener Center, University of Maryland, College Park, Maryland, USA

    Matthew Dellatorre

  5. Norbert Wiener Center, University of Maryland, College Park, Maryland, USA

    Kasso A. Okoudjou

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Cite this chapter

Boufounos, P.T., Rane, S., Mansour, H. (2017). Embedding-Based Representation of Signal Geometry. In: Balan, R., Benedetto, J., Czaja, W., Dellatorre, M., Okoudjou, K. (eds) Excursions in Harmonic Analysis, Volume 5. Applied and Numerical Harmonic Analysis. Birkhäuser, Cham. https://doi.org/10.1007/978-3-319-54711-4_7

Download citation

Publish with us

Policies and ethics

Search

Navigation