Skip to main content
SpringerLink
Log in

Logarithmic divergences from optimal transport and Rényi geometry

  • Research Paper
  • Published:
Information Geometry Aims and scope Submit manuscript

Abstract

Divergences, also known as contrast functions, are distance-like quantities defined on manifolds of non-negative or probability measures. Using the duality in optimal transport, we introduce and study the one-parameter family of \(L^{(\pm \alpha )}\)-divergences. They extrapolate between the Bregman divergence corresponding to the Euclidean quadratic cost, and the L-divergence introduced by Pal and the author in connection with portfolio theory and a logarithmic cost function. They admit natural generalizations of exponential family that are closely related to the \(\alpha \)-family and q-exponential family. In particular, the \(L^{(\pm \alpha )}\)-divergences of the corresponding potential functions are Rényi divergences. Using this unified framework we prove that the induced geometries are dually projectively flat with constant sectional curvatures, and a generalized Pythagorean theorem holds true. Conversely, we show that if a statistical manifold is dually projectively flat with constant curvature \(\pm \alpha \) with \(\alpha > 0\), then it is locally induced by an \(L^{(\mp \alpha )}\)-divergence. We define in this context a canonical divergence which extends the one for dually flat manifolds.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. The author thanks an anonymous referee for pointing this out.

  2. The author thanks Shun-ichi Amari for pointing this out.

References

  1. Adams, S., Dirr, N., Peletier, M.A., Zimmer, J.: From a large-deviations principle to the Wasserstein gradient flow: a new micro-macro passage. Commun. Math. Phys. 307(3), 791 (2011)

    Article  MathSciNet  Google Scholar 

  2. Amari, S., Nagaoka, H.: Methods of Information Geometry. Translations of Mathematical Monographs, vol. 191. Am. Math. Soc./Oxford University Press, Providence/London (2000)

  3. Amari, S.I.: Information Geometry and Its Applications. Springer, Berlin (2016)

    Book  Google Scholar 

  4. Amari, S.I., Karakida, R., Oizumi, M.: Information geometry connecting Wasserstein distance and Kullback–Leibler divergence via the entropy-relaxed transportation problem (2018) (to appear in Information Geometry)

  5. Amari, S.I., Ohara, A.: Geometry of \(q\)-exponential family of probability distributions. Entropy 13(6), 1170–1185 (2011)

    Article  MathSciNet  Google Scholar 

  6. Ambrosio, L., Gigli, N.: A User’s Guide to Optimal Transport. In: Modelling and Optimisation of Flows on Networks. Lecture Notes in Mathematics, vol 2062. Springer, Berlin, Heidelberg (2013). https://link.springer.com/chapter/10.1007/978-3-642-32160-3_1

  7. Ambrosio, L., Gigli, N., Savaré, G.: Gradient Flows in Metric Spaces and in the Spaces of Probability Measures, Second Edition. Lectures in Mathematics ETH Zürich. Birkhäuser Verlag (2008). https://www.springer.com/us/book/9783764387211

  8. Ay, N., Amari, S.I.: A novel approach to canonical divergences within information geometry. Entropy 17(12), 8111–8129 (2015)

    Article  Google Scholar 

  9. Ay, N., Jost, J., Lê, H., Schwachhöfer, L.: Information Geometry. Springer, Berlin (2017)

    Book  Google Scholar 

  10. Banerjee, A., Merugu, S., Dhillon, I.S., Ghosh, J.: Clustering with bregman divergences. J. Mach. Learn. Res. 6(Oct), 1705–1749 (2005)

    MathSciNet  MATH  Google Scholar 

  11. Brenier, Y.: Polar factorization and monotone rearrangement of vector-valued functions. Commun. Pure Appl. Math. 44(4), 375–417 (1991)

    Article  MathSciNet  Google Scholar 

  12. Calin, O., Udrişte, C.: Geometric modeling in probability and statistics. Springer, Berlin (2014)

    Book  Google Scholar 

  13. Chen, Y., Li, W.: Natural gradient in Wasserstein statistical manifold (2018). arXiv preprint arXiv:1805.08380

  14. Chizat, L., Peyré, G., Schmitzer, B., Vialard, F.X.: Unbalanced optimal transport: geometry and kantorovich formulation (2015). arXiv preprint arXiv:1508.05216

  15. Dillen, F., Nomizu, K., Vranken, L.: Conjugate connections and radon’s theorem in affine differential geometry. Monatshefte für Mathematik 109(3), 221–235 (1990)

    Article  MathSciNet  Google Scholar 

  16. Eguchi, S.: Second order efficiency of minimum contrast estimators in a curved exponential family. Ann. Stat. 11(3), 793–803 (1983)

    Article  MathSciNet  Google Scholar 

  17. Eguchi, S.: Geometry of minimum contrast. Hiroshima Math. J. 22(3), 631–647 (1992)

    MathSciNet  MATH  Google Scholar 

  18. Erbar, M., Kuwada, K., Sturm, K.T.: On the equivalence of the entropic curvature–dimension condition and Bochner’s inequality on metric measure spaces. Invent. Math. 201(3), 993–1071 (2015)

    Article  MathSciNet  Google Scholar 

  19. Felice, D., Ay, N.: Towards a canonical divergence within information geometry (2018). arXiv preprint arXiv:1806.11363

  20. Jordan, R., Kinderlehrer, D., Otto, F.: The variational formulation of the Fokker–Planck equation. SIAM J. Math. Anal. 29(1), 1–17 (1998)

    Article  MathSciNet  Google Scholar 

  21. Kurose, T.: Dual connections and affine geometry. Mathematische Zeitschrift 203(1), 115–121 (1990)

    Article  MathSciNet  Google Scholar 

  22. Kurose, T.: On the divergences of 1-conformally flat statistical manifolds. Tohoku Math. J. Second Ser. 46(3), 427–433 (1994)

    Article  MathSciNet  Google Scholar 

  23. Léonard, C.: From the Schrödinger problem to the Monge–Kantorovich problem. J. Funct. Anal. 262(4), 1879–1920 (2012)

    Article  MathSciNet  Google Scholar 

  24. Li, W., Montufar, G.: Ricci curvature for parametric statistics via optimal transport (2018). arXiv preprint arXiv:1807.07095

  25. Matsuzoe, H.: Geometry of contrast functions and conformal geometry. Hiroshima Math. J. 29(1), 175–191 (1999)

    MathSciNet  MATH  Google Scholar 

  26. Moreau, J.J.: Inf-convolutions, sous-additive, convexite des fonctions numeriques. J. Math. Anal. Appl. 49, 109–154 (1970)

    MATH  Google Scholar 

  27. Nagaoka, H., Amari, S.I.: Differential geometry of smooth families of probability distributions. Tech. Rep. METR 82–7. University of Tokyo, Tokyo (1982)

    Google Scholar 

  28. Naudts, J.: The \(q\)-exponential family in statistical physics. In: Journal of Physics: Conference Series, vol. 201, p. 012003. IOP Publishing (2010). http://iopscience.iop.org/article/10.1088/1742-6596/201/1/012003/meta

    Google Scholar 

  29. Naudts, J., Zhang, J.: Rho-tau embedding and gauge freedom in information geometry (2018). https://doi.org/10.1007/s41884-018-0004-6

  30. Pal, S.: Embedding optimal transports in statistical manifolds. Indian J. Pure Appl. Math. 48(4), 541–550 (2017)

    Article  MathSciNet  Google Scholar 

  31. Pal, S., Wong, T.K.L.: Energy, entropy, and arbitrage (2013). arXiv preprint arXiv:1308.5376

  32. Pal, S., Wong, T.K.L.: The geometry of relative arbitrage. Math. Fin. Econ. 10, 263–293 (2016)

    Article  MathSciNet  Google Scholar 

  33. Pal, S., Wong, T.K.L.: Exponentially concave functions and a new information geometry. Ann. Probab. 46(2), 1070–1113 (2018)

    Article  MathSciNet  Google Scholar 

  34. Pal, S., Wong, T.K.L.: Multiplicative Schrödinger problem and the Dirichlet transport (2018). arXiv preprint arXiv:1807.05649

  35. Rachev, S.T., Rüschendorf, L.: Mass Transportation Problems: Volume I: Theory, vol. 1. Springer, Berlin (1998)

    MATH  Google Scholar 

  36. Rényi, A.: On measures of entropy and information. In: Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics. The Regents of the University of California (1961)

  37. Santambrogio, F.: Optimal Transport for Applied Mathematicians. Springer, Berlin (2015)

    Book  Google Scholar 

  38. Van Erven, T., Harremos, P.: Rényi divergence and Kullback–Leibler divergence. IEEE Trans. Inf. Theory 60(7), 3797–3820 (2014)

    Article  Google Scholar 

  39. Villani, C.: Topics in Optimal Transportation. American Mathematical Society, Rhode Island (2003)

    Book  Google Scholar 

  40. Villani, C.: Optimal Transport: Old and New. Springer, Berlin (2008)

    MATH  Google Scholar 

  41. Wong, T.K.L.: Optimization of relative arbitrage. Ann. Financ. 11(3–4), 345–382 (2015)

    Article  MathSciNet  Google Scholar 

  42. Wong, T.K.L.: On portfolios generated by optimal transport (2017). arXiv preprint arXiv:1709.03169

  43. Zhang, J.: Divergence function, duality, and convex analysis. Neural Comput. 16(1), 159–195 (2004)

    Article  MathSciNet  Google Scholar 

  44. Zhang, J.: Referential duality and representational duality on statistical manifolds. In: Proceedings of the Second International Symposium on Information Geometry and Its Applications, Tokyo, Japan, vol. 1216, p. 5867 (2005)

  45. Zhang, J.: Reference duality and representation duality in information geometry. In: AIP Conference Proceedings, vol. 1641, pp. 130–146. AIP (2015)

Download references

Acknowledgements

Some of the results were obtained when the author was teaching a PhD topics course on optimal transport and information geometry at the University of Southern California. He thanks his students for their interest and feedbacks. He also thanks John Man-shun Ma, Jeremy Toulisse and Soumik Pal for helpful discussions. He is grateful to Shun-ichi Amari for his insightful comments and for pointing out the connection with the \(\alpha \)-divergence. Finally, he thanks the anonymous referees for their helpful comments which improved the paper.

Author information

Authors and Affiliations

  1. University of Toronto, Toronto, Canada

    Ting-Kam Leonard Wong

Authors
  1. Ting-Kam Leonard Wong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ting-Kam Leonard Wong.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wong, TK.L. Logarithmic divergences from optimal transport and Rényi geometry. Info. Geo. 1, 39–78 (2018). https://doi.org/10.1007/s41884-018-0012-6

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41884-018-0012-6

Keywords

Search

Navigation