Abstract
Our goal is to solve the large-scale linear programming (LP) formulation of Optimal Transport (OT) problems efficiently. Our key observations are: (i) the primal solutions of the LP problems are theoretically very sparse; (ii) the cost function is usually equipped with good geometric properties. The former motivates us to eliminate the majority of the variables, while the latter easily enables us to exploit a hierarchical multiscale structure. Each level in this structure corresponds to a standard OT problem, whose solution can be obtained by solving a series of restricted OT problems by fixing most of the primal variables to zeros and using the semi-smooth Newton method. We improve the performance of computing the semi-smooth Newton direction by forming and solving a much smaller symmetric positive-definite system whose matrix can be written explicitly according to the sparsity patterns. Extensive numerical experiments show that our algorithm is quite efficient compared to the state-of-the-art methods such as a multiscale implementation of the CPLEX’s Networkflow algorithm and the SparseSinkhorn method, due to its ability to solve problems at a much larger scale and obtain the optimal solution in less time.
Similar content being viewed by others
References
Agueh, Martial, Carlier, Guillaume: Barycenters in the wasserstein space. SIAM J. Math. Anal. 43, 904–924 (2011)
Ahujia, R.K., Magnanti, Thomas L., Orlin, James B.: Network flows: Theory, algorithms and applications. Rentice-Hall, New Jersey (1993)
Bertsekas, Dimitri P.: The auction algorithm: A distributed relaxation method for the assignment problem. Ann. Op. Res. 14, 105–123 (1988)
Cuturi, Marco: Sinkhorn distances: Lightspeed computation of optimal transport. In: Advances in neural information processing systems, pp. 2292–2300 (2013)
Cuturi, Marco, Doucet, Arnaud: Fast computation of wasserstein barycenters (2014)
Franklin, Joel, Lorenz, Jens: On the scaling of multidimensional matrices. Linear Algebra Appl. 114, 717–735 (1989)
Fu, Anthony Y., Wenyin, Liu, Deng, Xiaotie: Detecting phishing web pages with visual similarity assessment based on earth mover’s distance (emd). IEEE Trans. Depend. Secure Comput. 3, 301–311 (2006)
Gerber, Samuel, Maggioni, Mauro: Multiscale strategies for computing optimal transport. J. Mach. Learn. Res. 18, 2440–2471 (2017)
Grauman, Kristen, Darrell, Trevor: Fast contour matching using approximate earth mover’s distance, in Computer Vision and Pattern Recognition, 2004. CVPR 2004. In: Proceedings of the 2004 IEEE Computer Society Conference on, vol. 1, pp. I–I. IEEE (2004)
Kendal, Dave, Hauser, Cindy E., Garrard, Georgia E., Jellinek, Sacha, Giljohann, Katherine M., Moore, Joslin L.: Quantifying plant colour and colour difference as perceived by humans using digital images. PLoS One 8, e72296 (2013)
Knight, Philip A.: The sinkhorn-knopp algorithm: convergence and applications. SIAM J. Matrix Anal. Appl. 30, 261–275 (2008)
Kosowsky, Jeffrey J., Yuille, Alan L.: The invisible hand algorithm: Solving the assignment problem with statistical physics. Neural Netw. 7, 477–490 (1994)
Kuhn, Harold W.: The hungarian method for the assignment problem. Naval Res. Log. (NRL) 2, 83–97 (1955)
Li, Yongfeng, Wen, Zaiwen, Yang, Chao, Yuan, Ya.-xiang: A semismooth newton method for semidefinite programs and its applications in electronic structure calculations. SIAM J. Sci. Comput. 40, A4131–A4157 (2018)
Liu, Jialin, Yin, Wotao, Li, Wuchen, Chow, Yat Tin: Multilevel optimal transport: a fast approximation of wasserstein-1 distances. SIAM J. Sci. Comput. 43, A193–A220 (2021)
Mérigot, Quentin: A multiscale approach to optimal transport. In: Computer Graphics Forum, vol. 30, pp. 1583–1592. Wiley Online Library (2011)
Rubner, Yossi, Tomasi, Carlo, Guibas, Leonidas J.: The earth mover’s distance as a metric for image retrieval. Int. J. Comput. Vision 40, 99–121 (2000)
Schmitzer, Bernhard: A sparse multiscale algorithm for dense optimal transport. J. Math. Imag. Vision 56, 238–259 (2016)
Schmitzer, Bernhard: Stabilized sparse scaling algorithms for entropy regularized transport problems. SIAM J. Sci. Comput. 41, A1443–A1481 (2019)
Schrieber, Jörn., Schuhmacher, Dominic, Gottschlich, Carsten: Dotmark-a benchmark for discrete optimal transport. IEEE Access 5, 271–282 (2017)
Sharify, Meisam, Gaubert, Stéphane, Grigori, Laura: Solution of the optimal assignment problem by diagonal scaling algorithms (2011). arXiv preprint arXiv:1104.3830
Tarjan, Robert E.: Dynamic trees as search trees via euler tours, applied to the network simplex algorithm. Math. Progr. 78, 169–177 (1997)
Villani, Cédric.: Optical transport: old and new, vol. 338. Springer Science & Business Media, Berlin (2008)
Acknowledgements
The authors are grateful to the AE and two anonymous referees for their valuable comments and suggestions. They would like to thank Yongfeng Li and Xiang Meng for the valuable discussion on the semi-smooth Newton method for the OT problem.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Z. Wen: Research supported in part by the NSFC grant 11831002.
Rights and permissions
About this article
Cite this article
Liu, Y., Wen, Z. & Yin, W. A Multiscale Semi-Smooth Newton Method for Optimal Transport. J Sci Comput 91, 39 (2022). https://doi.org/10.1007/s10915-022-01813-y
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10915-022-01813-y