Abstract
The accurate estimation of optical flow is a challenging task, which is often posed as an energy minimization problem. Most top-performing methods approach this using continuous optimization algorithms. In many cases, the employed models are assumed to be convex to ensure tractability of the optimization problem. This is in contrast to the related problem of narrow-baseline stereo matching, where the top-performing methods employ powerful discrete optimization algorithms such as graph cuts and message-passing to optimize highly non-convex energies.
In this chapter, we demonstrate how similar non-convex energies can be formulated and optimized in the context of optical flow estimation using a combination of discrete and continuous techniques. Starting with a set of candidate solutions that are produced by either fast continuous flow estimation algorithms or sparse feature matching, the proposed method iteratively fuses these candidate solutions by the computation of minimum cuts on graphs. The obtained continuous-valued result is then further improved using local gradient descent. Experimentally, we demonstrate that the proposed energy is an accurate model and that the proposed discrete-continuous optimization scheme not only finds lower energy solutions than traditional discrete or continuous optimization techniques, but also leads to very accurate flow estimates.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Horn, B.K.P., Schunck, B.G.: Determining optical flow. Artificial Intelligence 17(1-3), 185–203 (1981)
Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: IJCAI, April 1981, pp. 674–679 (1981)
Papenberg, N., Bruhn, A., Brox, T., Didas, S., Weickert, J.: Highly accurate optic flow computation with theoretically justified warping. IJCV 67(2), 141–158 (2006)
Black, M.J., Anandan, P.: The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. CVIU 63(1), 75–104 (1996)
Roth, S., Black, M.J.: On the spatial statistics of optical flow. IJCV 74(1), 33–50 (2007)
Baker, S., Scharstein, D., Lewis, J.P., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. In: ICCV 2007 (2007), http://vision.middlebury.edu/flow/
Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. TPAMI 23(11), 1222–1239 (2001)
Sun, J., Zhen, N.N., Shum, H.Y.: Stereo matching using belief propagation. PAMI 25(7), 787–800 (2003)
Meltzer, T., Yanover, C., Weiss, Y.: Globally optimal solutions for energy minimization in stereo vision using reweighted belief propagation. In: ICCV 2005, vol. 1, pp. 428–435 (2005)
Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient belief propagation for early vision. In: CVPR 2004, vol. 1, pp. 261–268 (2004)
Glocker, B., Komodakis, N., Paragios, N., Tziritas, G., Navab, N.: Inter and intra-modal deformable registration: Continuous deformations meet efficient optimal linear programming. In: Karssemeijer, N., Lelieveldt, B. (eds.) IPMI 2007. LNCS, vol. 4584, pp. 408–420. Springer, Heidelberg (2007)
Lempitsky, V., Rother, C., Blake, A.: LogCut - Efficient graph cut optimization for Markov random fields. In: ICCV 2007 (2007)
Shekhovtsov, A., Kovtun, I., Hlavac, V.: Efficient MRF deformation model for non-rigid image matching. In: CVPR 2007 (2007)
Lempitsky, V., Roth, S., Rother, C.: FusionFlow: Discrete-continuous optimization for optical flow estimation. In: CVPR 2008 (2008)
Glocker, B., Paragios, N., Komodakis, N., Tziritas, G., Navab, N.: Optical flow estimation with uncertainties through dynamic MRFs. In: CVPR 2008 (2008)
Bruhn, A., Weickert, J., Schnörr, C.: Lucas/Kanade meets Horn/Schunck: Combining local and global optic flow methods. IJCV 61(3), 211–231 (2005)
Boros, E., Hammer, P.L., Tavares, G.: Preprocessing of unconstrained quadratic binary optimization. Technical Report RUTCOR RRR (2006)
Boros, E., Hammer, P.L.: Pseudo-boolean optimization. Discrete Applied Mathematics 123(1-3), 155–225 (2002)
Kolmogorov, V., Rother, C.: Minimizing non-submodular functions with graph cuts — A review. TPAMI 29(7), 1274–1279 (2006)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. IJCV 60(2), 91–110 (2004)
Liu, C., Yuen, J., Torralba, A.B., Sivic, J., Freeman, W.T.: SIFT flow: Dense correspondence across different scenes. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 28–42. Springer, Heidelberg (2008)
Brox, T., Bruhn, A., Weickert, J.: Variational motion segmentation with level sets. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 471–483. Springer, Heidelberg (2006)
Mémin, É., Pérez, P.: Hierarchical estimation and segmentation of dense motion fields. IJCV 46(2), 129–155 (2002)
Heitz, F., Bouthemy, P.: Multimodal estimation of discontinuous optical flow using Markov random fields. TPAMI 15(12), 1217–1232 (1993)
Konrad, J., Dubois, E.: Multigrid Bayesian estimation of image motion fields using stochastic relaxation. In: ICCV 1988, pp. 354–362 (1988)
Lempitsky, V., Rother, C., Roth, S., Blake, A.: Fusion moves for Markov random field optimization. TPAMI (in revision)
Woodford, O.J., Torr, P.H.S., Reid, I.D., Fitzgibbon, A.W.: Global stereo reconstruction under second order smoothness priors. In: CVPR 2008 (2008)
Trobin, W., Pock, T., Cremers, D., Bischof, H.: Continuous energy minimization via repeated binary fusion. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 677–690. Springer, Heidelberg (2008)
Birchfield, S., Natarjan, B., Tomasi, C.: Correspondence as energy-based segmentation. Image Vision Comp. 25(8), 1329–1340 (2007)
Sun, D., Roth, S., Lewis, J.P., Black, M.J.: Learning optical flow. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 83–97. Springer, Heidelberg (2008)
Li, Y., Huttenlocher, D.P.: Learning for optical flow using stochastic optimization. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 379–391. Springer, Heidelberg (2008)
Greig, D.M., Porteous, B.T., Seheult, A.H.: Exact MAP estimation for binary images. J. Roy. Stat. Soc. B 51(2), 271–279 (1989)
Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. TPAMI 26(9), 1124–1137 (2004)
Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts? TPAMI 24(2), 147–159 (2004)
Rother, C., Kolmogorov, V., Lempitsky, V., Szummer, M.: Optimizing binary MRFs via extended roof duality. In: CVPR 2007 (2007)
Ishikawa, H.: Exact optimization for Markov random fields with convex priors. TPAMI 25(10), 1333–1336 (2003)
Rasmussen, C.E.: minimize.m (September 2006), http://www.kyb.tuebingen.mpg.de/bs/people/carl/code/minimize/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Roth, S., Lempitsky, V., Rother, C. (2009). Discrete-Continuous Optimization for Optical Flow Estimation. In: Cremers, D., Rosenhahn, B., Yuille, A.L., Schmidt, F.R. (eds) Statistical and Geometrical Approaches to Visual Motion Analysis. Lecture Notes in Computer Science, vol 5604. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03061-1_1
Download citation
DOI: https://doi.org/10.1007/978-3-642-03061-1_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-03060-4
Online ISBN: 978-3-642-03061-1
eBook Packages: Computer ScienceComputer Science (R0)