Abstract
Since the Lucas-Kanade algorithm was proposed in 1981 image alignment has become one of the most widely used techniques in computer vision. Applications range from optical flow and tracking to layered motion, mosaic construction, and face coding. Numerous algorithms have been proposed and a wide variety of extensions have been made to the original formulation. We present an overview of image alignment, describing most of the algorithms and their extensions in a consistent framework. We concentrate on the inverse compositional algorithm, an efficient algorithm that we recently proposed. We examine which of the extensions to Lucas-Kanade can be used with the inverse compositional algorithm without any significant loss of efficiency, and which cannot. In this paper, Part 1 in a series of papers, we cover the quantity approximated, the warp update rule, and the gradient descent approximation. In future papers, we will cover the choice of the error function, how to allow linear appearance variation, and how to impose priors on the parameters.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Baker, S. and Kanade, T. 2000. Limits on super-resolution and how to break them. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 372–379.
Baker, S. and Matthews, I. 2001. Equivalence and efficiency of image alignment algorithms. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 1090–1097.
Baker, S. and Matthews, I. 2002. Lucas-kande 20 years on: A unifying framework: Part 1. Technical Report CMU-RI-TR-02-16, Carnegie Mellon University Robotics Institute.
Bergen, J.R., Anandan, P., Hanna, K.J., and Hingorani, R. 1992. Hierarchical model-based motion estimation. In Proceedings of the European Conference on Computer Vision, pp. 237–252.
Black, M. and Jepson, A. 1998. Eigen-tracking: Robust matching and tracking of articulated objects using a view-based representation. International Journal of Computer Vision, 36(2):101–130.
LaCascia, M., Sclaroff, S., and Athitsos, V. 2000. Fast, reliable head tracking under varying illumination: An approach based on registration of texture-mapped 3D models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(6):322–336.
Christensen, G.E. and Johnson, H.J. 2001. Image consistent registration. IEEE Transactions on Medical Imaging, 20(7):568– 582.
Cootes, T.F., Edwards, G.J., and Taylor, C.J. 1998. Active appearance models. In Proceedings of the European Conference on Computer Vision, vol. 2, pp. 484–498.
Dellaert, F. and Collins, R. 1999. Fast image-based tracking by selective pixel integration. In Proceedings of the ICCVWorkshop on Frame-Rate Vision, pp. 1–22.
Gill, P.E., Murray, W., and Wright, M.H. 1986. Practical Optimization. Academic Press.
Gleicher, M. 1997. Projective registration with difference decomposition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 331–337.
Hager, G.D. and Belhumeur, P.N. 1998. Efficient region tracking with parametric models of geometry and illumination. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(10):1025– 1039.
Lucas, B. and Kanade, T. 1981. An iterative image registration technique with an application to stereo vision. In Proceedings of the International Joint Conference on Artificial Intelligence, pp. 674– 679.
Press, W.H., Flannery, B.P., Teukolsky, S.A., and Vetterling, W.T. 1992. Numerical Recipes in C: The Art of Scientific Computing, 2nd edn., Cambridge University Press.
Sclaroff, S. and Isidoro, J. 1998. Active blobs. In Proceedings of the 6th IEEE International Conference on Computer Vision, pp. 1146–1153.
Shum, H.-Y. and Szeliski, R. 2000. Construction of panoramic image mosaics with global and local alignment. International Journal of Computer Vision, 16(1):63–84.
Szeliski, R. and Golland, P. 1998. Stereo matching with transparency and matting. In Proceedings of the 6th IEEE International Conference on Computer Vision, pp. 517– 524.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Baker, S., Matthews, I. Lucas-Kanade 20 Years On: A Unifying Framework. International Journal of Computer Vision 56, 221–255 (2004). https://doi.org/10.1023/B:VISI.0000011205.11775.fd
Issue Date:
DOI: https://doi.org/10.1023/B:VISI.0000011205.11775.fd