Skip to main content
SpringerLink
Log in

A Fast Fourier based Feature Descriptor and a Cascade Nearest Neighbour Search with an Efficient Matching Pipeline for Mosaicing of Microscopy Images

  • Applied Problems
  • Published:
Pattern Recognition and Image Analysis Aims and scope Submit manuscript

Abstract

Automatic mosaicing is an important image processing application and we propose several improvements and simplifications to the image registration pipeline used in microscopy to automatically construct large images of whole specimen samples from a series of images. First of all we propose a feature descriptor based on the amplitude of a few elements of the Fourier transform, which makes it fast to compute and that can be used for any image matching and registration applications where scale and rotation invariance is not needed. Secondly, we propose a cascade matching approach that will reduce the time for the nearest neighbour search considerably, making it almost independent on feature vector length. Moreover, several improvements are proposed that will speed up the whole matching process. These are: faster interest point detection, a regular sampling strategy and a deterministic false positive removal procedure that finds the transformation. All steps of the improved pipeline are explained and the results comparative experiments are presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. L. G. Brown, “A survey of image registration techniques,” ACM Comput. Surv. 24 (4), 325–376 (1992).

    Article  Google Scholar 

  2. B. Zitová and J. Flusser, “Image registration methods: A survey,” Image Vision Comput. 21, 977–1000 (2003).

    Article  Google Scholar 

  3. A. Elibol, J. Kim, J., N. Gracias, and R. Garcia, “Efficient image mosaicing for multi-robot visual underwater mapping,” Pattern Recogn. Lett. 46, 20–26 (2014).

    Article  Google Scholar 

  4. D. Capel, Image Mosaicing and Super-Resolution, Distinguished Dissertations (Springer, London, 2004).

    Book  MATH  Google Scholar 

  5. K. Nasrollahi and T. B. Moeslund, “Super-resolution: A comprehensive survey,” Mach. Vision Appl. 25 (6), 1423–1468 (2014).

    Article  Google Scholar 

  6. Y. Tian and K.-H. Yap, “Joint image registration and super-resolution from low-resolution images with zooming motion,” IEEE Trans. Circuits Syst. Video Technol. 23 (7), 1224–1234 (2013).

    Article  Google Scholar 

  7. Y. He, K.-H. Yap, L. Chen, and L.-P. Chau, “A nonlinear least square technique for simultaneous image registration and super-resolution,” IEEE Trans. Image Process. 16 (11), 2830–2841 (2007).

    Article  MathSciNet  Google Scholar 

  8. Y. Keller and A. Averbuch, “A projection-based extension to phase correlation image alignment,” Signal Process. 87 (1), 124–133 (2007).

    Article  MATH  Google Scholar 

  9. C. D. Kuglin and D. C. Hines, “The phase correlation image alignment method,” in Proc. 1975 IEEE Int. Conf. on Cybernetics and Society (San Francisco, CA, USA, 1975), pp. 163–165.

    Google Scholar 

  10. E. De Castro and C. Morandi, “Registration of translated and rotated images using finite fourier transforms,” IEEE Trans. Pattern Anal. Mach. Intell. PAMI-9 (5), 700–703 (1987).

    Article  Google Scholar 

  11. J. N. Sarvaiya, S. Patnaik, and K. Kothari, “Image registration using Log polar transform and phase correlation to recover higher scale,” J. Pattern Recogn. Res. 7 (1), 90–105 (2012).

    Article  Google Scholar 

  12. G. Wolberg and S. Zokai, “Robust image registration using log-polar transform,” in Proc. 2000 IEEE Int. Conf. on Image Processing (Vancouver, Canada, 2000), Vol. 1., pp. 493–496.

    Google Scholar 

  13. B. S. Reddy and B. N. Chatterji, “An FFT-based technique for translation, rotation, and scale-invariant image registration,” IEEE Trans. Image Process. 5 (8), 1266–1271 (1996).

    Article  Google Scholar 

  14. J. Shi and C. Tomasi, “Good features to track,” in Proc. 1994 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition (CVPR’94) (Seattle, WA, USA, 1994), pp. 593–600.

    Google Scholar 

  15. M. Brown and D. G. Lowe, “Automatic panoramic image stitching using invariant features,” Int. J. Comput. Vision 74 (1), 59–73 (2007).

    Article  Google Scholar 

  16. R. Szeliski, “Image alignment and stitching: A tutorial,” Found. Trends Comput. Graphics Vision 2 (1), 1–104 (2006).

    Article  MathSciNet  MATH  Google Scholar 

  17. H. Durrant-Whyte and T. Bailey, “Simultaneous localization and mapping (SLAM): Part I,” IEEE Rob. Autom. Mag. 13 (2), 99–110 (2006).

    Article  Google Scholar 

  18. H. Durrant-Whyte and T. Bailey, “Simultaneous localization and mapping (SLAM): Part II,” IEEE Rob. Autom. Mag. 13 (3), 108–117 (2006).

    Article  Google Scholar 

  19. W.-Y. Hsu, W.-F. P. Poon, and Y.-N. Sun, “Automatic seamless mosaicing of microscopic images: Enhancing appearance with colour degradation compensation and wavelet-based blending,” J. Microsc. 231 (3), 408–418 (2008).

    Article  MathSciNet  Google Scholar 

  20. C. Sun, R. Beare, R., V. Hilsenstein, and P. Jackway, “Mosaicing of microscope images with global geometric and radiometric corrections,” J. Microsc. 224 (2), 158–165 (2006).

    Article  MathSciNet  Google Scholar 

  21. D. E. Romo, J. Tarquino, J. D. García-Arteaga, and E. Romero, “Virtual slide mosaicing using feature descriptors and a registration consistency measure,” in IX Int. Seminar on Medical Information Processing and Analysis, Ed. by J. Brieva and B. Escalante-Ramírez, Proc. SPIE 8922, 89220Q-1–89220Q-8 (2013).

    Google Scholar 

  22. H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” Comput. Vision Image Understand. 110 (3), 346–359 (2008).

    Article  Google Scholar 

  23. W. Rong, H. Chen, J. Liu, Y. Xu, and R. Haeusler, “Mosaicing of microscope images based on surf,” in Proc. 2009 24th Image and Vision Computing New Zealand (IVCNZ 2009) (IEEE, 2009), p. 271–275.

    Google Scholar 

  24. Y. Liang, Q. Li, Z. Lin, and D. Chen, “A panoramic image registration algorithm based on SURF,” in Recent Advances in Computer Science and Information Engineering, Ed. by Z. Qian, L. Cao, W. Su, T. Wang, and H. Yang, Lecture Notes in Electrical Engineering, (Springer, Berlin, Heidelberg, 2012), Vol. 128, pp. 473–478.

    Google Scholar 

  25. P. V. Lukashevich, B. A. Zalesky, and S. V. Ablameyko, “Medical image registration based on SURF detector,” Pattern Recogn. Image Anal. 21 (3), 519–521 (2011).

    Article  Google Scholar 

  26. D. G. Lowe, “Distinctive image features from scaleinvariant key points,” Int. J. Comput. Vision 60 (2), 91–110 (2004).

    Article  Google Scholar 

  27. S. G. Stanciu, R. Hristu, R. Boriga, and G. A. Stanciu, “On the suitability of SIFT technique to deal with image modifications specific to confocal scanning laser microscopy,” Microsc. Microanal. 16 (5), 515–530 (2010).

    Article  Google Scholar 

  28. C. Tang, Y. Dong, and X. Su, “Automatic registration based on improved SIFT for medical microscopic sequence images,” in Proc. 2008 Second Int. Symp. on Intelligent Information Technology Application (IITA’08) (IEEE, 2008), Vol. 1, pp. 580–583.

    Article  Google Scholar 

  29. T. Botterill, S. Mills, and R. Green, “Real-time aerial image mosaicing,” in Proc. 2010 5th Int. Conf. of Image and Vision Computing New Zealand (Queenstown, NZ, 2010), pp. 1–8.

    Google Scholar 

  30. C. Harris and M. Stephens, “A combined corner and edge detector,” in Proc. Fourth Alvey Vision Conference (Manchester, 1988), pp. 147–151.

    Google Scholar 

  31. C. Schmid, R. Mohr, and C. Bauckhage, “Evaluation of interest point detectors,” Int. J. Comput. Vision 37 (2), 151–172 (2000).

    Article  MATH  Google Scholar 

  32. T. Tuytelaars and K. Mikolajczyk, “Local invariant feature detectors: A survey,” Found. Trends Comput. Graphics Vision 3 (3), 177–280 (2008).

    Article  Google Scholar 

  33. S. Gauglitz, T. Höllerer, and M. Turk, “Evaluation of interest point detectors and feature descriptors for visual tracking,” Int. J. Comput. Vision 94 (3), 335–360 (2011).

    Article  MATH  Google Scholar 

  34. M. Zuliani, C. Kenney, and B. S. Manjunath, “A mathematical comparison of point detectors,” in Proc. 2004 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition Workshops (CVPRW’04), Vol. 11, Second IEEE Workshop on Image and Video Registration (IVR) (Washington, DC, USA, 2004), p. 172; conference paper (7 pages) available at http://vision.ece.ucsb.edu/abstract/365

    Google Scholar 

  35. E. Rosten and T. Drummond, “Machine learning for high-speed corner detection,” in Computer Vision — ECCV 2006, Proc. 9th European Conf. on Computer Vision, Part I, Ed. by A. Leonardis, H. Bischof, and A. Pinz, Lecture Notes in Computer Science (Springer, Berlin, Heidelberg, 2006), Vol. 3951, pp. 430–443.

    Article  Google Scholar 

  36. S. M. Smith and J. M. Brady, “SUSAN—a new approach to low level image processing,” Int. J. Comput. Vision 23 (1), 45–78 (1997).

    Article  Google Scholar 

  37. J. Matas, O. Chum, M. Urban, and T. Pajdla, “Robust wide baseline stereo from maximally stable extremal regions,” in Proc. British Machine Vision Conf. (BMVC 2002), Ed. by D. Marshall and P. L. Rosin (BMVA Press, 2002), pp. 36.1–36.10 (384–393). DOI: 10.5244/C.16.3610.5244/C.16.36

    Google Scholar 

  38. P. F. Alcantarilla, A. Bartoli, and A. J. Davison, “KAZE features,” in Computer Vision — ECCV 2012, Proc. 12th European Conf. on Computer Vision, Part VI, Ed. by A. Fitzgibbon, S. Lazebnik, et al., Lecture Notes in Computer Science (Springer, Berlin, Heidelberg, 2012), Vol. 7577, pp. 214–227.

    Article  Google Scholar 

  39. P. F. Alcantarilla, J. Nuevo, and A. Bartoli, “Fast explicit diffusion for accelerated features in nonlinear scale spaces,” in Proc. British Machine Vision Conf.(BMVC 2013), Ed. by T. Burghardt, D. Damen, et al. (BMVA Press, 2013), pp. 13.1–13.11. DOI: doi 10.5244/C.27.1310.5244/C.27.13

    Google Scholar 

  40. N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in Proc. 2005 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition (CVPR’05) (San Diego, CA, USA, 2005), Vol. 1 (IEEE Computer Society, 2005), pp. 886–893.

    Article  Google Scholar 

  41. S. Leutenegger, M. Chli, and R. Y. Siegwart, “BRISK: Binary robust invariant scalable keypoints,” in Proc. 2011 IEEE International Conference on Computer Vision (ICCV’ 11) (Barcelona, Spain, 2011) (IEEE Computer Society, 2011), pp. 2548–2555

    Chapter  Google Scholar 

  42. A. Alahi, R. Ortiz, and P. Vandergheynst, “FREAK: Fast retina keypoint,” in Proc. 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (Providence, RI, USA, 2012), pp. 510–517.

    Chapter  Google Scholar 

  43. G. Carneiro and A. D. Jepson, “Phase-based local features,” in Computer Vision — ECCV 2002, Proc. 7th European Conf. on Computer Vision, Part. I, Ed. by A. Heyden, G. Sparr, M. Nielsen, and P. Johansen, Lecture Notes in Computer Science (Springer, Berlin, Heidelberg, 2002), Vol. 2350. P. 282–296.

    Article  MATH  Google Scholar 

  44. G. Carneiro and A. D. Jepson, “Multi-scale phasebased local features,” in Proc. 2003 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition (CVPR 2003) (Madison, WI, USA, 2003), Vol. 1 (IEEE Computer Society, 2003), pp. I-736–I-743.

  45. I. Ulusoy and E. R. Hancock, “A statistical approach to sparse multi-scale phase-based stereo,” Pattern Recogn. 40 (9), 2504–2520 (2007).

    Article  MATH  Google Scholar 

  46. A. Hast, “Robust and invariant phase based local feature matching,” in Proc. 22nd Int. Conf. on Pattern Recognition (ICPR 2014) (Stockholm, Sweden, 2014), pp. 809–814.

    Google Scholar 

  47. A. Hast and A. Marchetti, “Rotation invariant feature matching-based on Gaussian filtered log polar transform and phase correlation,” in Proc. 8th Int. Symp. on Image and Signal Processing and Analysis (ISPA 2013) (Trieste, Italy, 2013), pp. 107–112.

    Google Scholar 

  48. A. Andoni, Nearest Neighbor Search: the Old, the New, and the Impossible, Ph.D. thesis, (Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, MIT, 2009).

    Google Scholar 

  49. M. Muja and D. G. Lowe, “Fast matching of binary features,” in Proc. 2012 Ninth Conf. on Computer and Robot Vision (CRV 2012) (Toronto, Canada, 2012), pp. 404–410.

    Chapter  Google Scholar 

  50. A. Nakhmani and A. Tannenbaum, “A new distance measure based on generalized image normalized crosscorrelation for robust video tracking and image recognition,” Pattern Recogn. Lett. 34 (3), 315–321 (2013).`

    Article  Google Scholar 

  51. S.-H. Cha, “Comprehensive survey on distance/similarity measures between probability density functions,” Int. J. Math. Models Methods Appl. Sci. 1 (4), 300–307 (2007).

    Google Scholar 

  52. J. H. Friedman, J. L. Bentley, and R. A. Finkel, “An algorithm for finding best matches in logarithmic expected time,” ACM Trans. Math. Software 3 (3), 209–226 (1977).

    Article  MATH  Google Scholar 

  53. K. Fukunage and P. M. Narendra, “A branch and bound algorithm for computing k-nearest neighbors,” IEEE Trans. Comput., C-24 (7), 750–753 (1975).

    Google Scholar 

  54. M. Muja and D. G. Lowe, “Fast approximate nearest neighbors with automatic algorithm configuration,” in Proc. Fourth Int. Conf. on Computer Vision Theory and Applications (VISAPP 2009) (Lisboa, Portugal, 2009), Vol. 1, pp. 331–340.

    Google Scholar 

  55. M. A. Fischler and R. C. Bolles, “Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24 (6), 381–395 (1981).

    Article  MathSciNet  Google Scholar 

  56. R.I. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed. (Cambridge University Press, 2004).

    Book  MATH  Google Scholar 

  57. O. Chum, J. Matas, and J. Kittler, “Locally optimized RANSAC,” in Pattern Recognition, Proc. 25th DAGM Symposium (Magdeburg, Germany, 2003), Ed. by B. Michaelis and G. Krell, Lecture Notes in Computer Science (Springer, Berlin, Heidelberg, 2003), Vol. 2781, pp. 236–243.

    Google Scholar 

  58. R. Raguram, J.-M. Frahm, and M. Pollefeys, “A comparative analysis of RANSAC techniques leading to adaptive real-time random sample consensus,” in Computer Vision — ECCV 2008, Proc. 10th European Conf. on Computer Vision, Part. II, Ed. by D. Forsyth, P. Torr, and A. Zisserman, Lecture Notes in Computer Science (Springer, Berlin, Heidelberg, 2008), Vol. 5303, pp. 500–513.

    Article  Google Scholar 

  59. A. Hast, J. Nysjö, and A, Marchetti, “Optimal RANSAC − towards a repeatable algorithm for finding the optimal set,” J. WSCG 21 (1), 21–30 (2013).

    Google Scholar 

  60. A. Hast, V. A. Sablina, G. Kylberg, and I.-M. Sintorn, “A simple and efficient feature descriptor for fast matching,” in WSCG 2015, 23rd Int. Conf. in Central Europe on Computer Graphics, Visualization and Computer Vision (Plzen, Czech Republic, 2015), Full Papers Proceedings, pp. 135–142.

    Google Scholar 

  61. R. L. Burden and J. D. Faires, Numerical Analysis, 7th ed. (Brooks/Cole, Thomson Learning, Pacific Grove, 2001).

    MATH  Google Scholar 

  62. T. Barrera, A. Hast, and E. Bengtsson, “Incremental spherical linear interpolation,” in SIGRAD 2004, Proc. Annual SIGRAD Conf. — Environmental Visualization (Gävle, Sweden, 2004), pp. 7–10.

    Google Scholar 

  63. A. Hast and A. Marchetti, “Invariant interest point detection based on variations of the spinor tensor,” in WSCG 2014, 22nd Int. Conf. in Central Europe on Computer Graphics, Visualization and Computer Vision (Plzen, Czech Republic, 2014), Communication Papers Proceedings, pp. 49–56.

    Google Scholar 

  64. P. Viola and M. J. Jones, “Robust real-time face detection,” Int. J. Comput. Vision 57 (2), 137–154 (2004).

    Article  Google Scholar 

  65. A. Hast and G. Kylberg, “Clustering in 2D as a fast deterministic alternative to RANSAC,” in FEAST 2015: 2nd Workshop on Features and Structures, collocated with ICML 2015 (Lille, France, 2015), 1 p.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information Technology, Uppsala University, Uppsala, Sweden

    A. Hast & I.-M. Sintorn

  2. Department of Electronic Computers, Ryazan State Radio Engineering University, Ryazan, Russia

    V. A. Sablina

  3. Vironova AB, Stockholm, Sweden

    I.-M. Sintorn & G. Kylberg

Authors
  1. A. Hast
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. V. A. Sablina
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. I.-M. Sintorn
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. G. Kylberg
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to A. Hast.

Additional information

The article is published in the original.

Anders Hast was born in 1966. In 1996 he graduated from the University of Gävle and obtained the Bachelor of Science in Computer Science degree. In 2004 he defended the PhD thesis on topic “Improved Algorithms for Fast Shading and Lighting” in the Centre for Image Analysis at the Uppsala University. Besides computer graphics and mathematics, also parallel programming and visualisation were important parts of his PhD studies. For nine years he was an application expert in scientific visualisation at UPPMAX. In 2011 he spent one year at IIT, CNR, Pisa in Italy as an ERCIM fellow and after that he received a full time position as associate professor at Uppsala University (e-mail: anders.hast@it.uu.se). Since then the research has focused on computer vision and image processing, especially for cultural heritage applications. He published 15 journal papers and 18 short papers, took park in 29 conferences and wrote 11 book chapters. He is a member of the Swedish Society for automated image analysis, the International Association for Pattern Recognition and also a member of Eurographics.

Victoria Alexandrovna Sablina was born in 1983. In 2006 she graduated with honors from the Ryazan State Radio Engineering Academy. In 2009 she defended the thesis on topic “The Development and the Investigation of Image Restoration Algorithms by the Sequency Theory Methods” in specialty “System Analysis, Control and Information Processing (in Engineering Systems)” and obtained the degree of a Candidate of Engineering Sciences. At present, she works at the Electronic computers department of the Ryazan State Radio Engineering University as an Associate Professor (e-mail: sablina.v.a@evm.rsreu.ru). The area of her scientific interests includes computer vision systems, mathematical image processing methods, threevalued logic. After defending her thesis she continued to research image processing and analysis algorithms by the sequency analysis methods. At present, she does research in image matching and image superimposition in computer vision systems by the multiple view geometry methods. In all 69 scientific works were published and of them there are 56 papers (2 papers in the international peer-review journals and 19 papers in international conference proceedings) and 1 monograph “Image Processing in the Aviation Computer Vision Systems”. She is a member of the International Society for Optics and Photonics (SPIE).

Ida-Maria Sintorn was born in 1976. In 2000 she graduated from the Uppsala University and obtained the MSc degree in Molecular Biotechnology Engineering. In 2005 she also obtained the PhD degree in computerized image analysis and remote sensing at the Swedish University of Agricultural Science. Since 2012 Ida-Maria has a Docent degree from Uppsala University. She works at the Division of Visual Information and Interaction of the Uppsala University as an Associate Professor (e-mail: ida.sintorn@it.uu.se). Her fields of research are segmentation, texture analysis, automated electron and fluorescence microscopy. She published 16 journal papers and took part in 23 conferences. She is a member of the Swedish Society for Automated Image Analysis and the International Association for Pattern Recognition.

Bengt Gustaf Kylberg was born in 1983. In 2008 he obtained the MSc degree at the Uppsala University. In 2014 he also obtained the PhD degree in at the Centre for Image Analysis at the Uppsala University. At present, he works at the Vironova AB company in Stockholm at the position of the Technical Product Owner (e-mail: gustaf.kylberg@vironova. com). His fields of research are automation within electron microscopy, object segmentation, description and classification. He has 12 scientific publications. He is a member of the Swedish Society for Automated Image Analysis and the International Association for Pattern Recognition.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hast, A., Sablina, V.A., Sintorn, IM. et al. A Fast Fourier based Feature Descriptor and a Cascade Nearest Neighbour Search with an Efficient Matching Pipeline for Mosaicing of Microscopy Images. Pattern Recognit. Image Anal. 28, 261–272 (2018). https://doi.org/10.1134/S1054661818020050

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S1054661818020050

Keywords

Search

Navigation