Skip to main content
SpringerLink
Log in

Parallel hashing-based matching for real-time aerial image mosaicing

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

This paper presents a GPU-based real-time approach for generating high-definition (HD) aerial image mosaics. The cumbersome process of registering HD images is addressed by a parallel scheme that rapidly matches binary features. The proposed feature matcher takes advantage of the fast ORB (oriented FAST and rotated BRIEF) descriptor and its attainable arrangement into hash tables. By exploiting the best functionalities of binary descriptors and hashing-based data structures, the process of creating HD mosaics is accelerated. On average, real-time performance of 14.5 ms is achieved in a frame-to-frame process, for input images of 2.7 K resolution (2704 × 1521). For evaluation purposes in terms of robustness and speed, we selected two image registration methods for comparison. The first method uses the feature extractor and matcher modules of the well-known ORB-SLAM. The second comparison is carried out against the standard KNN-based matcher of OpenCV. The experiments were conducted under different conditions and scenarios, and the proposed approach exhibits a speed-up of 10.5 times compared to ORB-SLAM-based approach and 36.5 times compared to the OpenCV matcher. Therefore, this research widens the range of applications for aerial mosaicing, since the proposed system is capable of creating high-detail panoramas of large sites while acquiring data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Ham, Y., Han, K.K., Lin, J.J., Golparvar-Fard, M.: Visual monitoring of civil infrastructure systems via camera-equipped unmanned aerial vehicles (UAVs): a review of related works. Vis. Eng. 4(1), 1 (2016)

    Article  Google Scholar 

  2. Ezequiel, C.A.F., Cua, M., Libatique, N.C.,Tangonan, G.L., Alampay, R., Labuguen, R.T., Favila, C.M., Honrado, J.L.E., V. Canos, Devaney, C. et al.: UAV aerial imaging applications for post-disaster assessment, environmental management and infrastructure development. In: International Conference on Unmanned Aircraft Systems (ICUAS), pp. 274–283. IEEE (2014)

  3. Siebert, S., Teizer, J.: Mobile 3D mapping for surveying earth-work projects using an unmanned aerial vehicle (UAV) system. Autom. Constr. 41, 1–14 (2014)

    Article  Google Scholar 

  4. Leutenegger, S., Chli, M., Siegwart, R.Y.: Brisk: Binary robust invariant scalable keypoints. In: IEEE International Conference on Computer Vision (ICCV), pp. 2548–2555. IEEE (2011)

  5. Calonder, M., Lepetit, V., Strecha, C., Fua, P.: BRIEF: binary robust independent elementary features. Comput. Vis. ECCV 2010, 778–792 (2010)

    Google Scholar 

  6. Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: an efficient alternative to SIFT or SURF. In: IEEE International Conference on Computer Vision (ICCV), pp. 2564–2571. IEEE (2011)

  7. Datar, M., Immorlica, N., Indyk, P., Mirrokni, V.S.: Locality-sensitive hashing scheme based on p-stable distributions. In: Proceedings of the Twentieth Annual Symposium on Computational Geometry, pp. 253–262. ACM (2004)

  8. de Lima, R., Martinez-Carranza, J.: Real-time aerial image mosaicing using hashing-based matching. In: Workshop on Research, Education and Development of Unmanned Aerial Systems (RED-UAS), pp. 144–149. IEEE (2017)

  9. Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, New York (2012)

    MATH  Google Scholar 

  10. Lewis, J.P.: Fast normalized cross-correlation. Vis. Interface 10, 120–123 (1995)

    Google Scholar 

  11. Kokate, M.D., Wankhede, V.A., Patil, R.S.: Survey: image mosaicing based on feature extraction. Int. J Comput. Appl. 165(1), 26–30 (2017). https://doi.org/10.5120/ijca2017913776

    Article  Google Scholar 

  12. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  13. Khan, N., McCane, B., Mills, S.: Better than SIFT? Mach. Vis. Appl. 26(6), 819–836 (2015)

    Article  Google Scholar 

  14. Liqian, D., Yuehui, J.: Moon landform images fusion and mosaic based on SIFT method. In: International Conference on Computer and Information Application (ICCIA), pp. 29–32. IEEE (2010)

  15. Nemra, A., Aouf, N.: Robust invariant automatic image mosaicing and super resolution for UAV mapping. In: ISMA’09. 6th International Symposium on Mechatronics and its Applications, pp. 1–7. IEEE (2009)

  16. Bekele, D., Teutsch, M., Schuchert, T.: Evaluation of binary keypoint descriptors. In: 20th IEEE International Conference on Image Processing (ICIP), pp. 3652–3656. IEEE (2013)

  17. Kern, A., Bobbe, M., Bestmann, U.: Towards a real-time aerial image mosaicing solution. In: International Micro-Air Vehicle Conference and Competition (IMAV) (2016)

  18. Li, J., Yang, T., Yu, J., Lu, Z., Lu, P., Jia, X., Chen, W.: Fast aerial video stitching. Int. J. Adv. Robot. Syst. 11(10), 167 (2014)

    Article  Google Scholar 

  19. Wang, G., Zhai, Z., Xu, B., Cheng, Y.: A parallel method for aerial image stitching using ORB feature points. In: IEEE/ACIS 16th International Conference on Computer and Information Science (ICIS), pp. 769–773. IEEE (2017)

  20. Heise, P., Jensen, B., Klose, S., Knoll, A.: Fast dense stereo correspondences by binary locality sensitive hashing. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 105–110. IEEE (2015)

  21. Gálvez-López, D., Tardos, J.D.: Bags of binary words for fast place recognition in image sequences. IEEE Trans. Robot. 28(5), 1188–1197 (2012)

    Article  Google Scholar 

  22. Mur-Artal, R., Tardós, J.D.: ORB-SLAM2: an open-source slam system for monocular, stereo, and rgb-d cameras. IEEE Trans. Robot. 33(5), 1255–1262 (2017)

    Article  Google Scholar 

  23. de Lima, R., Martinez-Carranza, J., Morales-Reyes, A., Mayol-Cuevas, W.: Toward a smart camera for fast high-level structure extraction. J. Real-Time Image Proc 14, 685–699 (2018). https://doi.org/10.1007/s11554-017-0704-5

    Article  Google Scholar 

  24. Weberruss, J., Kleeman, L., Drummond, T.: ORB feature extraction and matching in hardware. In: Proceedings of the Australasian Conference on Robotics and Automation, pp. 2–4. The Australian National University, Canberra (2015)

  25. Agarwal, A., Jawahar, C., Narayanan, P.: A survey of planar homography estimation techniques. Technical Report IIIT/TR/2005/12. Centre for Visual Information Technology (2005)

  26. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  27. Quigley, M., Conley, K., Gerkey, B.P., Faust, J.: ROS: an open-source robot operating system. ICRA Workshop Open Source Softw. 3(3.2), 5 (2009)

    Google Scholar 

Download references

Acknowledgements

This work has been partially funded by the Royal Society through the Newton Advanced Fellowship with reference NA-140454 and by the CONACYT-INEGI research project 268528.

Author information

Authors and Affiliations

  1. KU Leuven, Leuven, Belgium

    Roberto de Lima

  2. INAOE, Puebla, Mexico

    Aldrich A. Cabrera-Ponce & Jose Martinez-Carranza

  3. University of Bristol, Bristol, UK

    Jose Martinez-Carranza

Authors
  1. Roberto de Lima
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aldrich A. Cabrera-Ponce
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jose Martinez-Carranza
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jose Martinez-Carranza.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

de Lima, R., Cabrera-Ponce, A.A. & Martinez-Carranza, J. Parallel hashing-based matching for real-time aerial image mosaicing. J Real-Time Image Proc 18, 143–156 (2021). https://doi.org/10.1007/s11554-020-00959-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-020-00959-y

Keywords

Search

Navigation