Skip to main content
SpringerLink
Log in

A Novel Visual Lane Line Detection System for a NMPC-based Path Following Control Scheme

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

Tracking a visual path to perform image-based control requires additional processing to extract valuable information even in the presence of inconveniences, such as failures or deformation in the lane line or even restrictive environmental conditions. Dealing with faulty in the path to be followed, non-homogeneous floors and bad lighting condition are some of the difficulties encountered that compromise the accurate of feature extraction and consequently the controller effectiveness. In this paper, a novel visual system is proposed to detect and extract useful lane line parameters and to use them in a NMPC-based Visual Path Following Control Scheme. To cope with the above-mentioned problems, the visual path is approximated by a quadratic function and to robustness improve, this novel algorithm was modified to use RANSAC as a model estimation approach instead of using the classical Least Square Method. Experimental results demonstrate the superiority of the proposed system with respect to two others, in environments with visual disturbance, faulty paths, noise and bad lighting conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Arakeri, M.P., Vijaya Kumar, B.P., Barsaiya, S., Sairam, H.V.: Computer vision based robotic weed control system for precision agriculture. In: 2017 International Conference on Advances in Computing, Communications and Informatics (ICACCI) (2017)

  2. Borkar, A., Hayes, M., Smith, M.T.: Robust lane detection and tracking with ransac and kalman filter. In: 2009 16th IEEE International Conference on Image Processing (ICIP), pp. 3261–3264 (2009)

  3. Castano, M., Tan, X.: Model predictive control-based path-following for tail-actuated robotic fish. J. Dyn. Syst. Meas. Control, 141 (2019)

  4. Cheong, H.W., Lee, H.: Concept design of agv (automated guided vehicle) based on image detection and positioning. Procedia Comput. Sci. 139, 104–107 (2018). https://doi.org/10.1016/j.procs.2018.10.224

    Article  Google Scholar 

  5. Derpanis, K.G.: Overview of the RANSAC Algorithm, vol. 4. ImageRochester, Rochester (2010)

    Google Scholar 

  6. Du, X., Tan, K.K., Htet, K.K.K.: Vision-based lane line detection for autonomous vehicle navigation and guidance. In: 2015 10th Asian Control Conference (ASCC), pp. 1–5 (2015)

  7. Faulwasser, T., Findeisen Rolf adn Magni, L., Raimondo, D.M., Allgöwer, F.: Nonlinear Model Predictive Path-Following Control, pp 335–343. Springer, Berlin (2009)

    Book  Google Scholar 

  8. Fischler, M.A., Bolles, R.C.: Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  9. Franco, I.J.P.B., Ribeiro, T.T., Conceicao, A.G.S.: A novel approach for parameter extraction of an nmpc-based visual follower model. In: 2019 19th International Conference on Advanced Robotics (ICAR), pp. 117–122 (2019)

  10. Gorbunov, V., Bobkov, V., Htet, N.W., Ionov, E.: Automated control system of fabrics parameters that uses computer vision. In: 2018 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus), pp 1728–1730 (2018)

  11. Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2004)

  12. Huang, A.S., Moore, D., Antone, M., Olson, E., Teller, S.: Finding multiple lanes in urban road networks with vision and lidar. Auton. Robot. 26(2-3), 103–122 (2009)

    Article  Google Scholar 

  13. Jang, H.J., Baek, S.H., Park, S.Y.: Lane marking detection in various lighting conditions using robust feature extraction. In: Computer Science (2014)

  14. Kim, Z.: Robust lane detection and tracking in challenging scenarios. IEEE Trans. Intell. Transp. Syst. 9(1), 16–26 (2008)

    Article  Google Scholar 

  15. Kuo, Y.C., Pai, N.S., Li, Y.F.: Vision-based vehicle detection for a driver assistance system. Advances in Nonlinear Dynamics, vol. 61, pp 2096–2100 (2011)

  16. Li, S., Xu, J., Wei, W., Qi, H.: Curve lane detection based on the binary particle swarm optimization. In: 2017 29th Chinese Control And Decision Conference (CCDC), pp. 75–80 (2017)

  17. Lu, W., Florez, S.A.R., Seignez, E., Reynaud, R.: An improved approach for vision-based lane marking detection and tracking. In: 2013 International Conference on Electrical, Control and Automation Engineering, Dec 2013, Hong Kong, China. DEStech Publications, pp. 382–386 (2014)

  18. Mammeri, A., Boukerche, A., Lu, G.: Lane detection and tracking system based on the mser algorithm, hough transform and kalman filter. In: MSWiM ’14 (2014)

  19. Mammeri, A., Boukerche, A., Tang, Z.: A real-time lane marking localization, tracking and communication system. Comput. Commun. 73, 132–143 (2016)

    Article  Google Scholar 

  20. Raković, S.V., Levine, W.: Handbook of Model Predictive Control. Springer, Berlin (2018)

  21. Ribeiro, T.T., Conceicao, A.G.S.: Nonlinear model predictive visual path following control to autonomous mobile robots. J. Intell. Robot. Syst. 95(2), 731–743 (2019)

    Article  Google Scholar 

  22. Ribeiro, T.T., Fernandez, R.O., Conceicao, A.G.S.: Nmpc-based visual leader-follower formation control for wheeled mobile robots. In: 2018 IEEE 16th International Conference on Industrial Informatics (INDIN), pp. 406–411 (2018)

  23. Sun, T.Y., Tsai, S.J., Chan, V.: Hsi color model based lane-marking detection. In: 2006 IEEE Intelligent Transportation Systems Conference, pp. 1168–1172 (2006)

  24. Vetrella, A.R., Savvaris, A., Fasano, G., Accardo, D.: Rgb-d camera-based quadrotor navigation in gps-denied and low light environments using known 3d markers. In: 2015 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 185–192 (2015)

  25. Yuan, C., Chen, H., Liu, J., Zhu, D., Xu, Y.: Robust lane detection for complicated road environment based on normal map. IEEE Access 6, 49,679–49,689 (2018)

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank the SEPIN/MCTI and the European Union’s Horizon 2020 Research and Innovation Programme through the Grant Agreement No̱ 777096, and the Brazilian funding agency (CNPq) grant number [306852/2017-9].

Author information

Authors and Affiliations

  1. LaR - Robotics Laboratory, Department of Electrical and Computer Engineering, Federal University of Bahia, Salvador, Bahia, Brazil

    Iago José Pattas Bastos Franco, Tiago Trindade Ribeiro & André Gustavo Scolari Conceição

Authors
  1. Iago José Pattas Bastos Franco
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tiago Trindade Ribeiro
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. André Gustavo Scolari Conceição
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to André Gustavo Scolari Conceição.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(MP4 28.0 MB)

(MP4 4.50 MB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Franco, I.J.P.B., Ribeiro, T.T. & Conceição, A.G.S. A Novel Visual Lane Line Detection System for a NMPC-based Path Following Control Scheme. J Intell Robot Syst 101, 12 (2021). https://doi.org/10.1007/s10846-020-01278-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10846-020-01278-x

Keywords

Search

Navigation