Abstract
This research investigates the use of Convolutional Neural Networks (CNN) and specifically, You Only Look Once ver. 4 (YOLOv4) to detect Racing Bib Numbers (RBNs) in images from running races and then to recognise the actual numbers using Optical Character Recognition (OCR) techniques. Pre-processing and Tesseract OCR were employed to achieve this. Using a self-acquired private dataset we achieve a recall of 0.91, precision of 0.88, an F1-measure of 0.89, and mean average precision (mAP) of 0.935 for detection. Full number recognition of 71% is then achieved on the successfully detected RBNs. Additionally, the proposed approach attains a very low average inference time of 23.5 ms compared to a previous best recorded time of 750 ms. This is achieved this with a relatively small training set of 1374 images, where previous research used 498,385 labelled images.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The YOLOv4 repository is available at: https://github.com/AlexeyAB/darknet.
- 2.
The Tesseract OCR repository is stored at: https://github.com/tesseract-ocr/tesseract.
- 3.
RBNR dataset is available at: https://people.csail.mit.edu/talidekel/RBNR.html.
References
Alexey, A.B.: AlexeyAB/darknet: Windows and Linux version of Darknet Yolo v3 v2 Neural Networks for object detection (Tensor Cores are used) (2019). https://github.com/AlexeyAB/darknet. Accessed 4 Oct 2020
Ben-Ami, I., Basha, T., Avidan, S.: Racing bib number recognition. In: BMVC 2012 - Electronic Proceedings of the British Machine Vision Conference 2012. Tel Aviv University (2012)
Bochkovskiy, A., Wang, C.Y., Liao, H.Y.M.: YOLOv4: optimal speed and accuracy of object detection (2020). http://arxiv.org/abs/2004.10934
Boonsim, N.: Racing bib number localization on complex backgrounds. Technical reports (2018)
Boonsim, N., Kanjaruek, S.: Racing bib number localization based on region convolutional neural networks. In: Proceedings of 2018 the 8th International Workshop on Computer Science and Engineering, pp. 293–297 (2018)
Everingham, M., Gool, L.V., Williams, C.K.I., Winn, J., Zisserman, A.: The pascal visual object classes (VOC) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)
Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 580–587. IEEE Computer Society (2014)
Goodfellow, I.J., Bulatov, Y., Ibarz, J., Arnoud, S., Shet, V.: Multi-digit number recognition from street view imagery using deep convolutional neural networks (2014). http://arxiv.org/abs/1312.6082
IAAF: Habos Gebrihwet lap miscount - IAAF Diamond League (2019). https://www.youtube.com/watch?v=hWGNFAlE-Rk. Accessed 4 Oct 2020
Ivarsson, E., Mueller, R.M.: Racing bib number recognition using deep learning racing bib number recognition using deep learning completed research. Technical reports, Berlin School of Economics and Law (2019)
Lienhart, R., Maydt, J.: An extended set of Haar-like features for rapid object detection. In: IEEE International Conference on Image Processing, vol. 1 (2002)
Tsung-Yi, L., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48
Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A.Y.: Reading digits in natural images with unsupervised feature learning. Technical reports (2011), http://ufldl.stanford.edu/housenumbers/
Redmon, J., Farhadi, A.: YOLOv3: an incremental improvement (2018). http://arxiv.org/abs/1804.02767
Roy, S., Shivakumara, P., Mondal, P., Raghavendra, R., Pal, U., Lu, T.: A new multi-modal technique for bib number/text detection in natural images. In: Ho, Y.-S., Sang, J., Ro, Y.M., Kim, J., Wu, F. (eds.) PCM 2015. LNCS, vol. 9314, pp. 483–494. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24075-6_47
Shivakumara, P., Quy Phan, T., Lim Tan, C.: New wavelet and color features for text detection in video (2010)
Shivakumara, P., Raghavendra, R., Qin, L., Raja, K.B., Lu, T., Pal, U.: A new multi-modal approach to bib number/text detection and recognition in marathon images. Pattern Recognit. 61, 479–491 (2017)
Smith, R.: An overview of the tesseract OCR engine. In: Proceedings of the International Conference on Document Analysis and Recognition, ICDAR, vol. 2, pp. 629–633 (2007)
Wong, Y.: Deep learning based racing bib number detection and recognition. Jordanian J. Comput. Inf. Technol. 05(03), 1 (2019)
Wrońska, A., Sarnacki, K., Saeed, K.: Athlete number detection on the basis of their face images. In: 2017 International Conference on Biometrics and Kansei Engineering (ICBAKE), pp. 84–89. IEEE (2017)
Yang, C.S., Yang, Y.H.: Improved local binary pattern for real scene optical character recognition. Pattern Recognit. Lett. 100, 14–21 (2017)
Yun, S., Han, D., Oh, S.J., Chun, S., Choe, J., Yoo, Y.: Cutmix: Regularization strategy to train strong classifiers with localizable features. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 6023–6032 (2019)
Acknowledgments
Avaya funded the course of study that gave rise to this research. The authors also wish to acknowledge the DJEI/DES/SFI/HEA Irish Centre for High-End Computing (ICHEC) for the provision of computational facilities and support. Access to the training images was provided by the Galway County Athletics board and the photographer John O’Connor.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Carty, G., Raja, M.A., Ryan, C. (2021). Running to Get Recognised. In: Thampi, S.M., Krishnan, S., Hegde, R.M., Ciuonzo, D., Hanne, T., Kannan R., J. (eds) Advances in Signal Processing and Intelligent Recognition Systems. SIRS 2020. Communications in Computer and Information Science, vol 1365. Springer, Singapore. https://doi.org/10.1007/978-981-16-0425-6_1
Download citation
DOI: https://doi.org/10.1007/978-981-16-0425-6_1
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-0424-9
Online ISBN: 978-981-16-0425-6
eBook Packages: Computer ScienceComputer Science (R0)