Abstract
Handwritten word spotting (HWS) is a task of retrieving word instances within handwritten documents, which is typically assisted by word annotations (word-level HWS). Previous methods following this paradigm are always in the manual feature modelling fashion, failing to capture sufficient discriminative information of the original input; also, they are always quite time-consuming in the artificial segmentation phase, limiting their applications in practice. To address these problems, we revisit HWS and model it on page-level via discriminative feature learning. Two distinct components modelled as neural networks are combined: word discriminative representation learning by Siamese Feature Network (SFNet) and the word discriminative spotting by Word Discriminative Spotting Network (WdsNet). Even without annotation of boxes, our WdsNet reaches impressive results on the IAM benchmark dataset with 76.8% mAP for the full page word spotting, revealing its superiority over other competitors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Almazan, J., Gordo, A., Fornes, A., Valveny, E.: Word spotting and recognition with embedded attributes. IEEE Trans. Pattern Anal. Mach. Intell. 36(12), 2552–2566 (2014). https://doi.org/10.1109/tpami.2014.2339814
Barakat, B.K., Alasam, R., El-Sana, J.: Word spotting using convolutional siamese network. In: 2018 13th IAPR International Workshop on Document Analysis Systems (DAS). IEEE (2018). https://doi.org/10.1109/das.2018.67
Bromley, J., et al.: Signature verification using a “siamese” time delay neural network. In: Advances in Neural Information Processing Systems, pp. 737–744 (1994). https://doi.org/10.1142/9789812797926_0003
Cai, Z., Vasconcelos, N.: Cascade r-CNN: Delving into high quality object detection. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE (2018). https://doi.org/10.1109/cvpr.2018.00644
Fei-Fei, L., Fergus, R., Perona, P.: One-shot learning of object categories. IEEE Trans. Pattern Anal. Mach. Intell. 28(4), 594–611 (2006). https://doi.org/10.1109/tpami.2006.79
Giotis, A.P., Sfikas, G., Gatos, B., Nikou, C.: A survey of document image word spotting techniques. Pattern Recogn. 68, 310–332 (2017). https://doi.org/10.1016/j.patcog.2017.02.023
Graves, A., Liwicki, M., Fernández, S., Bertolami, R., Schmidhuber, J.: A novel connectionist system for unconstrained handwriting recognition 31, 855–868 (2009). https://doi.org/10.1109/TPAMI.2008.137
He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 630–645. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_38
Krishnan, P., Dutta, K., Jawahar, C.: Word spotting and recognition using deep embedding. In: 2018 13th IAPR International Workshop on Document Analysis Systems (DAS). IEEE (2018). https://doi.org/10.1109/das.2018.70
Manmatha, R., Han, C., Riseman, E.: Word spotting: a new approach to indexing handwriting. In: Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE (1996). https://doi.org/10.1109/cvpr.1996.517139
Marti, U.V., Bunke, H.: The IAM-database: an English sentence database for offline handwriting recognition. Int. J. Doc. Anal. Recogn. 5(1), 39–46 (2002). https://doi.org/10.1007/s100320200071
Marti, U.V., Bunke, H.: Using a statistical language model to improve the performance of an HMM-based cursive handwriting recognition system. In: Hidden Markov Models: Applications in Computer Vision, pp. 65–90. World Scientific (2001). https://doi.org/10.1142/s0218001401000848
Perronnin, F., Dance, C.: Fisher kernels on visual vocabularies for image categorization. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition. IEEE (2007). https://doi.org/10.1109/cvpr.2007.383266
Rath, T., Manmatha, R.: Word image matching using dynamic time warping. In: 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings. IEEE Computer Society (2003). https://doi.org/10.1109/cvpr.2003.1211511
Ren, S., He, K., Girshick, R., Sun, J.: Faster r-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017). https://doi.org/10.1109/tpami.2016.2577031
Retsinas, G., Louloudis, G., Stamatopoulos, N., Sfikas, G., Gatos, B.: An alternative deep feature approach to line level keyword spotting. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2019). https://doi.org/10.1109/cvpr.2019.01294
Sudholt, S., Fink, G.A.: PHOCNet: A deep convolutional neural network for word spotting in handwritten documents. In: 2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR). IEEE (2013). https://doi.org/10.1109/icfhr.2016.0060
Sudholt, S., Fink, G.A.: Attribute CNNs for word spotting in handwritten documents. Int. J. Doc. Anal. Recogn. 21(3), 199–218 (2018). https://doi.org/10.1007/s10032-018-0295-0
Wahlberg, F., Dahllöf, M., Mårtensson, L., Brun, A.: Data mining medieval documents by word spotting. In: Proceedings of the 2011 Workshop on Historical Document Imaging and Processing - HIP. ACM Press (2011). https://doi.org/10.1145/2037342.2037355
Wei, H., Zhang, H., Gao, G.: Representing word image using visual word embeddings and RNN for keyword spotting on historical document images. In: 2017 IEEE International Conference on Multimedia and Expo (ICME). IEEE (2017). https://doi.org/10.1109/icme.2017.8019403
Wilkinson, T., Brun, A.: A novel word segmentation method based on object detection and deep learning. In: Bebis, G., et al. (eds.) ISVC 2015. LNCS, vol. 9474, pp. 231–240. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-27857-5_21
Wilkinson, T., Brun, A.: Semantic and verbatim word spotting using deep neural networks. In: 2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR). IEEE (2016). https://doi.org/10.1109/icfhr.2016.0065
Acknowledgements
This work is supported by National Natural Science Foundation of China under contract No. 61671025.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Gao, J., Guo, X., Shang, M., Sun, J. (2020). Page-Level Handwritten Word Spotting via Discriminative Feature Learning. In: Li, G., Shen, H., Yuan, Y., Wang, X., Liu, H., Zhao, X. (eds) Knowledge Science, Engineering and Management. KSEM 2020. Lecture Notes in Computer Science(), vol 12274. Springer, Cham. https://doi.org/10.1007/978-3-030-55130-8_32
Download citation
DOI: https://doi.org/10.1007/978-3-030-55130-8_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-55129-2
Online ISBN: 978-3-030-55130-8
eBook Packages: Computer ScienceComputer Science (R0)