Skip to main content
SpringerLink
Log in

Handwritten Digit Recognition Using GIST Descriptors and Random Oblique Decision Trees

  • Conference paper
  • First Online:
Some Current Advanced Researches on Information and Computer Science in Vietnam (NAFOSTED 2014)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 341))

Abstract

Our investigation aims at constructing random oblique decision trees to recognize handwritten digits. At the pre-processing step, we propose to use the GIST descriptor to represent digit images in large number of dimensions. And then we propose a multi-class version of random oblique decision trees based on the linear discriminant analysis and the Kolmogorov-Smirnov splitting criterion that is suited for classifying high dimensional datasets. The experimental results on USPS, MNIST datasets show that our proposal has very high accuracy compared to state-of-the-art algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. LeCun, Y., Boser, B., Denker, J., Henderson, D., Howard, R., Hubbard, W., Jackel, L.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989)

    Article  Google Scholar 

  2. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  3. Decoste, D., Schölkopf, B.: Training invariant support vector machines. Mach. Learn. 46(1–3), 161–190 (2002)

    Article  MATH  Google Scholar 

  4. Kégl, B., Busa-Fekete, R.: Boosting products of base classifiers. In: Proceedings of the 26th Annual International Conference on Machine Learning (ICML’09), pp. 497–504. ACM (2009)

    Google Scholar 

  5. Oliva, A., Torralba, A.: Modeling the shape of the scene: a holistic representation of the spatial envelope. Int. J. Comput. Vis. 42, 145–175 (2001)

    Article  MATH  Google Scholar 

  6. Abou-zeid, H., El-ghazal, A., Al-khatib, A.: Computer recognition of unconstrained handwritten numerals. In: 2003 IEEE 46th Midwest Symposium on Circuits and Systems, vol. 2, pp. 969–973 (2003)

    Google Scholar 

  7. Ranzato, M., Boureau, Y.L., Chopra, S., LeCun, Y.: A unified energy-based framework for unsupervised learning. In: AISTATS, pp. 371–379 (2007)

    Google Scholar 

  8. Lauer, F., Suen, C.Y., Bloch, G.: A trainable feature extractor for handwritten digit recognition. Pattern Recogn. 40(6), 1816–1824 (2007)

    Article  MATH  Google Scholar 

  9. Labusch, K., Barth, E., Martinetz, T.: Simple method for high-performance digit recognition based on sparse coding. Trans. Neural Netw. 19(11), 1985–1989 (2008)

    Article  Google Scholar 

  10. Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381(6583), 607–609 (1996)

    Article  Google Scholar 

  11. Cireşan, D., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. In: Computer Vision and Pattern Recognition (CVPR 2012), pp. 3642–3649 (2012). Arxiv preprint: arXiv:1202.2745

  12. Lowe, D.: Object recognition from local scale invariant features. In: Proceedings of the 7th International Conference on Computer Vision, pp. 1150–1157 (1999)

    Google Scholar 

  13. Lowe, D.: Distinctive image features from scale invariant keypoints. Int. J. Comput. Vis. 91–110 (2004)

    Google Scholar 

  14. Bosch, A., Zisserman, A., Muñoz, X.: Scene classification via pLSA. In: Proceedings of the European Conference on Computer Vision, pp. 517–530 (2006)

    Google Scholar 

  15. Li, X., Wu, C., Zach, C., Lazebnik, S., Frahm, J.: Modeling and recognition of landmark image collections using iconic scene graphs. In: Proceedings of the 10th European Conference on Computer Vision: Part I, pp. 427–440 (2008)

    Google Scholar 

  16. Douze, M., Jégou, H., Sandhawalia, H., Amsaleg, L., Schmid, C.: Evaluation of gist descriptors for web-scale image search. In: Proceedings of the ACM International Conference on Image and Video Retrieval, pp. 1–8 (2009)

    Google Scholar 

  17. Dietterich, T., Kong, E.B.: Machine learning bias, statistical bias, and statistical variance of decision tree algorithms. Technical report (1995)

    Google Scholar 

  18. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  MATH  Google Scholar 

  19. Freund, Y., Schapire, R.: A decision-theoretic generalization of on-line learning and an application to boosting. In: Computational Learning Theory: Proceedings of the Second European Conference, pp. 23–37 (1995)

    Google Scholar 

  20. Vapnik, V.: The Nature of Statistical Learning Theory. Springer (1995)

    Google Scholar 

  21. Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.: Classification and Regression Trees. Wadsworth International (1984)

    Google Scholar 

  22. Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo (1993)

    Google Scholar 

  23. Murthy, S., Kasif, S., Salzberg, S., Beigel, R.: OC1: Randomized induction of oblique decision trees. In: Proceedings of the Eleventh National Conference on Artificial Intelligence, pp. 322–327 (1993)

    Google Scholar 

  24. Do, T.N., Lenca, P., Lallich, S., Pham, N.K.: Classifying very-high-dimensional data with random forests of oblique decision trees. In: Advances in Knowledge Discovery and Management. Studies in Computational Intelligence, vol. 292, pp. 39–55. Springer, Berlin (2010)

    Google Scholar 

  25. Fisher, R.: The use of multiple measurements in taxonomic problems. Ann. Eugen. 7(2), 179–188 (1936)

    Article  Google Scholar 

  26. Lemmond, T.D., Chen, B.Y., Hatch, A.O., Hanley, W.G.: Discriminant random forests. In: DMIN, pp. 55–61 (2008)

    Google Scholar 

  27. Menze, B.H., Kelm, B.M., Splitthoff, D.N., Koethe, U., Hamprecht, F.A.: On oblique random forests. In: Proceedings of the 2011 European Conference on Machine Learning and Knowledge Discovery in Databases—Volume Part II (ECML PKDD’11), pp. 453–469 Springer (2011)

    Google Scholar 

  28. Do, T.N., Lenca, P., Lallich, S.: Enhancing network intrusion classification through the kolmogorov-smirnov splitting criterion. In: Proceedings of the 3rd International Conference on Theories and Applications of Computer Science, pp. 50–61 (2010)

    Google Scholar 

  29. Friedman, J.H.: A recursive partitioning decision rule for nonparametric classification. IEEE Trans. Comput. 26(4), 404–408 (1977)

    Article  MATH  Google Scholar 

  30. KreBel, U.: Pairwise classification and support vector machines, Advances in Kernel Methods: Support Vector Learning, pp. 255–268 (1999)

    Google Scholar 

  31. Platt, J., Cristianini, N., Shawe-Taylor, J.: Large margin DAGs for multiclass classification. Adv. Neural Inf. Process. Syst. 12, 547–553 (2000)

    Google Scholar 

  32. Vural, V., Dy, J.: A hierarchical method for multi-class support vector machines. In: Proceedings of the Twenty-First International Conference on Machine Learning, pp. 831–838 (2004)

    Google Scholar 

  33. Benabdeslem, K., Bennani, Y.: Dendogram-based SVM for multi-class classification. J. Comput. Inf. Technol. 14(4), 283–289 (2006)

    Google Scholar 

  34. MacQueen, J.: Some methods for classification and analysis of multivariate observations. In: Berkeley Symposium on Mathematical Statistics and Probability. Univ. Calif. Press 1, 281–297 (1967)

    MathSciNet  Google Scholar 

  35. Whaley, R., Dongarra, J.: Automatically tuned linear algebra software. In: Ninth SIAM Conference on Parallel Processing for Scientific Computing CD-ROM Proceedings (1999)

    Google Scholar 

  36. Chang, C.C., Lin, C.J.: LIBSVM—a library for support vector machines. http://www.csie.ntu.edu.tw/~cjlin/libsvm (2003)

  37. Bromley, J., Sackinger, E.: Neural-network and k-nearest-neighbor classifiers (1991)

    Google Scholar 

  38. Simard, P., LeCun, Y., Denker, J.: Efficient pattern recognition using a new transformation distance. In: Advances in Neural Information Processing Systems 5, [NIPS Conference], pp. 50–58 (1993)

    Google Scholar 

  39. Simard, P., Steinkraus, D., Platt, J.: Best practices for convolutional neural networks applied to visual document analysis. In: Proceedings of the Seventh International Conference on Document Analysis and Recognition, vol. 2, pp. 958–963 (2003)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Information Technology, Can Tho University, No 1, Ly Tu Trong Street, Ninh Kieu District, Can Tho, 92100, Vietnam

    Thanh-Nghi Do & Nguyen-Khang Pham

Authors
  1. Thanh-Nghi Do
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nguyen-Khang Pham
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thanh-Nghi Do .

Editor information

Editors and Affiliations

  1. Institute of Information Technology, Vietnam Academy of Science and Technology, Hanoi, Vietnam

    Quang A. Dang

  2. IT Research and Development Center, Hanoi University, Hanoi, Vietnam

    Xuan Hoai Nguyen

  3. Faculty of Information Technology, University of Science, Hochiminh, Vietnam

    Hoai Bac Le

  4. University of Engineering and Technology, Hanoi, Vietnam

    Viet Ha Nguyen

  5. Posts and Telecommunications Institute of Technology, Ho Chi Minh, Vietnam

    Vo Nguyen Quoc Bao

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Do, TN., Pham, NK. (2015). Handwritten Digit Recognition Using GIST Descriptors and Random Oblique Decision Trees. In: Dang, Q., Nguyen, X., Le, H., Nguyen, V., Bao, V. (eds) Some Current Advanced Researches on Information and Computer Science in Vietnam. NAFOSTED 2014. Advances in Intelligent Systems and Computing, vol 341. Springer, Cham. https://doi.org/10.1007/978-3-319-14633-1_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-14633-1_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-14632-4

  • Online ISBN: 978-3-319-14633-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation