Skip to main content
SpringerLink
Log in

Incremental Singular Value Decomposition Using Extended Power Method

  • Chapter
  • First Online:
Computer and Information Science (ICIS 2018)

Part of the book series: Studies in Computational Intelligence ((SCI,volume 791))

Included in the following conference series:

  • 462 Accesses

Abstract

In this paper we present a novel method to perform Incremental Singular Value Decomposition (ISVD) by using an adaptation of the power method for diagonalization of matrices. We find that the efficiency of the procedure depends on the initial values and present two alternative ways for initialization. Gram-Schmidt orthonormalization is employed to ensure the orthogonality of the generated singular vectors. The suggested procedure does not depend on any assumption regarding the nature of input matrix or the change (increment) in the input matrix. Moreover, the results do not deviate from exact values even after repeated increments to the original input matrix. In order to test the suggested technique we apply it to the task of Latent Semantic Indexing for query processing using the MEDLINE and TIME corpus. Seven hundred documents are used to build the initial matrix for MEDLINE and two hundred for TIME corpus. We then add one document at a time till the complete corpus is included. Our ISVD technique is applied each time a new document is added. The results obtained using both the methods for initialization are very similar to the exact results obtained using direct diagonalization routines. This implies that the proposed method remains stable even after hundreds of increments in the original matrix.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    \(UU^T = VV^T = I\), where I is an Identity Matrix.

  2. 2.

    Right singular vectors of \(\hat{A}\) are same as the eigenvectors of \(\hat{M}\).

  3. 3.

    http://ir.dcs.gla.ac.uk/resources/test_collections.

  4. 4.

    http://nlp.stanford.edu/software/stanford-corenlp-full-2015-12-09.zip.

References

  1. Baker, C.G., Gallivan, K.A., Dooren, P.V.: Low-rank incremental methods for computing dominant singular subspaces. Linear Algebra Appl. 436, 2866–2888 (2012)

    Article  MathSciNet  Google Scholar 

  2. Balsubramani, A., Dasgupta, S., Freund, Y.: The fast convergence of incremental PCA. In: Advances in Neural Information Processing Systems (2013)

    Google Scholar 

  3. Brand, M.: Incremental singular value decomposition of uncertain data with missing values. In: Proceedings of the 7th European Conference on Computer Vision-Part I, London, UK, pp. 707–720 (2002)

    Google Scholar 

  4. Brand, M.: Fast low-rank modifications of the thin singular value decomposition. Linear Algebra Appl. 415, 20–30 (2006)

    Article  MathSciNet  Google Scholar 

  5. Bhushan, A., Sharker, M.H., Karimi, H.A.: Incremental principal component analysis based outlier detection methods for spatiotemporal data streams. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. 2(4), 67 (2015)

    Article  Google Scholar 

  6. Chandrasekaran, S., Manjunath, B.S., Wang, Y.F., Winkeler, J., Zhang, H.: An Eigenspace update algorithm for image analysis. In: CVGIP: Graphical Model and Image Processing, vol. 59, pp. 321–332 (1997)

    Google Scholar 

  7. Davies, P.I., Smith, M.I.: Updating the singular value decomposition. J. Comput. Appl. Math. 170(1), 145–167 (2004)

    Article  MathSciNet  Google Scholar 

  8. Deerwester, S., Dumais, S.T., Furnas, G.W., Landauer, T.K., Harshman, R.: Indexing by latent semantic analysis. J. Am. Soc. Inf. Sci. 41, 391–407 (1990)

    Article  Google Scholar 

  9. Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. Johns Hopkins University Press, Baltimore (MD) (1996)

    MATH  Google Scholar 

  10. Hall, P.M., David Marshall, A., Martin, R.R.: Incremental eigenanalysis for classification. In: BMVC, vol. 98 (1998)

    Google Scholar 

  11. Hao, S., Xu, Y., Ke, D., Su, K., Peng, H.: SCESS: a WFSA-based automated simplified chinese essay scoring system with incremental latent semantic analysis. Nat. Lang. Eng. 22, 291–319 (2016)

    Article  Google Scholar 

  12. Jiang, H.Y., Nguyen, T.N., Chen, I.X., Jaygarl, H., Chang, C.K.: Incremental latent semantic indexing for automatic traceability link evolution management. In: 23rd IEEE/ACM International Conference on Automated Software Engineering, 2008. ASE 2008, pp. 59–68 (2008)

    Google Scholar 

  13. Kaufman, L.: Methods for updating the singular value decomposition. DAIMI Report Series 3 (1974)

    Google Scholar 

  14. King, J.D., Li, Y.: Web based collection selection using singular value decomposition. In: Web Intelligence, pp. 104–110 (2003)

    Google Scholar 

  15. Kwok, J.T., Zhao, H.: Incremental eigen decomposition. Matrix 100(C1), C2 (2003)

    Google Scholar 

  16. Levy, A., Lindenbaum, M.: Sequential Karhunen-Loeve Basis Extraction and Its Application to Images

    Google Scholar 

  17. Oyama, T., et al.: Incremental learning method of simple-PCA. In: International Conference on Knowledge-Based and Intelligent Information and Engineering Systems. Springer, Berlin, Heidelberg (2008)

    Google Scholar 

  18. Ozawa, S., Pang, S., Kasabov, N.: A modified incremental Principal Component Analysis for on-line learning of feature space and classifier. In: Pacific Rim International Conference on Artificial Intelligence. Springer, Berlin, Heidelberg (2004)

    Google Scholar 

  19. Press, W.H., Teukolsky, S.A., Vetterling, W.T., Flannery, B.P.: Numerical Recipes in C: The Art of Scientific Computing, 2nd edn. Cambridge University Press, New York, NY, USA (1992)

    Google Scholar 

  20. Rao, S., Medeiros, H., Kak, A.: Comparing incremental latent semantic analysis algorithms for efficient retrieval from software libraries for bug localization. SIGSOFT Softw. Eng. Notes. 40, 1–8 (2015)

    Google Scholar 

  21. Rehurek, R.: Subspace tracking for latent semantic analysis. In: Clough, P.D., Foley, C., Gurrin, C., Jones, G.J.F., Kraaij, W., Lee, H., Murdock, V. (eds) ECIR, pp. 289–300 (2011)

    Google Scholar 

  22. Ross, D.A., Lim, J., Lin, R.S., Yang, M.H.: Incremental learning for robust visual tracking. Int. J. Comput. Vis. 77(1–3), 125–141 (2008)

    Article  Google Scholar 

  23. Saad, Y.: Numerical methods for large eigenvalue problems. In: SIAM (2011)

    Google Scholar 

  24. Sabes, P.N.: Linear Algebraic Equations, SVD, and the Pseudo-Inverse. http://keck.ucsf.edu/

  25. Sarwar, B., et al.: Incremental singular value decomposition algorithms for highly scalable recommender systems. In: Fifth International Conference on Computer and Information Science (2002)

    Google Scholar 

  26. Stewart, G.W.: An updating algorithm for subspace tracking. IEEE Trans. Signal Process. 40(6), 1535–1541 (1992)

    Article  Google Scholar 

  27. Stewart, M., Van Dooren, P.: Updating a generalized URV decomposition. SIAM J. Matrix Anal. Appl. 22(2), 479–500 (2000)

    Article  MathSciNet  Google Scholar 

  28. Yang, J., et al.: A fast incremental algorithm for low rank approximations of matrices and its applications in facial images. J. Univ. Sci. Technol. China 39(9), 970–979 (2009)

    Google Scholar 

  29. Zhang, M., Hao, S., Xu, Y., Ke, D., Peng, H.: Automated essay scoring using incremental latent semantic analysis. J. Softw. 9 (2014)

    Google Scholar 

  30. Zha, H., Simon, H.D.: On updating problems in latent semantic indexing. SIAM J. Sci. Comput. 21, 782–791 (1999)

    Article  MathSciNet  Google Scholar 

  31. Zhao, H., Yuen, P.C., Kwok, J.T.: A novel incremental principal component analysis and its application for face recognition. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 36(4), 873–886 (2006)

    Google Scholar 

Download references

Acknowledgements

The authors gratefully acknowledge the infrastructural support provided by Indian Institute of Information Technology, Allahabad (IIIT-A). One of the authors (SG) also acknowledges the financial support from IIIT-A.

Author information

Authors and Affiliations

  1. Information Technology, Indian Institute of Information Technology, Allahabad, Allahabad, India

    Sharad Gupta

  2. Computer Science and Engineering, BML Munjal University, Gurgaon, India

    Sudip Sanyal

Authors
  1. Sharad Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sudip Sanyal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sharad Gupta .

Editor information

Editors and Affiliations

  1. Software Engineering and Information Technology Institute, Central Michigan University, Mt. Pleasant, MI, USA

    Roger Lee

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Gupta, S., Sanyal, S. (2019). Incremental Singular Value Decomposition Using Extended Power Method. In: Lee, R. (eds) Computer and Information Science. ICIS 2018. Studies in Computational Intelligence, vol 791. Springer, Cham. https://doi.org/10.1007/978-3-319-98693-7_7

Download citation

Publish with us

Policies and ethics

Search

Navigation