Abstract
Storage of indexing structures in the Vector Space Model (VSM) form has a number of advantages. In the case when text documents are considered, the indexing structure states the Term-By-Document (TBD) matrix. Its size is proportional to the product of the indexed documents number and the keywords number. In the case of large text documents databases, the size of the indexing structure is a serious limitation. Too large TBD matrix may not be able to be stored in memory or the process of searching for documents may take too much time. The article presents a methodology that allows to reduce the size of the large TBD matrix. The operation performed on the TBD matrix is the Singular Value Decomposition (SVD). It allows to transform the original indexing structure vectors into a space with fewer dimensions. As a result of the operation, keywords used in the indexing process are generalized. This is a desirable effect, methods for generalizing the keywords are called the Latent Sematic Indexing (LSI) methods. Despite the undeniable advantages of the SVD decomposition, it has a big disadvantage. Its computational complexity is O(n3). In practice, this prevents the application of the method to a large indexing structure. The methodology presented in the article assumes the use of the Epsilon decomposition in order to divide the original TBD matrix into parts before the reduction process. The proposed modification allows the use of the SVD decomposition for the indexing structure of any size.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Zhao, Y., Shi, X.: The application of vector space model in the information retrieval system. In: Zhang, W. (eds.) Software Engineering and Knowledge Engineering: Theory and Practice, Advances in Intelligent and Soft Computing, vol. 162, pp. 43–49. Springer, Heidelberg (2012)
Raczyński, D., Stanisławski, W.: SVD based Latent Semantic Indexing with use of the GPU computations. Int. J. Soft Comput. Math. Control (IJSCMC) 6(2/3), 1–14 (2017)
Gao, J., Zhang, J.: Clustered SVD strategies in Latent Semantic Indexing. Inf. Process. Manag. 41(5), 1051–1063 (2005)
Raczyński, D., Stanisławski, W.: Decomposition and reduction of indexing structures with use of the GPU computations. In: Grzech, A., Świątek, J., Wilimowska, Z., Borzemski, L. (eds.) Information Systems Architecture and Technology: Proceedings of 37th International Conference on Information Systems Architecture and Technology – ISAT 2016 – Part II, Advances in Intelligent Systems and Computing, vol. 522, pp. 225–237. Springer (2017)
Zečević, A., Šiljak, D.: Control of Complex Systems. Structural Constraints and Uncertainty. Springer, London (2010)
Šiljak, D.: Decentralized Control of Complex Systems. Academic Press, New York (1991)
Sezer, M., Šiljak, D.: Nested epsilon decompositions of linear systems: weakly coupled and overlapping blocks. SIAM. J. Matrix Anal. Appl. 12(3), 521–533 (1991)
Raczyński, D., Stanisławski, W.: Use of the modified EPSILON decomposition for the LTI models reduction. In: Świątek, J., Borzemski, L., Wilimowska, Z. (eds.) Information Systems Architecture and Technology: Proceedings of 38th International Conference on Information Systems Architecture and Technology – ISAT 2017, ISAT 2017, Advances in Intelligent Systems and Computing, vol. 656, pp. 3–16. Springer, Cham (2018)
Czyszczoń, A., Zgrzywa, A.: Latent Semantic Indexing for web service retrieval. In: Hwang, D., Jung, J.J., Nguyen, N.T. (eds.) Computational Collective Intelligence, Technologies and Applications, ICCCI 2014, Lecture Notes in Computer Science, vol. 8733, pp. 694–702. Springer, Cham (2014)
Rattanapanich, R., Sriharee, G.: Auto-tagging articles using Latent Semantic Indexing and ontology. In: Nguyen, N.T., Attachoo, B., Trawiński, B., Somboonviwat, K. (eds.) Intelligent Information and Database Systems, ACIIDS 2014, Lecture Notes in Computer Science, vol. 8397, pp. 153–162. Springer, Cham (2014)
Saad, M., Langlois, D., Smaïli, K.: Cross-lingual semantic similarity measure for comparable articles. In: Przepiórkowski, A., Ogrodniczuk, M. (eds.) Advances in Natural Language Processing, NLP 2014, Lecture Notes in Computer Science, vol. 8686, pp. 105–11. Springer, Cham (2014)
Rahman, N.A., Mabni, Z., Omar, N., Hanum, H.F.M., Rahim, N.N.A.T.M.: A parallel Latent Semantic Indexing (LSI) algorithm for malay hadith translated document retrieval. In: Berry, M., Mohamed, A., Yap, B. (eds.) Soft Computing in Data Science, SCDS 2015, Communications in Computer and Information Science, vol. 545, pp. 154–163. Springer, Singapore (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Raczyński, D., Stanisławski, W. (2019). Use of the EPSILON Decomposition and the SVD Based LSI Techniques for Reduction of the Large Indexing Structures. In: Świątek, J., Borzemski, L., Wilimowska, Z. (eds) Information Systems Architecture and Technology: Proceedings of 39th International Conference on Information Systems Architecture and Technology – ISAT 2018. ISAT 2018. Advances in Intelligent Systems and Computing, vol 853. Springer, Cham. https://doi.org/10.1007/978-3-319-99996-8_27
Download citation
DOI: https://doi.org/10.1007/978-3-319-99996-8_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-99995-1
Online ISBN: 978-3-319-99996-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)