Towards an Architecture for Managing Big Semantic Data in Real-Time

Abstract

Big Data Management has become a critical task in many application systems, which usually rely on heavyweight batch processes to process large amounts of data. However, batch architectures are not an adequate choice for the design of real-time systems, where expected response times are several orders of magnitude underneath. This paper outlines the foundations for defining an architecture able to deal with such an scenario, fulfilling the specific needs of real-time systems which expose big RDF datasets. Our proposal (Solid) is a tiered architecture which separates the complexities of Big Data management from their real-time data generation and consumption. Big semantic data are stored and indexed in a compressed way following the Rdf/Hdt  proposal; while at the same time, real-time requirements are addressed using NoSQL technology. Both are efficient layers, but their approaches are quite different and their combination is not easy. Two additional layers are required to achieve an overall high performance, satisfying real-time needs, and able to work even in a mobile context.

This work has been partially funded by the Spanish Ministry of Economy and Competitiveness through Projects TIN2012-31104, TIN2009-13838 and TIN2009-14009-C02-0; and also by Chilean Fondecyt Grant 1-110066, the Regional Government of Castilla y Leon and the ESF.