Design of a Cross-Media Indexing System

Abstract

There is a lack of an integrated technology that will increase effective usage of the vast and heterogeneous multi-lingual and multimedia digital content. The need is being expressed insistently by endusers, and professionals in content business. The EU-IST Framework 6 Reveal-This (R-T) project aims at developing a complete and integrated content programming technology able to capture, semantically index, categorise, multimedia and multilingual digital content, whilst providing search, summarisation and translation functionalities. In order to fulfill this, the project proposes an architectural unit called Cross-Media Indexing Component (CMIC). CMIC leverages the individual potential of each indexing information generated by the analyzers of diverse modalities such as speech, text and image. It hypothesises that a system which combines and cross analyses different high-level modal descriptions of the same audio-visual content will perform better at retrieval and filtering time. The initial prototype utilises the Multiple Evidence approach by establishing links among the modality specific descriptions in order to depict topical similarity in the semantic textual space. This paper gives an overview of the project, CMIC’s enrichment approach and its support for retrieval.