In the last decade, substantial progress has been made in multimedia creation, transmission, presentation and analysis to facilitate the development of large-scale multimedia information systems. Together with the maturation and deployment of semantic web technologies, it is now possible to build a new generation of multimedia applications that enables large-scale semantic representation, analysis, and delivery of multimedia data from heterogeneous data sources. However, there is still a long way to go for mature solutions of multimedia database systems that are capable of processing semantics-rich, large-volume of multimedia data. It could be even more challenging if such systems are under stringent functional and non-functional (e.g., QoS) requirements.

The goal of this special issue is to invite high quality scientific contributions in multimedia data semantics, with a focus on how to apply semantic technologies to the acquisition, generation, transmission, storage, processing, and retrieval of large-scale multimedia information. Discussions on future challenges in multimedia information processing, as well as practical solutions for the design and implementation of multimedia database software systems have also been encouraged. The topics of interest include, but are not limited to, practical areas that span both semantic technologies and multimedia processing and computing.

This special issue follows the very successful IEEE International Workshop on Data Semantics for Multimedia Systems and Applications (IEEE-DSMSA) held in 2009 and for which the best authors were invited to submit extended versions of their papers. An open call has been also issued resulting in a very diverse pool of twenty four (24) submissions, well spread around the world since the contact authors were from Austria (1), Belgium (1), Brazil (1), China (1), Finland (1), France (1), Germany (2), India (2), Ireland (2), Italy (1), Japan (1), Netherlands (1), Poland (1), Portugal (1), South Korean (1), Taiwan (2), Tunisia (1), United Kingdom (1) and United States (2).

Due to the interdisciplinary nature of the Call for Paper, the papers represented a wide range of approaches and scenarios that have in common the area of multimedia data semantics. All the papers were reviewed by at least three experts. We have carefully selected reviewers with track record publications and review experiences in the particular sub-area. The guest editors have sent out more than 100 review invitations and most of the invited reviewers finished their reviews on time. We are very grateful to the reviewers for their excellent job. In addition, the guest editors examined the reviews for each paper and extended discussions were performed among the organizers and reviewers to reach a consensus for each paper. In the end, six (6) papers were recommended for acceptance. These papers can be split equally into two main topics: semantic multimedia analysis and interaction and evaluation of semantic multimedia. The key ideas and contributions of these papers are summarized as below.

1 Semantic multimedia analysis

In the paper entitled “Semantic Scalability Using Tennis Videos as Examples”, Jui-Hsin Lai and Shao-Yi Chien propose a semantic scalability scheme with four levels and employ tennis videos as examples in experiments to test the scheme. Instead of detecting shot categories to determine suitable scaling options for Scalable Video Coding (SVC) as in previous studies, the proposed method analyzes a video, transmits video content according to semantic priority, and reintegrates the extracted contents in the receiver. In addition, the proposed scalability scheme can be extended to various video categories. In the paper entitled “Social Network Analysis in a Movie using Character-net”, Seung-bo Park, Kyeong-Jin Oh, and Geun-Sik Jo investigate the problem of analyzing movie stories using social network. They propose a Character-net that can represent the relationships between characters using dialogs, and a method that can extract the sequences via clustering communities composed of characters. Their experiments show that the proposed method can efficiently detect sequences. The paper “Probabilistic Semantic Component Descriptor” by Cheng-Chieh Chiang, Jia-Wei Wu and Greg C. Lee introduces the probabilistic semantic component descriptor (PSCD) for automatically extracting semantic information in a set of images. The basic idea of the PSCD is first to identify what kind of hidden semantic concepts can be associated with regions in a set of images and then to construct an image-based descriptor by integrating hidden concepts of regions in an image. Quantitative experiments have been presented to demonstrate the performance of the proposed PSCD approach.

2 Interaction and evaluation of semantic multimedia

In the paper entitled “Towards Exploratory Video Search Using Linked Data”, Jörg Waitelonis and Harald Sack propose a prototype implementation of exploratory video search and show how traditional keyword-based search can be augmented by the use of Linked Open Data. This paper highlights a strong and rigorous user evaluation. Mathias Lux revisits metrics for relevance assessment based on the MPEG-7 Semantic Description Scheme in the context of information retrieval in his paper “How to search in MPEG-7 based semantic descriptions: An evaluation of metrics”. The author evaluates them in a digital photo retrieval scenario and investigates correlation of similarity and distance metrics to user perception in a user study. This paper has been very controversial during the review process, yielding long and passionate debates, and we believe that it will also trigger more discussions within the entire community. Finally the paper “A URI-Based Approach for Addressing Fragments of Media Resources on the Web” by Erik Mannens, Davy Van Deursen, Raphaël Troncy, Silvia Pfeier, Conrad Parker, Yves Lafon, Jack Jansen, Michael Hausenblas and Rik Van de Walle, introduces Media Fragments as a media-format independent, standard means of addressing media resources using URIs. Moreover, they explain how the HTTP protocol can be used and extended to serve Media Fragments and what the impact is for current Web-enabled media formats. This paper is the result of the W3C Media Fragments Working Group’s efforts to develop the Media Fragment protocol and guidelines that will put back the video as a primary citizen in the Web.

We are very grateful for all of the contributing authors and the hard work of more than 75 reviewers. We would also like to thank the anonymous reviewers of the proposal and Editor-in-Chief Dr. Borko Furht for sharing our vision. Special thanks got to the MTAP editorial staff (Mhanilet de Leon, Laura Lander, and Emma Kalb) for their technical and administrative assistance.