Mental Search in Image Databases: Implicit Versus Explicit Content Query

Abstract

In comparison with the classic query-by-example paradigm, the “mental image search” paradigm lifts the strong assumption that the user has a relevant example at hand to start the search. In this chapter, we review different methods that implement this paradigm, originating from both the content-based image retrieval and the object recognition fields. In particular, we present two complementary methods. The first one allows the user to reach the target mental image by relevance feedback, using a Bayesian inference. The second one lets the user specify the mental image visual composition from an automatically generated visual thesaurus of segmented regions. In this scenario, the user formulates the query with an explicit representation of the image content, as opposed to the first scenario which accommodates an implicit representation. In terms of usage, we will show that the second approach is particularly suitable when the mental image has a well-defined visual composition. On the other hand, the Bayesian approach can handle more “semantic” queries, such as emotions for which the visual characterization is more implicit.