Graphics-Based Retrieval of Color Image Databases Using Hand-Drawn Query Sketches

Abstract

This paper presents a novel approach to graphics-based information retrieval validated with an experimental system that is able to perform integrated shape and color based image retrieval with hand-drawn sketches which can be presented in rotation-, scale-, and translation-invariant mode. Due to the use of Hidden Markov Models (HMMs), an elastic matching of shapes can be performed, which allows the retrieval of shapes by applying simple sketches. Since these sketches represent hand-made line drawings and can be augmented with color features, the resulting user query represents a complex graphics structure that has to be analyzed for retrieving the image database. The database elements (mostly images of hand tools) are represented by HMMs which have been modified in order to achieve the desired rotation invariance property. Invariance with respect to scaling and translation is achieved by the feature extraction, which is a polar sampling technique, with the center of the sampling raster positioned at the shapes’s center of gravity. The outcome of the feature extraction step is also known as a shape matrix, which is a shape descriptor that has been already used occasionally in image processing tasks. The image retrieval system showed good retrieval results even with unexperienced users, which is demonstrated by a number of query sketches and corresponding retrieval images in this paper.