Superimposed Image Description and Retrieval for Fish Species Identification

Abstract

Fish species identification is critical to the study of fish ecology and management of fisheries. Traditionally, dichotomous keys are used for fish identification. The keys consist of questions about the observed specimen. Answers to these questions lead to more questions till the reader identifies the specimen. However, such keys are incapable of adapting or changing to meet different fish identification approaches, and often do not focus upon distinguishing characteristics favored by many field ecologists and more user-friendly field guides. This makes learning to identify fish difficult for Ichthyology students. Students usually supplement the use of the key with other methods such as making personal notes, drawings, annotated fish images, and more recently, fish information websites, such as Fishbase. Although these approaches provide useful additional content, it is dispersed across heterogeneous sources and can be tedious to access. Also, most of the existing electronic tools have limited support to manage user created content, especially that related to parts of images such as markings on drawings and images and associated notes. We present SuperIDR, a superimposed image description and retrieval tool, developed to address some of these issues. It allows users to associate parts of images with text annotations. Later, they can retrieve images, parts of images, annotations, and image descriptions through text- and content-based image retrieval. We evaluated SuperIDR in an undergraduate Ichthyology class as an aid to fish species identification and found that the use of SuperIDR yielded a higher likelihood of success in species identification than using traditional methods, including the dichotomous key, fish web sites, notes, etc.