Abstract
This article presents an approach to performing the task of visual search in the context of descriptive topological spaces. The presented algorithm forms the basis of a descriptive visual search system (DVSS) that is based on the guided search model (GSM) that is motivated by human visual search. This model, in turn, consists of the bottom-up and top-down attention models and is implemented within the DVSS in three distinct stages. First, the bottom-up activation process is used to generate saliency maps and to identify salient objects. Second, perceptual objects, defined in the context of descriptive topological spaces, are identified and associated with feature vectors obtained from a VGG deep learning convolutional neural network. Lastly, the top-down activation process makes decisions on whether the object of interest is present in a given image through the use of descriptive patterns within the context of a descriptive topological space. The presented approach is tested with images from the ImageNet ILSVRC2012 and SIMPLIcity datasets. The contribution of this article is a descriptive pattern-based visual search algorithm.
Jiajie Yu—This research has been supported by the Natural Sciences and Engineering Research Council of Canada (NSERC) Discovery Grant 418413, and the Faculty of Graduate Studies at the University of Winnipeg. Also, special thanks to Keith Massey for developing the code that produced the VGG object descriptions.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The term perceptual object has specific meaning in descriptive set theory and perceptual systems. Hence, we will use visual object to represent any salient object in an FOV.
References
Yu, Y., Mann, G.K.I., Gosine, R.G.: A goal-directed visual perception system using object-based top-down attention. IEEE Trans. Auton. Ment. Dev. 4(1), 87–103 (2012)
Duncan, J., Humphreys, G.W.: Visual search and stimulus similarity. Psychol. Rev. 96(3), 433 (1989)
Wolfe, J.M.: Guided search 2.0 a revised model of visual search. Psychon. Bull. Rev. 1, 202–238 (1994)
Peters, J.F., Naimpally, S.A.: Applications of near sets. Not. Am. Math. Soc. 59(4), 536–542 (2012)
Naimpally, S.A., Peters, J.F.: Topology with Applications Topological Spaces via Near and Far. World Scientific, Singapore (2013)
Peters, J.F.: Topology of Digital Images. Visual Pattern Discovery in Proximity Spaces. Intelligent Systems Reference Library, vol. 63. Springer, Berlin (2014). https://doi.org/10.1007/978-3-642-53845-2
Peters, J.: Computational Proximity: Excursions in the Topology of Digital Images. Intelligent Systems Reference Library. Springer, Berlin (2016). https://doi.org/10.1007/978-3-319-30262-1
Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: Advances in Neural Information Processing Systems, pp. 545–552 (2006)
Yu, J.: A descriptive topological framework for performing visual search, Masters thesis, University of Winnipeg (2017)
Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)
Treisman, A.M., Gelade, G.: A feature-integration theory of attention. Cogn. Psychol. 12(1), 97–136 (1980)
Dismone, R., Duncan, J.: Neural mechanisms of selective visual attention. Annu. Rev. Neurosci. 18, 193–222 (1995)
Duncan, J., Humphreys, G., Ward, R.: Competitive brain activity in visual attention. Curr. Opin. Neurobiol. 7(2), 255–261 (1997)
Fink, G.R., Dolan, R.J., Halligan, P.W., Marshall, J.C., Frith, C.D.: Space-base and object-based visual attention: shared and specific neural domains. Brain 120(11), 2013–2028 (1997)
Peters, J.F.: Near sets. In: Henry, C.J. (ed.) Wikipedia, The Free Encyclopaedia (2015)
Sossinsky, A.B.: Tolerance space theory and some applications. Acta Applicandae Mathematicae: Int. Surv. J. Appl. Math. Math. Appl. 5(2), 137–167 (1986)
Poincaré, H.: Science and Hypothesis. The Mead Project, Brock University (1905). L. G. Ward’s translation
Benjamin Jr., L.T.: A Brief History of Modern Psychology. Blackwell Publishing, Malden (2007)
Hergenhahn, B.R.: An Introduction to the History of Psychology. Wadsworth Publishing, Belmont (2009)
Zeeman, E.C.: The topology of the brain and the visual perception. In: Fort, K.M. (ed.) Topoloy of 3-Manifolds and Selected Topics, pp. 240–256. Prentice Hall, New Jersey (1965)
Naimpally, S.A.: Near and far. A centennial tribute to Frigyes Riesz. Siberian Electron. Math. Rep. 6, A.1–A.10 (2009)
Naimpally, S.A., Warrack, B.D.: Proximity spaces. In: Cambridge Tract in Mathematics No. 59. Cambridge University Press, Cambridge (1970)
Pawlak, Z., Peters, J.F.: Jak Blisko (how near). Systemy Wspomagania Decyzji I, 57–109 (2002)
Mozzochi, C.J., Naimpally, S.A.: Uniformity and proximity. In: Allahabad Mathematical Society Lecture Note Series, vol. 2, p. 153 pp. The Allahabad Mathematical Society, Allahabad (2009)
Naimpally, S.A.: Proximity Approach to Problems in Topology and Analysis. Oldenburg Verlag, München (2009). ISBN 978-3-486-58917-7
Hocking, J.G., Naimpally, S.A.: Nearness-a better approach to continuity and limits. In: Allahabad Mathematical Society Lecture Note Series, vol. 3, p. 153 pp. The Allahabad Mathematical Society (2009)
Peters, J.F.: Near sets. General theory about nearness of objects. Appl. Math. Sci. 1(53), 2609–2629 (2007)
Peters, J.F.: Near sets. Special theory about nearness of objects. Fundamenta Informaticae 75(1–4), 407–433 (2007)
Peters, J.F.: Tolerance near sets and image correspondence. Int. J. Bio-Inspired Comput. 1(4), 239–245 (2009)
Peters, J.F.: Corrigenda and addenda: tolerance near sets and image correspondence. Int. J. Bio-Inspired Comput. 2(5), 310–318 (2010)
İnan, E., Öztürk, M.A.: Near groups on nearness approximation spaces. Hacettepe J. Math. Stat. 41(4), 545–558 (2012)
Peters, J.F., İnan, E., Öztürk, M.A.: Spatial and descriptive isometries in proximity spaces. Gen. Math. Notes 21(2), 1–10 (2014)
Peters, J.F., Wasilewski, P.: Foundations of near sets. Inf. Sci. 179(18), 3091–3109 (2009)
Peters, J.F.: Classification of perceptual objects by means of features. Int. J. Inf. Technol. Intell. Comput. 3(2), 1–35 (2008)
Li, F., Karpathy, A.: CS231n: Convolutional Neural Networks for Visual Recognition, Course Lecture Notes, Standford University (2015)
Henry, C.J., Smith, G.: Proximity system: a description-based system for quantifying the nearness or apartness of visual rough sets. In: Peters, J.F., Skowron, A. (eds.) Transactions on Rough Sets XVII. LNCS, vol. 8375, pp. 48–73. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-642-54756-0_3
Henry, C.J.: Near sets: theory and applications, Ph.D. thesis, University of Manitoba (2010)
Henry, C.J.: Metric free nearness measure using description-based neighbourhoods. Math. Comput. Sci. 7(1), 51–69 (2013)
Peters, J.F.: Computational proximity. In: Computational Proximity. ISRL, vol. 102, pp. 1–62. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-30262-1_1
C̆ech, E.: Topological Spaces. Wiley, London (2014). fr seminar, Brno, 1936–1939; rev. ed. Z. Frolik, M. Katĕtov
Efremovic̆, V.A.: The geometry of proximity I (in Russian). Mat. Sb. (N.S.) 31(73)(1), 189–200 (1952)
Lodato, M.: On topologically induced generalized proximity relations, Ph.D. dissertation, Rutgers University (1962). supervisor: S. Leader
Wallman, H.: Lattices and topological spaces. Ann. Math. 39(1), 112–126 (1938)
Peters, J.F.: Local near sets. Pattern discovery in proximity spaces. Math. Comput. Sci. 7(1), 87–106 (2013)
Peters, J.F., Wasilewski, P.: Tolerance spaces: origins, theoretical aspects and applications. Inf. Sci. 195, 211–225 (2012)
Henry, C.J.: Perceptual indiscernibility, rough sets, descriptively near sets, and image analysis. In: Peters, J.F., Skowron, A. (eds.) Transactions on Rough Sets XV. LNCS, vol. 7255, pp. 41–121. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-31903-7_3
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR, pp. 1–14 (2015). http://arxiv.org/abs/1409.1556
Karpathy, A.: CS231n: Convolutional Neural Networks for Visual Recognition. Stanford University, Stanford (2015)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1–9 (2012)
Perone, C.: Deep learning - convolutional neural networks and feature extraction with python (2015). http://blog.christianperone.com/2015/08/convolutional-neural-networks-and-feature-extraction-with-python//
Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Mach. Intell. 22(12), 1349–1380 (2000)
Yates-Baeza, R., Ribeiro-Neto, B.: Modern Information Retrieval. ACM Press/Pearson Addison Wesley, New York (1999)
Deng, J.D.J., et al.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2–9 (2009)
Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vision (IJCV) 115(3), 211–252 (2015)
Wang, J.Z., Li, J., Wiederholdy, G.: SIMPLIcity: semantics-sensitive integrated matching for picture libraries. In: Laurini, R. (ed.) VISUAL 2000. LNCS, vol. 1929, pp. 360–371. Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-40053-2_32
Jhanwar, N., Chaudhuri, S., Seetharaman, G., Zavidovique, B.: Content based image retrieval using motif cooccurrence matrix. Image Vis. Comput. 22(14), 1211–1220 (2004)
Subrahmanyam, M., Jonathan Wu, Q.M., Maheshwari, R.P., Balasubramanian, R.: Modified color motif co-occurrence matrix for image indexing and retrieval. Comput. Electr. Eng. 39(3), 762–774 (2013)
Vadivel, A., Sural, S., Majumdar, A.K.: An integrated color and intensity co-occurrence matrix. Pattern Recogn. Lett. 28(8), 974–983 (2007)
Lin, C.-H., Chen, R.-T., Chan, Y.-K.: A smart content-based image retrieval system based on color and texture feature. Image Vis. Comput. 27(6), 658–665 (2009). https://doi.org/10.1016/j.imavis.2008.07.004
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer-Verlag GmbH Germany, part of Springer Nature
About this chapter
Cite this chapter
Yu, J., Henry, C.J. (2019). Descriptive Topological Spaces for Performing Visual Search. In: Peters, J., Skowron, A. (eds) Transactions on Rough Sets XXI. Lecture Notes in Computer Science(), vol 10810. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-58768-3_2
Download citation
DOI: https://doi.org/10.1007/978-3-662-58768-3_2
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-58767-6
Online ISBN: 978-3-662-58768-3
eBook Packages: Computer ScienceComputer Science (R0)