Abstract
Context understanding is established from the content, analysis, and guidance from query-based coordination between users and machines. In this chapter, a live video computing (LVC) structure is presented for access of a database management of information for context assessment. Context assessment includes multimedia fusion of query-based text, images, and exploited tracks which can be utilized for content-based image retrieval (CBIR). In this chapter, we explore the developments in database systems to enable context to be utilized in user-based queries (e.g., Level 5 fusion) for information fusion content extraction. Using a common video dataset, we demonstrate time savings in the analysis from user queries to provide a context, privacy, and semantic-aware information fusion.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
E. Blasch, A. Steinberg, S. Das, J. Llinas, C.-Y. Chong, O. Kessler, E. Waltz, F. White, Revisiting the JDL model for information exploitation, in International Conference on Info Fusion (2013)
C.J. Date, An Introduction to Database Systems, 2nd edn. (Addison-Wesley Publishing Company Inc, 1977)
M.A. Tantaoui, K.A. Hua, T.T. Do, BroadCatch: a periodic broadcast technique for heterogeneous video-on-demand. Broadcast. IEEE Trans. 50(3), 289–301 (2004)
Z. Liu, E. Blasch, Z. Xue, E. Langaniere, W. Wu, Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision: a comparative survey. IEEE Trans. Pattern Analysis Mach. Intell. 34(1), 94–109 (2012)
E. Blasch, S. Plano, Cognitive fusion analysis based on context, in Proceedings of SPIE, vol. 5434 (2004)
E.P. Blasch, E. Bosse, D.A. Lambert, High-Level Information Fusion Management and Systems Design (Artech House, Norwood, MA, 2012)
S. Ezekiel, M.G. Alford, D. Ferris, E. Jones et al., Multi-scale decomposition tool for content based image retrieval, in IEEE Applied Imagery Pattern Recognition Workshop (2013)
E. Blasch, I. Kadar, K. Hintz, J. Biermann, C. Chong, S. Das, Resource management coordination with level 2/3 fusion issues and challenges. IEEE Aerosp. Electron. Syst. Mag. 23(3), 32–46 (2008)
S. Hoberman, Data Modeling Made Simple: A Practical Guide for Business and Information Technology Professionals (Technics Publications, 2005)
J. Grimes, M. Potel, What is multimedia? Comput. Graph. Appl. IEEE 11(1), 49–52 (1991)
Z. Zhang, R. Zhang, Multimedia Data Mining: A Systematic Introduction to Concepts and Theory (Chapman & Hall/CRC, 2008)
H. Ling, L. Bai, E. Blasch, X. Mei, Robust infrared vehicle tracking across target pose change using L1 regularization, in International Conference on Info Fusion (2010)
Z. Liu, E. Blasch, Z. Xue, R. Langaniere, W. Wu, Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision: a comparative survey. IEEE Trans. Pattern Anal. Mach. Intell. 34(1), 94–109 (2012)
M.K. Hu, Visual pattern recognition by moment invariants. Inf. Theory IRE Trans. 8, 179–187 (1962)
E. Bribiesca, A. Guzman, How to describe pure form and how to measure differences in shapes using shape numbers. Pattern Recogn. 12, 101–112 (1980)
M. Nixon, A.S. Aguado, Feature Extraction and Image Processing for Computer Vision (Academic Press, 2012)
C. Harris, M. Stephens, A combined corner and edge detector, in Alvey Vision Conference, (Manchester, UK, 1988) p. 50
D.G. Lowe, Object recognition from local scale-invariant features. IEEE International Conference on Computer Vision (1999), pp. 1150–1157
Y. Wu, E. Blasch, G. Chen, L. Bai, H. Ling, Multiple source data fusion via sparse representation for robust visual tracking, in International Conference on Info Fusion (2011)
H. Ling, Y. Wu, E. Blasch, G. Chen, L. Bai, Evaluation of visual tracking in extremely low frame rate wide area motion imagery, in International Conference on Info Fusion
C.R. Wren, A. Azarbayejani, T. Darrell, A.P. Pentland, Pfinder: real-time tracking of the human body. IEEE Trans. Pattern Anal. Mach. Intell. 19, 780–785 (1997)
C. Stauffer, W.E.L. Grimson, Adaptive background mixture models for real-time tracking. in IEEE Conferernce on Computer Vision and Pattern Recognition (1999)
K. Kim, T.H. Chalidabhongse, D. Harwood, L. Davis, Real-time foreground–background segmentation using codebook model. Real-Time Imaging 11, 172–185 (2005)
M. Piccardi, Background subtraction techniques: a review, in Presented at the 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE, 2004), pp. 3099–3104
S.C.S. Cheung, C. Kamath, Robust techniques for background subtraction in urban traffic video, in Presented at the Proceedings of SPIE (2004) pp. 881–892
D.H. Parks, S.S. Fels, Evaluation of background subtraction algorithms with post-processing, in Presented at the IEEE Fifth International Conference on Advanced Video and Signal Based Surveillance (IEEE, 2008), pp. 192–199
R.O. Duda, P.E. Hart, D.G. Stork, Pattern Classification and Scene Analysis, 2nd edn. (1995)
H. Samet, The Design and Analysis of Spatial Data Structures (Addison-Wesley Reading MA, 1990)
H. Samet, Foundations of Multidimensional and Metric Data Structures (Morgan Kaufmann, 2006)
R.M. Bolle, B.L. Yeo, M. Yeung, Video query: research directions. IBM J. Res. Dev. 42, 233–252 (1998)
R. Brunelli, O. Mich, C.M. Modena, A survey on the automatic indexing of video data. J. Vis. Commun. Image Represent. 10, 78–112 (1999)
C.G.M. Snoek, M. Worring, Multimodal video indexing: a review of the state-of-the-art. Multimed. Tools Appl. 25, 5–35 (2005)
Y. Wang, Z. Liu, J.C. Huang, Multimedia content analysis-using both audio and visual clues. Signal Process. Mag. IEEE 17, 12–36 (2000)
J.M. Boggs, The Art of Watching Films (ERIC, 1996)
R. Jain, A. Hampapur, Metadata in video databases. ACM Sigmod Rec. 23, 27–33 (1994)
R. Brunelli, O. Mich, C.M. Modena, A survey on the automatic indexing of video data. J. Vis. Commun. Image Represent. 10, 78–112 (1999)
A.K. Jain, R.P.W. Duin, J. Mao, Statistical pattern recognition: a review. Pattern Anal. Mach. Intell. IEEE Trans. 22, 4–37 (2000)
I. Ide, K. Yamamoto, H. Tanaka, Automatic video indexing based on shot classification. Adv. Multimed. Content Process. 87–102 (1999)
J. Foote, An overview of audio information retrieval. Multimed. Syst. 7, 2–10 (1999)
E. Wold, T. Blum, D. Keislar, J. Wheaten, Content-based classification, search, and retrieval of audio. Multimed. IEEE 3, 27–36 (1996)
J. Nievergelt, H. Hinterberger, K.C. Sevcik, The grid file: an adaptable, symmetric multikey file structure. ACM Trans. Database Syst. TODS 9, 38–71 (1984)
A.V. Aho, J.E. Hopcroft, J. Ullman, Data Structures and Algorithms (Addison-Wesley Longman Publishing Co., Inc, 1983)
T.H. Cormen, C.E. Leiserson, R.L. Rivest, C. Stein, Introduction to Algorithms. MIT press
Pieprzyk, J., Sadeghiyan, B. Design of Hashing Algorithms (Springer, New York, 2001)
J.L. Bentley, Multidimensional binary search trees used for associative searching. Commun. ACM 18, 509–517 (1975)
P. Scheuermann, M. Ouksel, Multidimensional B-trees for associative searching in database systems. Inf. Syst. 7, 123–137 (1982)
W.G. Aref, I.F. Ilyas, Sp-gist: an extensible database index for supporting space partitioning trees. J. Intell. Inf. Syst. 17, 215–240 (2001)
A. Guttman, R-trees: a dynamic index structure for spatial searching (ACM, 1984)
G. Hristescu, M. Farach-Colton, Cluster-preserving embedding of proteins. Technical Report 99-50, Computer Science Department, Rutgers University
J.T.L. Wang, X. Wang, K.I. Lin, D. Shasha, B.A. Shapiro, K. Zhang, Evaluating a class of distance-mapping algorithms for data mining and clustering, in Presented at the Proceedings of the Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (ACM, 1999), pp. 307–311
W. Aref, H. Samet, Uniquely reporting spatial objects: yet another operation for comparing spatial data structures, in Presented at the Proceedings of the Fifth International Symposium on Spatial Data Handling (1992) pp. 178–189
W.G. Aref, H. Samet, Hashing by proximity to process duplicates in spatial databases, in Presented at the Proceedings of the Third International Conference on Information and Knowledge Management (ACM, 1994), pp. 347–354
H. Samet, Spatial data structures. Mod. Database Syst. Object Model Interoperability Beyond, 361–385 (1995)
M. Flickner, H. Sawhney, W. Niblack, J. Ashley, Q. Huang, B. Dom, M. Gorkani, J. Hafner, D. Lee, D. Petkovic, Query by image and video content: the QBIC system. Computer 28, 23–32 (1995)
A. Blaser, Data base techniques for pictorial applications (Springer, Florence, 1979)
N.S. Chang, K.S. Fu, Query-by-pictorial-example. IEEE Trans. Softw. Eng. 519–524 (1980)
S.F. Chang, A. Eleftheriadis, R. McClintock, Next-generation content representation, creation, and searching for new-media applications in education. Proc. IEEE 86, 884–904 (1998)
J.R. Bach, C. Fuller, A. Gupta, A. Hampapur, B. Horowitz, R. Humphrey, R. Jain, C.F. Shu, The virage image search engine: an open framework for image management, in Presented at the SPIE Storage and Retrieval for Image and Video Databases IV (1996) pp. 76–87
A. Pentland, R.W. Picard, S. Sclaroff, Photobook: content-based manipulation of image databases. Int. J. Comput. Vis. 18, 233–254 (1996)
T. Huang, S. Mehrotra, K. Ramchandran, Multimedia analysis and retrieval system (MARS) project
Y. Rui, T.S. Huang, S.F. Chang, Image retrieval: current techniques, promising directions, and open issues. J. Vis. Commun. Image Represent. 10, 39–62 (1999)
R. Zhao, W.I. Grosky, Negotiating the semantic gap: from feature maps to semantic landscapes. Pattern Recognit. 35, 593–600 (2002)
Y. Liu, D. Zhang, G. Lu, W.Y. Ma, A survey of content-based image retrieval with high-level semantics. Pattern Recognit. 40, 262–282 (2007)
D. Durkee, Why cloud computing will never be free. Queue 8, 20 (2010)
T. Sato, T. Kanade, E.K. Hughes, M.A. Smith, S. Satoh, Video OCR: indexing digital news libraries by recognition of superimposed captions. Multimed. Syst. 7, 385–395 (1999)
P. Geetha, V. Narayanan, A survey of content-based video retrieval. J. Comput. Sci. 4, 474–486 (2008)
J. Foote, Content-based retrieval of music and audio, in Presented at the Proceedings of SPIE (1997) pp. 138–147
Z. Liu, Q. Huang, Content-based indexing and retrieval-by-example in audio, in Presented at the IEEE International Conference on Multimedia and Expo (IEEE, 2000), pp. 877–880
J. Makhoul, F. Kubala, T. Leek, D. Liu, L. Nguyen, R. Schwartz, A. Srivastava, Speech and language technologies for audio indexing and retrieval. Proc. IEEE 88, 1338–1353 (2000)
G. Bradski, The OpenCV Library. Dr Dobbs J. Software Tools (2000)
S. Taylor, Optimizing Applications for Multi-Core Processors, Using the Intel Integrated Performance Primitives (Intel Press, 2007)
E. Blasch, Z. Wang, H. Ling, K. Palaniappan, G. Chen, D. Shen, A. Aved, G. Seetharaman, Video-based activity analysis using the L1 tracker on VIRAT data, in IEEE Applied Imagery Pattern Recognition Workshop (2013)
A.J. Aved, Scene Understanding for Real Time Processing of Queries over Big Data Streaming Video. Ph.D. dissertation, University of Central Florida, 2013
H. Zha, X. He, C. Ding, H. Simon, M. Gu, Bipartite graph partitioning and data clustering, in Presented at the Proceedings of the Tenth International Conference on Information and Knowledge Management (ACM, 2001), pp. 25–32
H.W. Kuhn, The Hungarian method for the assignment problem. Nav. Res. Logist. Q. 2, 83–97 (1955)
J. Munkres, Algorithms for the assignment and transportation problems. J. Soc. Ind. Appl. Math. 5, 32–38 (1957)
G.A. Mills-Tettey, A. Stentz, M.B. Dias, The Dynamic Hungarian Algorithm for the Assignment Problem with Changing Costs (No. CMU-RI-TR-07-27) (Robotics Institute, Pittsburgh, PA, 2007)
R. Fisher, CAVIAR: context aware vision using image-based active recognition (WWW Document). URL http://homepages.inf.ed.ac.uk/rbf/CAVIAR/. Accessed 11 Dec 2011
Acknowledgments
This work is partly supported by the Air Force Office of Scientific Research (AFOSR) under the Dynamic Data Driven Application Systems program and the Air Force Research Lab.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland (outside the USA)
About this chapter
Cite this chapter
Aved, A.J., Blasch, E. (2016). Context Understanding from Query-Based Streaming Video. In: Snidaro, L., GarcÃa, J., Llinas, J., Blasch, E. (eds) Context-Enhanced Information Fusion. Advances in Computer Vision and Pattern Recognition. Springer, Cham. https://doi.org/10.1007/978-3-319-28971-7_19
Download citation
DOI: https://doi.org/10.1007/978-3-319-28971-7_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-28969-4
Online ISBN: 978-3-319-28971-7
eBook Packages: Computer ScienceComputer Science (R0)