Abstract
Event detection is used to locate the frames corresponding to events of interest in given videos. Real-world videos contain multiple events of interest, and they are rarely segmented. Existing online methods can only detect segments containing single event instances, and this is not suitable for processing videos with several event instances. There are multiple event detection methods, but they are all relatively inefficient and offline methods. To handle the online detection of several events, we propose a novel framework with three modules that are: the event proposal generation, aggregated-event representation, and refined detection modules. The first module can locate time intervals that are likely to contain target events, termed as proposals. The second module can aggregate all events before the current time to form a temporal context that will be used to generate initial detection results of multiple events. The refined detection module finally refines the results based on event proposals and object detection. The proposed method achieves a detection accuracy of 24.88% on a multi-event dataset - Charades, which is higher than state-of-the-art methods.
Supported by Natural Science Foundation of Tianjin (Grant No. 16JCYBJC42300, 17JCQNJC00100, 18JCYBJC44000, 18JCYBJC15300) and National Natural Science Foundation of China (Grant No. 6180021345, 61771340).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Buch, S., Escorcia, V., Shen, C., Ghanem, B., Carlos Niebles, J.: SST: single-stream temporal action proposals. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017
Carreira, J., Zisserman, A.: Quo vadis, action recognition? A new model and the kinetics dataset. CoRR abs/1705.07750 (2017). http://arxiv.org/abs/1705.07750
Du, T., Bourdev, L., Fergus, R., Torresani, L., Paluri, M.: Learning spatiotemporal features with 3D convolutional networks. In: IEEE International Conference on Computer Vision, pp. 4489–4497 (2015)
Gao, J., Yang, Z., Nevatia, R.: Cascaded boundary regression for temporal action detection. CoRR abs/1705.01180 (2017). http://arxiv.org/abs/1705.01180
De Geest, R., Gavves, E., Ghodrati, A., Li, Z., Snoek, C., Tuytelaars, T.: Online action detection. CoRR abs/1604.06506 (2016). http://arxiv.org/abs/1604.06506
Ghosh, P., Yao, Y., Davis, L.S., Divakaran, A.: Stacked spatio-temporal graph convolutional networks for action segmentation. CoRR abs/1811.10575 (2018). http://arxiv.org/abs/1811.10575
Lin, T., et al.: Microsoft COCO: common objects in context. CoRR abs/1405.0312 (2014). http://arxiv.org/abs/1405.0312
Piergiovanni, A.J., Ryoo, M.S.: Activity detection with latent sub-event hierarchy learning. CoRR abs/1803.06316 (2018). http://arxiv.org/abs/1803.06316
Piergiovanni, A., Ryoo, M.S.: Learning latent super-events to detect multiple activities in videos. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018
Ren, S., He, K., Girshick, R.B., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. CoRR abs/1506.01497 (2015). http://arxiv.org/abs/1506.01497
Shou, Z., et al.: Online detection of action start in untrimmed, streaming videos. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11207, pp. 551–568. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01219-9_33
Sigurdsson, G.A., Divvala, S., Farhadi, A., Gupta, A.: Asynchronous temporal fields for action recognition. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017
Simonyan, K., Zisserman, A.: Two-stream convolutional networks for action recognition in videos. CoRR abs/1406.2199 (2014). http://arxiv.org/abs/1406.2199
Tang, K., Yao, B., Fei-Fei, L., Koller, D.: Combining the right features for complex event recognition. In: The IEEE International Conference on Computer Vision (ICCV), December 2013
Xu, H., Das, A., Saenko, K.: R-C3D: region convolutional 3D network for temporal activity detection. CoRR abs/1703.07814 (2017). http://arxiv.org/abs/1703.07814
Yeung, S., Russakovsky, O., Ning, J., Andriluka, M., Mori, G., Li, F.F.: Every moment counts: dense detailed labeling of actions in complex videos. Int. J. Comput. Vis. 126(2–4), 1–15 (2018)
Zhao, Y., Xiong, Y., Wang, L., Wu, Z., Lin, D., Tang, X.: Temporal action detection with structured segment networks. CoRR abs/1704.06228 (2017). http://arxiv.org/abs/1704.06228
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Mleya, M.V. et al. (2019). Online Aggregated-Event Representation for Multiple Event Detection in Videos. In: Li, J., Wang, S., Qin, S., Li, X., Wang, S. (eds) Advanced Data Mining and Applications. ADMA 2019. Lecture Notes in Computer Science(), vol 11888. Springer, Cham. https://doi.org/10.1007/978-3-030-35231-8_36
Download citation
DOI: https://doi.org/10.1007/978-3-030-35231-8_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-35230-1
Online ISBN: 978-3-030-35231-8
eBook Packages: Computer ScienceComputer Science (R0)