Abstract
Pedestrian trajectory prediction in crowd scenes is very useful in many applications such as video surveillance, self-driving cars, and robotic systems; however, it remains a challenging task because of the complex interactions and uncertainties of crowd motions. In this paper, a novel trajectory prediction method called the Attention-based Interaction-aware Spatio-temporal Graph Neural Network (AST-GNN) is proposed. AST-GNN uses an Attention mechanism to capture the complex interactions among multiple pedestrians. The attention mechanism allows for a dynamic and adaptive summary of the interactions of the nearby pedestrians. When the attention matrix is obtained, it is formulated into a propagation matrix for graph neural networks. Finally, a Time-extrapolator Convolutional Neural Network (TXP-CNN) is used in the temporal dimension of the aggregated features to predict the future trajectories of the pedestrians. Experimental results on benchmark pedestrian datasets (ETH and UCY) reveal the competitive performances of AST-GNN in terms of both the final displace error (FDE) and average displacement error (ADE) as compared with state-of-the-art trajectory prediction methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alahi, A., Goel, K., Ramanathan, V., Robicquet, A., Li, F., Savarese, S.: Social LSTM: human trajectory prediction in crowded spaces. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016
Cui, H., et al.: Multimodal trajectory predictions for autonomous driving using deep convolutional networks. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 2090–2096, May 2019
Gao, J., et al.: VectorNet: encoding HD maps and agent dynamics from vectorized representation. ArXiv abs/2005.04259 (2020)
Gupta, A., Johnson, J., Fei-Fei, L., Savarese, S., Alahi, A.: Social GAN: socially acceptable trajectories with generative adversarial networks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2255–2264, June 2018
Huang, Y., Bi, H., Li, Z., Mao, T., Wang, Z.: STGAT: modeling spatial-temporal interactions for human trajectory prediction. In: The IEEE International Conference on Computer Vision (ICCV), October 2019
Kosaraju, V., Sadeghian, A., Mart ın-Mart ın, R., Reid, I., Rezatofighi, H., Savarese, S.: Social-BiGAT: multimodal trajectory forecasting using bicycle-GAN and graph attention networks. In: Advances in Neural Information Processing Systems, vol. 32, pp. 137–146 (2019)
Lefevre, S., Laugier, C., Ibanezguzman, J.: Exploiting map information for driver intention estimation at road intersections. In: 2011 IEEE Intelligent Vehicles Symposium (IV), pp. 583–588, June 2011
Lerner, A., Chrysanthou, Y., Lischinski, D.: Crowds by example. Comput. Graph. Forum 26(3), 655–664 (2007)
Li, X., Ying, X., Chuah, M.C.: GRIP: graph-based interaction-aware trajectory prediction. In: 2019 IEEE Intelligent Transportation Systems Conference (ITSC), pp. 3960–3966 (2019)
Liang, J., Jiang, L., Niebles, J.C., Hauptmann, A.G., Fei-Fei, L.: Peeking into the future: predicting future person activities and locations in videos. In: 2019IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5718–5727, June 2019
Mohamed, A., Qian, K., Elhoseiny, M., Claudel, C.: Social-STGCNN: a social spatio-temporal graph convolutional neural network for human trajectory prediction. arXiv e-prints arXiv:2002.11927 (2020)
Pellegrini, S., Ess, A., Schindler, K., van Gool, L.: You’ll never walk alone: modeling social behavior for multi-target tracking. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 261–268, September 2009
Pellegrini, S., Ess, A., Van Gool, L.: Improving data association by joint modeling of pedestrian trajectories and groupings. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6311, pp. 452–465. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15549-9_33
Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine earning (Adaptive Computation and Machine Learning). The MIT Press, Cambridge (2005)
Sadeghian, A., Kosaraju, V., Sadeghian, A., Hirose, N., Rezatofighi, H., Savarese, S.: SoPhie: an attentive GAN for predicting paths compliant to social and physical constraints. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
Toledo-Moreo, R., Zamora-Izquierdo, M.A.: Imm-based lane-change prediction in highways with low-cost gps/ins. IEEE Trans. Intell. Transp. Syst. 10(1), 180–185 (2009)
Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Li, P., Bengio, Y.: Graph attention networks. In: International Conference on Learning Representations (2018)
Xu, K., Hu, W., Leskovec, J., Jegelka, S.: How powerful are graph neural networks? In: International Conference on Learning Representations (2019)
Xu, Y., Piao, Z., Gao, S.: Encoding crowd interaction with deep neural network for pedestrian trajectory prediction. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018
Zhang, L., She, Q., Guo, P.: Stochastic trajectory prediction with social graph network. arXiv preprint arXiv:1907.10233 (2019)
Zhang, P., Ouyang, W., Zhang, P., Xue, J., Zheng, N.: SR-LSTM: state refinement for LSTM towards pedestrian trajectory prediction. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019
Acknowledgements
This work is supported partly by the National Natural Science Foundation (NSFC) of China (grants 61973301, 61972020, 61633009, 51579053, 61772373 and U1613213), partly by the National Key R&D Program of China (grants 2016YFC0300801 and 2017YFB1300202), partly by the Field Fund of the 13th Five-Year Plan for Equipment Pre-research Fund (No. 61403120301), partly by Beijing Science and Technology Plan Project, partly by the Key Basic Research Project of Shanghai Science and Technology Innovation Plan (No. 15JC1403300), partly by Beijing Science and Technology Project. (No. Z181100008918018), partly by Beijing Nova Program (No. Z201100006820046), and partly by Meituan Open R&D Fund.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhou, H., Ren, D., Xia, H., Fan, M., Yang, X., Huang, H. (2020). An Attention-Based Interaction-Aware Spatio-Temporal Graph Neural Network for Trajectory Prediction. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Communications in Computer and Information Science, vol 1333. Springer, Cham. https://doi.org/10.1007/978-3-030-63823-8_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-63823-8_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63822-1
Online ISBN: 978-3-030-63823-8
eBook Packages: Computer ScienceComputer Science (R0)