Abstract
The Blood-Oxygen-Level-Dependent (BOLD) signal of resting-state fMRI (rs-fMRI) records the temporal dynamics of intrinsic functional networks in the brain. However, existing deep learning methods applied to rs-fMRI either neglect the functional dependency between different brain regions in a network or discard the information in the temporal dynamics of brain activity. To overcome those shortcomings, we propose to formulate functional connectivity networks within the context of spatio-temporal graphs. We train a spatio-temporal graph convolutional network (ST-GCN) on short sub-sequences of the BOLD time series to model the non-stationary nature of functional connectivity. Simultaneously, the model learns the importance of graph edges within ST-GCN to gain insight into the functional connectivities contributing to the prediction. In analyzing the rs-fMRI of the Human Connectome Project (HCP, N = 1,091) and the National Consortium on Alcohol and Neurodevelopment in Adolescence (NCANDA, N = 773), ST-GCN is significantly more accurate than common approaches in predicting gender and age based on BOLD signals. Furthermore, the brain regions and functional connections significantly contributing to the predictions of our model are important markers according to the neuroscience literature.
S. Gadgil and Q. Zhao—Equal contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The code is available at https://github.com/sgadgil6/cnslab_fmri.
- 2.
The 87% accuracy in [19] on 131 HCP subjects was based on multi-modal data.
References
Allen, E., Damaraju, E., Plis, S., Erhardt, E., Eichele, T., Calhoun, V.: Tracking whole-brain connectivity dynamics in the resting state. Cereb. Cortex 24(3), 663–676 (2014). https://doi.org/10.1093/cercor/bhs352
Brown, S., Brumback, T., Tomlinson, K., et al.: The national consortium on alcohol and neurodevelopment in adolescence (NCANDA): a multisite study of adolescent development and substance use. J. Stud. Alcohol Drugs. 76(6), 895–908 (2015). https://doi.org/10.15288/jsad.2015.76.895
Buckner, R., Krienen, F., Yeo, B.: Opportunities and limitations of intrinsic functional connectivity MRI. Nat. Neurosci. 16(7), 832–837 (2013)
Conrin, S.D., et al.: From default mode network to the basal configuration: sex differences in the resting-state brain connectivity as a function of age and their clinical correlates. Front. Psychiatry 9, 365 (2018)
Covert, I., et al.: Temporal graph convolutional networks for automatic seizure detection. In: Machine Learning for HealthCare (2019)
Cui, Y., et al.: Identifying brain networks of multiple time scales via deep recurrent neural network. In: 21st International Conference, Granada, Spain, September 16–20, 2018, Proceedings, Part III, pp. 284–292 (September 2018)
Dvornek, N.C., Ventola, P., Pelphrey, K.A., Duncan, J.S.: Identifying autism from resting-state fMRI using long short-term memory networks. In: Wang, Q., Shi, Y., Suk, H.-I., Suzuki, K. (eds.) MLMI 2017. LNCS, vol. 10541, pp. 362–370. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67389-9_42
Glasser, M., et al.: A multi-modal parcellation of human cerebral cortex. Nature 536, 171–178 (2016)
Glasser, M., et al.: The minimal preprocessing pipelines for the human connectome project. NeuroImage 80, 105 (2013). https://doi.org/10.1016/j.neuroimage.2013.04.127
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: ICLR (2017)
Klein, A., Tourville, J.: 101 labeled brain images and a consistent human cortical labeling protocol. Front. Neurosci. 6, 171 (2012). https://doi.org/10.3389/fnins.2012.00171
Ktena, S.I., et al.: Distance metric learning using graph convolutional networks: application to functional brain networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10433, pp. 469–477. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66182-7_54
Li, H., Fan, Y.: Brain decoding from functional MRI using long short-term memory recurrent neural networks. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11072, pp. 320–328. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00931-1_37
Li, H., Satterthwaite, T.D., Fan, Y.: Brain age prediction based on resting-state functional connectivity patterns using convolutional neural networks. In: ISBI (2018)
Li, X., Dvornek, N.C., Zhou, Y., Zhuang, J., Ventola, P., Duncan, J.S.: Graph neural network for interpreting Task-fMRI biomarkers. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11768, pp. 485–493. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32254-0_54
Marrelec, G., et al.: Partial correlation for functional brain interactivity investigation in functional MRI. Neuroimage 32(1), 228–237 (2006)
Müller-Oehring, E., et al.: Influences of age, sex, and moderate alcohol drinking on the intrinsic functional architecture of adolescent brains. Cereb. Cortex 28, 1–15 (2017). https://doi.org/10.1093/cercor/bhx014
Smith, S.M., Vidaurre, D., Beckmann, C.F., et al.: Functional connectomics from resting-state fMRI. Trends Cogn. Sci. 17(12), 666–682 (2013)
Taghia, J., Ryali, S., et al.: Bayesian switching factor analysis for estimating time-varying functional connectivity in fMRI. NeuroImage 155, 271–290 (2017). https://doi.org/10.1016/j.neuroimage.2017.02.083
Van Essen, D.C., et al.: The WU-Minn human connectome project: an overview. Neuroimage 80, 62–79 (2013)
Weis, S., Patil, K.R., Hoffstaedter, F., Nostro, A., Yeo, B.T.T., Eickhoff, S.B.: Sex classification by resting state brain connectivity. Cereb. Cortex 30, 824 (2019). https://doi.org/10.1093/cercor/bhz129
Westlye, L., et al.: Life-span changes of the human brain white matter: diffusion tensor imaging (DTI) and volumetry. Cereb. Cortex 20, 2055–68 (2010). https://doi.org/10.1093/cercor/bhp280
Yan, S., Xiong, Y., Lin, D.: Spatial temporal graph convolutional networks for skeleton-based action recognition. In: AAAI (2018)
Yu, B., Yin, H., Zhu, Z.: Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. In: IJCAI (2018)
Zhang, Y., Bellec, P.: Functional annotation of human cognitive states using graph convolution networks. In: NeurIPS 2019 Workshop Neuro AI (2019)
Zhao, Q., Adeli, E., Pfefferbaum, A., Sullivan, E.V., Pohl, K.M.: Confounder-aware visualization of ConvNets. In: Suk, H.-I., Liu, M., Yan, P., Lian, C. (eds.) MLMI 2019. LNCS, vol. 11861, pp. 328–336. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32692-0_38
Zhao, Q., Honnorat, N., Adeli, E., Pfefferbaum, A., Sullivan, E.V., Pohl, K.M.: Variational autoencoder with truncated mixture of Gaussians for functional connectivity analysis. In: Chung, A.C.S., Gee, J.C., Yushkevich, P.A., Bao, S. (eds.) IPMI 2019. LNCS, vol. 11492, pp. 867–879. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20351-1_68
Zhao, Q., et al.: Longitudinally consistent estimates of intrinsic functional networks. Hum. Brain Mapp. 40, 2511–2528 (2019). https://doi.org/10.1002/hbm.24541
Zhao, Y., et al.: Modeling 4D fMRI data via spatio-temporal convolutional neural networks (ST-CNN). In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11072, pp. 181–189. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00931-1_21
Acknowledgment
This research was supported in part by NIH grants AA021697, AA005965, AA013521, and AA010723.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Gadgil, S., Zhao, Q., Pfefferbaum, A., Sullivan, E.V., Adeli, E., Pohl, K.M. (2020). Spatio-Temporal Graph Convolution for Resting-State fMRI Analysis. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12267. Springer, Cham. https://doi.org/10.1007/978-3-030-59728-3_52
Download citation
DOI: https://doi.org/10.1007/978-3-030-59728-3_52
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59727-6
Online ISBN: 978-3-030-59728-3
eBook Packages: Computer ScienceComputer Science (R0)
