Skip to main content
SpringerLink
Log in

Deep Learning and Deep Knowledge Representation of fMRI Data

  • Chapter
  • First Online:
Time-Space, Spiking Neural Networks and Brain-Inspired Artificial Intelligence

Part of the book series: Springer Series on Bio- and Neurosystems ((SSBN,volume 7))

Abstract

The chapter presents first background information about functional magnetic-resonance imaging (fMRI) and then introduces methods for deep learning and deep knowledge representation from fMRI data using brain-inspired SNN. These methods are applied to develop specific methods for fMRI data analysis related to cognitive processes.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 229.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 299.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 299.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. R.C. DeCharms, Application of real-time fMRI. Nat. Rev. Neurosci. 9, 720–729 (2008)

    Article  Google Scholar 

  2. J.P. Mitchell, C.N. Macrae, M.R. Banaji, Encoding specific effects of social cognition on the neural correlates of subsequent memory. J. Neurosci. 24(21), 4912–4917 (2004)

    Article  Google Scholar 

  3. K.H. Brodersen, K. Wiech, E.I. Lomakina, C.S. Lin, J.M. Buhmann, U. Bingel, I. Tracey, Decoding the perception of pain from fMRI using multivariate pattern analysis. NeuroImage 63(3), 1162–1170 (2012). https://doi.org/10.1016/j.neuroimage.2012.08.035

    Article  Google Scholar 

  4. R.B. Buxton, K. Uludağ, D.J. Dubowitz, T.T. Liu, Modeling the hemodynamic response to brain activation. NeuroImage 23(Suppl 1), S220–S233 (2004). https://doi.org/10.1016/j.neuroimage.2004.07.013

    Article  Google Scholar 

  5. S. Ogawa, T.M. Lee, A.R. Kay, D.W. Tank, Brain magnetic resonance imaging with contrast dependent on blood oxygenation. Proc. Natl. Acad. Sci. 87(24), 9868–9872 (1990)

    Article  Google Scholar 

  6. J.V. Haxby, M.I. Gobbini, M.L. Furey, A. Ishai, J.L. Schouten, P. Pietrini, Distributed and overlapping representation of faces and objects in ventral temporal cortex. Science 293(5539), 2425–2430 (2001)

    Article  Google Scholar 

  7. M.A. Lindquist, The statistical analysis of fMRI data. Stat. Sci. 23(4), 439–464 (2008). https://doi.org/10.1214/09-STS282

    Article  MathSciNet  MATH  Google Scholar 

  8. K.J. Friston, C.D. Frith, R.S. Frackowiak, R. Turner, Characterizing dynamic brain responses with fMRI: a multivariate approach. NeuroImage (1995). Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/9343599

  9. M.K. Carroll, G.A. Cecchi, I. Rish, R. Garg, A.R. Rao, Prediction and interpretation of distributed neural activity with sparse models. NeuroImage 44(1), 112–122 (2009)

    Article  Google Scholar 

  10. T. Schmah, R.S. Zemel, G.E. Hinton, S.L. Small, S.C. Strother, Comparing classification methods for longitudinal fMRI studies. Neural Comput. 22(11), 2729–2762 (2010). https://doi.org/10.1162/NECO_a_00024

    Article  MATH  Google Scholar 

  11. D.D. Cox, R.L. Savoy, Functional magnetic resonance imaging (fMRI) “brain reading”: detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage 19(2), 261–270 (2003)

    Article  Google Scholar 

  12. Y. Kamitani, F. Tong, Decoding the visual and subjective contents of the human brain. Nat. Neurosci. 8(5), 679–685 (2005)

    Article  Google Scholar 

  13. M. Misaki, Y. Kim, P.A. Bandettini, N. Kriegeskorte, Comparison of multivariate classifiers and response normalizations for pattern-information fMRI. NeuroImage 53(1), 103–118 (2010). https://doi.org/10.1016/j.neuroimage.2010.05.051.Comparison

    Article  Google Scholar 

  14. J. Mourão-Miranda, A.L.W. Bokde, C. Born, H. Hampel, M. Stetter, Classifying brain states and determining the discriminating activation patterns: support vector machine on functional MRI data. NeuroImage 28(4), 980–995 (2005). https://doi.org/10.1016/j.neuroimage.2005.06.070

    Article  Google Scholar 

  15. T.M. Mitchell, R. Hutchinson, M.A. Just, R.S. Niculescu, F. Pereira, X. Wang, in Classifying Instantaneous Cognitive States from FMRI Data. AMIA, Annual Symposium Proceedings/AMIA Symposium, AMIA Symposium (2003), pp. 465–469. Retrieved from http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1479944&tool=pmcentrez&rendertype=abstract

  16. I. Rustandi, in Classifying Multiple-Subject fMRI Data Using the Hierarchical Gaussian Naïve Bayes Classifier. 13th Conference on Human Brain Mapping (2007a), pp. 4–5

    Google Scholar 

  17. I. Rustandi, in Hierarchical Gaussian Naive Bayes Classifier for Multiple-Subject fMRI Data. Submitted to AISTATS, (1), 2–4 (2007b)

    Google Scholar 

  18. S.M. Polyn, G.J. Detre, S. Takerkart, V.S. Natu, M.S. Benharrosh, B.D. Singer, J.D. Cohen, J.V. Haxby, K.A. Norman, A Matlab-based toolbox to facilitate multi-voxel pattern classification of fMRI data (2005)

    Google Scholar 

  19. Y. Fan, D. Shen, C. Davatzikos, in Detecting Cognitive States from fMRI Images by Machine Learning and Multivariate Classification. In Conference on Computer Vision and Pattern Recognition Workshop 2006 (IEEE, 2006), pp. 89–89. https://doi.org/10.1109/cvprw.2006.64

  20. S.J. Hanson, T. Matsuka, J.V. Haxby, Combinatorial codes in ventral temporal lobe for object recognition: Haxby (2001) revisited: is there a “face” area? NeuroImage 23(1), 156–166 (2004). https://doi.org/10.1016/j.neuroimage.2004.05.020

    Article  Google Scholar 

  21. G. Yourganov, T. Schmah, N.W. Churchill, M.G. Berman, C.L. Grady, S.C. Strother, Pattern classification of fMRI data: applications for analysis of spatially distributed cortical networks. NeuroImage 96, 117–132 (2014). https://doi.org/10.1016/j.neuroimage.2014.03.074

    Article  Google Scholar 

  22. J.D. Haynes, G. Rees, Predicting the orientation of invisible stimuli from activity in human primary visual cortex. Nat. Neurosci. 8(5), 686–691 (2005). https://doi.org/10.1038/nn1445

    Article  Google Scholar 

  23. S. Ku, A. Gretton, J. Macke, N.K. Logothetis, Comparison of pattern recognition methods in classifying high-resolution BOLD signals obtained at high magnetic field in monkeys. Magn. Reson. Imaging 26(7), 1007–1014 (2008). https://doi.org/10.1016/j.mri.2008.02.016

    Article  Google Scholar 

  24. T. Schmah, G.E. Hinton, R.S. Zemel, S.L. Small, S. Strother, Generative versus discriminative training of RBMs for classification of fMRI images, in Advances in Neural Information Processing Systems, vol. 21, ed. by D. Koller, D. Schuurmans, Y. Bengio, L. Bottou (MIT Press, Cambridge, MA, 2009), pp. 1409–1416

    Google Scholar 

  25. S. LaConte, S. Strother, V. Cherkassky, J. Anderson, X. Hu, Support vector machines for temporal classification of block design fMRI data. NeuroImage 26(2), 317–329 (2005). https://doi.org/10.1016/j.neuroimage.2005.01.048

    Article  Google Scholar 

  26. N. Mørch, L. Hansen, S. Strother, C. Svarer, D. Rottenberg, B. Lautrup, in Nonlinear vs. linear models in functional neuroimaging: Learning curves and generalization crossover. Proceedings of the 15th international conference on information processing in medical imaging, volume 1230 of Lecture Notes in Computer Science (Springer, 1997) pp. 259–270

    Google Scholar 

  27. J. Mourão-Miranda, K.J. Friston, M. Brammer, Dynamic discrimination analysis: a spatial-temporal SVM. NeuroImage 36(1), 88–99 (2007). https://doi.org/10.1016/j.neuroimage.2007.02.020

    Article  Google Scholar 

  28. M.A.J. Van Gerven, B. Cseke, F.P. de Lange, T. Heskes, Efficient Bayesian multivariate fMRI analysis using a sparsifying spatio-temporal prior. NeuroImage 50(1), 150–161 (2010). https://doi.org/10.1016/j.neuroimage.2009.11.064

    Article  Google Scholar 

  29. B. Ng, R. Abugharbieh, in Modeling Spatiotemporal Structure in fMRI Brain Decoding Using Generalized Sparse Classifiers. 2011 International Workshop on Pattern Recognition in NeuroImaging (2011b), pp. 65–68. https://doi.org/10.1109/prni.2011.10

  30. P. Avesani, H. Hazan, E. Koilis, L. Manevitz, D. Sona, in Learning BOLD Response in fMRI by Reservoir Computing. 2011 International Workshop on Pattern Recognition in NeuroImaging (2011), pp. 57–60. https://doi.org/10.1109/prni.2011.16

  31. N. Kasabov, To spike or not to spike: a probabilistic spiking neuron model. Neural Netw. 23(1), 16–19 (2010). https://doi.org/10.1016/j.neunet.2009.08.010

    Article  Google Scholar 

  32. M. Åberg, L. Löken, J. Wessberg, in An Evolutionary Approach to Multivariate Feature Selection for fMRI Pattern Analysis (2008)

    Google Scholar 

  33. T. Niiniskorpi, M. Bj, J. Wessberg, in Particle Swarm Feature Selection for fMRI Pattern Classification. In BIOSIGNALS (2009), pp. 279–284

    Google Scholar 

  34. J. Kennedy, R. Eberhart, in Particle Swarm Optimization. Proceedings of ICNN’95—International Conference on Neural Networks, 4, 1942–1948. https://doi.org/10.1109/icnn.1995.488968. C. Koch, R.C. Reid, in Observatories of the mind. Nature, 483(22 March 2012), 397–398 (2012). https://doi.org/10.1038/483397a

  35. X. Wang, J. Yang, X. Teng, W. Xia, R. Jensen, Feature selection based on rough sets and particle swarm optimization. Pattern Recogn. Lett. 28(4), 459–471 (2007). https://doi.org/10.1016/j.patrec.2006.09.003

    Article  Google Scholar 

  36. S. Ryali, K. Supekar, D.A. Abrams, V. Menon, Sparse logistic regression for whole-brain classification of fMRI data. NeuroImage 51(2), 752–764 (2010)

    Article  Google Scholar 

  37. O. Yamashita, M. Sato, T. Yoshioka, F. Tong, Y. Kamitani, Sparse estimation automatically selects voxels relevant for the decoding of fMRI activity patterns. NeuroImage 42(4), 1414–1429 (2008). https://doi.org/10.1016/j.neuroimage.2008.05.050

    Article  Google Scholar 

  38. B. Ng, A. Vahdat, G. Hamarneh, R. Abugharbieh, Generalized Sparse Classifiers for Decoding Cognitive States in fMRI. Machine Learning in Medical Imaging (Springer, 2010), pp. 108–115

    Google Scholar 

  39. B. Ng, R. Abugharbieh, Generalized group sparse classifiers with application in fMRI brain decoding. Cvpr 2011, 1065–1071 (2011). https://doi.org/10.1109/CVPR.2011.5995651

    Article  Google Scholar 

  40. N. Kasabov, M. Doborjeh, Z. Doborjeh, Mapping, learning, visualization, classification, and understanding of fMRI data in the NeuCube evolving spatiotemporal data machine of spiking neural networks. IEEE Trans. Neural Netw. Learn. Syst. https://doi.org/10.1109/tnnls.2016.2612890, Manuscript Number: TNNLS-2016-P-6356, 2016

  41. R. Brette et al., Simulation of networks of spiking neurons: a review of tools and strategies. J. Comput. Neurosci. 23(3), 349–398 (2007)

    Article  MathSciNet  Google Scholar 

  42. E.M. Izhikevich, Polychronization: computation with spikes. Neural Comput. 18(2), 245–282 (2006)

    Article  MathSciNet  Google Scholar 

  43. N. Scott, N. Kasabov, G. Indiveri, in NeuCube Neuromorphic Framework for Spatio-Temporal Brain Data and Its Python Implementation. Proc. ICONIP. Springer LNCS, vol 8228 (2013), pp. 78–84

    Google Scholar 

  44. S.B. Furber, F. Galluppi, S. Temple, L.A. Plana, The SpiNNaker project. Proc. IEEE 102(5), 652–665 (2014)

    Article  Google Scholar 

  45. P.A. Merolla et al., A million spiking-neuron integrated circuit with a scalable communication network and interface. Science 345(6197), 668–673 (2014)

    Article  Google Scholar 

  46. G. Indiveri et al., Neuromorphic silicon neuron circuits. Front. Neurosci. (2011) May 2011 [Online]. Available: http://dx.doi.org/10.3389/fnins.2011.00073

  47. A. van Schaik, S.-C. Liu, AER EAR: a matched silicon cochlea pair with address event representation interface. Proc. IEEE Int. Symp. Circuits Syst. 5, 4213–4216 (2005)

    Google Scholar 

  48. T. Delbruck. jAER, accessed on 15 Oct 2014. [Online] (2007). Available: http://sourceforge.net

  49. P. Lichtsteiner, C. Posch, T. Delbruck, A dB using latency asynchronous temporal contrast vision sensor. IEEE J SolidState Circ. 43(2), 566–576 (2008)

    Article  Google Scholar 

  50. S. Song, K.D. Miller, L.F. Abbott, Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nature Neurosci. 3(9), 919–926 (2000)

    Article  Google Scholar 

  51. N. Kasabov, K. Dhoble, N. Nuntalid, G. Indiveri, Dynamic evolving spiking neural networks for on-line spatio- and spectro-temporal pattern recognition. Neural Netw. 41, 188–201 (2013)

    Article  Google Scholar 

  52. N. Kasabov, Evolving Connectionist Systems (Springer, New York, NY, USA, 2007)

    MATH  Google Scholar 

  53. S.G. Wysoski, L. Benuskova, N. Kasabov, Evolving spiking neural networks for audiovisual information processing. Neural Netw. 23(7), 819–835 (2010)

    Article  Google Scholar 

  54. W. Maass, T. Natschläger, H. Markram, Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14(11), 2531–2560 (2002)

    Article  Google Scholar 

  55. N.K. Kasabov, NeuCube: a spiking neural network architecture for mapping, learning and understanding of spatio temporal brain data. Neural Netw. 52, 62–76 (2014)

    Article  Google Scholar 

  56. J. Talairach, P. Tournoux, Co-Planar Stereotaxic Atlas of the Human Brain: 3-Dimensional Proportional System: An Approach to Cerebral Imaging (Thieme Medical Publishers, New York, NY, USA, 1998)

    Google Scholar 

  57. M. Brett, K. Christoff, R. Cusack, J. Lancaster, Using the Talairach atlas with the MNI template. NeuroImage 13(6), 85 (2001)

    Article  Google Scholar 

  58. N. Kasabov et al., Evolving spatio-temporal data machines based on the NeuCube neuromorphic framework: design methodology and selected applications. Neural Netw. 78, 1–14 (2016). https://doi.org/10.1016/j.neunet.2015.09.011.2015

    Article  Google Scholar 

  59. E.M. Izhikevich, Which model to use for cortical spiking neurons? IEEE Trans. Neural Netw. 15(5), 1063–1070 (2004)

    Article  Google Scholar 

  60. E. Tu et al., in NeuCube(ST) for Spatio-Temporal Data Predictive Modeling with a Case Study on Ecological Data, in Proceedings of International Joint Conference on Neural Networks (IJCNN), Beijing, China (2014), Jul 2014, pp. 638–645

    Google Scholar 

  61. E. Tu, N. Kasabov, J. Yang, Mapping temporal variables into the NeuCube for improved pattern recognition, predictive modeling, and understanding of stream data. IEEE Trans. Neural Netw. Learn. Syst., vol. PP(99), 1–13 (2016). https://doi.org/10.1109/tnnls.2016.2536742.2016

  62. E. Bullmore, O. Sporns, Complex brain networks: graph theoretical analysis of structural and functional systems. Nature Rev. Neurosci. 10(3), 186–198 (2009)

    Article  Google Scholar 

  63. V. Braitenberg, A. Schuz, Cortex: Statistics and Geometry of Neuronal Connectivity (Springer, Berlin, Germany, 1998)

    Book  Google Scholar 

  64. S. Schliebs, N. Kasabov, Evolving spiking neural network—a survey. Evolving Syst. 4(2), 87–98 (2013)

    Article  Google Scholar 

  65. M. Just, StarPlus fMRI data, accessed on 13 Jul 2014. [Online] (2014). Available: http://www.cs.cmu.edu/afs/cs.cmu.edu/project/theo-81/www

  66. F. Pereira, (13 Feb 2002), E-print network, accessed on 13 Jul 2014 [Online]. Available: http://www.osti.gov/eprints/topicpages/documents/record/181/3791737.html

  67. NEUCOM-KEDRI, Available: http://www.theneucom.com

  68. K.A. Johnson, J.A. Becker, in The Whole Brain Atlas. Accessed on 16 Oct 2014 [Online]. Available: http://www.med.harvard.edu/AANLIB/home.html

  69. L. Koessler et al., Automated cortical projection of EEG sensors: anatomical correlation via the international 10–10 system. NeuroImage 46(1), 64–72 (2009)

    Article  Google Scholar 

  70. S. Thorpe, J. Gautrais, in Rank Order Coding. Computational Neuroscience (Plenum Press, New York, NY, USA, 1998), pp. 113–118

    Google Scholar 

  71. M. Yuasa, K. Saito, N. Mukawa, Brain activity when reading sentences and emoticons: an fMRI study of verbal and nonverbal communication. Electron. Commun. Jpn. 94(5), 17–24 (2011)

    Article  Google Scholar 

  72. R.K. Christensen, Negative and affirmative sentences increase activation in different areas in the brain. J. Neurolinguist. 22(1), 1–17 (2009)

    Article  Google Scholar 

  73. D. Zhou, O. Bousquet, T.N. Lal, J. Weston, B. Schölkopf, Learning with local and global consistency. Proc. Adv. Neural Inf. Process. Syst. 16, 321–328 (2004)

    Google Scholar 

  74. M. Behroozi, M.R. Daliri, RDLPFC area of the brain encodes sentence polarity: a study using fMRI. Brain Imag. Behav. 9(2), 178–189 (2015)

    Article  Google Scholar 

  75. N. Kasabov, L. Zhou, M. Gholami Doborjeh, J. Yang, New algorithms for encoding, learning and classification of fMRI data in a spiking neural network architecture: a case on modelling and understanding of dynamic cognitive processes. IEEE Trans. Cogn. Dev. Syst. (2017). https://doi.org/10.1109/tcds.2016.2636291

  76. J. Sjöström, W. Gerstner, Spike-timing dependent plasticity. Front. Synaptic Neurosci. 5(2), 35–44 (2010)

    Google Scholar 

  77. T. Masquelier, R. Guyonneau, S. J. Thorpe, Spike timing dependent plasticity finds the start of repeating patterns in continuous spike trains. PLoS ONE 3(1), Art. no. e1377 (2008)

    Google Scholar 

  78. M.G. Doborjeh, E. Capecci, N. Kasabov, in Classification and Segmentation of fMRI Spatio-Temporal Brain Data with a NeuCube Evolving Spiking Neural Network Model. Proceedings of IEEE Symposium on Evolving and Autonomous Learning Systems (EALS), (Orlando, FL, USA, 2014), pp. 73–80

    Google Scholar 

  79. N. Kasabov (ed.), in Springer Handbook of Bio-/Neuroinformatics (Springer, 2014)

    Google Scholar 

  80. G.E. Hinton, R.R. Salakhutdinov, Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)

    Article  MathSciNet  Google Scholar 

  81. G.E. Hinton, Learning multiple layers of representation. Trends Cogn. Sci. 11(10), 428–434 (2007)

    Article  Google Scholar 

  82. Y. Bengio, Learning deep architectures for AI. Found. Trends Mach. Learn. 2(1), 1–127 (2009)

    Article  Google Scholar 

  83. Y. LeCun, Y. Bengio, in Convolutional Networks for Images, Speech, and Time Series. The Handbook of Brain Theory and Neural Networks, vol 3361 (MIT Press, Cambridge, MA, USA, 1995), p. 1995

    Google Scholar 

  84. J. Schmidhuber, Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)

    Article  Google Scholar 

  85. M. Just, StarPlus fMRI data. Accessed on 7 May 2016 [Online] (2016). Available: http://www.cs.cmu.edu/afs/cs.cmu.edu/project/theo-81/www/

  86. N. Sengupta, C. McNabb, N. Kasabov, B. Russel, Integrating space, time and orientation in spiking neural networks: a case study on multimodal brain data modelling. IEEE Trans. Neural Netw. Learn. Syst. (2018)

    Google Scholar 

Download references

Acknowledgements

Some of the material in this chapter is previously published as referenced in the corresponding sections. I would like to thank my co-authors of these publications Maryam Gholami, Lei Zhou, Norhanifah Murli, Jie Yang, Zohreh Gholami.

Author information

Authors and Affiliations

  1. Knowledge Engineering and Discovery Research Institute (KEDRI), Auckland University of Technology, Auckland, New Zealand

    Nikola K. Kasabov

Authors
  1. Nikola K. Kasabov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nikola K. Kasabov .

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer-Verlag GmbH Germany, part of Springer Nature

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Kasabov, N.K. (2019). Deep Learning and Deep Knowledge Representation of fMRI Data. In: Time-Space, Spiking Neural Networks and Brain-Inspired Artificial Intelligence . Springer Series on Bio- and Neurosystems, vol 7. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-57715-8_10

Download citation

Publish with us

Policies and ethics

Search

Navigation