Skip to main content

Advertisement

SpringerLink
Log in

Approaches to Information-Theoretic Analysis of Neural Activity

  • Published:
Biological Theory Aims and scope Submit manuscript

Abstract

Understanding how neurons represent, process, and manipulate information is one of the main goals of neuroscience. These issues are fundamentally abstract, and information theory plays a key role in formalizing and addressing them. However, application of information theory to experimental data is fraught with many challenges. Meeting these challenges has led to a variety of innovative analytical techniques, with complementary domains of applicability, assumptions, and goals.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Abbott LF, Varela JA, Sen K, Nelson SB (1997) Synaptic depression and cortical gain control. Science 275: 220–224.

    Google Scholar 

  • Abeles M, Prut Y (1996) Spatio-temporal firing patterns in the frontal cortex of behaving monkeys. Journal of Physiology Paris 90: 249–250.

    Google Scholar 

  • Amari S-I (2001) Information geometry on hierarchy of probability distributions. IEEE Transaction Information Theory 47: 1701–1711.

    Google Scholar 

  • Amigo JM, Szczepanski J, Wajnryb E, Sanchez-Vives M (2004) Estimating the entropy rate of spike trains via Lempel-Ziv complexity. Neural Computation 16: 717–736.

    Google Scholar 

  • Aronov D (2003) Fast algorithm for the metric-space analysis of simultaneous responses of multiple single neurons. Journal of Neuroscientific Methods 124: 175–179.

    Google Scholar 

  • Aronov D, Reich DS, Mechler F, Victor JD (2001) Multidimensional representation of spatial phase in V1. Investigations in Ophthalmology and Vision Science 42: 405.

    Google Scholar 

  • Baker SN, Lemon RN (2000) Precise spatiotemporal repeating patterns in monkey primary and supplementary motor areas occur at chance levels. Journal of Neurophysiology 84: 1770–1780.

    Google Scholar 

  • Berry MJ, Warland DK, Meister M (1997) The structure and precision of retinal spike trains. Proceedings of the National Academy of Sciences USA 94: 5411–5416.

    Google Scholar 

  • Bialek W, Rieke F, de Ruyter van Steveninck RR, Warland D (1991) Reading a neural code. Science 252: 1854–1857.

    Google Scholar 

  • Bliss TV, Collingridge GL (1993) A synaptic model of memory: long-term potentiation in the hippocampus. Nature 361: 31–39.

    Google Scholar 

  • Bourne H, Nicoll R (1993). Molecular machines integrate coincident synaptic signals. Cell 72/Neuron 10 (Suppl), 65–85.

    Google Scholar 

  • Buracas GT, Zador AM, DeWeese MR, Albright TD (1998). Efficient discrimination of temporal patterns by motion-sensitive neurons in primate visual cortex. Neuron 20: 959–969.

    Google Scholar 

  • Carlton AG (1969). On the bias of information estimates. Psychological Bulletin 71: 108–109.

    Google Scholar 

  • Chao A, Shen TJ (2003) Nonparametric estimate of Shannon’s index of diversity when there are unseen species in a sample. Environmental and Ecological Statistics 10: 429–443.

    Google Scholar 

  • Chee-Orts MN, Optican LM (1993) Clustermethod for analysis of transmitted information in multivariate neuronal data. Biological Cybernetics 69: 29–35.

    Google Scholar 

  • Cline H (1997) Coincidence detection in the nervous system. Trends in Neurosciences 19: 566–567.

    Google Scholar 

  • Dan Y, Alonso JM, Usrey WM, Reid RC (1998) Coding of visual information by precisely correlated spikes in the lateral geniculate nucleus. Nature Neuroscience 1: 501–507.

    Google Scholar 

  • de Ruyter van Steveninck RR, Lewen GD, Strong SP, Koberle R, Bialek W (1997). Reproducibility and variability in neural spike trains. Science 275: 1805–1808.

    Google Scholar 

  • Di Lorenzo PM, Victor JD (2003) Taste response variability and temporal coding in the nucleus of the solitary tract of the rat. Journal of Neurophysiology 90: 1418–1431.

    Google Scholar 

  • Efron B (1982) The Jackknife, the Bootstrap and Other Resampling Plans.Philadelphia: SIAM.

    Google Scholar 

  • Efron B, Tibshirani RJ (1998) An Introduction to the Bootstrap. Monographs on Statistics and Applied Probability, Vol. 57, 436. Boca Raton, FL: Chapman and Hall/CRC Press.

    Google Scholar 

  • Farach M, Noordewier M, Savari S, Shepp L, Wyner A, Ziv J (1995) Ontheentropy of DNA: Algorithms and measurements based on memory and rapid convergence. Proceedings of the 6th Annual ACM-SIAM Symposium on Discrete Algorithms, 48–57.

  • Gawne TJ (2000) The simultaneous coding of orientation and contrast in the responses of V1 complex cells. Experimental Brain Research 133: 293–302.

    Google Scholar 

  • Grassberger P (1988) Finite sample corrections to entropy and dimension estimates. Physics Letters A 128: 369–373.

    Google Scholar 

  • Grassberger P, Procaccia I (1983) Measuring the strangeness of strange attractors. Physica D 9: 189–208.

    Google Scholar 

  • Gray CM, Singer W (1989) Stimulus-specific neuronal oscillations in orientation columns of cat visual cortex. Proceedings of the National Academy of Sciences the USA 86: 1698–1702.

    Google Scholar 

  • Hirata Y, Mees AI (2003) Estimating topological entropy via a symbolic data compression technique. Physical Review E 67(2 Pt 2): 026205.

    Google Scholar 

  • Jarvis MR, Mitra PP (2001) Sampling properties of the spectrum and coherency of sequences of action potentials. Neural Computation 13: 717–749.

    Google Scholar 

  • Johnson DH, Gruner CM, Baggerly K, Seshagiri C (2001) Information-theoretic analysis of neural coding. Journal of Computational Neuroscience 10: 47–69.

    Google Scholar 

  • Kennel MB, Shlens J, Abarbanel HD, Chichilnisky EJ (2005) Estimating entropy rates with Bayesian confidence intervals. Neural Computation 17: 1531–1576.

    Google Scholar 

  • Kontoyiannis I, Algoet PH, Suhov YM, Wyner AJ (1998) Nonparametric entropy estimation for stationary processes and random fields, with applications to English text. IEEE Transactions on Information Theory 44: 1319–1327.

    Google Scholar 

  • Kozachenko LF, Leonenko NN (1987) Sample estimate of the entropy of a random vector. Problemy Peredachi Informatsii 23: 9–16.

    Google Scholar 

  • Kraskov A, Stogbauer H, Grassberger P (2004) Estimating mutual information. Physical Review E 69(6 Pt 2): 066138.

    Google Scholar 

  • Kreiman G, Krahe R, Metzner W, Koch C, Gabbiani F (2000) Robustness and variability of neuronal coding by amplitude-sensitive afferents in the weakly electric fish eigenmannia. Journal of Neurophysiology 84: 189–204.

    Google Scholar 

  • Krichevsky R, Trofimov V (1981) The performance of universal coding. IEEE Transactions on Information Theory 27: 199–207.

    Google Scholar 

  • Lestienne R, Tuckwell HC (1997) The significance of precisely replicating patterns in mammalian CNS spike trains. Neuroscience 82: 315–336.

    Google Scholar 

  • Levy WB (2000) Experiences, thoughts, and conjectures on implementing a Lempel-Ziv-type algorithm to measure information in a spike train. In: Neural Information Processing Systems Workshop on Information and Statistical Structure in Spike Trains. Breckenridge, CO December 1–2, 2000.

  • London M, Schreibman A, Hausser M, Larkum ME, Segev I (2002) The information efficacy of a synapse. Nature Neuroscience 5: 332–340.

    Google Scholar 

  • Luck SJ, Chelazzi L, Hillyard SA, Desimone R (1997) Neural mechanisms of spatial selective attention in areas V1, V2, and V4 of macaque visual cortex. Journal of Neurophysiology 77: 24–42.

    Google Scholar 

  • Markram H, Lubke J, Frotscher M, Sakmann B (1997) Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science 275: 213–215.

    Google Scholar 

  • Marmarelis PZ, Marmarelis VZ (1978) Analysis of physiological systems: The white-noise approach. In: Computers in Biology and Medicine, 487. New York: Plenum.

    Google Scholar 

  • McClurkin JW, Optican LM, Richmond BJ, Gawne TJ (1991) Concurrent processing and complexity of temporally encoded neuronal messages in visual perception. Science 253: 675–677.

    Google Scholar 

  • McFadden JA (1965) The entropy of a point process. Journal of the Society for Industrial and Applied Mathematics 13: 988–994.

    Google Scholar 

  • Meister M, Lagnado L, Baylor DA (1995) Concerted signaling by retinal ganglion cells. Science 270: 1207–1210.

    Google Scholar 

  • Mel BW (1993) Synaptic integration in an excitable dendritic tree. Journal of Neurophysiology 70: 1086–1101.

    Google Scholar 

  • Middlebrooks JC, Clock AE, Xu L, Green DM (1994) A panoramic code for sound location by cortical neurons. Science 264: 842–844.

    Google Scholar 

  • Miller GA (1955) Note on the bias on information estimates. In: Information Theory in Psychology: Problems and Methods (Quastler H, ed), II-B, Glencoe, IL: Free Press, 95–100.

    Google Scholar 

  • Mitra PP, Pesaran B (1999) Analysis of dynamic brain imaging data. Biophysical Journal 76: 691–708.

    Google Scholar 

  • Nakahara H, Amari S (2002) Information-geometric measure for neural spikes. Neural Computation 14: 2269–2316.

    Google Scholar 

  • Nemenman I, Bialek W, de Ruyter van Steveninck R (2004) Entropy and information in neural spike trains: Progress on the sampling problem. Physical Review E 69 (5 Pt 2): 056111.

    Google Scholar 

  • Nirenberg S, Carcieri SM, Jacobs AL, Latham PE (2001) Retinal ganglion cells act largely as independent encoders. Nature 411: 698–701.

    Google Scholar 

  • Nirenberg S, Jacobs A, Fridman G, Latham P, Douglas R, Alam N, Prusky G (2006) Ruling out and ruling in neural codes. Journal of Vision 6: 889a.

    Google Scholar 

  • Nirenberg S, Latham PE (2003) Decoding neuronal spike trains: how important are correlations? Proceedings of the National Academy of Sciences USA 100: 7348–7353.

    Google Scholar 

  • Optican LM, Richmond BJ (1987) Temporal encoding of two-dimensional patterns by single units in primate inferior temporal cortex: III. Information theoretic analysis. Journal of Neurophysiology 57: 162–178.

    Google Scholar 

  • Oram MW, Wiener MC, Lestienne R, Richmond BJ (1999). Stochastic nature of precisely timed spike patterns in visual system neuronal responses. Journal of Neurophysiology 81: 3021–3033.

    Google Scholar 

  • Paninski L (2003) Estimation of entropy and mutual information. Neural Computation 15: 1191.

    Google Scholar 

  • Panzeri S, Petersen RS, Schultz SR, Lebedev M, Diamond ME (2001) The role of spike timing in the coding of stimulus location in rat somatosensory cortex. Neuron 29: 769–777.

    Google Scholar 

  • Panzeri S, Schultz SR (2001) A unified approach to the study of temporal, correlational, and rate coding. Neural Computation 13: 1311–1349.

    Google Scholar 

  • Panzeri S, Schultz SR, Treves A, Rolls ET (1999) Correlations and the encoding of information in the nervous system. Proceedings of the Royal Society of London B 266: 1001–1012.

    Google Scholar 

  • Perkel DH, Gerstein GL, Moore GP (1967) Neuronal spike trains and stochastic point processes. II. Simultaneous spike trains. Biophysical Journal 7: 419–440.

    Google Scholar 

  • Petersen RS, Panzeri S, Diamond ME (2001) Population coding of stimulus location in rat somatosensory cortex. Neuron 32: 503–514.

    Google Scholar 

  • Pfurtscheller G, Andrew C (1999) Event-related changes of band power and coherence: methodology and interpretation. Journal of Clinical Neurophysiology 16: 512–519.

    Google Scholar 

  • Quiroga RQ, Rosso OA, Basar E, Schurmann M (2001) Wavelet entropy in event-related potentials: A new method shows ordering of EEG oscillations. Biological Cybernetics 84: 291–299.

    Google Scholar 

  • Rapp PE, Zimmerman ID, Vining EP, Cohen N, Albano AM, Jimenez-Montano MA (1994) The algorithmic complexity of neural spike trains increases during focal seizures. Journal of Neuroscience 14: 4731–4739.

    Google Scholar 

  • Reich DS, Mechler F, Victor JD (2001a) Formal and attribute-specific information in primary visual cortex. Journal of Neurophysiology 85: 305–318.

    Google Scholar 

  • Reich DS, Mechler F, Victor JD (2001b) Independent and redundant information in nearby cortical neurons. Science 294: 2566–2568.

    Google Scholar 

  • Reich DS, Mechler F, Victor JD (2001c) Temporal coding of contrast in primary visual cortex: When, what, and why. Journal of Neurophysiology 85: 1039–1041.

    Google Scholar 

  • Reinagel P, Reid RC (2000) Temporal coding of visual information in the thalamus. Journal of Neuroscience 20: 5392–5400.

    Google Scholar 

  • Reynolds JH, Pasternak T, Desimone R (2000) Attention increases sensitivity of V4 neurons. Neuron 26: 703–714.

    Google Scholar 

  • Richmond BJ, Optican LM (1987) Temporal encoding of two-dimensional patterns by single units in primate inferior temporal cortex: II. Quantification of response waveform. Journal of Neurophysiology 57: 147–161.

    Google Scholar 

  • Rieke F, Warland D, de Ruyter van Steveninck R, Bialek W (1997) Spikes: Exploring the Neural Code. Cambridge, MA: MIT Press.

    Google Scholar 

  • Rissanen J (1989) Stochastic Complexity in Statistical Inquiry. Singapore: World Scientific.

    Google Scholar 

  • Rodriguez E, George N, Lachaux JP, Martinerie J, Renault B, Varela FJ (1999) Perception’s shadow: long-distance synchronization of human brain activity. Nature 397: 430–433.

    Google Scholar 

  • Roelfsema PR, Lamme VA, Spekreijse H (2004) Synchrony and covariation of firing rates in the primary visual cortex during contour grouping. Nature Neuroscience 7: 982–991.

    Google Scholar 

  • Salzman CD, Newsome WT (1994) Neural mechanisms for forming a perceptual decision. Science 264: 231–237.

    Google Scholar 

  • Samonds JM, Bonds AB (2004) From another angle: Differences in cortical coding between fine and coarse discrimination of orientation. Journal of Neurophysiology 91: 1193–1202.

    Google Scholar 

  • Samonds JM, Zhou Z, Bernard MR, Bonds AB (2006) Synchronous activity in cat visual cortex encodes collinear and cocircular contours. Journal of Neurophysiology 95: 2602–2616.

    Google Scholar 

  • Schiff ND, Kalik SF, Purpura KP (2000) Episodic dynamics of cortical processing in the ventral stream during free-viewing: Analysis of local field potentials in striate/extrastriate and inferotemporal cortices. Society for Neuroscience Abstracts 26: 1199.

    Google Scholar 

  • Schiff ND, Kalik SF, Purpura KP (2001) Sustained activity in the central thalamus and extrastriate areas during attentive visuomotor behavior: Correlation of single unit activity and local field potentials. Society for Neuroscience Abstracts 27: 1910.

    Google Scholar 

  • Schiff SJ, Aldroubi A, Unser M, Sato S (1994) Fast wavelet transformation of EEG. Electroencephalography and Clinical Neurophysiology 91: 442–455.

    Google Scholar 

  • Schiff SJ, So P, Chang T, Burke RE, Sauer T (1996) Detecting dynamical interdependence and generalized synchrony through mutual prediction in a neural ensemble. Physical Review E 54: 6708–6724.

    Google Scholar 

  • Schlens J, Kennel M, Abarbanel H, Chichilnisky EJ (2006) Estimating information rates in neural spike trans with confidence intervals. Neural Computation.

  • Schneidman E, Berry MJ II, Segev R, Bialek W (2006) Weak pairwise correlations imply strongly correlated network states in a neural population. Nature 440: 1007–1012.

    Google Scholar 

  • Schultz SR, Panzeri S (2001) Temporal correlations and neural spike train entropy. Physical Review Letters 86: 5823–5826.

    Google Scholar 

  • Schurmann T, Grassberger P (1996) Entropy estimation of symbol sequences. Chaos 6: 414–427.

    Google Scholar 

  • Sellers P (1974) On the theory and computation of evolutionary distances. SIAM Journal of Applied Mathematics 26: 787–793.

    Google Scholar 

  • Sen K, Jorge-Rivera JC, Marder E, Abbott LF (1996) Decoding synapses. Journal of Neuroscience 16: 6307–6318.

    Google Scholar 

  • Shadlen MN, Newsome WT (1998) The variable discharge of cortical neurons: Implications for connectivity, computation, and information coding. Journal of Neuroscience 18: 3870–3896.

    Google Scholar 

  • Shannon CE, Weaver W (1949) The Mathematical Theory of Communication. Urbana: University of Illinois Press.

    Google Scholar 

  • Shlens J, Field GD, Gauthier JL, Grivich MI, Petrusca D, Sher A, Litke AM, Chichilnisky EJ (2006) Probing the structure of multi-neuron firing patterns in the primate retina using maximum entropy methods. CoSyNe, Salt Lake City, UT.

    Google Scholar 

  • Shlens J, Kennel MB, Abarbanel HD, Chichilnisky EJ (in press 2006) Estimating information rates with confidence intervals in neural spike trains. Neural Computation.

  • Slepian D (1976) On bandwidth. Proceedings of the IEEE 64: 292–300.

    Google Scholar 

  • Softky W (1994) Sub-millisecond coincidence detection in active dendritic trees. Neuroscience 58: 13–41.

    Google Scholar 

  • Softky WR, Koch C (1993) The highly irregular firing of cortical cells is inconsistent with temporal integration of random EPSPs. Journal of Neuroscience 13: 334–350.

    Google Scholar 

  • Solomonoff R (1964) A formal theory of inductive inference: Part I. Information and Control 7: 1–22.

    Google Scholar 

  • Stopfer M, Bhagavan S, Smith BH, Laurent G (1997) Impaired odour discrimination on desynchronization of odour-encoding neural assemblies. Nature 390: 70–74.

    Google Scholar 

  • Strong SP, Koberle R, Ruyter van Steveninck RR, Bialek W (1998) Entropy and information in neural spike trains. Physical Review Letters 80: 197–200.

    Google Scholar 

  • Tallon C, Bertrand O, Bouchet P, Pernier J (1995) Gamma-range activity evoked by coherent visual stimuli in humans. European Journal of Neuroscience 7: 1285–1291.

    Google Scholar 

  • Theiler J, Galdrikian B, Longtin A, Farmer J (1991) Testing for nonlinearity in time series: The method of surrogate data. Los Alamos National Laboratory Preprint, LA-UR-91-3343.

  • Theiler J, Rapp PE (1996) Re-examination of the evidence for low-dimensional, nonlinear structure in the human electroencephalogram. Electroencephalography and Clinical Neurophysiology 98: 213–222.

    Google Scholar 

  • Théunissen F, Miller JP (1995) Temporal encoding in nervous systems: A rigorous definition. Journal of Computational Neuroscience 2: 149–162.

    Google Scholar 

  • Theunissen F, Roddey JC, Stufflebeam S, Clague H, Miller JP (1996) Information theoretic analysis of dynamical encoding by four identified primary sensory interneurons in the cricket cercal system. Journal of Neurophysiology 75: 1345–1364.

    Google Scholar 

  • Thomson DJ (1982) Spectrum estimation and harmonic analysis. Proceedings of the IEEE 70: 1055–1096.

    Google Scholar 

  • Treves A, Panzeri S (1995). The upward bias in measures of information derived from limited data samples. Neural Computation 7: 399–407.

    Google Scholar 

  • Usrey WM, Reppas JB, Reid RC (1998) Paired-spike interactions and synaptic efficacy of retinal inputs to the thalamus. Nature 395: 384–387.

    Google Scholar 

  • van der Togt C, Kalitzin S, Spekreijse H, Lamme VA, Super H (2006) Synchrony dynamics in monkey V1 predict success in visual detection. Cerebral Cortex 16: 136–148.

    Google Scholar 

  • Victor JD (2002). Binless strategies for estimation of information from neural data. Physical Review E 66: 51903.

    Google Scholar 

  • Victor JD (2005) Spike train metrics. Current Opinion in Neurobiology 15: 585–592.

    Google Scholar 

  • Victor JD, Purpura KP (1996a) Nature and precision of temporal coding in visual cortex: A metric-space analysis. Journal of Neurophysiology 76: 1310–1326.

    Google Scholar 

  • Victor JD, Purpura KP (1996b) Nature and precision of temporal coding in visual cortex: A metric-space analysis. Journal of Neurophysiology 76: 1310–1326.

    Google Scholar 

  • Victor JD, Purpura KP (1997) Metric-space analysis of spike trains: theory, algorithms and application. Network 8: 127–164.

    Google Scholar 

  • Willems FMJ, Shtarkov YM, Tjalkens TJ (1995) The context-tree weighting method: basic properties. IEEE Transactions on Information Theory 41: 653–664.

    Google Scholar 

  • Wolpert DH, Wolf DR (1995) Estimating functions of probability distributions from a finite set of samples. Physical Review E 52: 6841–6854.

    Google Scholar 

  • Wu M, David SV, Gallant J (2006) Complete functional characterization of sensory neurons by system identification. Annual Review of Neuroscience 29: 477–505.

    Google Scholar 

  • Wyner AD, Ziv J (1989) Some asymptotic properties of entropy of a stationary ergodic data source with applications to data compression. IEEE Transactions on Information Theory 35: 1250–1258.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Neurology and Neuroscience, Weill Medical College of Cornell University, New York, NY, USA

    Jonathan D. Victor

Authors
  1. Jonathan D. Victor
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jonathan D. Victor.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Victor, J.D. Approaches to Information-Theoretic Analysis of Neural Activity. Biol Theory 1, 302–316 (2006). https://doi.org/10.1162/biot.2006.1.3.302

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1162/biot.2006.1.3.302

Keywords

Search

Navigation