Abstract
A system with some degree of biological plausibility is developed to categorise items from a widely used machine learning benchmark. The system uses fatiguing leaky integrate and fire neurons, a relatively coarse point model that roughly duplicates biological spiking properties; this allows spontaneous firing based on hypo-fatigue so that neurons not directly stimulated by the environment may be included in the circuit. A novel compensatory Hebbian learning algorithm is used that considers the total synaptic weight coming into a neuron. The network is unsupervised and entirely self-organising. This is relatively effective as a machine learning algorithm, categorising with just neurons, and the performance is comparable with a Kohonen map. However the learning algorithm is not stable, and behaviour decays as length of training increases. Variables including learning rate, inhibition and topology are explored leading to stable systems driven by the environment. The model is thus a reasonable next step toward a full neural memory model.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
This elevated firing in the internal subnets contrasts with the firing in those nets when inhibition is added. Figure 5c, d reflect the firing behaviour when extra inhibition is added (see “Homeostasis” section).
It should be noted that the three input task is different, and naturally performance will be lower as there is less input. Performance on the full twofold task at 20,000 cycles with 100 nets measuring by firing is 79.65 %.
References
Abbott L (1999) Lapicque’s introduction of the integrate-and-fire model neuron (1907). Brain Res 50:303–304
Ackley D, Hinton G, Sejnowski T (1985) A learning algorithm for Boltzmann machines. Cogn Sci 9:147–169
Amit D (1989) Modelling brain function: the world of attractor neural networks. Cambridge University Press, Cambridge
Athitsos V, Sclaroff S (2004) Boosting nearest neighbor classifiers for multiclass recognition. Technical report, Boston University
Bache K, Lichman M (2013) UCI machine learning repository. School of Information and Computer Science, University of California, Irvine. http://archive.ics.uci.edu/ml
Bi G, Poo M (1998) Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J Neurosci 18:24:10464–10472
Bienenstock E, Cooper L, Munro P (1982) Theory for the development of neuron selectivity: orientation specificity and binocular interaction in the visual cortex. J Neurosci 2:1:32–48
Brette R, Rudolph M, Carnevale T, Hines M, Beeman D, Bower J, Diesmann M, Morrison A, Goodman P, Harris F, Zirpe M, Natschalager T, Pecevski D, Ermentrout B, Djurfeldt M, Lansner A, Rochel O, Vieville T, Muller E, Dafison A, ElBoustani S, Destexhe A (2007) Simulation of networks of spiking neurons: a review of tools and strategies. J Comput Neurosci 23:349–398
Bush D, Philippides A, Husbands P, O’Shea M (2010) Reconciling the stdp and bcm models of synaptic plasticity in a spiking recurrent neural network. Neural Comput 22:2059–2085
Freedman D, Riesenhuber M, Poggio T, Miller E (2001) Categorical representation of visual stimuli in the primate prefrontal cortex. Science 291:312–316
Fyfe C (2005) Hebbian learning and negative feedback networks. Springer, Berlin
Hebb D (1949) The organization of behavior. Wiley, London
Hodgkin A, Huxley A (1952) A quantitative description of membrane current and its application to conduction and excitation in nerve. J Physiol 117:500–544
Hsu D, Tan A, Hsu M, Beggs J (2007) A simple spontaneously active hebbian learning model: homeostasis of activity and connectivity, and consequences for learning and epileptogensis. Phys Rev E 76:041909
Huyck C (2007) Creating hierarchical categories using cell assemblies. Connect Sci 19:1:1–24
Huyck C, Mitchell I (2013) Compensatory hebbian learning for categorisation in simulated biological neural nets. Biol Inspir Cogn Arch 6:3–7
Huyck C, Orengo V (2005) Information retrieval and categorisation using a cell assembly network. Neural Comput Appl 14:282–289
Huyck C, Parvizi A (2012) Parameter values and fatigue mechanisms for flif neurons. J Syst Cybern Inf 10:4:80–86
Izhikevich E (2004) Which model to use for cortical spiking neurons? IEEE Trans Neural Netw 15:5:1063–1070
Izhikevich E, Desai N (2003) Relating stdp to bcm. Neural Comput 15:1511–1523
Kohn A (2007) Visual adaptation: Physiology, mechanisms, and functional benefits. J Neurophysiol 97:3155–3164
Kohonen T (1997) Self-organizing maps. Springer, London
McCulloch W, Pitts W (1943) A logical calculus of ideas immanent in nervous activity. Bull Math Biophys 5:115–133
Mitchell I, Huyck C (2013) Self organising maps with a point neuron model. In 17th international conference on cognitive and neural systems
O’Reilly R (1996) The Leabra Model of Neural Interactions and Learning in the Neocortex. PhD thesis, Carnegie Mellon University, Pittsburgh, PA
Wehrens R, Buydens L (2007) Self- and super-organizing maps in R: the Kohonen package. J Stat Softw 21(5):1–9
Wolpert D, Macready W (1997) No free lunch theorems for optimization. IEEE Trans Evol Comput 1:67–82
Wu Q, Maguire L, Glackin B, Belatreche A (2006) Learning under weight constraints in networks of temporal encoding spiking neurons. Neurocomputing 69:1912–1922
Acknowledgments
Thanks to Zhijun Yang and Dan Diaper for comments on this paper.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Huyck, C.R., Mitchell, I.G. Post and pre-compensatory Hebbian learning for categorisation. Cogn Neurodyn 8, 299–311 (2014). https://doi.org/10.1007/s11571-014-9282-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11571-014-9282-4