Coherence detection in a spiking neuron via hebbian learning
and more learning
a comparison with SparseNet (see Publications/Perrinet06cns)
a complete code architecture (see SparseHebbianLearning)
get a preprint
It is generally assumed that neurons in the central nervous system communicate through temporal firing patterns. As a first step, we will study the learning of a layer of realistic neurons in the particular case where the relevant messages are formed by temporally correlated patterns, or synfire patterns. The model is a layer of Integrate-and-Fire (IF) neurons with synaptic current dynamics that adapts by minimizing a cost according to a gradient descent scheme. This leads to a rule similar to Spike-Time Dependent Hebbian Plasticity (STDHP). Our results show that the rule that we derive is biologically plausible and leads to the detection of the coherence in the input in an unsupervised way. An application to shape recognition is shown as an illustration.
- Laurent Perrinet, Manuel Samuelides. Coherence detection in a spiking neuron via hebbian learning, URL URL2 . Neurocomputing, 44--46(C):817--22, 2002 abstractIt is generally assumed that neurons in the central nervous system communicate throughtemporal firing patterns. As a first step, we will study the learning of a layer of realisticneurons in the particular case where the relevant messages are formed by temporally cor-related patterns, or synfire patterns. The model is a layer of Integrate-and-Fire (IF) neuronswith synaptic current dynamics that adapts by minimizing a cost according to a gradientdescent scheme. This leads to a rule similar to Spike-Time Dependent Hebbian Plasticity(STDHP). Our results show that the rule that we derive is biologically plausible and leadsto the detection of the coherence in the input in an unsupervised way. An application toshape recognition is shown as an illustration..
- D. Debanne, D. Shulz, Yves Frégnac. Temporal constraints in associative synaptic plasticity in hippocampus and neocortex.. Can. J. Physiol. and Pharmacol., 73:1295--311, 1995 abstract
- Wulfram Gerstner, Larry F. Abbott. Learning Navigational Maps Through Potentiation and Modulation of Hippocampal Place Cells. Journal of Computational Neuroscience, 4:79-94, 1996 abstract
- Wulfram Gerstner, Richard Kempter, J. Leo van Hemmen, Hermann Wagner. A Neuronal Learning Rule for Sub-Millisecond Temporal Coding. Nature, 384:76-8, 1996 abstract
- Henry Markram, Misha Tsodyks. Redistribution of Synaptic Efficacy Between Neocortical Pyramidal Neurons. Nature, 382:807-810, 1996 abstract
- CC. Bell, VZ. Han, Y. Sugawara, K. Grant. Synaptic plasticity in a cerebellum-like structure depends on temporal order.. Nature, 387:278--81, 1997 abstract
- Henry Markram, J. Luebke, M. Frotscher, B. Sakmann. Regulation of Synaptic Efficacy by Concidence of Postsynaptic APs and EPSPs. Science, 275:213-215, 1997 abstract
- Walter Senn, Misha Tsodyks, Henry Markram. An Algorithm for Synaptic Modification Based on Exact Timing of Pre- and Post-Synaptic Action Potentials. In ICANN'97, pages 121--6. 1997 abstract
- G-Q Bi, Mu-ming Poo. Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type, URL . Journal of Neuroscience, 18:10464--72, 1998 abstract
- Wulfram Gerstner, Richard Kempter, J. Leo van Hemmen, Hermann Wagner. Hebbian Learning of Pulse Timing in the \emphBarn Owl Auditory Sytem. pages 353-75. abstract
- Richard Kempter, Wulfram Gerstner, J. Leo van Hemmen. Hebbian Learning and Spiking Neurons. Physical Review Letters, E 59:4498--514, 1999 abstract
- Richard Kempter, Wulfram Gerstner, J. Leo van Hemmen. Spike-based compared to rate-based hebbian learning. 1999 abstract
- Laurent Perrinet. Apprentissage hebbien d'un reseau de neurones asynchrone a codage par rang, URL . Technical report, Rapport de stage du DEA de Sciences Cognitives, CERT, Toulouse, France, 1999.
- Larry F. Abbott, Sacha B. Nelson. Synaptic plasticity: taming the beast, URL . Nature Neuroscience, 3:1178--83, 2000 abstract
- Mark C. W. van Rossum, G. Q. Bi, G. G. Turrigiano. Stable Hebbian Learning from Spike Timing-Dependent Plasticity. Journal of Neuroscience, 20(23):8812-21, 2000 abstract
- Sen Song, Kenneth D. Miller, Larry F. Abbott. Competitive Hebbian Learning Through Spike-Timing Dependent Synaptic Plasticity. Nature Neuroscience, 3:919--26, 2000 abstract
- Murat Okatan, Stephen Grossberg. Frequency-Dependent Synaptic Potentiation, Depression, and Spike Timing Induced by Hebbian Pairing in Cortical Pyramidal Neurons. Neural Networks, 2001 abstract
- C. Panchev, S. Wermter. Hebbian Spike-Timing Dependent Self-Organization in Pulsed Neural Networks. In Proceedings of World Congress on Neuroinformatics., Vienna, Austria, 2001 abstract
- Rajesh Rao, Terrence J. Sejnowski. Spike-timing-dependent Hebbian plasticity as temporal difference learning.. Neural Computation, 13:2221-37, 2001 abstract
- Hideyuki Cateau, Tomoki Fukai. A stochastic method to predict the consequence of arbitrary forms of spike-timing-dependent plasticity. Neural Computation, 2002 abstract
- Wulfram Gerstner, Werner M. Kistler. Mathematical formulations of Hebbian learning. Biological Cybernetics, 87:404---15, 2002 abstract
- Laurent Perrinet, Manuel Samuelides. Coherence detection in a spiking neuron via hebbian learning. Neurocomputing, 44--46(C):817--22, 2002 abstractIt is generally assumed that neurons in the central nervous system communicate through temporal firing patterns. As a first step, we will study the learning of a layer of realistic neurons in the particular case where the relevant messages are formed by temporally cor- related patterns, or synfire patterns. The model is a layer of Integrate-and-Fire (IF) neurons with synaptic current dynamics that adapts by minimizing a cost according to a gradient descent scheme. This leads to a rule similar to Spike-Time Dependent Hebbian Plasticity (STDHP). Our results show that the rule that we derive is biologically plausible and leads to the detection of the coherence in the input in an unsupervised way. An application to shape recognition is shown as an illustration..
- Rudy Guyonneau, Rufin van Rullen, Simon J. Thorpe. Neurons Tune to the Earliest Spikes Through STDP, URL . Neural Computation, 17(4):859--79, 2005 abstractSpike timing-dependent plasticity (STDP) is a learning rule that modifies the strength of a neuron's synapses as a function of the precise temporal relations between input and output spikes. In many brains areas, temporal aspects of spike trains have been found to be highly reproducible. How will STDP affect a neuron's behavior when it is repeatedly presented with the same input spike pattern? We show in this theoretical study that repeated inputs systematically lead to a shaping of the neuron's selectivity, emphasizing its very first input spikes, while steadily decreasing the postsynaptic response latency. This was obtained under various conditions of background noise, and even under conditions where spiking latencies and firing rates, or synchrony, provided conflicting informations. The key role of first spikes demonstrated here provides further support for models using a single wave of spikes to implement rapid neural processing..
- Eugene M Izhikevich. Polychronization: computation with spikes. Neural Computation, 18(2):245--82, 2006 abstractWe present a minimal spiking network that can polychronize, that is, exhibit reproducible time-locked but not synchronous firing patterns with millisecond precision, as in synfire braids. The network consists of cortical spiking neurons with axonal conduction delays and spike-timing-dependent plasticity (STDP); a ready-to-use MATLAB code is included. It exhibits sleeplike oscillations, gamma (40 Hz) rhythms, conversion of firing rates to spike timings, and other interesting regimes. Due to the interplay between the delays and STDP, the spiking neurons spontaneously self-organize into groups and generate patterns of stereotypical polychronous activity. To our surprise, the number of coexisting polychronous groups far exceeds the number of neurons in the network, resulting in an unprecedented memory capacity of the system. We speculate on the significance of polychrony to the theory of neuronal group selection (TNGS, neural Darwinism), cognitive neural computations, binding and gamma rhythm, mechanisms of attention, and consciousness as "attention to memories.".
- Patrick J. Drew, Larry F. Abbott. Extending the effects of spike-timing-dependent plasticity to behavioral timescales. Proceedings of the National Academy of Sciences USA, 103(23):8876 -- 8881, 2006 abstractActivity-dependent modification of synaptic strengths due to spike-timing-dependent plasticity (STDP) is sensitive to correla- tions between pre- and postsynaptic firing over timescales of tens of milliseconds. Temporal associations typically encountered in behavioral tasks involve times on the order of seconds. To relate the learning of such temporal associations to STDP, we must account for this large discrepancy in timescales. We show that the gap between synaptic and behavioral timescales can be bridged if the stimuli being associated generate sustained responses that vary appropriately in time. Synapses between neurons that fire this way can be modified by STDP in a manner that depends on the temporal ordering of events separated by several seconds even though the underlying plasticity has a much smaller temporal window..
- Tim Masquelier, Simon J. Thorpe. Unsupervised Learning of Visual Features through Spike Timing Dependent Plasticity.. PLoS Computational Biology, 3(2):e31, 2007 abstractSpike timing dependent plasticity (STDP) is a learning rule that modifies synaptic strength as a function of the relative timing of pre- and postsynaptic spikes. When a neuron is repeatedly presented with similar inputs, STDP is known to have the effect of concentrating high synaptic weights on afferents that systematically fire early, while postsynaptic spike latencies decrease. Here we use this learning rule in an asynchronous feedforward spiking neural network that mimics the ventral visual pathway and shows that when the network is presented with natural images, selectivity to intermediate-complexity visual features emerges. Those features, which correspond to prototypical patterns that are both salient and consistently present in the images, are highly informative and enable robust object recognition, as demonstrated on various classification tasks. Taken together, these results show that temporal codes may be a key to understanding the phenomenal processing speed achieved by the visual system and that STDP can lead to fast and selective responses..
- Taro Toyoizumi, Jean-Pascal Pfister, Kazuyuki Aihara, Wulfram Gerstner. Optimality model of unsupervised spike-timing-dependent plasticity: synaptic memory and weight distribution.. Neural Computation, 19(3):639--71, 2007 abstractWe studied the hypothesis that synaptic dynamics is controlled by three basic principles: (1) synapses adapt their weights so that neurons can effectively transmit information, (2) homeostatic processes stabilize the mean firing rate of the postsynaptic neuron, and (3) weak synapses adapt more slowly than strong ones, while maintenance of strong synapses is costly. Our results show that a synaptic update rule derived from these principles shares features, with spike-timing-dependent plasticity, is sensitive to correlations in the input and is useful for synaptic memory. Moreover, input selectivity (sharply tuned receptive fields) of postsynaptic neurons develops only if stimuli with strong features are presented. Sharply tuned neurons can coexist with unselective ones, and the distribution of synaptic weights can be unimodal or bimodal. The formulation of synaptic dynamics through an optimality criterion provides a simple graphical argument for the stability of synapses, necessary for synaptic memory..
All material (c) L. Perrinet. Please check the copyright notice.