Temporal difference learning does not always lead to STDP
It has been previously shown that a form of temporal difference (TD) learning for predicting the value of the membrane potential of a neuron, at a fixed delay after the neuron received a presynaptic spike, results in a learning rule that is very similar to hebbian STDP (Rao and Sejnowski, 2001). Since this result was obtained using a relatively complex neural model (two-compartmental, with Hodgkin-Huxley-like currents) and a simple setup (a single presynaptic spike followed by a single current pulse), we investigated whether it holds for simpler neural models and more general situations. This is relevant both for the theoretical understanding of this phenomenon and for verifying that it holds in common simulations of spiking neural networks.
We studied the same phenomenon using both integrate-and-fire (IAF) and Izhikevich neurons, through simulations and, for the IAF neuron, also analytically. Postsynaptic spikes were generated by single current pulses (as in the original study), by the irregular firing of synaptic afferents, or by a constant input current.
For the IAF neuron, we found a hebbian, STDP-like plasticity rule only when postsynaptic spikes were generated by a single current pulse and the reset potential of the neuron was positive. For the same input and negative reset potential, as well as for constant input current (regardless of the reset potential), the resulting plasticity rule was anti-hebbian.
For the Izhikevich neuron we obtained hebbian STDP-like plasticity for both constant and pulsed input current. There is a qualitative difference with respect to the IAF case because the Izhikevich neuron incorporates the dynamics of the membrane potential during the onset of the action potential. By adding an action potential of non-zero duration to the IAF model, the shape of the plasticity function changes significantly and becomes similar to hebbian STDP. This shows that the plasticity function resulted from TD learning depends critically on whether the neuron adapts its synapses to learn the shape of its action potential or not. However, the shape of the action potential is commonly considered not to carry information. When we consider just the TD learning of the sub-threshold dynamics of the membrane potential, the shape of the resulted learning function can loose its similarity with hebbian STDP.
For both neural models, in the case of irregular synaptic input there was no clear relationship between the plastic changes predicted by TD learning and the temporal delay between the pre- and postsynaptic spikes. Moreover, the sign of these plastic changes did not depend uniquely on the sign of the temporal delay.
In conclusion, TD learning in spiking neurons does not always lead to hebbian STDP.
References
1. R.P.N. Rao and T.J. Sejnowski (2001), Spike-timing-dependent hebbian plasticity as temporal difference learning, Neural Computation 13, pp. 2221-2237
Conference:
Computational and systems
neuroscience 2009, Salt Lake City, UT, United States, 26 Feb - 3 Mar, 2009.
Presentation Type:
Poster Presentation
Topic:
Poster Presentations
Citation:
(2009). Temporal difference learning does not always lead to STDP.
Front. Syst. Neurosci.
Conference Abstract:
Computational and systems
neuroscience 2009.
doi: 10.3389/conf.neuro.06.2009.03.205
Copyright:
The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers.
They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.
The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.
Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.
For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.
Received:
03 Feb 2009;
Published Online:
03 Feb 2009.