Biological neuron models, also known as spiking neuron models, are mathematical descriptions of the conduction of electrical signals in neurons. Neurons (or nerve cells) are electrically excitable cells within the nervous system, able to fire electric signals, called action potentials, across a neural network. These mathematical models describe the role of the biophysical and geometrical characteristics of neurons on the conduction of electrical activity. Central to these models is the description of how the membrane potential (that is, the difference in electric potential between the interior and the exterior of a biological cell) across the cell membrane changes over time. In an experimental setting, stimulating neurons with an electrical current generates an action potential (or spike), that propagates down the neuron's axon. This axon can branch out and connect to a large number of downstream neurons at sites called synapses. At these synapses, the spike can cause the release of neurotransmitters, which in turn can change the voltage potential of downstream neurons. This change can potentially lead to even more spikes in those downstream neurons, thus passing down the signal. As many as 95% of neurons in the neocortex, the outermost layer of the mammalian brain, consist of excitatory pyramidal neurons, and each pyramidal neuron receives tens of thousands of inputs from other neurons. Thus, spiking neurons are a major information processing unit of the nervous system. One such example of a spiking neuron model may be a highly detailed mathematical model that includes spatial morphology. Another may be a conductance-based neuron model that views neurons as points and describes the membrane voltage dynamics as a function of trans-membrane currents. A mathematically simpler "integrate-and-fire" model significantly simplifies the description of ion channel and membrane potential dynamics (initially studied by Lapicque in 1907).
Biological background, classification, and aims of neuron models Non-spiking cells, spiking cells, and their measurement Not all the cells of the nervous system produce the type of spike that defines the scope of the spiking neuron models. For example, cochlear hair cells, retinal receptor cells, and retinal bipolar cells do not spike. Furthermore, many cells in the nervous system are not classified as neurons but instead are classified as glia. Neuronal activity can be measured with different experimental techniques, such as the "Whole cell" measurement technique, which captures the spiking activity of a single neuron and produces full amplitude action potentials. With extracellular measurement techniques, one or more electrodes are placed in the extracellular space. Spikes, often from several spiking sources, depending on the size of the electrode and its proximity to the sources, can be identified with signal processing techniques. Extracellular measurement has several advantages:
It is easier to obtain experimentally; It is robust and lasts for a longer time; It can reflect the dominant effect, especially when conducted in an anatomical region with many similar cells. Overview of neuron models Neuron models can be divided into two categories according to the physical units of the interface of the model. Each category could be further divided according to the abstraction/detail level:
Electrical input–output membrane voltage models – These models produce a prediction for membrane output voltage as a function of electrical stimulation given as current or voltage input. The various models in this category differ in the exact functional relationship between the input current and the output voltage and in the level of detail. Some models in this category predict only the moment of occurrence of the output spike (also known as "action potential"); other models are more detailed and account for sub-cellular processes. The models in this category can be either deterministic or probabilistic. Natural stimulus or pharmacological input neuron models – The models in this category connect the input stimulus, which can be either pharmacological or natural, to the probability of a spike event. The input stage of these models is not electrical but rather has either pharmacological (chemical) concentration units, or physical units that characterize an external stimulus such as light, sound, or other forms of physical pressure. Furthermore, the output stage represents the probability of a spike event and not an electrical voltage. Although it is not unusual in science and engineering to have several descriptive models for different abstraction/detail levels, the number of different, sometimes contradicting, biological neuron models is exceptionally high. This situation is partly the result of the many different experimental settings, and the difficulty to separate the intrinsic properties of a single neuron from measurement effects and interactions of many cells (network effects). Aims of neuron models Ultimately, biological neuron models aim to explain the mechanisms underlying the operation of the nervous system. However, several approaches can be distinguished, from more realistic models (e.g., mechanistic models) to more pragmatic models (e.g., phenomenological models). Modeling helps to analyze experimental data and address questions. Models are also important in the context of restoring lost brain functionality through neuroprosthetic devices.
Electrical input–output membrane voltage models The models in this category describe the relationship between neuronal membrane currents at the input stage and membrane voltage at the output stage. This category includes (generalized) integrate-and-fire models and biophysical models inspired by the work of Hodgkin–Huxley in the early 1950s using an experimental setup that punctured the cell membrane and allowed to force a specific membrane voltage/current. Most modern electrical neural interfaces apply extra-cellular electrical stimulation to avoid membrane puncturing, which can lead to cell death and tissue damage. Hence, it is not clear to what extent the electrical neuron models hold for extra-cellular stimulation (see e.g.).
Hodgkin–Huxley
The Hodgkin–Huxley model (H&H model) is a model of the relationship between the flow of ionic currents across the neuronal cell membrane and the membrane voltage of the cell. It consists of a set of nonlinear differential equations describing the behavior of ion channels that permeate the cell membrane of the squid giant axon. Hodgkin and Huxley were awarded the 1963 Nobel Prize in Physiology or Medicine for this work. The voltage-current relationship, with multiple voltage-dependent currents charging the cell membrane of capacity Cm
C m d V ( t ) d t = − ∑ i I i ( t , V ) . {\displaystyle C_{\mathrm {m} }{\frac {dV(t)}{dt}}=-\sum _{i}I_{i}(t,V).}
The above equation is the time derivative of the law of capacitance, Q = CV where the change of the total charge must be explained as the sum over the currents. Each current is given by
I ( t , V ) = g ( t , V ) ⋅ ( V − V e q ) {\displaystyle I(t,V)=g(t,V)\cdot (V-V_{\mathrm {eq} })}
where g(t,V) is the conductance, or inverse resistance, which can be expanded in terms of its maximal conductance ḡ and the activation and inactivation fractions m and h, respectively, that determine how many ions can flow through available membrane channels. This expansion is given by
g ( t , V ) = g ¯ ⋅ m ( t , V ) p ⋅ h ( t , V ) q {\displaystyle g(t,V)={\bar {g}}\cdot m(t,V)^{p}\cdot h(t,V)^{q}}
and our fractions follow the first-order kinetics
d m ( t , V ) d t = m ∞ ( V ) − m ( t , V ) τ m ( V ) = α m ( V ) ⋅ ( 1 − m ) − β m ( V ) ⋅ m {\displaystyle {\frac {dm(t,V)}{dt}}={\frac {m_{\infty }(V)-m(t,V)}{\tau _{\mathrm {m} }(V)}}=\alpha _{\mathrm {m} }(V)\cdot (1-m)-\beta _{\mathrm {m} }(V)\cdot m}
with similar dynamics for h, where we can use either τ and m∞ or α and β to define our gate fractions. The Hodgkin–Huxley model may be extended to include additional ionic currents. Typically, these include inward Ca2+ and Na+ input currents, as well as several varieties of K+ outward currents, including a "leak" current. The result can be at the small end of 20 parameters which one must estimate or measure for an accurate model. In a model of a complex system of neurons, numerical integration of the equations are computationally expensive. Careful simplifications of the Hodgkin–Huxley model are therefore needed. The model can be reduced to two dimensions thanks to the dynamic relations which can be established between the gating variables. it is also possible to extend it to take into account the evolution of the concentrations (considered fixed in the original model).
Perfect Integrate-and-fire One of the earliest models of a neuron is the perfect integrate-and-fire model (also called non-leaky integrate-and-fire), first investigated in 1907 by Louis Lapicque. A neuron is represented by its membrane voltage V which evolves in time during stimulation with an input current I(t) according
I ( t ) = C d V ( t ) d t {\displaystyle I(t)=C{\frac {dV(t)}{dt}}}
which is just the time derivative of the law of capacitance, Q = CV. When an input current is applied, the membrane voltage increases with time until it reaches a constant threshold Vth, at which point a delta function spike occurs and the voltage is reset to its resting potential, after which the model continues to run. The firing frequency of the model thus increases linearly without bound as input current increases. The model can be made more accurate by introducing a refractory period tref that limits the firing frequency of a neuron by preventing it from firing during that period. For constant input I(t)=I the threshold voltage is reached after an integration time tint=CVthr/I after starting from zero. After a reset, the refractory period introduces a dead time so that the total time until the next firing is tref+tint . The firing frequency is the inverse of the total inter-spike interval (including dead time). The firing frequency as a function of a constant input current, is therefore
f ( I ) = I C
V t h + t r e f I . {\displaystyle \,\!f(I)={\frac {I}{C_{\mathrm {} }V_{\mathrm {th} }+t_{\mathrm {ref} }I}}.}
A shortcoming of this model is that it describes neither adaptation nor leakage. If the model receives a below-threshold short current pulse at some time, it will retain that voltage boost forever - until another input later makes it fire. This characteristic is not in line with observed neuronal behavior. The following extensions make the integrate-and-fire model more plausible from a biological point of view.
Leaky integrate-and-fire The leaky integrate-and-fire model, which can be traced back to Louis Lapicque, contains a "leak" term in the membrane potential equation that reflects the diffusion of ions through the membrane, unlike the non-leaky integrate-and-fire model. The model equation looks like
C m d V m ( t ) d t = I ( t ) − V m ( t ) R m {\displaystyle C_{\mathrm {m} }{\frac {dV_{\mathrm {m} }(t)}{dt}}=I(t)-{\frac {V_{\mathrm {m} }(t)}{R_{\mathrm {m} }}}}
where Vm is the voltage across the cell membrane and Rm is the membrane resistance. (The non-leaky integrate-and-fire model is retrieved in the limit Rm to infinity, i.e. if the membrane is a perfect insulator). The model equation is valid for arbitrary time-dependent input until a threshold Vth is reached; thereafter the membrane potential is reset. For constant input, the minimum input to reach the threshold is Ith = Vth / Rm. Assuming a reset to zero, the firing frequency thus looks like
f ( I ) = { 0 , I ≤ I t h [ t r e f − R m C m log ( 1 − V t h I R m ) ] − 1 , I > I t h {\displaystyle f(I)={\begin{cases}0,&I\leq I_{\mathrm {th} }\\\left[t_{\mathrm {ref} }-R_{\mathrm {m} }C_{\mathrm {m} }\log \left(1-{\tfrac {V_{\mathrm {th} }}{IR_{\mathrm {m} }}}\right)\right]^{-1},&I>I_{\mathrm {th} }\end{cases}}}
which converges for large input currents to the previous leak-free model with the refractory period. The model can also be used for inhibitory neurons. The most significant disadvantage of this model is that it does not contain neuronal adaptation, so that it cannot describe an experimentally measured spike train in response to constant input current. This disadvantage is removed in generalized integrate-and-fire models that also contain one or several adaptation-variables and are able to predict spike times of cortical neurons under current injection to a high degree of accuracy.
Adaptive integrate-and-fire
Neuronal adaptation refers to the fact that even in the presence of a constant current injection into the soma, the intervals between output spikes increase. An adaptive integrate-and-fire neuron model combines the leaky integration of voltage V with one or several adaptation variables wk (see Chapter 6.1. in the textbook Neuronal Dynamics)
τ m d V m ( t ) d t = R I ( t ) − [ V m ( t ) − E m ] − R ∑ k w k {\displaystyle \tau _{\mathrm {m} }{\frac {dV_{\mathrm {m} }(t)}{dt}}=RI(t)-[V_{\mathrm {m} }(t)-E_{\mathrm {m} }]-R\sum _{k}w_{k}}
τ k d w k ( t ) d t = − a k [ V m ( t ) − E m ] − w k + b k τ k ∑ f δ ( t − t f ) {\displaystyle \tau _{k}{\frac {dw_{k}(t)}{dt}}=-a_{k}[V_{\mathrm {m} }(t)-E_{\mathrm {m} }]-w_{k}+b_{k}\tau _{k}\sum _{f}\delta (t-t^{f})}
where τ m {\displaystyle \tau _{m}} is the membrane time constant, wk is the adaptation current number, with index k, τ k {\displaystyle \tau _{k}} is the time constant of adaptation current wk, Em is the resting potential and tf is the firing time of the neuron and the Greek delta denotes the Dirac delta function. Whenever the voltage reaches the firing threshold the voltage is reset to a value Vr below the firing threshold. The reset value is one of the important parameters of the model. The simplest model of adaptation has only a single adaptation variable w and the sum over k is removed.
Integrate-and-fire neurons with one or several adaptation variables can account for a variety of neuronal firing patterns in response to constant stimulation, including adaptation, bursting, and initial bursting. Moreover, adaptive integrate-and-fire neurons with several adaptation variables are able to predict spike times of cortical neurons under time-dependent current injection into the soma.
Fractional-order leaky integrate-and-fire Recent advances in computational and theoretical fractional calculus lead to a new form of model called Fractional-order leaky integrate-and-fire. An advantage of this model is that it can capture adaptation effects with a single variable. The model has the following form
I ( t ) − V m ( t ) R m = C m d α V m ( t ) d α t {\displaystyle I(t)-{\frac {V_{\mathrm {m} }(t)}{R_{\mathrm {m} }}}=C_{\mathrm {m} }{\frac {d^{\alpha }V_{\mathrm {m} }(t)}{d^{\alpha }t}}}
Once the voltage hits the threshold it is reset. Fractional integration has been used to account for neuronal adaptation in experimental data.
'Exponential integrate-and-fire' and 'adaptive exponential integrate-and-fire'
In the exponential integrate-and-fire model, spike generation is exponential, following the equation:
d V d t − R τ m I ( t ) = 1 τ m [ E m − V + Δ T exp ( V − V T Δ T ) ] . {\displaystyle {\frac {dV}{dt}}-{\frac {R}{\tau _{m}}}I(t)={\frac {1}{\tau _{m}}}\left[E_{m}-V+\Delta _{T}\exp \left({\frac {V-V_{T}}{\Delta _{T}}}\right)\right].}
where V {\displaystyle V} is the membrane potential, V T {\displaystyle V_{T}} is the intrinsic membrane potential threshold, τ m {\displaystyle \tau _{m}} is the membrane time constant, E m {\displaystyle E_{m}} is the resting potential, and Δ T {\displaystyle \Delta _{T}} is the sharpness of action potential initiation, usually around 1 mV for cortical pyramidal neurons. Once the membrane potential crosses V T {\displaystyle V_{T}} , it diverges to infinity in finite time. In numerical simulation the integration is stopped if the membrane potential hits an arbitrary threshold (much larger than V T {\displaystyle V_{T}} ) at which the membrane potential is reset to a value Vr . The voltage reset value Vr is one of the important parameters of the model. Importantly, the right-hand side of the above equation contains a nonlinearity that can be directly extracted from experimental data. In this sense the exponential nonlinearity is strongly supported by experimental evidence. In the adaptive exponential integrate-and-fire neuron the above exponential nonlinearity of the voltage equation is combined with an adaptation variable w
τ m d V d t = R I ( t ) + [ E m − V + Δ T exp ( V − V T Δ T ) ] − R w {\displaystyle \tau _{m}{\frac {dV}{dt}}=RI(t)+\left[E_{m}-V+\Delta _{T}\exp \left({\frac {V-V_{T}}{\Delta _{T}}}\right)\right]-Rw}
τ d w ( t ) d t = − a [ V m ( t ) − E m ] − w + b τ δ ( t − t f ) {\displaystyle \tau {\frac {dw(t)}{dt}}=-a[V_{\mathrm {m} }(t)-E_{\mathrm {m} }]-w+b\tau \delta (t-t^{f})}
where w denotes the adaptation current with time scale τ {\displaystyle \tau } . Important model parameters are the voltage reset value Vr, the intrinsic threshold V T {\displaystyle V_{T}} , the time constants τ {\displaystyle \tau } and τ m {\displaystyle \tau _{m}} as well as the coupling parameters a and b. The adaptive exponential integrate-and-fire model inherits the experimentally derived voltage nonlinearity of the exponential integrate-and-fire model. But going beyond this model, it can also account for a variety of neuronal firing patterns in response to constant stimulation, including adaptation, bursting, and initial bursting. However, since the adaptation is in the form of a current, aberrant hyperpolarization may appear. This problem was solved by expressing it as a conductance.
Adaptive Threshold Neuron Model In this model, a time-dependent function θ ( t ) {\displaystyle \theta (t)} is added to the fixed threshold, v t h 0 {\displaystyle v_{th0}} , after every spike, causing an adaptation of the threshold. The threshold potential, v t h {\displaystyle v_{th}} , gradually returns to its steady state value depending on the threshold adaptation time constant τ θ {\displaystyle \tau _{\theta }} . This is one of the simpler techniques to achieve spike frequency adaptation. The expression for the adaptive threshold is given by:
v t h ( t ) = v t h 0 + ∑ θ ( t − t f ) f = v t h 0 + ∑ θ 0 exp [ − ( t − t f ) τ θ ] f {\displaystyle v_{th}(t)=v_{th0}+{\frac {\sum \theta (t-t_{f})}{f}}=v_{th0}+{\frac {\sum \theta _{0}\exp \left[-{\frac {(t-t_{f})}{\tau _{\theta }}}\right]}{f}}}
where θ ( t ) {\displaystyle \theta (t)} is defined by: θ ( t ) = θ 0 exp [ − t τ θ ] {\displaystyle \theta (t)=\theta _{0}\exp \left[-{\frac {t}{\tau _{\theta }}}\right]}
When the membrane potential, u ( t ) {\displaystyle u(t)} , reaches a threshold, it is reset to v r e s t {\displaystyle v_{rest}} :
u ( t ) ≥ v t h ( t ) ⇒ v ( t ) = v rest {\displaystyle u(t)\geq v_{th}(t)\Rightarrow v(t)=v_{\text{rest}}}
A simpler version of this with a single time constant in threshold decay with an LIF neuron is realized in to achieve LSTM like recurrent spiking neural networks to achieve accuracy nearer to ANNs on few spatio temporal tasks.
Double Exponential Adaptive Threshold (DEXAT) The DEXAT neuron model is a flavor of adaptive neuron model in which the threshold voltage decays with a double exponential having two time constants. Double exponential decay is governed by a fast initial decay and then a slower decay over a longer period of time. This neuron used in SNNs through surrogate gradient creates an adaptive learning rate yielding higher accuracy and faster convergence, and flexible long short-term memory compared to existing counterparts in the literature. The membrane potential dynamics are described through equations and the threshold adaptation rule is:
v t h ( t ) = b 0 + β 1 b 1 ( t ) + β 2 b 2 ( t ) {\displaystyle v_{th}(t)=b_{0}+\beta _{1}b_{1}(t)+\beta _{2}b_{2}(t)}
The dynamics of b 1 ( t ) {\displaystyle b_{1}(t)} and b 2 ( t ) {\displaystyle b_{2}(t)} are given by
b 1 ( t + δ t ) = p j 1 b 1 ( t ) + ( 1 − p j 1 ) z ( t ) δ ( t ) {\displaystyle b_{1}(t+\delta t)=p_{j1}b_{1}(t)+(1-p_{j1})z(t)\delta (t)} ,
b 2 ( t + δ t ) = p j 2 b 2 ( t ) + ( 1 − p j 2 ) z ( t ) δ ( t ) {\displaystyle b_{2}(t+\delta t)=p_{j2}b_{2}(t)+(1-p_{j2})z(t)\delta (t)} , where p j 1 = exp [ − δ t τ b 1 ] {\displaystyle p_{j1}=\exp \left[-{\frac {\delta t}{\tau _{b1}}}\right]} and p j 2 = exp [ − δ t τ