A Quick Overview of Models Neurons for Neuromorphic Applications.

There are many options to choose when deciding what single neuron model to use when constructing a neural network. Here, I have compiled a list of several choices  one can use when building neural networks. The models presented will prioritize neural models that have been, or easily can be, embedded on neuromorphic chips.

Leaky Integrate-and-Fire Neurons

With no refractory period

The leaky integrate-and-fire neuron is the simplest spiking neuron possible. It contains a leak current, and spikes whenever it reaches a specific threshold. More specifically,

$$
C_m \frac{dV}{dt} = - g_L(V-E_L) + \sum I^{syn}_j(t)
$$
$$
\text {when } V>V_{Th} \text{ then } V\rightarrow V_{reset}
$$
where $V$ is the membrane voltage, $C_m$ is the membrane capacitance, $g_L$ is the leak conductance, $E_L$ is the leak current, $V_{Th}$ is the voltage threshold, and $V_{reset}$ is the reset voltage. Here $I^{syn}_j(t)$ is the synaptic current. There are three possible models that I will describe be below.

Whenever $V$ exceeds the voltage threshold $V_{Th}$ The neuron fires a spike. This is used to calculate the synaptic current in the post synaptic neuron. The LIF model without refractory is used in IBM's TrueNorth, and Intel's Loihi. Models without refractory are easier to build with real hardward as refractory periods are difficult to construct.

With Refractory Period

The same equation as above however after the spike there is a refractory period with no spikes for $t_{re}$ milliseconds.

$$
C_m \frac{dV}{dt} = - g_L(V-E_L) + \sum I^{syn}_j(t)
$$
$$
\text {when } V>V_{Th} \text{ then } V\rightarrow V_{reset}\text{, } t\rightarrow t+t_{re}
$$

Quadratic Integrate-and-Fire

The Quadratic Integrate-and-fire model uses a quadratic non-linearity to model the upstroke of the action potential. This means the refractory period is "built in " to the model. This model also has two equilibrium points, one stable that corresponds to the rest potential, and one unstable the corresponds to the voltage threshold. Depending on the the reset potential, this model can be can be bistable (ie once it starts spiking, it does not stop until it receives an inhibitory input).


$$
C_m \frac{dV}{dt} = - k (V-V_{rest})(V-V_{thresh}) + \sum I^{syn}_j(t)
$$
$$
\text {when } V>V_{Spike} \text{ then } V\rightarrow V_{reset}
$$
where $V$ is the membrane voltage, $C_m$ is the membrane capacitance, $k$ is strength of the quadratic non-linearity, $V_R$ is the rest potential, $V_{Th}$ is the voltage threshold, $V_{Spike}$ is the voltage maximum of the action potential, and $V_{reset}$ is the reset voltage. Again $I^{syn}_j(t)$ is the synaptic current, and the three models are the same as the LIF model. This model is used by Stanford's Neurogrid , and is easy to implement in hardware, but is slower than LIF when using a numerical solver. Our lab is pursuing a simplified version of QIF.
 

 Izhikevich Neuron

The Izhikevich model neuron  adds an adaptive $W$-gate that represents a linearized potassium current, and allows for spike adaption (slower firing rate after first spike). Furthermore, it can capture sub threshold voltage oscillations, and bursting dynamics. 

$$ C_m \frac{dV}{dt} = k (V-V_R)(V-V_{Th}) - W + \sum I^{syn}_j(t)$$
$$ \frac{dW}{dt} = a(b(V-V_R) -  W)$$
$$ V > V_{spike}, V\rightarrow V_{reset}, W \rightarrow W+d$$

Again,  $V$ is the membrane voltage, $C_m$ is the membrane capacitance, $k$ is strength of the quadratic non-linearity, $V_R$ is the rest potential, $V_{Th}$ is the voltage threshold, $V_{Spike}$ is the voltage maximum of the action potential, and $V_{reset}$ is the reset voltage. Here $W$ is the linearized potassium gate, the parameter $a$ represents the time constant for the $W$ gate. It's value dictates how quickly the $W$ gate exponentially decays to rest. Likewise, $b$ represents how much voltage $V$ influences $W$'s decay, and $d$ represents the opening of potassium gates after a spike.

Again $I^{syn}_j(t)$ is the synaptic current, and the three models are the same as the LIF model. This model is used by Cambridge's Bluehive , and is easy to implement in hardware than a LIF with Refractory, but is slower when using a numerical solver than LIF or QIF.

Synapse Models

In this section I will describe three possible currents for the  $I^{syn}_j(t)$ term above. All integrate-and-fire neurons can use any of the synaptic options. 

Delta Pulse

Here the synaptic current is just a delta pulse every time the presynaptic neuron fires a spike.
 $I^{syn}_j(t) = g_{ij} \delta(t-t_{spike})$ 
whenever $g_{ij}>0$ then the synapse is excitatory, otherwise the synapse is inhibitory. This is the easiest synapse to implement, but is the least realistic. 

Synaptic Current

These are currents that exponentially decay overtime. They are an approximation of AMPA and GABA synapses in the brain.
 Here $I^{syn}_j(t) = g_{ij} s_j(t)$ where $s$ is the synaptic gate. Its dynamics are controlled by a separate differential equation given by
 $$
 \tau_{syn} \frac{ds}{dt} = -s + \delta(t-t_{spike})
 $$
 where $\tau_{syn}$ is the synaptic decay rate for the synapse (around 2 to 5 milliseconds). Presynaptic input is again controlled by a delta impulses, but are smoothed out by the differential equation. Again, $g_{ij}>0$ then the synapse is excitatory, otherwise the synapse is inhibitory. 
 

Synaptic Conductance

These are now conductance's that exponentially decay overtime. They are a more accurate approximation to AMPA and GABA synapses. Here $I^{syn}_j(t) = g_{ij} s_j(t) (V-E_{syn})$ where $s$ is the synaptic gate, and $E_{syn}$ is the reversal potential of the ion channel. The synapse is inhibitory for $E_{syn}\leq V_{Rest}$, otherwise it is excitatory. Again  $s$ dynamics are controlled by a separate differential equation given by
 $$
 \tau_{syn} \frac{ds}{dt} = -s + \delta(t-t_{spike})
 $$
 where $\tau_{syn}$ is the synaptic decay rate for the synapse (around 2 to 5 milliseconds). Presynaptic input is again controlled by a delta impulses, but are smoothed out by the differential equation.

Author: Alexander White