Spiking Neural Networks:
The Machine Learning Approach
Prof.Nikola Kasabov, FIEEE, FRSNZ
Knowledge Engineering and Discovery Research Institute (KEDRI),
Auckland University of Technology, New Zealand
nkasabov@aut.ac.nz
www.kedri.info
PRESENTATION OUTLINE
Content
Part 1. SNN Methods
Part 2. SNN Systems. STDM.
Part 3. SNN Applications.
Part 4. Advanced topics
References
SNN
Methods
SNN:
Systems
Brain
Advanced
Topics
SNN:
Applications
for SSTD
nkasabov@aut.ac.nz
www.kedri.info
PRESENTATION OUTLINE
Content
Biological motivations for neurocomputation
Spiking neuron models
Part 1. SNN Methods
1.
2.
3. Data and information representation as spikes
4.
Learning methods for SNN
Part 2. SNN Systems. STDM.
5. SNN systems for pattern recognition, classification and regression
6. Neuromorphic space-time data machines. NeuCube
7. Neuromorphic hardware systems.
Part 3. SNN Applications.
8. Spatio-temporal brain data
9. Audio-/ video data and moving object recognition
10. Ecological and environmental data
11. Bioinformatics
12. Predictive modelling on financial and business streaming data
Part 4. Advanced topics
13. Computational neurogenetic modelling.
14. Quantum inspired evolutionary computation for SNN optimisation
15.Discussions and future directions
References
nkasabov@aut.ac.nz
www.kedri.info
Part I: SNN Methods
1. Biological motivation for neurocomputation
A single neuron is very rich of information
processes: time; frequency; phase;
field potentials; molecular (genetic)
information; space.
Three, mutually interacting, memory types
-
-
-
short term;
long term
genetic
SNN can accommodate both spatial and
temporal information as location of
neurons/synapses and their spiking
activity over time.
nkasabov@aut.ac.nz
www.kedri.info
Spiking activities of neurons
Electric synaptic potentials and axonal ion channels responsible for spike generation
and propagation: EPSP = excitatory postsynaptic potential, IPSP = inhibitory
postsynaptic potential, = excitatory threshold for an output spike generation.
nkasabov@aut.ac.nz
EPSP IPSP EPSP?IPSP Spike train Na+ K+ Voltage-gated ion channels in the neuron membrane
How does a synapse work?
Abbreviation:
NT: neurotransmitter,
R : AMPA-receptor-
gated ion channel
for sodium,
N: NMDA-receptor-
gated ion channel
for sodium and
calcium.
- Ion channels with quantum properties affect spiking activities in a stochastic
way. “To spike or not to spike?” is a matter of probability.
- Transmission of electric signal in a chemical synapse upon arrival of action
potential into the terminal is probabilistic
- Emission of a spike on the axon is also probabilistic
- Prior art on stochastic modelling of neuronal processes : D. Colguhoun, B.
Sakmann, E. Neher, SShoman, SWang, DTank , JHopfield
nkasabov@aut.ac.nz
NT R Ca2+ Ca2+ Na+ Na+ Ca2+ a b presynaptic terminal synaptic cleft postsynaptic membrane 106 m N vesicles
2. Spiking neuron models
Information processing principles in SNN:
– LTP and LTD
– Trains of spikes
– Time, frequency and space
– Synchronisation and stochasticity
– Evolvability…
Models of a spiking neuron and SNN
– Hodgkin- Huxley
– Spike response model
– Integrate-and-fire ---------------->
– Leaky integrator
– Izhikevich model
– Probabilistic and neurogenetic models
They offer the potential for:
– Bridging neuronal functions and “lower” level genetics
– Bridging spiking activities with quantum properties
– Integration of modalities
– Temporal or spatio-temporal data modelling
nkasabov@aut.ac.nz
… Models of Spiking Neurons
• Spiking neurons represent the 3rd generation of neural models,
incorporating the concepts of spaceand time trough neural
connectivity and plasticity
• Neural modeling can be described at several levels of abstraction
• Microscopic Level: Modeling of ion channels, that depend on
presence/absence of various chemical messenger molecules
Hodgkin-Huxley Model
Izhikevich model
Compartment models describe small segments of a neuron
separately by a set of ionic equations
• Macroscopic Level: A neuron is a homogenous unit, receiving and
emitting spikes according to defined internal dynamics
Integrate-and-Fire models
Probabilistic models
nkasabov@aut.ac.nz