Electrophysiological studies indicate that neurons in the Middle
Temporal (MT) area of the primate brain are selective for the velocity
(both direction and speed) of visual stimuli. We've developed a descriptive computational model of MT
physiology, in which local image velocities are represented via the
distribution of MT neuronal responses. The computation is performed
in two stages of identical architecture, corresponding to neurons in
cortical areas V1 and MT. Each stage computes a weighted linear sum
of inputs, followed by rectification and divisive normalization. The
output of the model corresponds to the steady-state firing rates of a
population of MT neurons, which form a distributed representation
(population encoding) of image velocity for each local spatial region
of the visual stimulus. One can think of the distributed set of responses
as representing a probability density over local image velocity.
The model accounts for a wide range of
physiological data.
Model Structure
The diagram shows the essential aspects of the model. In the first
(V1) stage, each neuron computes an inner product of the image
contrast with a space-time oriented receptive field. Our receptive
fields are directional third derivatives of a Gaussian. The use of
directional derivatives is a fundamental aspect of the model, as it
allows us to interpolate responses at arbitrary space-time
orientations from a fairly small (28) fixed population of V1 neurons.
[NOTE: The
Gaussian is chosen for simplicity: alternative choices would lead to
beneficial properties such as causality or spatio-temporal
separability, without fundamentally changing the steady-state responses].
These linear outputs are half-squared (halfwave-rectified and
squared), and then divisively normalized. The normalization factor is
a sum of a semisaturation constant (sigma) and the responses of
neurons at all space-time orientations, and within a local spatial
neighborhood.
The second (MT) stage of the model performs a summation over V1
afferents consistent with a given pattern velocity. These are
illustrated in the spatio-temporal frequency domain. This is
essentially a neural implementation of the
"intersection-of-constraints" construction (see references). This
construction is appropriate for so-called "pattern cells" in area MT.
As shown in the paper below (Vis. Res. 1998), variants can be
constructed for "component cells", by
summing afferents of the subset of V1 cells with the same direction
tuning.
Note: this is a model of steady-state responses to
spatio-temporally homogeneous stimuli.
Software
A matlab implementation of the model is
available here.
A brief description may be found in the
README file, and
updates and changes are listed in the
ChangeLog file.
Partial List of References
This Model
N C Rust, V Mante, E P Simoncelli and J A Movshon,
How MT cells analyze the motion of visual patterns
Nature Neuroscience, 9(11):1421-1431, Nov 2006.
Abstract
/
Reprint
E P Simoncelli, W D Bair, J R Cavanaugh, and J A Movshon.
Testing and Refining a Computational Model of Neural
Responses in Area MT. ARVO, 1996.
Abstract
/
Presentation slides (pdf)
E P Simoncelli and D J Heeger.
A Velocity-representation Model for MT Cells. ARVO, 1994.
Abstract
Eero P Simoncelli.
Distributed Analysis and Representation of Visual Motion.
PhD thesis, Massachusetts Institute of Technology, Department of
Electrical Engineering and Computer Science,
Cambridge MA, January 1993.
Abstract /
Reprint
E P Simoncelli and D J Heeger.
A Computational Model for Representation of Image Velocities.
ARVO, 1993.
Abstract
Related Computational Motion Models
S Nishimoto, JL Gallant
A three-dimensional spatiotemporal receptive field model explains responses of area MT neurons to naturalistic movies
The Journal of Neuroscience, 31(41), 14551-14564, 2011.
A A Stocker and E P Simoncelli
Noise characteristics and prior expectations in human visual speed perception.
Nature:Neuroscience, 9(4): 578-585, Apr 2006,
[abstract/reprint]
E P Simoncelli
Local analysis of visual motion.
Chapter 109 in: The Visual Neurosciences, Eds: LM Chalupa and JS Werner,
pp. 1616--1623. MIT Press, Jan 2003.
[abstract/reprint]
Y Weiss, E P Simoncelli and E H Adelson
Motion illusions as optimal percepts.
Nature:Neuroscience, 5(6): 598-604, June 2002,
[abstract/reprint]
S J Nowlan and T J Sejnowski.
A Selection Model for Motion Processing in Area MT of
Primates.
Journal of Neuroscience, vol 15, pp 1195-1214, 1995.
D J Heeger and E P Simoncelli.
Model of Visual Motion Sensing.
Chapter 19 in Spatial Vision in Humans and Robots, ed
L Harris and M Jenkin. Cambridge University Press, 1994,
[abstract/reprint]
J A Smith and N M Grzywacz.
A Local Model for Transparent Motion Based on Spatio-temporal
Filtering.
Computation and Neural Systems, ed. F H Eeckman and J M Bower.
Kluwer Academic Press, 1993.
M E Sereno.
Neural Computation of Pattern Motion: Modeling Stages of
Motion Anaysis in the Primate Visual Cortex.
MIT Press, 1993.
E P Simoncelli.
Distributed Representation and Analysis of Visual Motion.
Ph.D. Thesis, Jan 1993. MIT Dept. of Electrical Engineering and Computer Science.
[abstract/reprint]
E P Simoncelli and D J Heeger.
A computational Model for Perception of
Two-dimensional Pattern Velocities.
ARVO, 1992.
[abstract/reprint]
H R Wilson and V P Ferrara and C Yo.
A Psychophysically Motivated Model for Two-dimensional Motion
Perception.
Visual Neuroscience, vol 9, pp 79-97, 1992.
N M Grzywacz and A L Yuille.
A Model for the Estimate of Local Image Velocity by Cells in
the Visual Cortex.
Proc. Royal Society of London A, vol 239, pp 129-161, 1990.
D J Heeger.
Model for the Extraction of Image Flow.
Journal Optical Society of America, vol 4, pp 1455-1471, 1987.
E H Adelson and J R Bergen.
Spatiotemporal Energy Models for the Perception of Motion.
Journal Optical Society of America, vol 2, pp 284-299, 1985.