Perception Lecture Notes: Visual Motion Perception

Motion is a perceptual attribute: the visual system infers motion from the changing pattern of light in the retinal image. Often the inference is correct. Sometimes it is not. In class I showed you a number of demonstrations in which motion is misestimated. Consider, for example, the motion aftereffect. Stare at the center of the following animation for about a minute, as it expands continuously (you may need to reload the page to get it moving again after it stops), then fix your gaze on another, static object (e.g., a friend's face).

Motion aftereffect

After viewing continuous motion in the same direction for a long time, if you look at a stationary object, it appears to move in the direction opposite to the one you were viewing. This is sometimes called the "waterfall illusion" - if you look at a waterfall for a while, then look at a tree next to it, the tree appears to move upward. The demonstration above shows that this adaptation is local in the retina (to the right of where you were looking, you were adapting to rightward motion, to the left you adapted to leftward, and so on). We take this as evidence for the existence of neurons that are sensitive to motion and selective for the direction of motion, which adapt to the stimulus (analogous to color adaptation after-effects).

Role of motion perception: Motion perception serves lots of helpful functions.

Simply detecting that something is moving, draws your attention to it.
Segmentation of foreground from background.
Compute the 3D shape of an object.
Compute the distance to various objects in the scene and estimate the direction in which you are heading within the scene. For example, hold up two fingers (one on each hand) at different distances, and move your head slowly from side to side while fixating an object on a far wall. Things that are further away slide across the retina more slowly. When there is strong motion on your retina, especially in peripheral regions, you can misattribute that motion and perceive yourself as moving (called "vection"). Movies (especially with large screens as in an IMAX theater) can give this illusion that you are moving.
Recognition actions, such as movements of a human (in the "point light displays" shown in class of people walking, dancing, etc., displayed as the motion of a small number of dots attached to the joints of the person).

Object motion versus observer motion: Two different kinds of events can cause visual motion. Observer can move through an otherwise stationary environment - entire retinal image changes over time. Or an object can move while the observer is stationary - so that a small region of the retinal image changes over time. Often, of course, both of these things happen at once, but it is helpful to consider the two separately.

Optical Flow: Here is a representation of the actual physical motion in the image.

Optical Flow Fields

Each arrow represents the speed and direction of motion for each little patch of the visual field. Near points move fast (long arrows), far points move slowly (short arrows). In this example, the arrows point away from a single point called the focus of expansion that corresponds to where the observer is heading.

The first step in motion perception is for the visual system to estimate optical flow from the changing pattern of light in the retinal image. Then, the 3D motions of observer and objects can be inferred from the optical flow.

J. J. Gibson hypothesized that there's sufficient information in the visual stimulus to specify a unique, unambiguous interpretation of 3D motion and depth. Recently, mathematicians have proven that this hypothesis is basically correct. There is a caveat, however: distance and speed are ambiguous (i.e., they trade off). That is, a small, close object when you are moving slowly creates the identical retinal images over time as a large, distant object when you are moving quickly. That's why you need a speedometer in your car. You are lousy at making absolute speed and distance judgements. But, you are very good at relative speed/direction and relative distance.

As time progresses, the red object appears progressively further to the right. In other words, this is a representation of rightward motion. Leftward motion would be represented in this diagram by an orientation down-and-left. A change in the orientation represents a change in speed. A vertical contour would represent an object that was moving. This diagram only represents one spatial dimension (left-right, not up-down); a true space-time plot for retinal motion is 3-dimensional. Thus, to sens

Visual Motion in the Brain

Functional specialization hypothesis: there are specific brain area(s) that are involved in visual motion perception.

Evidence for this includes a patient known as LM who, following a stroke, had great difficulty perceiving certain types of motion. Color vision and acuity remained normal, and there was no difficulty recognizing faces or objects, no difficulty with stereo. But LM cannot see coffee flowing into a cup: appears frozen like a glacier, does not perceive the fluid rising, and often lets the coffee spill or overflows. LM feels uncomfortable in a crowded room or on a street. "People were suddenly here or there, but I have not seen them moving... When I'm looking at the car first it seems far away, but then when I want to cross the road suddenly the car is very near". LM's lesion extends over a substantial region of visual cortex, so one can not localize sharply the regions relevant to LM's motion deficit. This makes it particularly surprising that loss of motion perception can be so cleanly dissociated from other visual abilities.

MT: Area MT is one of the most studied regions of the cortex of the brain, probably second only to V1. Current opinion is that the optical flow field is represented by neurons in area MT. MT neurons receive inputs from direction-selective neurons in V1. MT neurons are velocity selective, each responds best to a preferred velocity (speed and direction) within its receptive field, pretty much independent of stimulus pattern. By contrast, a direction-elective V1 neuron confounds motion with pattern. A typical V1 neuron responds to a particular orientation (edge or bar) moving in a particular direction. The response of the V1 neuron also increases with contrast. A typical MT neuron responds to almost any pattern with almost any contrast, as long as it moves with the right velocity.

Direction columns in MT

There also appears to be a columnar architecture in MT for stimulus motion; neurons with similar motion preferences lie nearby one another, with an orderly progression from one motion direction to the next as you move through MT, analogous to orientation columns in V1.

MT and motion perception: Bill Newsome (Stanford Neurobiologist) and colleagues trained monkeys to perform a difficult motion discrimination task.

Monkey chair

Newsome stimulus

Microstimulation data

The monkeys viewed moving dots and decided which way they went, e.g., either up or down. In the stimulus, only a subset of the dots moved in the indicated direction; the others moved randomly. The percentage of dots moving in the given direction was varied (in the graph, a negative percentage means the dots were moved in the opposite direction). The graph (open circles) shows the percentage of times the monkey indicated the dots were moving in the indicated direction as a function of the percentage of moving dots. The investigators then inserted microelectrodes into MT to electrically stimulate the monkeys' brains while they were performing the task. When the electrode was in a column selective for upward motion, stimulation biases the monkey to report "up" more often. This is a very cool result: stimulating MT neurons directly influences behavior, presumably because it influences the monkeys' conscious percepts.

<>Human MT: A homologous area has been localized in the human brain. A flickering checkerboard stimulus evokes brain activity in V1 at the back of head. A moving dot stimulus again evokes brain activity in V1, but also in a lateral region just behind your ears which is believed to be the human homologue of monkey area MT.

Human MT and the motion aftereffect: Tootell used fMRI to measure the neuronal correlates of the motion aftereffect.

fMRI motion aftereffect data

Subjects viewed two types of stimuli. (1) The stimulus moved constantly in one direction (light blue) for 40 seconds then was stationary (black) for the next 40 seconds. (2) The stimulus moved in and out alternating direction every other second (dark blue) for 40 seconds then was stationary (black) for the next 40 seconds. Observers experience the motion aftereffect for (1) but not for (2). They found elevated levels of MT activity for the motion aftereffect condition (compare the white shaded regions following the light blue and dark blue regions). This is another example linking motion perception with activity in area MT. This is particularly compelling because this is an illusion of motion for a physically stationary stimulus! This supports the functional specialization hypothesis, specifically that area MT is tightly coupled with the conscious perception of visual motion.

MST: Monkey area MST is right next to MT.

MST responses

<>MST neurons have very large receptive fields, respond selectively to complex optical flow fields: expansion, contraction, rotation. Perhaps MST is involved in 3D motion perception, inferring 3D motion of objects/observer from optical flow. Neurons in area STS respond selectively to biological motion, like Johannson's point-light walkers.

Eye Movements and Motion Perception

When an observer is moving, the visual system uses the changing retinal image to infer the observer's trajectory and 3D structure. When an observer is stationary, but the object is moving, the visual system infers the motion and structure of the object. How does the visual system keep track of what's moving? Are you moving or is it the scene that you're looking at that is moving? Sometimes your visual system gets it wrong (sitting in a train at the station when the neighboring train pulls away, an example of vection) but most of the time your visual system gets it right.

Answer: visual images are combined with other information to inform you about the motion of your eyes, head, and body. The vestibular system provides information about the motion of your head and body. A copy of the eye movement command from the eye movement centers in the brain stem provides information about eye movements. Vision is combined with vestibular and eye movement signals in Area MST.

Corrolary discharge model

Push gently on the side of your eye and the world appears to jiggle around. Helmholtz did this experiment and came up with a theory of how eye movement information is combined with change in the retinal image to yield the motion percept. Generally speaking, the brain is divided into motor areas and sensory areas. The corrolary discharge is a copy of the motor signal that is transmitted to a comparator - a hypothetical structure that receives both the corrolary discharge and the sensory movement signal (maybe in MST). If the visual motion signal is the same as the eye movement command, then you don't "see" motion. If the visual motion signal is different from the eye movement command, then you do see motion.

David Heeger