Perception Lecture Notes: Attention and Awareness

Professor David Heeger

1. Not all operations of the brain correspond to conscious processes.

2. Consciousness is closely associated with attention

3. Consciousness involves some form of memory

Hemi-field Neglect

• Extinction: The patient can see normally in the left hemifield. Put an object to their left, they can see it and identify it. But, if you show two objects, one in each hemifield they don't see the object on the left. They deny that it was there.
• Neglect: The patient actually ignores half of their body. They get dressed on one side of the body, put makeup on one side of their face.






Example: copy a clock face



Example: Self-portraits

Anecdote about an Italian patient. There is a plaza in his home town with a church at one end, a market at the other, and shops along both sides. The patient was asked: "Imagine you are in the plaza facing the church. Describe what you see." He named all the shops on the right side, but none of those on the left. "Now imagine that you turn around and face the market. What do you see?" He now named all the shops on the other side of the plaza (those that are now on his right, but that used to be on the left).

• Denial: Patients are sometimes unaware that they have a problem; when confronted they will deny it and confabulate (come up with some kind of story, no matter how absurd). Example: grab a patient's right arm and ask them what it is. "It's my arm." Grab their left arm and ask them what it is. "It's AN arm." "Whose arm is it?" "I don't know. It's not mine. It must be yours."
• Spatial orientation deficit: Patients can be confused in familiar surroundings. Example: a person goes to/from work the same way every day for 20 years. On the way home on the bus they suffer a mild right parietal lobe stroke. They get off the bus. They see a drug store in front of them. They know it's the store near their house. They know this is the bus stop they get off at every day but they have no idea how to get home. There's no "vision" problem - they can see and recognize the store.

Summary: These patients suffer from loss of some form(s) of awareness, particularly having to do with the relationship between their own body and the visual world around them. There is lots of evidence that the parietal lobe is involved in "sensory-motor" integration, turning perceptual information into motor action. That's why some of these symptoms show a loss of awareness of the visual world and some of the symptoms show a loss of body awareness.

Patients with temporal lobe lesions (causing face and object agnosia) are fully aware that there's a problem and they develop strategies to compensate for it.  Parietal patients are often unaware of their deficits and confabulate (denial) when forced to confront it.

In hemi-field neglect, loss of attention and awareness can result in devastating behavioral deficits.  However, there are some equally striking examples of residual behavioral performance in the absence of awareness.

Action without Awareness

The black region marks the blind region of the visual field. One has the patient do a series of forced-choice trials. One presents a light on half the trials, and forces the subject to guess on each trial whether or not the light has been presented. Numbered insets indicate percent correct. There is near-chance (43% correct) performance when the light was flashed in the blind spot (labeled "Disc"). However, performance was well above chance for all other positions within the blind region. As far as the subject is concerned, none of the lights were seen, although for some positions he had a non-visual feeling that something happened (hatched area). This non-visual awareness of a visual stimulus has been described by subject GY as "The nearest I ever get, and it is not a fair comparison, is waving your hand in front of your eyes when they are closed. You are kind of aware that something happened but you don't quite see it."

Split brain patients: Neurosurgeons sometimes cut the corpus colossum (the massive bundle of nerve fiber axons connecting the two hemispheres) to treat epilepsy. Such an operation results in subtle, but very interesting deficits.  One of the most interesting effects is another example of residual behavioral performance without awareness.

Language is centered (in most right-handed subjects) in the left hemisphere. The subject fixates the center point, a word flashes up on one side or the other. The subject reports (through the speaking left hemisphere) only the words flashed to the right visual hemifield and denies seeing the left field stimuli. Even so, when instructed to pick up the object corresponding to the flashed word, the left hand correctly retrieves objects for which the subject verbally denies having any knowledge. Sperry won the Nobel prize in 1981 for work on split-brain patients

Consciousness, what is it good for? Blindsight patients' residual vision allows for rudimentary discriminations for action, but they can't use their residual capacity for "thinking" or "imagery" (mental manipulation of a percept). They can't, for example, compare what they are looking at now with what they saw yesterday (because they don't "see" what they are looking at now).

Visual Attention

This is the most direct way to shift attention, called overt attention.  Poor resolution in the periphery means that you are aware primarily of things near the center of gaze. There are lots of things you can't do without moving your eyes (for example: count a bunch of briefly flashed dots, even if they're bright enough to give a strong afterimage)

Covert attention: You can also shift attention without eye/head movements to "filter out" unattended locations.

In a typical covert attention experiment, subjects perform a difficult (threshold) visual discrimination task. There is a cue just before each trial that indicates where they should focus their attention. Sometimes the cue is correct, sometimes it is misleading (it cues the wrong location). Performance is often measured as reaction time, but sometimes (as I would prefer) as d' (discriminability via signal detection theory). On average, performance is better on trials that are cued correctly with enough lead time. Here is a summary of the results of this sort of experiment:

1. You can shift spatial attention without eye movements.
2. The shifts can be controlled by the subject's decision to allocate attention (e.g., with instructions, or with a central arrow as a cue) or by automatic attraction (e.g., with a preceding peripheral box cue).
3. Performance falls off smoothly with distance when cued to the wrong location.
4. Shifts of attention are linked in time to the occurrence of the cue. This yields best performance right after (50-300 msec) the cue, then performance is worse if there is a long delay (> 300 msec) between cue and target. The drop-off in performance for long delays is sometimes called inhibition of return.

In this example of texture segmentation, it is easy to see the pluses on the background of L's, but to see the region of T's is hard, and requires item by item scrutiny (i.e., allocation of visual attention).

It is hard to search for a vertical rectangle among horizontal rectangles and ovals of both orientations. It would be easy to search for a vertical rectangles if the only distractors were horizontal rectangles. The conjuction of two features (vertical and rectangle) make it hard.  This visual search task is like the "Where's Waldo" books.

Difficult discriminations require attention - spotlight or zoom lens, and item-by-item serial search. They show a set-size effect whereby reaction time is increased linearly with an increase in the number of distractors (non-target items). Not only that, this increase for each irrelevant item is twice as big, on average, on trials where the target is absent (and the observer should say "no") than on trials where the target is present (and the observer should say "yes"). This is because the observer needs to attend every item to be assured that "no" is the correct answer, but on average will only have to look at half the items to find the target, when it is indeed present.

Feature integration theory (Treisman): This is a theory for explaining which patterns are easily discriminable and why.  The idea is that the front-end of the visual system breaks the stimulus down into its constituent parts. It separately analyzes each local patch of the visual field to determine: pattern; motion; shape (depth and size); color, etc. Attention is the glue in feature integration theory. Example: seeing red is preattentive, and seeing rightward motion is preattentive, but seeing a red thing moving to the right requires attention to connect the red thing with the moving thing. The theory does not go so far as to say how attention accomplishes this gluing. Feature integration theory is based in part on the functional specialization hypothesis, e.g., that "red" and "moving rightward" are represented in separate visual brain centers. One needs attention to bring those separate neuronal representations together.

Illusory conjuction: Suppose we briefly flash a picture of woman with black hair and a red sweater. Some subjects will incorrectly report seeing a woman with red hair. This sort of "illusory conjunction" is taken as evidence for feature integration theory. In a brief display, there's not enough time for the attentive processing to combine the features correctly.

Attention in the Brain

Attention affects neural activity

Goldberg & Wurtz (NIH) were the first to demonstrate that attention can affect neural responses. They recorded from neurons in the parietal lobe in awake monkeys while the monkeys were performing various tasks that they had been trained on.

Top: The monkey was trained to fixate. There is a small response from the parietal lobe neuron when a light comes on in the periphery.

Middle: The monkey was trained to fixate until the peripheral light comes on, then it must move its eyes to look at the light. The retinal stimulation is identical, but there are much bigger responses (3x).

Bottom: The monkey was trained to move its arm instead of its eyes. One gets the same result (bigger response), i.e., this is not simply an eye movement control signal.  It is rather more like the neural correlate of attention, salience or task-relevance.

Attentional modulation of V1 brain activity: The classical (Hubel and Weisel) view of V1 is that it acts as a passive, automatic, image processing machine. An emerging view is that V1 does much more than that.

Here is an example of an fMRI experiment performed in Prof. Heeger's lab. Subjects viewed moving stimuli presented within a pair of apertures, one positioned to the left and one to the right of the center of fixation. The shape of the fixation point cued subjects to attend alternately to a series of motion discrimination trials on the right and to a series of trials on the left. Because the discrimination was difficult, subjects had to allocate spatial attention to the relevant (cued) side of the visual field in order to perform as well as possible.

V1 brain activity, measured with fMRI, modulates as subjects alternate attention. The "Attend left" condition increases brain activity in the right hemisphere. The "Attend right" condition increases activity in the left hemisphere.  Attention reaches all the way back to the first visual area in the cortex.

Mental Imagery

Subjects looked at real motion.  Then, after a 20 second break, subjects imagined (while viewing a blank screen) the moving stimulus they had just seen.  The graph plots fMRI signal in human area MT over time. Activity in MT is very similar for real motion and imagined motion.

Once every 12 seconds, subject heard the name of a person or a familiar place (a building on the MIT campus).  They were instructed to imagine it.  The investigator recorded fMRI signals in the "face" area (that responds strongly to pictures of faces) and the "place" area (that responds strongly to pictures of familiar places, buildings, etc.).  The red curve shows the time-course of response in the "face area".  The blue curve shows the time-course of response in the the "place area".  fMRI response is bigger in the "face" area when imagining a face.  Response is bigger in the "place" area when imagining a place.

Scientific Study of Consciousness

1. Brain activity in V1 fluctuates with perceptual alternations during binocular rivalry.
2. Microstimulation in MT biases monkey's judgements of motion.
3. Brain activity in MT is correlated with the perception of illusory motion during the motion aftereffect.
4. Neural responses in V2 are correlated with the perception of  illusory contours.
5. Brain activity in many visual areas including V1 modulates with shifts in attention.
6. Imagining different attributes of visual stimuli (e.g., color, motion, faces) selectively activates the specific brain pathways involved in the perceptual processing of those stimulus attributes.

To find out more about current scientific research on consciousness, go the Association for the Scientific Study of Consciousness web page.