G89.2223
Perception and Attention
Fall 2023
Tuesdays & Thursdays
2 pm - 3:15 pm
Meyer (4-6 Washington Place), Room 159
Last updated: September 19, 2023
The textbook for this course is Foundations of Vision by Brian Wandell. The book is out of print but the entire book is available online (https://foundationsofvision.stanford.edu) and pdf's of the assigned chapters are available by following the links provided below. Additional readings are also available by following the links provided below.
This course will be hybrid. I have been struggling with long covid for over a year so I will be on zoom only. Please plan to attend class in person as long as you are healthy. But if you cannot attend in person, we will be using zoom to stream and record the lectures:
Zoom link:
https://nyu.zoom.us/j/97706780287
9/5, 9/7, 9/12, 9/14 Color: trichromacy, color opponency, & chromatic adaptation (Heeger) Readings: Wandell Chs 1, 3, 4, & 9.
Lecture slides:
Color matching & trichromacy lecture slides (5.9MB pdf)
Color opponency & adaptation lecture slides (4.6MB pdf)Lecture notes:
Lecture notes on color (from undergrad perception course)
Lecture notes on the retina (from undergrad perception course)Zoom lecture recordings:
Color matching tutorial (100KB, zip archive of matlab code)
9/19, 9/21, 9/26, 9/28 Linear systems theory: spatial vision (Winawer) Readings: Wandell Chs 2, 3, 5, 6, 7, 8, & App 1; Signals, Linear Systems, & Convolution Handout.
An interactive guide to the Fourier transform
Tutorial on complex numbers
Supplementary readings:
- Blakemore C & Sutton P (1969). Size adaptation: A new aftereffect. Science, 166, 245-247.
- Campbell FW & Gubisch RW (1966). Optical quality of the human eye. Journal of Physiology, 186, 558-578.
- Campbell FW & Robson JG (1968). Application of Fourier analysis to the visibility of gratings. Journal of Physiology, 197, 551-566.
- Graham N (1980). Spatial-frequency channels in human vision: Detecting edges without edge detectors. In Harris, C. (Ed.), Visual Coding and Adaptability (pp. 215-252). Hillsdale, NJ: Erlbaum.
- Graham N & Nachmias J (1971). Detection of grating patterns containing two spatial frequencies: A comparison of single-channel and multiple-channels models. Vision Research, 11, 251-259.
- Watson AB & Robson JG (1981). Discrimination at threshold: Labelled detectors in human vision. Vision Research, 21, 1115-1122.
- Wilson HR, McFarlane DK & Phillips GC (1983). Spatial frequency tuning by orientation selective units estimated by oblique masking. Vision Research, 23, 873-882.
- Bracewell RN (2003). Fourier Analysis and Imaging. New York: Kluwer/Plenum.
Lecture slides (12.6MB pdf)
Zoom lecture recordings:
Matlab tutorials (1.6MB zipped archive, requires matlabPyrTools).
10/3, 10/5, 10/12 Detection and signal detection theory (Landy) Readings: Wandell App 3; Signal Detection Theory handout; Poisson handout; Hecht et al. (1942); Geisler (1989).
Supplementary readings:
- Parker A. J. & Newsome W. T. (1998). Sense and the single neuron: Probing the physiology of perception, Annu. Rev. Neurosci., 227-277.
- Cornsweet, T. N. (1970). Visual Perception. New York: Academic Press (chs. 2-4).
- Duda, R. O., Hart, P. E. & Stork, D. G. (2001). Pattern Classification. New York: Wiley (chs. 2-3).
- Green, D. M. & Swets, J. A. (1966/1974) Signal Detection Theory and Psychophysics. New York: Robert E. Krieger.
- Macmillan, N. A. & Creelman, C. D. (1991). Detection Theory: A User's Guide. New York: Cambridge.
- Wickens, T. D. (2002). Elementary Signal Detection Theory. New York: Oxford.
- Coombs, C. H., Dawes, R. M. & Tversky, A. (1970). Mathematical Psychology, An Elementary Introduction. Englewood Cliffs, NJ: Prentice-Hall (ch. 6).
Additional readings about retinal responses near absolute threshold: Field, Sampath & Rieke (2005); Chichilnisky & Rieke (2005).
Signal detection tutorial (zip archive of matlab code)
Lecture slides part 1 (3.8MB pdf)
Lecture slides part 2 (5.6MB pdf)Zoom lecture recordings:
10/10 No class (classes meet on a Monday schedule) 10/17, 10/19 Reaction time and diffusion models (Landy) Readings: Gold & Shadlen (2007); Palmer, Huk, & Shadlen (2005).
Supplementary readings:
Lecture slides (4.7MB pdf)
- Ratcliff, R. & Smith, P.L. (2004). A comparison of sequential sampling models for two-choice reaction time. Psychological Review 111:333-367.
- Shadlen, M.N., Hanks, T.D., Churchland, A.K., Kiani, R., & Yang, T. (2006). The speed and accuracy of a simple perceptual decision: a mathematical primer. In Bayesian Brain: Probabilistic Approaches to Neural Coding, K. Doya, S. Ishii, A. Pouget, and R.P.N. Rao, eds. (Cambridge: MIT Press).
- Shadlen, M.N. & Kiani, R. (2013). Decision making as a window on cognition. Neuron 80:791-806.
- Link, S.W. (1992). The Wave Theory of Difference and Similarity (Hillsdale, NJ: Erlbaum).
Zoom lecture recordings:
10/24, 10/26 V1 and normalization (Heeger) Readings: Wandell Ch 6, Carandini & Heeger (2011).
Lecture slides (6.1MB pdf)
Zoom lecture recordings:
10/31, 1/2, 11/7, 11/9
Attention (Carrasco) Readings:
Carrasco (2011); Reynolds & Heeger (2009); Herrmann et al (2010)Supplementary readings:
- Carrasco (2006).
- Lu & Dosher (2004).
- Dosher & Lu (2000).
- Carrasco & Yeshurun (1998).
- Wolfe (1998).
- Palmer (1995).
- Reynolds, Pasternak, & Desimone (2000).
Lecture slides:
Lecture slides - part 1 (19MB PDF)
Lecture slides - part 2 (8MB PDF)
Lecture slides - part 3 (6MB PDF)
Lecture slides - part 4 (5.2MB PDF)Zoom lecture recordings:
- 11/4/2021
- 11/9/2021
- 11/11/2021
- 11/16/2021
11/14, 11/16, 11/21 Visual Motion Perception (Heeger) Readings: Wandell Ch 10 & App 5; Adelson & Bergen (1985); Adelson & Movshon (1982); Weiss, Simoncelli, & Adelson (2002).
Supplementary readings:
- Simoncelli & Heeger (1998). A model of neuronal responses in visual area MT. Vis Res, 38:743-761.
- Huk & Heeger (2002); Pattern-motion responses in human visual cortex, Nat Neurosci, 5:72-75.
- Motion estimation handout.
Lecture slides:
Motion intro lecture slides (2.3MB pdf)
Functional specialization lecture slides (5.7MB pdf)
Computational theory lecture slides (9.1MB pdf)Lecture notes:
Lecture notes on motion (from undergrad perception course)
Lecture notes on the visual cortex (from undergrad perception course)Zoom lecture recordings:
Matlab code: Motion tutorial (160KB zipped archive, requires matlabPyrTools)
MT model (matlab code available for download)11/23 No class (Thanksgiving) 11/28, 11/30
Cue combination and Bayesian decision theory (Maloney) Readings: Landy, Banks & Knill (2011), pp. 1-10; Ernst & Banks (2002); Maloney & Zhang (2010); Najemnik & Geisler (2005); Stocker & Simoncelli (2006).
Supplementary readings: Saunders & Knill (2005); Hudson, Maloney & Landy (2008); Zhang, Morvan & Maloney (2010).
Books for background reading on Bayesian estimation and decision theory (optional): Leanard & Hsu, Bayesian Methods: An analysis for statisticians and interdisciplinary researchers; Sivia, Data Analysis: A Bayesian Tutorial.
Lecture slides:
Cue combination (5.2MB pdf)
Bayesian decision theory (5.4MB pdf)Lecture notes:
Lecture notes on depth (from undergrad perception course)Zoom lecture recordings:
12/5, 12/7 Recognition (Pelli) Readings: Rosch et al. (1976); Pelli & Tillman (2008) with supplementary material; Majaj et al. (2015); Ranzato, Huang, Boureeau, & LeCun (2007).
Lecture slides (24.3MB ppt)Zoom lecture recordings:
| Marisa Carrasco, Rm. 971, 8-8328 marisa.carrasco@nyu.edu |
| David J. Heeger david.heeger@nyu.edu |
| Michael Landy, Rm. 961, 8-7857 landy@nyu.edu |
| Laurence Maloney, Rm. 877, 8-7851 laurence.maloney@nyu.edu |
| Denis Pelli, Rm. 279, 646-258-7524 denis.pelli@nyu.edu |
| Jon Winawer, Rm. 960, 8-7922 jonathan.winawer@nyu.edu |
| Hsing-Hao Lee, Rm. 974 hl3967@nyu.edu Office hours: Mon 11am-1pm and Fri 1-3pm |
Submit each assignment by email to the instructors listed. Please submit a pdf file (only pdf files will be accepted - not MS Word documents), along with matlab *.m files when relevant.
Assignment 1 (Heeger): Color (due 9/21)
For this assignment, you will need the
color matching tutorial
(100KB, zip archive), the
rod spectral sensitivity
(1KB, matlab file), and the
cone spectral sensitivities
(1KB, matlab file).
Please submit to Prof. Heeger.
Assignment 2 (Landy): SDT, RT (due 10/26)
For this assignment, you will need the
signal detection tutorial (89KB, zip archive).
Please submit to Prof. Landy.
Assignment 3 (Carrasco & Winawer): Spatial vision, attention (due 11/16)
It is hypothesized that one of the effects of covert attention is
to change the bandwidth of spatial frequency and/or orientation
channels.
(1) Given that arguably the function of attention is to improve
discrimination at the attended location, would it be in the
observer's best interest to broaden or to narrow spatial frequency
and/or orientation bandwidths at the attended location? Justify
your choice with reference to a model of a particular task that
would predict improved performance with the hypothesized change in
bandwidth.
(2) Design a psychophysical experiment to test this hypothesis:
(a) Describe the experimental design
including what kind of attention (spatial or feature, endogenous
or exogenous) you are manipulating and how you are doing so, the
particular stimuli, task, and procedure.
(b) Explain how you will analyze the data.
(c) Describe the potential outcomes and how
they will either prove or disprove the hypothesis.
(d) Include description of relevant control
experiments that indicate you can measure a bandwidth and you have
changed performance with your attentional manipulation.
Extra credit. Matlab simulations are welcome (you could generate a
simulated data set assuming that the hypothesis is true, another
simulated data set assuming that it is false, and walk through the
analysis, results, and interpretation for each).
Please submit to Profs. Carrasco and Winawer.
Assignment 4 (Heeger): Motion and normalization (due 11/30)
Please submit to Prof. Heeger.
Assignment 5 (Maloney): Cue combination and Bayesian decision theory (due 12/14)
Please submit to Prof. Maloney.
Students must adhere to NYU's principles of academic integrity, and the CAS Honor Code. Academic integrity means that all of the work that you submit is original.
Violations of academic integrity include:
Consequences of a single violation of the principles of academic integrity are a grade of F on the scored assessment, and a formal report to the Dean's Office. The grade of F must be factored into the course's final grade. In cases of serious violation(s) of these standards, the department will seek dismissal of the student from the university.
Please plan to attend class in person as long as you are healthy. If you feel ill (even slightly ill) or if you think you might have been exposed (e.g., you attended at a party that was more crowded than you expected it to be), please skip class and get yourself tested. The TA or instructor will meet with you 1:1 if needed to make sure that you don't get behind.
Please stay home even if you are ill even if you think you don't have COVID (i.e., you got a negative test result). Think of the consequences. If you come to class with a cold and transmit it to someone else, they will have to assume that they might have COVID. So they will need to get tested and they will need to quarantine, skip classes and cancel all their plans, risk infecting their friends and family. So...
STAY HOME IF YOU FEEL ILL - EVEN IF YOU ARE CERTAIN THAT YOU DON'T HAVE COVID.
At this point in time, everyone should know how to wear a mask. But just in case... Wearing a mask is effective only if it covers both your mouth and your nose. There's little metal strip at the top of the mask that should be pinched over your nose to the shape of your face, so that the mask doesn't slide down. We should not be able to see the tip of your nose.