NEURL-GA 3042.005 -- Spring, 2014
(cross-listed: MATH-GA 2855, CSCI-GA 2715.001, PSYCH-GA 3405.005)

Representation and Analysis of Visual Images

Instructor: Eero Simoncelli
Time: Tue/Thu 10-11:50 am
Location: Meyer Hall (4 Washington Place), rm 851
Prerequisites: linear algebra, vector calculus, linear systems theory, basic probability/statistics. Some matlab programming experience.

Brief Description: A graduate-level lecture course on theory and tools for representing, manipulating and analyzing visual images. Topics include: imaging and optics, color, estimation and representation of position, alignment, displacement, and local orientation, multi-scale image decomposition (wavelets, multi-scale frames), statistical image modeling and its use in compression, estimation, enhancement, synthesis and classification. Grades are based on homework, which relies heavily on matlab programming.

Date Topic Handouts/Reading Homework
Tue, 28 Jan Intro / the visual (plenoptic) world / imaging Course description, Background poll,
Plenoptic chapter,
Slides: Introduction 		 
30 Jan Color, reflectance    
4 Feb
(ending 11:30)		 White balance / Color constancy    
6 Feb [no class]    
11 Feb Color Slides: Plenoptic function, color  
13 Feb Linear Systems, Convolution
   
18 Feb Fourier representation Homework 1, Due 4 March
20 Feb [no class]    
25 Feb Sampling
27 Feb [no class] Slides: Linear Systems  
4 Mar Spectral models Part 3 of HW1 reposted, due 11 Mar
6 Mar Spectral estimation: Wiener filter Slides: Spectral Modeling
11 Mar Matched filter, Derivatives and oriented filtering
13 Mar [no class]
18 Mar / 20 Mar [no class (NYU spring break)]
25 Mar Differentiation of discrete signals Slides: Matched filter / differentiation / orientation
27 Mar Analysis/representation of local orientation
1 Apr Differential matching / optic flow Slides: differential matching Homework 2, Due 15 April
3 Apr Heavy-tailed image models / ICA Chapter on differential optic flow
8 Apr Sparse image decomposition Slides: ICA, sparsity
10 Apr Scale, zooming, interpolation
15 Apr Pyramids, multi-resolution
17 Apr Coarse-to-Fine matching
22 Apr Subband Decompositions Slides: multi-scale representations / pyramids
24 Apr Wavelets
29 Apr Multi-scale oriented decompositions
1 May Joint statistics Slides: multi-scale statistical models
Book chapter on denoising 		Homework 3, Due 13 May
6 May Classification/recognition: classifiers Slides: classification
8 May Classification/recognition: features

Auxilliary Reading: Image Processing, Computer Vision, Biological Vision

  • Pattern Classification, Duda, Hart and Storck. Wiley, 2001.
  • Computer Vision: Algorithms and Applications. Richard Szeliski Springer, 2010.
  • Computer Vision: A Modern Approach. David Forsyth and Jean Ponce. Prentice Hall, 2003.
  • Digital Image Processing . Bernd Jähne. Springer, 1997.
  • A Wavelet Tour of Signal Processing. Stephane Mallat. Academic Press, 1998.
  • Foundations of Vision. Brian Wandell. Sinauer, 1995.
  • Natural Image Statistics -- A probabilistic approach to early computational vision . Aapo Hyvärinen, Jarmo Hurri, and Patrik O. Hoyer. Springer-Verlag, 2009.
  • Two-dimensional Imaging. Ronald N. Bracewell. Prentice Hall, 1995.

    Background Material (mathematics/engineering)

  • My linear algebra notes: linearAlgebra, least squares estimation.
  • Linear algebra appendix from the PDP series, by Michael Jordan: linearAlgebra
  • My convolution/Fourier notes: linSys
  • Linear Algebra and Its Applications. Gilbert Strang. Academic Press, 1980.
  • Discrete-Time Signal Processing. Alan Oppenheim and Ronald Schafer. Prentice-Hall, 1989.
  • The Scientist and Engineer's Guide to Digital Signal Processing. Steven W. Smith.
  • Elements of Information Theory. Thomas Cover and Joy Thomas. Wiley Series in Telecommunication, 1991.
  • Probability, Random Variables, and Stochastic Processes. A. Papoulis. 3rd edition, McGraw-Hill, 1991.
  • Probability and Statistics, Morris DeGroot and Mark Schervish. Addison-Wesley, 2002.
  • Convex Optimization, Boyd and Vandenberghe. Cambridge University Press, 2004.