Eero P. Simoncelli
Computational Vision		 Home page  |  Lab home page  |  HHMI home page
Vision @ NYU  |  Computational Vision Links

My research addresses a variety of basic issues in the analysis and representation of visual images. The work is interdisciplinary, spanning computational neuroscience, image processing, and computer vision. Three broad goals provide the motivation for this work: 1) construction of mathematical theories for the representation of visual information, 2) development of functional models for biological visual processing, and 3) creation of novel algorithms for image processing and computer vision applications. Current major research topics include:

  • Statistical models of visual images. When coupled with a statistical characterization of their coefficients, wavelets can provide a "prior" model for natural images. We have used such prior models in a number of image processing applications (e.g., image compression, noise removal and image enhancement, analysis and synthesis of texture). Furthermore, the computations required to properly decompose signals generated according to these statistical models are remarkably similar to the divisive normalization computation used in neural models (see above). As such, these models of natural image statistics provide a theoretical justification for these neural models, and offer the opportunity to test directly (through physiological measurements) the ecological hypothesis that visual neural computations are optimally matched to the statistics of images.
  • A computational model for visual motion processing. This is a functional model, derived from fundamental properties of motion information encoded in visual signals. The computation is performed in two stages of identical architecture, corresponding to neurons in cortical areas V1 and MT. Each stage computes a weighted linear sum of inputs, followed by rectification and divisive normalization. The output of the model corresponds to the steady-state firing rates of a population of MT neurons, which form a distributed representation (population encoding) of image velocity for each local spatial region of the visual stimulus. The model accounts for a wide range of physiological data, and a related model is consistent with a surprising variety of human perceptual phenomena.
    • Image statistics and cortical normalization models
  • Multi-scale multi-orientation image representation. Starting with the work of my Master's Thesis in 1986, I've devoted a large proportion of my research efforts toward development of representations of this type, suitable for visual image processing. Recent work includes the development of the steerable pyramid representation that, unlike classical "wavelet" representations, has both translation- and rotation-invariance properties. This is essentially a tool for performing multi-scale differential analysis of imagery. Current work includes applications of these representations to problems of orientation analysis and rotation-invariant pattern matching, and texture and motion processing.

    • I started my higher education as a physics major at Harvard, went to Cambridge University on a Knox Fellowship to study Mathematics for a year and a half, and then returned to the States to pursue a doctorate in Electrical Engineering and Computer Science at MIT. I received my Ph.D. in 1993, and joined the faculty of the Computer and Information Science department at U Pennsylvania. I came to NYU in September of 1996, as part of the Sloan Center for Theoretical Visual Neuroscience. I received an NSF Faculty Early Career Development (CAREER) grant in September '96, for research and teaching in "Visual Information Processing", and a Sloan Research Fellowship in February of 1998. In August 2000, I became an Associate Investigator of the Howard Hughes Medical Institute, under their new program in Computational Biology.

    Representative Publications (Reprints available online):

    E P Simoncelli and J Pillow and L Paninski and O Schwartz. Characterization of Neural Responses with Stochastic Stimuli. In The New New Cognitive Neurosciences, 3rd edition, Ed: M Gazzaniga. MIT Press. To appear: 2004.

    Z Wang, A C Bovik, H R Sheikh and E P Simoncelli. Image quality assessment: From error measurement to structural similarity. To appear: IEEE Trans. Image Processing, 2003.

    J Portilla, V Strela, M Wainwright and E P Simoncelli. Image denoising using a scale mixture of Gaussians in the wavelet domain. To appear: IEEE Trans. Image Processing, 12(12), 2003.

    N C Rust, O Schwartz, J A Movshon and E P Simoncelli. Spike-triggered characterization of excitatory and suppressive stimulus dimensions in monkey V1 directionally selective neurons. Presented at: Annual Meeting, Computational Neuroscience, July 2003.

    J W Pillow, L Paninski and E P Simoncelli. Maximum likelihood estimation of a stochastic integrate-and-fire neural model. Presented at: Annual Meeting, Computational Neuroscience, July 2003.

    E P Simoncelli. Local analysis of visual motion. In The Visual Neurosciences, Eds. L M Chalupa and J S Werner, MIT Press. To appear: 2003.

    E P Simoncelli. Vision and the statistics of the visual environment Current Opinion in Neurobiology, 13, Apr 2003.

    Y Weiss, E P Simoncelli, and E H Adelson. Motion illusions as optimal percepts. Nature Neuroscience, 5(6):598-604, Jun 2002.

    J Pillow and E Simoncelli. Biases in white noise analysis due to non-Poisson spike generation. Neurocomputing, 52-54:109-115, 2003.

    O Schwartz, EJ Chichilnisky, and E P Simoncelli. Characterizing neural gain control using spike-triggered covariance. In Adv. Neural Information Processing Systems (NIPS*01), v14, May 2002.

    M Wainwright, O Schwartz and E P Simoncelli. Natural image statistics and divisive normalization: Modeling nonlinearities and adaptation in cortical neurons. In Statistical Theories of the Brain, eds. R Rao, B Olshausen and M Lewicki, MIT Press. Spring 2002.

    O Schwartz and E P Simoncelli, Natural signal statistics and sensory gain control. Nature:Neuroscience, 4(8):819-825, Aug 2001.

    M Wainwright, E Simoncelli, and A Willsky. Random cascades on wavelet trees and their use in modeling and analyzing natural imagery. Journal of Applied and Computational Harmonic Analysis. 11(1): Jul 2001.

    E P Simoncelli and B Olshausen. Natural image statistics and neural representation. Annual Review of Neuroscience, 24:1193-1216, May 2001.

    P Schrater, D Knill, and E Simoncelli. Perceiving visual expansion without optic flow. Nature. 410:816-819, 12 Apr 2001.

    J Portilla and E P Simoncelli. A parametric texture model based on joint statistics of complex wavelet coefficients. Int'l Journal of Computer Vision. 40(1):49-71, Dec 2000.

    P R Schrater, D C Knill, and E P Simoncelli. Mechanisms of visual motion detection. Nature:Neuroscience. 3(1):64-68, Jan 2000.

    M J Wainwright and E P Simoncelli. Scale mixtures of Gaussians and the statistics of natural images. In Adv. Neural Information Processing Systems (NIPS*99), 12, May 2000.

    R W Buccigrossi and E P Simoncelli. Image compression via joint statistical characterization in the wavelet domain. IEEE Trans. Image Processing, 8(12):1688-1701, Dec 1999.

    E P Simoncelli. Bayesian denoising of visual images in the wavelet domain. Chapter 18 of Bayesian Inference in Wavelet Based Models. eds. P Müller and B Vidakovic. pp. 291-308, Springer-Verlag, New York. Lecture Notes in Statistics 141, Jun 1999.

    E P Simoncelli. Bayesian multi-scale differential optic flow. Volume 2, chapter 14 of Handbook on Computer Vision and Applications. eds. B Jähne, H Haussecker, and P Geissler. pp 397-422, Apr 1999. Academic Press, San Diego.

    E P Simoncelli. Statistical models for images: Compression, restoration and synthesis. In 31st Asilomar Conf on Signals, Systems and Computers Pacific Grove, CA. Nov 1997.

    H Farid and E P Simoncelli. Range estimation by optical differentiation. J. Optical Society of America, A. 15(7):1777-1786, Jul 1998.

    P R Schrater and E P Simoncelli. Local velocity representation: Evidence from motion adaptation. Vision Research. 38(24):3899-3912, 1998.

    E P Simoncelli and D J Heeger. A model of neural responses in visual area MT. Vision Research, 38(5):743-761, 1998.

    E P Simoncelli and W T Freeman. The steerable pyramid: A flexible architecture for multi-scale derivative computation. In 2nd IEEE Int'l Conf on Image Processing, Washington, DC, Oct 1995.

    E P Simoncelli. Design of multi-dimensional derivative filters. In First IEEE Int'l Conf on Image Processing. Austin TX, Nov 1994.

    J Nimeroff, E Simoncelli, and J Dorsey. Efficient re-rendering of naturally illuminated environments. Proc. Fifth Annual Eurographics Symposium on Rendering. Darmstadt Germany, Jun 1994.

    E P Simoncelli. Distributed analysis and representation of visual motion. Ph.D. Thesis, Dept. Electrical Engineering and Computer Science, MIT, Jan 1993.

    E P Simoncelli, W T Freeman, E H Adelson, and D J Heeger. Shiftable multi-scale transforms [or "What's wrong with orthonormal wavelets"]. IEEE Trans. Information Theory, 38(2):587-607, Mar 1992.

    E P Simoncelli. Orthogonal sub-band image transforms [or "Orthonormal wavelets for image representation"]. Masters Thesis, Dept. Electrical Engineering and Computer Science, MIT, May 1988.

       The Center for Neural Science   Related Organizations
    Faculty Other Members    Events      Contact Information  

    All contents copyright © New York University.   All rights reserved.
    Revised: 30 September 2004.