Bill Freeman, Mitsubishi Electric Research Labs (MERL)
March 9, 2001
How to Tell Shading from Paint
Abstract:
When people study a picture, they can judge whether it depicts a
shaded, 3-dimensional surface, or simply a flat surface with markings
or paint on it. This task--distinguishing shading from paint--is
essential for interpreting images. We seek to get a computer to make
the same judgements. We use as "ground truth" a database of pictures
that human subjects had labelled according to their "shadedness" (from
Freeman and Viola '98).
We use a machine learning approach. We generate a training set of
synthetic examples of images that are either caused by shading or
paint, from which we derive probabilistic interpretations for a given
local patch of image. We use a Markov network to model the images and
underlying scenes, and use Bayesian belief propagation to efficiently
propagate the local probabilistic evidence across the image.
The machine learning approach focusses attention on representations.
We contrast two different approaches. One uses a pixel-based image
representation and solves for the shape and reflectance at each
position. The second approach represents image data by a cascaded
energy model, and represents the scene only by a label for the cause
of the image information at each position, scale, and orientation of a
steerable pyramid. We compare the methods, and show results from each
approach.
Joint work with Egon Pasztor (MIT Media Lab) and Matt Bell (Stanford).
References:
Bell and Freeman,
Learning local evidence
for shading and reflectance
Freeman, Pasztor, and Carmichael,
Learning Low-Level
Vision, Intl. J. Computer Vision, October, 2000.
Freeman and Viola,
Bayesian model of
surface perception, NIPS 1998