Human perception is context-dependent. In addition to sensory context, data from recent psychophysical studies suggest that context can also include previous perceptual decisions (Baldassi etal. PLoS, 4(3):e56, 2006; Jazayeri and Movshon, Nature, 446:912ff, 2007). In both studies, subjects were asked to estimate a stimulus parameter (e.g., the exact orientation angle of a Gabor patch) after being forced to make a binary decision about that parameter (e.g., orientation to the left/right of vertical). On each individual trial, the subjects' estimates were consistent with their preceding decision (i.e., a decision of "left of vertical" was followed by an estimated direction left of vertical). In addition, the distributions of estimates were bimodal, indicating repulsion away from the decision boundary.

We present a probabilistic observer model that accounts for this perceptual behavior. Specifically, we adopt the general hypothesis that the brain attempts to perform optimal estimation of stimulus parameters based on noisy sensory evidence and prior expectations. However, we augment this hypothesis by assuming that the observer performs the secondary estimation task in the belief that his/her previous decision regarding the data was correct. Noisy sensory evidence may initially support both decisions, although with different probability. After making the decision, however, the observer discards all potential estimates that are not in agreement with the choice. This leads to the observed repulsive bias away from the decision boundary. The model fits the data well and makes quantitative predictions for novel experiments.

It is worth noting that the behavior of the model is suboptimal in terms of estimation performance. An optimal (Bayesian) observer model would compute estimates from the sensory evidence under each possible decision taken, and then average these estimates, weighting each according to the probability that the corresponding decision is correct. Thus, our model implies that humans sacrifice performance in order to maintain self-consistency.

Human perception is context-dependent. In addition to sensory context, two recent psychophysical studies have shown that context can also include previous perceptual decisions [1, 2]. In both studies, subjects were asked to estimate a stimulus parameter (orientation [1], or direction of motion [2]) after being forced to make a categorical decision (orientation to the left or right of vertical [1], direction of motion to left or right of a visual reference [2]). On each individual trial, the subjects' estimates were consistent with their preceding decision (i.e., a decision of left of the reference was followed by an estimated direction to the left of the reference). The distribution of estimates were bimodal, indicating repulsion away from the decision boundary (middle panel - data from [2] for a single subject).

We present an observer model that can account for this perceptual behavior. Specifically, we adopt the general hypothesis that the brain attempts to perform optimal estimation of stimulus parameters based on noisy sensory evidence and prior expectations. However, we augment this hypothesis by assuming that the brain performs the secondary estimation task in the belief that its previous decision regarding the data was correct. Noisy sensory evidence may initially support both decisions, although with different probability (left panel: a portion of the likelihood falls under each decision region). However, after being forced to make a decision (e.g. decision B), the observer discards all potential estimates that are not in agreement with the choice (thus all values in the gray shaded area), performing inference conditioned only on the decision made. This leads to the observed repulsive bias away from the decision boundary (right panel). It is worth noting that the behavior of the model (and the human subjects) is suboptimal in terms of estimation performance. An optimal (Bayesian) observer model [3] would compute estimates from the sensory evidence under each possible decision value, and then average these estimates, weighting each according to the probability that the corresponding decision is correct. Thus, our model implies that humans sacrifice performance in order to maintain self-consistency.

[1] S. Baldassi, N. Megna, D.C. Burr. Visual clutter causes high-magnitude errors. PLoS, 4(3):e56, 2006.
[2] M. Jazayeri and J.A. Movshon. A new perceptual illusion reveals mechanisms of sensory decoding. Nature, 446:912ff, April 2007.
[3] D.C. Knill and W. Richards, eds. Perception as Bayesian Inference. Cambridge University Press, 1996.

When visual scenes are ambiguous, the visual system must choose from multiple perceptual interpretations. A classic stimulus that illustrates this idea is a moving plaid, composed by adding two drifting one-dimensional gratings (left panel). Such a stimulus is typically perceived as either a single coherently moving pattern or two transparent components sliding over one another [1]. Previous studies have identified many factors that can influence which of these percepts occurs. But these studies typically explored the effects of a single stimulus variable in isolation.

We performed a series of psychophysical experiments to examine the joint effects of the component speed and the coherent pattern speed on perceptual interpretation. The coherent pattern speed is determined by the component speed and the angle between the two gratings’ directions: vc = vp cos(è/2). We used a forced-choice experimental design in which subjects indicated their percept after a brief presentation (1.5 s) of the stimulus. We found that for a range of intermediate component speeds, the transparent interpretation became more likely when a plaid’s pattern speed was significantly faster than its component speed (middle panel). For both higher and lower component speeds, the percept remains coherent up to higher pattern speeds.

We hypothesize that these behaviors arise from a competition between two visual preferences: the system prefers a single motion to a two-motion interpretation, but the system also prefers slower speed interpretations to faster ones. These two preferences arise naturally in the context of a Bayesian decision model that incorporates a prior that favors slower speeds. We write the probability of the observed stimulus, conditioned on both the interpretation (transparent vs. coherent) and the relevant velocity (of either components or pattern, respectively) using a lognormal likelihood function derived in a previous psychophysical study [2]. The probability conditioned on each interpretation is then multiplied by a power-law prior for slower speeds (also derived in [2]), and integrated over all velocities to generate the likelihoods of the interpretations, which are then compared to determine the “percept.” The perceptual interpretations predicted by this simple model (right panel) are well-matched to the experimental data.

[1] Adelson, E. H. & Movshon, J. A. Phenomenal coherence of moving visual patterns. Nature 300, 523-5, 1982.
[2] Stocker, A. A. & Simoncelli, E. P. Noise characteristics and prior expectations in human visual speed perception. Nat Neurosci 9, 578-85, 2006.

The response properties of sensory neurons change dynamically with the spatial and temporal context. Sensory adaptation, for example, is known to induce a decrease in neurons’ responsivity [1]. How does the rest of the brain interpret the responses of the adaptation-altered neurons? In particular, does the read-out of an adapted population undergo complementary changes to compensate for the adaptation? Perceptually, adaptation leads to distortions in the perception of subsequently presented stimuli [2]. Are these perceptual effects the signature of a particular type of read-out? Although these questions have been raised in the past [1-4], the link between the physiological and perceptual effects of adaptation remains unclear.

We explore these issues in the context of motion direction and contrast adaptation. In our framework, perception is modeled as resulting from an encoder-decoder cascade. The encoder is embodied in the mapping from stimuli to the responses of a population of noisy neurons, which changes adaptively in response to the recent stimulus history. We choose the simplest models for this change: a decrease in the response gain of neurons sensitive to the adaptor for direction adaptation, and a shift in contrast-response curves for contrast adaptation. Several different decoders are considered, which are either fixed and 'unaware' of the adaptation state, or which change dynamically with the encoder, being thus 'aware' of the adaptation state. In each class, the decoders can also either be optimal (e.g. maximum likelihood (ML) and minimum mean-squared error (MMSE) estimators) or suboptimal (e.g. constrained to be linear (OLE) or based on the activity of the most active neuron (winner-take-all)). Using Estimation Theory, we systematically compared the predictions of these read-outs with the psychophysical data for estimation (direction after-effect, changes in apparent contrast) and discrimination tasks. They are also compared with the predictions of Fisher Information for discriminability.

We find that the 'aware' decoders that we tested (ML,MMSE with flat prior, OLE) predict direction and contrast estimates that are matched on average to the physical stimulus, inconsistent with the reported perceptual biases after adaptation. On the other hand, simple models of neural adaptation coupled with 'unaware' readouts (population vector, ML optimized for the pre-adaptation responses) account well for both estimation and discrimination psychophysical performance. We also find that Fisher information, while being naturally linked with the variance of optimal unbiased read-outs, still provides a meaningful lower bound for the discrimination threshold of biased 'unaware' read-outs. We discuss the significance of having read-outs that would be fixed on a short time-scale compatible with adaptation as well as and their possible relevance in other phenomena.

[1] Visual adaptation: physiology, mechanisms, and functional benefits. A. Kohn, J Neurophysiol, 97(5):3155-64, 2007.
[2] Perceptual adaptation: motion parallels orientation. C.W. Clifford. TICS, 6(3):136-143, 2002.
[3] Efficiency and ambiguity in an adaptive neural code. A. Fairhall et al, Nature, 412:787-92, 2001.
[4] Space and time in visual context. O. Schwartz et al, Nat Rev Neurosci, 8(7):522-35, 2007.

There is ample evidence that humans have the ability to estimate local retinal motion. These estimates are typically not veridical, but are biased by non-motion stimulus characteristics (e.g. contrast, spatial pattern) and the system's contextual state (e.g. attention, adaptation). A complete characterization of human speed perception should thus incorporate all of these effects. Here, we focus on adaptation, and characterize its influence on both the bias (i.e. shift in perceived speed) and variance (i.e. discrimination threshold) of subsequent estimates. We measured the perceived speed of a spatially broadband noise stimulus with veridical speed chosen from the range 0.5-16 deg/s in either horizontal direction, for several different adaptor speeds. Subjective responses were gathered using a 2AFC discrimination paradigm, with a simultaneous presentation of a reference and test stimulus within 3deg apertures on either side of fixation. The reference location was adapted, initially for 40s, and for an additional 5s between each trial.

We find that adaptation affects the subsequent estimation of stimulus speed over the entire range of speeds tested and across direction boundaries. The bias, relative to the unadapted percept, is repulsive yet asymmetric, with a perceived speed at the adaptor that is typically reduced. Discrimination thresholds, measured as the slope of the psychometric function gathered under each reference/test condition, typically increase around the adaptor speed. However, using signal detection theory, we can infer the change in variability and bias of the estimate of the reference speed due to adaptation and predict the discriminability that would result if both the test and reference locations were adapted. We predict a clear increase in discriminability around the adaptor, consistent with some previous literature.

We discuss the relationship of these findings to our previously proposed Bayesian model of speed perception, as well as the implications for the brain's internal representation of retinal speed.

With the goal of arriving at a population model that would link the physiology with the psychophysics, we measured the effect of contrast on velocity tuning in anesthetized macaque area MT. We chose broadband gratings, consisting of three sinusoidal gratings of fixed spatial frequency (0.5 c/deg, 1 c/deg, 2 c/deg) summed with randomized phases, moving at 8 different speeds (over the range [3,60 deg/sec]),at three contrasts (5%, 20% and 80%) on a gray background. We presented the same set of stimuli to every cell, without optimizing for the spatial frequency or speed preferences.

Across the population of recorded neurons (n= 41), we find that reducing the contrast from 80% to 20% reduces the preferred velocity of the neuron by a factor of ~2.7. Although the measurements were noisier, reducing the contrast to 5% produced an even larger shift. This effect was more pronounced in cells that preferred high speeds at high contrast. By recording the same data set on every neuron without optimizing for the neuron's preference, and using the same stimuli used in a recent psychophysical study [5], our data set supports the formulation of a quantitative model linking the physiology with behavior.

[1] Perceived rate of movement depends on contrast. P. Thompson, Vision Research 22:377-380, 1982.
[2] Estimating target speed from the population response in visual area MT. N. Priebe and S. Lisberger, Journal of Neuroscience 24, 1907-1916.
[3] Contrast dependence of suppressive influences in cortical area MT of alert macaque. C. Pack, J. Hunter, and R. Born, Journal of Neurophysiology 93:1809-1815, 2005.
[4] Interactions between speed and contrast tuning in the middle temporal area: implications for the neural code for speed. B. Krekelberg, R. J. van Wezel, and T. D. Albright, Journal of Neuroscience 26(35):8988-98, 2006.
[5] Noise characteristics and prior expectations in human visual perception. A. A. Stocker and E. P. Simoncelli, Nature Neuroscience 9(4):578-585, 2006.

A growing number of studies support the notion that humans apply an optimal or near-optimal strategy when performing a perceptual estimation task that combines the sensory observations with a priori knowledge as defined by Bayes' rule. A Bayesian framework provides a principled yet simple computational framework for perception that can account for a large number of known perceptual effects and illusions.

Adaptation is a fundamental phenomenon in sensory perception that seems to occur at all processing levels and modalities. A variety of computational principles have been suggested as explanations for adaptation. Many of these are based on the concept of maximizing the sensory information an observer can obtain about a stimulus despite limited sensory resources, which is similar to the concept of redundancy reduction and efficient representation. More mechanistically, adaptation can be interpreted as the attempt of the sensory system to adjusts its limited dynamic range such that it is maximally informative with respect to the statistics of the stimulus. Perceptually, adaptation seems to have two fundamental effects. First, subsequent stimuli are repelled by the adaptor stimulus, i.e. the perceived values of the stimulus variable that is subject to the perception task are more distant to the adaptor value after adaptation. Second, adaptation leads to an increase in the observer's discrimination ability around the adaptor value, whereas it decreases further away from the adaptor.

If a Bayesian framework is to provide a valid computational explanation of perceptual processes, then it needs to account for the behavior of a perceptual system, regardless of its adaptation state. So far, it has not been shown how adaptation could be in agreement with a Bayesian framework of perception. We extend our previously developed Bayesian framework for perception [1] to account for adaptation.

We first note that the perceptual effects of adaptation seems inconsistent with an adjustment of the internally represented prior distribution. Instead, we postulate that adaptation increases the signal-to-noise ratio of the measurements by re-allocating the limited resources of the measurement stage to the input range. We show that this re-allocation changes the likelihood function in such a way that the Bayesian estimator model accounts for reported perceptual behavior. In particular, we compare the model's predictions to human motion direction discrimination data and demonstrate that the model well predicts the characteristics of the observed adaption effects of repulsion and changes in discrimination threshold. The proposed model shows for the first time how adaptation can be incorporated into a Bayesian framework of perception. It represents a link between bottom-up processing guided by efficient coding and a top-down Bayesian estimation process.

[1] A.A. Stocker and E.P. Simoncelli. Constraining a Bayesian Model of Human Visual Speed Perception. In: Advances in Neural Information Processing and Systems NIPS, vol.17, May 2005, p.1361-1368