A foveated model of visual discrimination based on windowed texture statistics

J Kurzawski, W F Broderick, E P Simoncelli and J Winawer

Published in Workshop on Computational and Mathematical Models in Vision (ModVis), Vision Sciences Society, May 2024.
Most information from visual scenes is discarded by the human nervous system and thus cannot influence visual behavior. Here, we investigated the loss of spatial pattern information. This loss increases dramatically with eccentricity. Recently, models have been developed that average image features in pooling windows whose diameters scale with eccentricity. These models can be used to synthesize "metamers" of natural scenes, images which are physically different but perceptually indistinguishable. Psychophysical experiments have identified the maximum window scaling for which model discrimination abilities match human performance ("critical scaling"). If images are synthesized with pooling windows that exceed critical scaling ("supercritical scaling"), the models make no prediction about performance. The models therefore do not account for the observation that, at supercritical scaling, performance is far better when discriminating between a synthesized image and a natural image, rather than between two synthesized images. We hypothesized that this performance difference arises from differential information loss in downstream processing. To test this, we implemented an observer model that pools image features (high-order texture statistics), followed by divisive normalization and additive noise. We then compared model performance to human performance on images synthesized with pooling models of earlier stage image features (local luminance or local spectral energy) at supercritical scaling. The observer model qualitatively accounted for the primary psychophysical effects, including a performance advantage for discriminating synthesized from natural images. Accurately predicting discriminability of stimuli whose early visual representations do not match requires accounting for information loss in later stages of processing.
  • Listing of all publications