Published in Annual Meeting, Neuroscience, Nov 2018.
As visual information propagates along the ventral pathway, individual neurons respond selectively to stimulus features of increasing complexity. Most neurons in primary visual cortex (V1) respond to oriented gratings, while many neurons of the second visual area (V2) respond to visual textures, with single neurons preferring pseudo-periodic features common among natural scenes. Specifically, recent results indicate that single unit V2 activity is strongly modulated by the presence of higher-order statistics in visual textures, while V1 neurons, driven primarily by local spectral (second-order) content, are largely insensitive to these higher-order statistics. However, it has proven difficult to interpret these findings in the context of physiological mechanisms, or to refine such computations into precise statements of selectivity for individual V2 neurons. Toward these goals we construct a physiologically-plausible model of local texture representation for discriminating naturalistic textures and their spectrally-matched counterparts, optimized over a database of natural textures. We begin by decomposing texture patches with a multi-scale oriented filter bank (a "steerable pyramid"), whose linear basis functions mimic V1 selectivity for orientation and spatial frequency. We then perform principal component analysis (PCA) over the rectified filter responses, and find that a low-dimensional basis is sufficient to capture essential texture statistics. A classification experiment then demonstrates that a small number (approx. 3-5) of these PCA components are sufficient to distinguish naturalistic textures from noise, with remarkably high accuracy given the low number of free parameters. Furthermore, we find that our basis components resemble localized differences (derivatives) spanning four dimensions of V1 selectivity: horizontal and vertical position, orientation, and spatial frequency. Our results suggest that a simple canonical mechanism, operating on V1 afferents with local derivative filters followed by rectifying or squaring nonlinearities (analogous to the construction of V1 simple and complex cells, respectively, from LGN afferents) mimics the enhanced responses of single V2 neurons to higher-order texture statistics. We discuss implications for fitting this model to physiological responses of individual V2 cells, as well as generalizations of our findings to other domains of ventral visual computation (e.g. curvature or border ownership).