The perception of complex visual patterns emerges from neuronal activity in a cascade of areas in the cerebral cortex. As information propagates along this hierarchy, neuronal responses become more selective for particular features of natural images and more tolerant to image transformations that preserve those features. Despite this generally accepted framework for visual processing, we lack satisfying descriptions of the neural computations supporting visual representations at intermediate stages of this pathway. The role of the second visual area (V2) in pattern vision has been particularly enigmatic, partly because no simple response properties robustly distinguish V2 neurons from their inputs in primary visual cortex (V1). Previous approaches to midlevel vision and V2 have employed intuitions about features of intermediate complexity in natural images, or have attempted to build models to predict neuronal responses to arbitrary natural images. Here, we employ a new approach, constructing targeted, naturalistic stimuli by building on insights from models of the V1 inputs into V2, the statistics of natural images, and perception. We found that V2 neurons, but not V1 neurons, responded more vigorously to naturalistic texture stimuli than to control stimuli that lacked the statistical dependencies found in natural images. V2 responses were also linked to both the detection and discrimination of naturalistic structure in perceptual experiments in humans and nonhuman primates. Together, our results suggest a specific role for V2 in visual perception, and a framework for approaching midlevel computations in hierarchical sensory transformations.