Plenoptic: A python library for synthesizing model-optimized visual stimuli

W F Broderick, Eddoardo Balzani, K Bonnen, H Dettki, L Duong, P-E Fiquet, D Herrera-Esposito, N Parthasarathy, T Yerxa, X Zhao, E P Simoncelli.

Published in Annual Meeting, Vision Sciences Society, vol.25 May 2025.
In sensory perception and neuroscience, new computational models are most often tested and compared in terms of their ability to fit existing data sets. However, experimental data are inherently limited and complex models often saturate their explainable variance, resulting in similar performance across models. Here, we present "Plenoptic", a python software library for synthesizing model-optimized visual stimuli for understanding, testing, and comparing models. Plenoptic provides a unified framework containing three previously-published synthesis methods -- model metamers (Freeman and Simoncelli, 2011), Maximum Differentiation (MAD) competition (Wang and Simoncelli, 2008), and eigen-distortions (Berardino et al. 2017) -- which enable visualization of different aspects of model representations. The resulting images can then be used to experimentally test model alignment with biological visual systems. Plenoptic leverages modern machine-learning methods to enable application of these synthesis methods to any computational model that satisfies a small set of common requirements: the model must be image-computable, implemented in PyTorch, and end-to-end differentiable. The package includes examples of several previously-published low- and mid-level visual models, as well as a set of perceptual quality metrics, and is compatible with the pre-trained machine learning models included in PyTorch's torchvision library. Plenoptic is open source, tested, documented, modular, and extensible, allowing the broader research community to contribute new models, examples, and methods. In summary, Plenoptic leverages machine learning tools to tighten the scientific hypothesis-testing loop, facilitating the development of computational models aligned with biological visual representations.
  • Listing of all publications