Maximum a posteriori natural scenes reconstruction from retinal ganglion cells with deep denoiser priors

E G Wu, N Brackbill, A Sher, A M Litke, E P Simoncelli and EJ Chichilnisky

bioRxiv, Technical Report 2022.05.19.492737, May 2022.

DOI: 10.1101/2022.05.19.492737

This paper has been superseded by:
Maximum a posteriori natural scenes reconstruction from retinal ganglion cells with deep denoiser priors
E G Wu, N Brackbill, A Sher, A M Litke, E P Simoncelli and E J Chichilnisky.
Adv. Neural Information Processing Systems (NeurIPS), vol.35 Dec 2022.


Download:

  • Reprint (pdf)
    • A fraction of the visual information arriving at the retina is transmitted to the brain by signals in the optic nerve, and the brain must rely solely on these signals to make inferences about the visual world. Previous work has probed the visual information contained in retinal signals by reconstructing images from retinal activity using linear regression and nonlinear regression with neural networks. Maximum a posteriori (MAP) reconstruction offers a more general and principled approach. We develop a novel method for approximate MAP reconstruction by combining a generalized linear model of light responses in retinal neurons and their dependence on spike history and spikes of neighboring cells, with an image prior implicitly embedded in a deep convolutional neural network trained for image denoising. We use this method to reconstruct natural images from ex vivo simultaneously-recorded spikes of hundreds of ganglion cells uniformly sampling a region of the retina. The method produces reconstructions that match or exceed the state-of-the-art in perceptual similarity and exhibit additional fine detail, while using substantially fewer model parameters than previous approaches. The use of more rudimentary encoding models (a linear-nonlinear-Poisson cascade) or image priors (a 1/F spectral model) significantly reduces reconstruction performance, indicating the essential role of both components in achieving high-quality reconstructed images from the retinal signal.
  • Related Publications: Zhang22a, Kadkhodaie20a, Pillow08
  • Listing of all publications