Neurons transmit information about their input using spikes that are noisy and metabolically costly. In order to maximize efficiency, a neuron's transfer function should balance transmitted information against cost by adapting to the noise and input distributions. For bounded output and small noise, the optimal nonlinearity equalizes the frequency of output levels, consistent with the behavior of certain non-spiking neurons (Laughlin, 1981). More generally, in the limit of vanishing noise variance, the nonlinearity that maximizes mutual information (MI) between input and output can be expressed in terms of their relative noise levels and the input distribution (Nadal and Parga, 1994; McDonnell and Stocks, 2008). Here we extend these results by deriving the nonlinearity that maximizes MI in the presence of non-neglible Gaussian input and output noise and metabolic cost (in bits) that is proportional to firing rate. Provided the nonlinearity is smooth at the scale of the noise, the optimum is a polynomial function of the input/output noise, the metabolic cost parameter, and the input distribution. In contrast to the vanishing noise case, hard rectification (i.e. zero mean firing rate for inputs below some threshold) is optimal for some parameter settings.
We applied this theory to data by testing the optimality of retinal ganglion cell (RGC) responses to binary white noise (Pillow et al, 2008). For each cell, we solved for three parameters (input and output noise and metabolic cost) that maximized the likelihood of observed firing rates. An additional parameter optimized the variance of a Gaussian input distribution across all ON-center cells, and another across OFF-center cells. The model yielded good fits to RGC responses and their variability, offering a potential explanation for observed differences in ON/OFF cell nonlinearities (Chichilnisky and Kalmar, 2002) in terms of noise levels and spiking cost.