Liam Paninski, Feb. 8, 2001
<br>
<br>
I will discuss some work in progress on the estimation of the entropy
of a random variable and the mutual information between a pair of
random variables, where the variables can take values in abstract
spaces (such as the space of all possible spike trains, for example).
I've proven some novel results on the bias, variance, consistency,
sufficiency, and asymptotic behavior of some standard estimators, as
well as a few results which hold in general, for any estimator.  In
addition, I propose a new (minimax) estimator and give an algorithm
for its computation.
<br>
<br>
The presentation will be split up into two parts: first, I will
introduce the necessary statistical concepts (defining the terms
mentioned above), outlining my approach and a few of the major results
(emphasizing the implications for data analysis and for some of the
results that have appeared in the literature).  This will take about
30-40 minutes. Then we can take a break for questions (providing a
convenient exit for anyone less interested in the technical aspects of
the work to follow), and I'll proceed with some of the precise
statements and (brief) sketches of some proofs, concentrating more on
the mathematical content of the work than the applicability to neural
data.
<br>
<br>
<br>
refs:
<br>
<br>
\bibitem[Cover and Thomas, `91]{CT91} Cover TM, Thomas JA.
\underline{Elements of information theory}.  John Wiley and Sons, New
York (1991).
<br>
<br>
\bibitem[Devroye et al., `96]{DEV96} Devroye, L., Gy\"orfi, L. \&
Lugosi, G.  \underline{A Probabilistic Theory of Pattern
Recognition}. Springer-Verlag, NYC, NY (1996).
<br>
<br>
\bibitem[Treves and Panzeri, `95]{TP95} Treves, A \& Panzeri, S.  The
upward bias in measures of information derived from limited data
samples.  Neural Computation 7: 399-407 (1995).
<br>
<br>
\bibitem[Strong et al., `98]{STR98} Strong, S. Koberle, R., de Ruyter
van Steveninck R. \& Bialek, W.  Entropy and information in neural spike
trains.  Physical Review Letters 80: 197-202 (1998).
<br>
<br>
\bibitem[Bura\v cas et al., `98]{BUR98} Bura\v cas, G. T., Zador,
A. M., DeWeese, M. R.  \& Albright, T.D.  Efficient discrimination of
temporal patterns by motion-sensitive neurons in primate visual
cortex.  Neuron 5, 959-969 (1998).