|
 |
Nathaniel Daw
Computational Neuroscience of Learning and Decision Making
I study how people and animals learn from trial and error (and from
rewards and punishments) to make decisions, combining computational,
neural, and behavioral perspectives. Having trained partly in
computer science, I am particularly interested in machine learning,
reinforcement learning, and Bayesian techniques as frameworks for understanding
and analyzing biological decision making. I therefore focus on
how the brain copes with the sorts of computationally demanding
decision situations that these methods address, such as choice under
uncertainty and in tasks (such as mazes or chess) requiring many decisions
to be made in sequence.
After studying philosophy and physics as an undergraduate at Columbia
University, I went to graduate school in the Computer Science Department
at Carnegie Mellon University. Concurrently, I received training in
cognitive neuroscience at the Center for the Neural Basis of Cognition.
After completing my Ph.D. in 2003 (on reinforcement learning in the
dopamine system) I received a Royal Society USA Research Fellowship to
pursue postdoctoral research at the Gatsby Computational Neuroscience
Unit, University College London. I arrived at NYU in 2007 as an
assistant professor in the Center for Neural Science and the Psychology
Department. Some ongoing projects include:
Computational models in neuroscientific experiments.
Computational models are more than cartoons: they can provide
detailed trial-by-trial hypotheses about how subjects might approach
tasks such as decision making. By fitting such models to raw behavioral
and neural data, and comparing different candidates, we can understand
in detail the processes underlying subjects' choices. Such models
can also quantify otherwise subjective phenomena (such as the
expectation of reward or punishment), allowing the study of their
neural representations. Methodologically, I am interested in
developing these techniques for issues such as how to pool heterogeneous
data sources (e.g., simultaneously obtained choice behavior,
eye monitoring, and BOLD signals from multiple brain areas).
Practically, I have applied these methods in behavioral and functional
imaging experiments to study learned choice in humans.
Interactions between multiple decision-making systems. The idea
that the brain contains multiple, separate decision systems is as
ubiquitous (in psychology, neuroscience, and even economics) as it is
bizarre. For instance, much evidence points to competition between a
reflective or cognitive planning system centered in prefrontal cortex,
and a more reflexive 'habitual' controller associated with dopamine
and the basal ganglia. Such competition has often been implicated in
self-control issues such as dieting or drug addiction. But, as these
examples suggest, having multiple decision systems actually compounds
the decision problem, by requiring the brain to choose between
the systems. The computational underpinnings and neural substrates for this sort
of arbitration are poorly understood. I have developed computational models
of how (and why) multiple decision making systems interact; armed with
such a detailed characterization, we are beginning to search for the
fingerprints of these interactions in human behavior and functional imaging.
Learning and neuromodulation. There is much evidence for
the idea that dopamine serves as a teaching signal for reinforcement
learning. This relatively good understanding can now provide a
foothold for investigations in a number of new directions.
These include computational (e.g., how can this system balance the need
to explore and learn about unfamiliar options versus exploiting known good ones),
behavioral (how is dopaminergically mediated learning manifest; how is it deficient in
pathologies such as drug addiction or Parkinson's disease), and neural (what is the
contribution of systems that interact with dopamine, such as serotonin and the
prefrontal cortex). One example that crosscuts these categories is the interaction
of appetitive and aversive learning. Psychologists have long suggested that the
brain contains parallel, opponent motivational systems for reward and punishment;
the identification of the former with dopamine allowed us to suggest an account
of serotonin as its opponent for aversive learning. We are presently investigating
these ideas using human imaging and pharmacological manipulations in tasks
involving decision making for both reward and punishment.
Representative Publications
Courvile, A.C., Daw, N.D., and Touretzky, D.S. (2004), Similarity and
discrimination in classical conditioning: A latent variable account, Advances in Neural Information Processing Systems 17:313-320.
Daw, N.D., Niv, Y., and Dayan, P. (2005) Uncertainty-based competition
between prefrontal and dorsolateral striatal systems for behavioral control,
Nature Neuroscience 8:1704-1711.
Daw, N.D., O'Doherty, J.P., Dayan, P., Seymour, B., and Dolan, R.J.
(2006) Cortical substrates for exploratory decisions in humans, Nature 441:876-879.
Courville, A.C., Daw, N.D., and Touretzky, D.S. (2006) Bayesian
theories of conditioning in a changing world, Trends in Cognitive Sciences 10:294-300.
Daw, N.D., Courville, A.C., and Touretzky, D.S. (2006) Representation and timing in theories of the dopamine system, Neural Computation 18:1637-1677.
Dayan, P.D., Niv, Y., Seymour, B., and Daw, N.D. (2006) The misbehavior
of value and the discipline of the will, Neural Networks 19:1153-1160."
Niv, Y., Daw, N.D., and Dayan, P. (2006) Tonic dopamine: Opportunity
costs and the control of response vigor, In press, Psychopharmacology.
|
|