Resources

Computational modeling of brain and behavior

Tutorials and textbooks:

• Ten simple rules for the computational modeling of behavioral data Wilson and Collins in eLife (2019) A must-read for anyone who wants to start working on computational modeling of brain and behavior
• Theoretical neuroscience: computational and mathematical modeling of neural systems Dayan and Abbot 2001
• Neuronal dynamics: from single neurons to networks and models of cognition Gerstner et al. 2014
• Computational foundations of cognitive neuroscience Gershman 2025

Example papers:

• Model-based influences on humans' choices and striatal prediction errors Daw et al. in Neuron (2011) A classic example of a theory-guided experimental design to dissociate the predictions of different models
• Human inferences about sequences: a minimal transition probability model Meyniel et al. in PLOS Computational Biology (2016) A neat example of Bayesian modeling of neural and behavioral data
• Reconciling novelty and complexity through a rational analysis of curiosity Dubey and Griffiths in Psychological Review (2020) A great example of how normative modeling reconciles fragmented theories
• Using large-scale experiments and machine learning to discover theories of human decision-making Peterson et al. in Science (2021) An excellent example of using modern machine learning methods for modeling human behavior
• Computational models of episodic-like memory in food-caching birds Brea et al. in Nature Communications (2023) A successful example of integrating and testing traditional theories via computational models

Reinforcement learning and optimal control

Tutorials and textbooks:

• Markov Decision Processes: Discrete Stochastic Dynamic Programming Puterman 1994 A math-heavy textbook to study the theoretical foundation of reinforcement learning
• Reinforcement Learning: An Introduction Sutton and Barto 2018 The go-to reinforcement learning textbook
• Algorithms for Reinforcement Learning Szepesvari 2019 A short textbook summarizing several key reinforcement learning algorithms along with their theoretical properties

Example papers:

• Near-Bayesian exploration in polynomial time Kolter and Ng in ICML (2009) A classic example of theoretically justified algorithm design for efficient exploration
• Unifying count-based exploration and intrinsic motivation Bellemare et al. in NeurIPS (2016) An elegant example of translating theoretical ideas into deep reinforcement learning
• Learning to reinforcement learn Wang et al. on arXiv (2017) The classic paper introducing meta reinforcement learning
• Diversity is all you need: learning skills without a reward function Eysenbach et al. in ICLR (2019) A great example of unsupervised, general-purpose skill-discovery
• Discovering and achieving goals via world models Menconda et al. in NeurIPS (2021) An excellent example of integrating several algorithmic modules for goal-conditioned reinforcement learning

My general go-to textbooks

• Elements of Information Theory Thomas and Cover 1991 This is among the most well-written textbooks I have ever seen!
• Information Theory, Inference and Learning Algorithms MacKay 2003
• All of Statistics: A Concise Course in Statistical Inference Wasserman 2004 It's pretty much exactly what its title claims!
• The Elements of Statistical Learning Friedman et al. 2009 Deep learning and reinforcement learning aside, this can reasonably be regarded as a "complete" textbook on machine learning
• Bayesian Reasoning and Machine Learning Barber 2012 This is particularly useful for probabilistic graphical models
• Computer Age Statistical Inference: Algorithms, Evidence, and Data Science Efron and Hastie 2016 I LOVE this book! Particularly, the first few chapters on Frequentist, Bayesian, and Fisherian approaches to statistics

Some general resources to become a better researcher

• To become better at presenting: “The Sense of Style” by Steven Pinker and “How to Speak” by Patrick Winston
• To become better at doing research: “You and Your Research” by Richard Hamming and “Advice for Young Investigators” by Sam Gershman
• To become better at understanding how science works: “The Structure of Scientific Revolutions” by Thomas Kuhn

A non-exhaustive list of other cool papers!

Neuroscience and psychology

• The role of the exploratory drive in learningMontgomery in Journal of Comparative and Physiological Psychology (1954)
• The effect of stimulus sequence on the waveform of the cortical event-related potentialSquires et al. in Science (1976)
• Prospect theory: an analysis of decision under riskKahneman and Tversky in Econometrica (1979)
• Statistical computations underlying the dynamics of memory updatingGershman et al. in PLOS Computational Biology (2014)
• Performance-optimized hierarchical models predict neural responses in higher visual cortexYamins et al. in PNAS (2014)
• Diverse motives for human curiosityKobayashi et al. in Nature Human Behaviour (2019)
• Pinpointing the neural signatures of single-exposure visual recognition memoryMehrpour et al. in PNAS (2021)
• A primate temporal cortex–zona incerta pathway for novelty seekingOgasawara et al. in Nature Neuroscience (2022)
• A prospective code for value in the serotonin systemHarkin et al. in Nature (2025)
• Emergent rate-based dynamics in duplicate-free populations of spiking neuronsSchmutz et al. in Physical Review Letters (2025)

Machine learning and statistics

• Learning to achieve goalsKaelbling in IJCAI (1993)
• Bayesian online changepoint detectionAdams and MacKay on arXiv (2007)
• An analysis of model-based interval estimation for Markov Decision ProcessesStrehl and Littman in Journal of Computer and System Sciences (2008)
• Intrinsically motivated reinforcement learning: an evolutionary perspectiveSingh et al. in IEEE Transactions on Autonomous Mental Development (2010)
• A distributional perspective on reinforcement learningBellemare et al. in ICML (2017)
• C-Learning: learning to achieve goals via recursive classificationEysenbach et al. in ICLR (2021)
• Geometry of the loss landscape in overparameterized neural networks: symmetries and invariancesSimsek et al. in ICML (2021)
• Cross-validation: what does it estimate and how well does it do it?Bates et al. in Journal of the American Statistical Association (2023)
• Learning to assist humans without inferring rewardsMyers et al. in NeurIPS (2024)

Other domains

• A triangular theory of loveSteinberg in Psychological Review (1986)
• What might cognition be, if not computation?van Gelder in the Journal of Philosophy (1995)
• Doing better but feeling worse: looking for the “best” job undermines satisfactionIyengar et al. in Psychological Science (2006)
• The optimism biasSharot in Current Biology (2011)
• Language left behind on social media exposes the emotional and cognitive costs of a romantic breakupSeraj et al. in PNAS (2020)
• OperationalismChang in Stanford Encyclopedia of Philosophy (2021)
• The effect of seeing scientists as intellectually humble on trust in scientists and their researchKoetke et al. in Nature Human Behaviour (2025)
• Large teams develop and small teams disrupt science and technologyWu et al. in Nature (2026)