Copyright © Division of Cognitive Neuroscience
Teaching
MACHINE LEARNING MEETS HUMAN LEARNING (Spring 2016)
Gedi Luksys
Overview
Machine learning has recently emerged as one of the most successful and practically applicable fields of artificial intelligence: its algorithms are used in search engines, image and text recognition, medical systems, financial markets, and more recently in cognitive neuroscience and genetic studies. This seminar will cover fundamentals of machine learning and review its most important techniques. The main emphasis will be on learning how to use techniques using standard software packages (such as Matlab) and discussing their applications to cognitive neuroscience and behavioral genetics.
Structure
The seminar will take place during the first week of February, between 2pm and 6pm (including 1 hour break during which students will be encouraged to solve short exercises that will discussed at the beginning of the second session) in Nebenhaus (64a) Seminarraum 2.
For each method, we will first discuss the theoretical basis, followed by exercises, applications and/or demos. On February 5 there will be a short quiz to test the understanding of essential concepts.
There will also be an additional session, provisionally scheduled for February 19 2-4pm and March 11 3-6pm, when students will present their selected papers. Please select the papers you would like to present here.
Tentative schedule, topics, and papers to be presented
February 1
14:00-15:45 Introduction to the seminar, fundamentals of supervised learning
16:45-18:00 Artificial neural networks (ANN)
February 2
14:00-14:30 ANN applications
16:00-18:00 ANN demo, support vector machines
February 3
14:00-15:45 Support vector regression, multi-voxel pattern analysis (searchlights)
16:45-18:00 Introduction to unsupervised learning, K-means demo
February 4
14:00-15:30 Probabilistic approaches (Gaussian mixtures, expectation-maximization), Neurosynth
16:30-18:00 Neurosynth demo, component analyses (PCA/ICA)
February 5
14:00-15:45 Quiz, Introduction to reinforcement learning
16:45-18:00 Model-based analysis of learning, memory and decision making
Papers for presentation:
- Kamitani & Tong, "Decoding the visual and subjective contents of the human brain", Nature Neuroscience (2005) - (pdf, 13 pts)
- Dosenbach et al., "Prediction of Individual Brain Maturity Using fMRI", Science (2010) - (pdf, 15 pts)
- Miyawaki et al., "Visual Image Reconstruction from Human Brain Activity using a Combination of Multiscale Local Image Decoders", Neuron (2008) - (pdf, 16 pts)
- Wiestler & Diedrichsen, "Skill learning strengthens cortical representations of motor sequences", eLIFE (2013) - (pdf, 14 pts)
- Helfinstein et al., "Predicting risky choices from brain activity patterns", PNAS (2014) - (pdf, 12 pts)
- Novembre et al., "Genes mirror geography within Europe", Nature (2008) - (pdf, 10 pts)
- Mueller et al., "Dopamine effects on human error processing depend on catechol-O-methyltransferase VAL158MET genotype", Journal of Neuroscience (2011) - (pdf, 12 pts)
- Tanaka et al., "Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops", Nat. Neurosci. (2004) - (pdf, 11 pts)
- Li et al., "Differential roles of human striatum and amygdala in associative learning", Nat. Neurosci. (2011) - (pdf, 8 pts)
- Chib et al., "Neural Mechanisms Underlying Paradoxical Performance for Monetary Incentives Are Driven by Loss Aversion", Neuron (2012) - (pdf, 13 pts)
- Set et al., "Dissociable contribution of prefrontal and striatal dopaminergic genes to learning in economic games", PNAS (2014) - (pdf, 15 pts)
- Zhu et al., "Dissociable neural representations of reinforcement and belief prediction errors underlie strategic learning", PNAS (2012) - (pdf, 13 pts)
Evaluation
As this seminar has no exam, active participation is essential for a successful completion of the course.
Your performance will be evaluated based on points that can be received in a number of different ways:
- Attendance: you earn 1 point for showing up each day and another point for active participation.
- Exercises: one point can be earned by solving exercises each day.
- Papers: by presenting a paper you can earn up to 10-16 points, depending on the length & difficulty of the paper and the quality of your presentation.
- Quiz: by answering questions and solving simple exercises on February 5 you can earn up to 8 points.
- For 4 credits, you may also earn up to 10-25 points by writing an essay (Referat) about applications of machine learning to a field of your choice or by completing a miniproject where you would apply one or more machine learning methods to a problem of your choice (for the latter, some proficiency with Matlab or other package you would use for the analysis is necessary. I could help you with the miniproject, but I would not teach you Matlab/programming basics)
For a pass with 2 ECTS credits you need to earn at least 24 pts, which can be achieved by regular attendance, exercises / quiz and 1 paper presentation.
For 4 ECTS credits, you need 38-58 pts. Your grade will then be points / 10, rounded to the nearest half.
THE PREDICTIVE BRAIN: COMPUTATIONAL MODELS OF LEARNING, MEMORY AND DECISION MAKING (Autumn 2014)
Gedi Luksys
Overview
Learning and memory allows humans and animals to become more capable of making decisions that maximize their evolutionary success. During the recent years it has become popular to analyze neural and behavioural data using computational models that provide more detailed insights into these cognitive processes. In this seminar we will review recent experimental and computational studies of reinforcement learning, associative memory, and decision making. We will also discuss how the influence of genes, emotion, motivation and stress can be modelled.
Structure
The seminar will consist of 13 classes, taking place on Wednesdays at 16:15-17:45, from September 24 till December 17 in Nebenhaus (64a) Seminarraum 4.
Most classes will consist of an introductory lecture, followed by presentation of 1 or 2 papers with the follow-up discussion. On November 5 and December 10 there will also be short quizzes to test the understanding of essential concepts.
Tentative schedule, topics, and papers to be presented
Please select paper(s) to present here.
September 24. Introduction to the seminar, methodology, and overview of brain systems in learning, memory and decision making.
(No papers)
October 1. Fundamentals of Hebbian learning and associative memory
Papers:
Bakker et al., "Pattern separation in the human hippocampal CA3 and dentate gyrus", Science (2008) - (pdf, 8 pts)
Pasupathy & Miller, "Different time courses of learning-related activity in the prefrontal cortex and striatum", Nature (2005) - (pdf, 8 pts)
October 8. Working memory: mechanisms and models.
Papers:
Mongillo et al., "Synaptic theory of working memory", Science (2008) - (pdf, 12 pts)
Lara & Wallis, "Executive control processes underlying multi-item working memory", Nat. Neurosci. (2014) - (pdf, 14 pts)
October 15. Synaptic plasticity and its computational models
Papers:
Kirkwood et al., "Experience-dependent modification of synaptic plasticity in visual cortex", Nature (1996) - (pdf, 9 pts)
Kim & Yoon, "Stress: metaplastic effects in the hippocampus", Trends Neurosci. (1998) - (pdf, 8 pts)
October 22. Neural codes of spatial and episodic memory
Papers:
Wills et al., "Attractor Dynamics in the Hippocampal Representation of the Local Environment", Science (2005) - (pdf, 9 pts)
MacDonald et al., "Hippocampal 'time cells' bridge the gap in memory for discontiguous events", Neuron (2011) - (pdf, 14 pts)
October 29. Consolidation and stability of long-term memory
Papers:
Pastalkova et al., "Storage of spatial information by the maintenance mechanism of LTP", Science (2006) - (pdf, 9 pts)
Tse et al., "Schemas and memory consolidation", Science (2007) - (pdf, 12 pts)
November 5. Heterosynaptic modulation of plasticity and three-factor learning
Quiz #1. Paper:
Seung, "Learning in spiking neural networks by reinforcement of stochastic synaptic transmission", Neuron (2003) - (pdf, 15 pts)
November 12. Dopamine, reward prediction and reinforcement learning
Papers:
Tobler et al., "Adaptive coding of reward value by dopamine neurons", Science (2005) - (pdf, 9 pts)
Samejima et al., "Representation of action-specific reward values in the striatum", Science (2005) - (pdf, 9 pts)
November 19. Action control: balancing exploration and exploitation
Papers:
Daw et al., "Cortical substrates for exploratory decisions in humans", Nature (2006) - (pdf, 8 pts)
Morris et al., "Action-value comparisons in the dorsolateral prefrontal cortex control choice between goal-directed actions", Nat. Comm. (2014) - (pdf, 12 pts)
November 26. Valuation and future discounting in the brain
Papers:
Tanaka et al., "Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops", Nat. Neurosci. (2004) - (pdf, 11 pts)
Yang & Shadlen, "Probabilistic reasoning by neurons", Nature (2007) - (pdf, 11 pts)
December 3. Uncertainty and its neural correlates
Papers:
Hsu et al., "Neural systems responding to degrees of uncertainty in human decision-making", Science (2005) - (pdf, 8 pts)
Yu & Dayan, "Uncertainty, neuromodulation, and attention", Neuron (2005) - (pdf, 15 pts)
December 10. Stress, motivation and neuromodulators in learning and memory
Quiz #2. Paper:
Luksys et al., "Stress, genotype and norepinephrine in the prediction of mouse behavior using reinforcement learning", Nat. Neurosci. (2009) - (pdf, 15 pts)
December 17. Brain-computer interface, patterns and prediction
Papers:
Xue et al., "Greater neural pattern similarity across repetitions is associated with better memory", Science (2010) - (pdf, 9 pts)
Cerf et al., "On-line, voluntary control of human temporal lobe neurons", Nature (2010) - (pdf, 9 pts)
Note: new papers may be added to the list later. If you would like to present another good paper (related to these topics) that is not on the list, please let me know.
Evaluation
As this seminar has no exam, active participation is essential for a successful completion of the course. Questions regarding the main points of each class will be provided in order to facilitate understanding and encourage critical thinking. Answers to these questions will be discussed at the beginning of next class.
Your performance will be evaluated based on points that can be received in a number of different ways:
- For attendance you earn 1 point per class.
- For particularly active and thoughtful participation, 1 additional point per class may be earned.
- By presenting a paper you earn up to 8-15 points, depending on the length & difficulty of the paper and the quality of your presentation.
- By answering questions and solving simple exercises in two quizzes you can earn up to 10 points.
- For 4 credits, you may also earn up to 10-20 points by writing an essay (Referat) on a topic of your interest, related to the seminar.
For a pass with 2 credits you need to earn at least 24 pts, which can be easily achieved by regular attendance, quizzes and 1 paper presentation.
For 4 credits, you need 38-58 pts. Your grade will then be points / 10, rounded to the nearest half.
Dr. Gediminas Luksys, PhD
Postdoctoral Assistant
Computational modelling
