Deep Learning Fall 2019: Foundations of Deep Learning


Mondays 5:00 pm-7:30pm at Room 130, Sociology-Psychology Building, Duke University, beginning Monday, August 26, 2019
No class during week of Thanksgiving (November 25, 2019)
Last class: Monday, December 2, 2019

Instructor: This course will be taught by David Banks, Professor of the Practice of Statistics, Duke University & Director, SAMSI

Course Outline:
This course is being offered in conjunction with the SAMSI semester-long research program on Deep Learning. The course will start with a review of standard neural networks, and then progress to modern deep learning, including convolutional neural networks, recursive neural networks, generative adversarial networks, and various kinds of autoencoders. We shall discuss training strategies, architecture search, regularization and quantization.

There will be mathematics in the course, and a degree of mathematical sophistication is expected from the students, but the material will all be self-contained. The emphasis will be upon heuristics and applications. There will be projects and presentations at the end of the semester, and students will work on those in small groups. Each group will need to have at least one member who can program in Python or a comparable language.

Assignment Collaboration Folder

** Click HERE to post homework assignments or other deliverables assigned by instructor **


  • NCSU – MA 810.001 / ST 810 005
  • UNC-CH – STOR  894.001 and MATH 892.001
  • Duke STA 790.01 / MA 790-71.01

Monday, August 26, 2019

Date Topic Speaker Slides
August 26, 2019 Deep Learning – Applications and Overview David Banks
Homework Assignment – Data Access Word Doc
Team Sign Up Information Word Doc
September 2, 2019 Probability and Information Theory David Banks
September 9, 2019 Numerical Computation David Banks
September 16, 2019 Machine Learning Basics
September 23, 2019 Deep Feed Forward Networks
September 30, 2019 Regularization
Oct 7, 2019 Fall Break – NO CLASS
October 14, 2019 Optimization
October 21, 2019 Convolutional Neural Networks
October 28, 2019 Recurrent and Recursive Neural Networks
November 4, 2019 Guest Lecture – Empirical Risk
November 11, 2019 Autoencoders
November 18, 2019 Generative Adversarial Networks
November 25, 2019 Practical Issues/Tricks