“Deep Learning” systems, typified by deep neural networks, are increasingly taking over all AI tasks, ranging from language understanding, and speech and image recognition, to machine translation, planning, and even game playing and autonomous driving. As a result, expertise in deep learning is fast changing from an esoteric desirable to a mandatory prerequ isite in many advanced academic settings, and a large advantage in the industrial job market.
In this course we will learn about the basics of deep neural networks, and their applications to various AI tasks. By the end of the course, it is expected that students will have significant familiarity with the subject, and to be able to apply to them to a variety of tasks. They will also be positioned to understand much of the current literature on the topic and extend their knowledge through further study.
Instructor: Bhiksha Raj
Time: Mondays, Thursdays, 9.00am-10.20am
This course is worth 12 units.
Grading will be based on weekly quizzes, homework assignments and a final project. There will be six assignments in all.
|Quizzes||12 or 13, total contribution to grade 25%|
|Assignments||6, total contribution to grade 50%|
|Project||Total contribution to grade: 25%|
Deep learning is a relatively new, fast developing topic, and there are no standard textbooks on the subject that cover the state-of-art, although there are several excellent tutorial books that one can refer to. The topics in this course are collected from a variety of sources, including recent papers. As a result, we do not specify a single standard textbook. However, we list a number of useful books at the end of this page, which we greatly encourage students to read, as they will provide much of the background for the course. We will also put up links to relevant reading material for each class. Students are expected to familiarize themselves with the material before the class. The readings will sometimes be arcane and difficult to understand; if so, do not worry, we will present simpler explanations in class.
We will use Piazza for discussions. Here is the link. Please sign up.
|Lecture||Start date||Topics||Lecture notes/Slides||Additional readings, if any||Quizzes/Assignments|
|7||September 20||Review of neural network training|
||slides available on Piazza Please do not distribute.|
||slides||Goodfellow Chapter 9|
|12||October 9||Class cancelled|
||slides||Goodfellow Chapter 10|
|19||November 1||Pulkit Agarwal
|22||November 13||Graham Neubig
|24||November 17 (Make up class)||
|25||November 20||Reinforcement Learning (part 1)
|26||November 27||Reinforcement Learning (part 2)
||slides: same deck as RL part 1|
|27||November 29||Reinforcement Learning (part 3)
A nice catalog of the various types of neural network models that are current in the literature can be found here. We expect that you will be in a position to interpret, if not fully understand many of these architectures by the end of the course.