“Deep Learning” systems, typified by deep neural networks, are increasingly taking over all AI tasks, ranging from language understanding, and speech and image recognition, to machine translation, planning, and even game playing and autonomous driving. As a result, expertise in deep learning is fast changing from an esoteric desirable to a mandatory prerequisite in many advanced academic settings, and a large advantage in the industrial job market.
In this course we will learn about the basics of deep neural networks, and their applications to various AI tasks. By the end of the course, it is expected that students will have significant familiarity with the subject, and to be able to apply to them to a variety of tasks. They will also be positioned to understand much of the current literature on the topic and extend their knowledge through further study.
Instructor: Bhiksha Raj
Lecture: Monday and Wednesday, 9.00am-10.20am
Recitation: Friday, 9.00am-10.20am, Newell Simon 3002
This course is worth 12 units.
Grading will be based on weekly quizzes, homework assignments and a final project.
There will be five assignments in all. Note that assignments 4 and 5 are released simultaneously. They will also be due on the same date.
|Quizzes||13 quizzes (bottom 3 quiz scores will be dropped), total contribution to grade 25%|
|Assignments||5 assignments, total contribution to grade 50%|
|Project||1 project, total contribution to grade 25%|
The course will not follow a specific book, but will draw from a number of sources. We list relevant books at the end of this page. We will also put up links to relevant reading material for each class. Students are expected to familiarize themselves with the material before the class. The readings will sometimes be arcane and difficult to understand; if so, do not worry, we will present simpler explanations in class.
We will use Piazza for discussions. Here is the link. Please sign up.
We have created an experimental wiki explaining the types of neural networks in use today. Here is the link.
You can also find a nice catalog of models that are current in the literature here. We expect that you will be in a position to interpret, if not fully understand many of the architectures on the wiki and the catalog by the end of the course.
Kaggle is a popular data science platform where visitors compete to produce the best model for learning or analyzing a data set.
For assignments 4 and 5 you will be submitting your evaluation results to a Kaggle leaderboard.
|Lecture||Start date||Topics||Lecture notes/Slides||Additional readings, if any||Quizzes/Assignments|
||slides||Quiz 1 (Due 20th)|
||slides||Assignment 1Quiz 2 (Due 27th)|
||slides||Quiz 3 (Due Feb 3rd)|
|7||February 7||Guest Lecture (Scott Fahlman)||Quiz 4 (Due 10th)|
||slides||Quiz 5 (Due 17th)|
||slides||Quiz 6 (Due 24th)|
||slides||Assignment 3Quiz 7 (Due 3rd)|
||slides||Quiz 8 (Due 10th)|
|16||March 12||Spring break|
|17||March 14||Spring break|
|18||March 19||NNets in Speech Recognition, Guest Lecture (Stern)|
||slides||Quiz 9 (Due 24th)
Assignments 4 and 5
||slides||Quiz 10 (Due 31st)|
||slides||Quiz 11 (Due 7th)|
|24||April 9||Guest lecture (Graham Neubig)|
||Quiz 12 (Due 14th)|
||Quiz 13 (Due 21st)|
|29||April 25||Guest Lecture (TBD)|
|1||January 19||Amazon Web Services (AWS)|
|2||January 26||Practical Deep Learning in Python|
|3||February 2||Optimization methods|
|4||February 9||Tuning methods|
|7||March 2||RNN's and LSTM's|
|10||March 23||Practical implementation of VAEs|
|11||March 30||Practical implementation of GANs|
|12||April 6||Practice with BMs and RBMs|