About OH Course Work Class Notes Lectures Recitations Assignments Docs & Tools Resources S22 F21 S21
Course Work
Class Notes
Docs & Tools
S22 F21 S21
11-785 Introduction to Deep Learning
Fall 2021
Class Streaming Link

In-Person Venue: Baker Hall A51

Bulletin and Active Deadlines

Assignment Deadline Description Links
Final Project Report Dec. 10th, 11:59 ET - Canvas Submission
HW4P1 Dec 12th, 11:59 PM EST Word-based Language Model Writeup (*.pdf)
HW3P2 Early: Nov 18th, 11:59 PM ET
Final: Dec 12th, 11:59 PM ET
Listen, Attend and Spell Writeup (*.pdf)
Project Gallery
Here's an example of a successful project from Fall 2020.
The team developed an AI Limmerick generator,
and compiled a book from the AI Poet's creations.
Project Report, Project Video, Book (Amazon)
This piece is performed by the Chinese Music Institute at
Peking University (PKU) together with PKU's Chinese orchestra.
This is an adaptation of Beethoven: Serenade in D major, Op.25 - 1.
Entrata (Allegro),for Chinese transverse flute (Dizi), clarinet and flute.

CoVID-19 Related Announcement

In the event that the course is moved online due to CoVID-19, we will continue to deliver lectures via zoom. In the event that an instructor is unable to deliver a lecture in person, we will broadcast that lecture over zoom or, in extreme situations, expect you to view pre-recorded lectures from prior semesters. You will be notified through Piazza should any of these eventualities arise.

The Course

“Deep Learning” systems, typified by deep neural networks, are increasingly taking over all AI tasks, ranging from language understanding, and speech and image recognition, to machine translation, planning, and even game playing and autonomous driving. As a result, expertise in deep learning is fast changing from an esoteric desirable to a mandatory prerequisite in many advanced academic settings, and a large advantage in the industrial job market.

In this course we will learn about the basics of deep neural networks, and their applications to various AI tasks. By the end of the course, it is expected that students will have significant familiarity with the subject, and be able to apply Deep Learning to a variety of tasks. They will also be positioned to understand much of the current literature on the topic and extend their knowledge through further study.

If you are only interested in the lectures, you can watch them on the YouTube channel.

Course description from student point of view

The course is well rounded in terms of concepts. It helps us understand the fundamentals of Deep Learning. The course starts off gradually with MLPs and it progresses into the more complicated concepts such as attention and sequence-to-sequence models. We get a complete hands on with PyTorch which is very important to implement Deep Learning models. As a student, you will learn the tools required for building Deep Learning models. The homeworks usually have 2 components which is Autolab and Kaggle. The Kaggle components allow us to explore multiple architectures and understand how to fine-tune and continuously improve models. The task for all the homeworks were similar and it was interesting to learn how the same task can be solved using multiple Deep Learning approaches. Overall, at the end of this course you will be confident enough to build and tune Deep Learning models.


  1. We will be using Numpy and PyTorch in this class, so you will need to be able to program in python3.
  2. You will need familiarity with basic calculus (differentiation, chain rule), linear algebra and basic probability.


Courses 11-785, 18-786, and 11-685 are equivalent 12-unit graduate courses, and have a final project. Course 11-485 is the undergraduate version worth 9 units, the only difference being that there is no final project.


Your Supporters



Pittsburgh Schedule (Eastern Time)

Lecture: Monday and Wednesday, 8:35 a.m. - 9:55 a.m.

Recitation: Friday, 8:35 am - 9:55 am

Office hours: We will be using OHQueue for the zoom related Office hours, others would be in-person. See the schedule below.

Day Time (Eastern Time) TA Zoom/In Person Venue
Monday 2:00 - 4:00 pm Hao Chen WEH 3110
2:00 - 3:00 pm Urvil Kenia Zoom
5:00 - 7:00 pm Yuxin Pei GHC 5417
6:30 - 8:30 pm Chaoran Zhang WEH 3110
Tuesday 10:30 - 11:30 am Ojas Bhargave GHC 5417
11:30 - 12:30 pm Ojas Bhargave Zoom
10:00 - 12:00 pm Xiang Li WEH 3110
3:00 - 4:00 pm Clay Yoo GHC 5417
Wednesday 1:00 - 3:00 pm Sheila Mbadi Zoom
4:30 - 6:30 pm Jinhyung (David) Park Zoom
Thursday 10:00 - 12:00 pm Dijing Zhang GHC 5417
12:00 - 1:00 pm Urvil Kenia Zoom
12:00 - 2:00 pm Omisa Jinsi GHC 5417
6:00 - 8:00 pm Zhe Chen GHC 5417
Friday 10:00 - 12:00 pm Rukayat Sadiq Zoom
4:00 - 6:00 pm Manish Mishra WEH 3110
5:00 - 7:00 pm Zilin Si WEH 3110
Saturday 10:00 - 11:00 am Diksha Agarwal Zoom
10:00 - 12:00 pm Mehar Goli GHC 5417
4:00 - 5:00 pm Clay Yoo Zoom
Sunday 11:00 - 12:00 pm Diksha Agarwal Zoom
12:00 - 2:00 pm Fan Zhou GHC 5417
5:00 - 7:00 pm Joseph Konan Zoom

Course Work

Score Assignment      Grading will be based on weekly quizzes (24%), homeworks (50%) and a course project (25%). Note that 1% of your grade is assigned to Attendance.
Quizzes      There will be weekly quizzes.
  • We will retain your best 12 out of the remaining 14 quizzes.
  • Quizzes will generally (but not always) be released on Friday and due 72 hours later.
  • Quizzes are scored by the number of correct answers.
  • Quizzes will be worth 24% of your overall score.
Assignments There will be five assignments in all. Assignments will include autolab components, where you must complete designated tasks, and a kaggle component where you compete with your colleagues.
  • Autolab components are scored according to the number of correctly completed parts.
  • We will post performance cutoffs for A (100%), B (80%), C (60%), D (40%) and F (0%) for Kaggle competitions. Scores will be interpolated linearly between these cutoffs.
  • Assignments will have a “preliminary submission deadline”, an “on-time submission deadline” and a “late-submission deadline.”
    • Early submission deadline: You are required to make at least one submission to Kaggle by this deadline. People who miss this deadline will automatically lose 10% of subsequent marks they may get on the homework. This is intended to encourage students to begin working on their assignments early.
    • On-time deadline: People who submit by this deadline are eligible for up to five bonus points. These points will be computed by interpolation between the A cutoff and the highest performance obtained for the HW. The highest performance will get 105.
    • Late deadline: People who submit after the on-time deadline can still submit until the late deadline. There is a 10% penalty applied to your final score, for submitting late.
    • Slack days: Everyone gets up to 7 slack days, which they can distribute across all their homework P2s only. Once you use up your slack days you will fall into the late-submission category by default. Slack days are accumulated over all parts of all homeworks, except HW0, to which no slack applies.
    • Kaggle scoring: We will use max(max(on-time score), max(slack-day score), .0.9*max(late-submission score)) as your final score for the HW. If this happens to be a slack-days submission, slack days corresponding to the selected submission will be counted.
  • Assignments carry 50% of your total score. HW0 is not graded (but is mandatory), while each of the subsequent four are worth 12.5%.
  • A fifth HW, HW5, will be released later in the course and will have the same weight as a course project. Please see Project section below for more details.
  • All students taking a graduate version of the course are required to do a course project. The project is worth 25% of your grade. These points are distributed as follows: 10% - Proposal; 15% - Midterm Report; 30% - Project Video; 5% - Responding to comments on Piazza; 40% - Project paper/report.
  • Note that a Project is mandatory for 11-785/18-786 students. In the event of a catastrophe (remember Spring 2020), the Project may be substititued with HW5. 11-685 Students may choose to do a Project instead of HW5. Either your Project OR HW5 will be graded.
  • If you are in section A you are expected to attend in-person lectures. We will track attendance.
  • If you are in any of the other (out-of-timezone) sections, you may either watch the real-time zoom lectures or the recorded lectures on mediatech
    • If viewed on mediatech, the lectures of each week must be viewed before 8AM of the Monday following the following week (Otherwise, it doesn’t count)
  • At the end of the semester, we will select a random subset of 50% of the lectures and tabulate attendance
  • If you have attended at least 70% of these (randomly chosen) lectures, you get the attendance point
Final grade
Final grade The end-of-term grade is curved. Your overall grade will depend on your performance relative to your classmates.
Pass/Fail Students registered for pass/fail must complete all quizzes, HWs and if they are in the graduate course, the project. A grade equivalent to B- is required to pass the course.
Auditing Auditors are not required to complete the course project, but must complete all quizzes and homeworks. We encourage doing a course project regardless.
End Policy

Study groups

This semester we will be implementing study groups. It is highly recommended that you join a study group; see the forms on the bulletin.

Piazza: Discussion Board

Piazza is what we use for discussions. You should be automatically signed up if you're enrolled at the start of the semester. If not, please sign up here. Also, please follow the Piazza Etiquette when you use the piazza.

AutoLab: Software Engineering

AutoLab is what we use to test your understand of low-level concepts, such as engineering your own libraries, implementing important algorithms, and developing optimization methods from scratch.

Kaggle: Data Science

Kaggle is where we test your understanding and ability to extend neural network architectures discussed in lecture. Similar to how AutoLab shows scores, Kaggle also shows scores, so don't feel intimidated -- we're here to help. We work on hot AI topics, like speech recognition, face recognition, and neural machine translation.

Media Services/YouTube: Lecture and Reciation Recordings

CMU students who are not in the live lectures should watch the uploaded lectures at Media Services in order to get attendance credit. Links to individual videos will be posted as they are uploaded.

YouTube is where non-CMU folks can view all lecture and recitation recordings. Videos marked “Old“ are not current, so please be aware of the video title.

Books and Other Resources

The course will not follow a specific book, but will draw from a number of sources. We list relevant books at the end of this page. We will also put up links to relevant reading material for each class. Students are expected to familiarize themselves with the material before the class. The readings will sometimes be arcane and difficult to understand; if so, do not worry, we will present simpler explanations in class.

You can also find a nice catalog of models that are current in the literature here. We expect that you will be in a position to interpret, if not fully understand many of the architectures on the wiki and the catalog by the end of the course.

Academic Integrity

You are expected to comply with the University Policy on Academic Integrity and Plagiarism.
  • You are allowed to talk with and work with other students on homework assignments.
  • You can share ideas but not code. You should submit your own code.
Your course instructor reserves the right to determine an appropriate penalty based on the violation of academic dishonesty that occurs. Violations of the university policy can result in severe penalties including failing this course and possible expulsion from Carnegie Mellon University. If you have any questions about this policy and any work you are doing in the course, please feel free to contact your instructor for help.

Class Notes

A book containing class notes is being developed in tandem with this course; check it out.

Schedule of Lectures

Lecture Date Topics Slides and Video Additional Materials Quiz
0 -
  • Course Logistics
  • Learning Objectives
  • Grading
  • Deadlines
Slides (*.pdf)
Video (YT)
No Quiz
1 Monday
Aug  30
  • Introduction
Slides (*.pdf)
Video (MT)
The New Connectionism (1988)
On Alan Turing's Anticipation of Connectionism
Quiz 1
2 Wednesday
Sept  1
  • Neural Nets as Universal Approximators
Slides (*.pdf)
Video (MT)
Hornik et al. (1989)
Shannon (1949)
On the Bias-Variance Tradeoff
- Monday
Sept 6
  • No class
Quiz 2
3 Wednesday
Sept  8
  • Learning a Neural Net
Slides (*.pdf)
Video (MT)
Widrow and Lehr (1992)
Adaline and Madaline
Convergence of perceptron algorithm
4 Monday
Sept 13
  • Backpropagation
  • Calculus of backpropagation
Slides (*.pdf)
Video (MT)
Werbos (1990)
Rumelhart, Hinton and Williams (1986)
Quiz 3
5 Wednesday
Sept  15
  • Backpropagation, continued
  • Calculus of backpropagation, continued
Slides (*.pdf)
Video (MT)
Werbos (1990)
Rumelhart, Hinton and Williams (1986)
6 Monday
Sept 20
  • Convergence issues
  • Loss Surfaces
  • Momentum
Slides (*.pdf)
Video (MT)
Backprop fails to separate, where perceptrons succeed, Brady et al. (1989)
Why Momentum Really Works
Quiz 4
7 Wednesday
Sept 22
  • Batch Size, SGD, Minibatch, second-order methods
Slides (*.pdf)
Video (MT)
Momentum, Polyak (1964)
Nestorov (1983)
Derivatives and Influences
8 Monday
Sept 27
  • Optimizers and Regularizers
    • Choosing a divergence (loss) function
    • Batch normalization
    • Dropout
Slides (*.pdf)
Video (MT)
ADAGRAD, Duchi, Hazan and Singer (2011)
Adam: A method for stochastic optimization, Kingma and Ba (2014)
Quiz 5
9 Wednesday
Sept 29
  • Shift invariance and Convolutional Neural Networks
Slides (*.pdf)
Video (MT)
10 Monday
Oct 4
  • Models of vision, Convolutional Neural Networks
Slides (*.pdf)
Video (MT)
Quiz 6
11 Wednesday
Oct 6
  • Learning in Convolutional Neural Networks
Slides (*.pdf)
Video (MT)
CNN Explainer
12 Monday
Oct 11
  • Learning in CNNs, transpose Convolution
Slides (*.pdf)
Video (MT)
Quiz 7
13 Wednesday
Oct 13
  • Time Series and Recurrent Networks
Slides (*.pdf)
Video (MT)
Fahlman and Lebiere (1990)
How to compute a derivative, extra help for HW3P1 (*.pptx)
14 Monday
Oct 18
  • Stability and Memory, LSTMs
Slides (*.pdf)
Bidirectional Recurrent Neural Networks Quiz 8
15 Wednesday
Oct 20
  • Loss Functions in RNNs, Sequence Prediction
Slides (*.pdf)
Video (MT)
16 Monday
Oct 25
  • Connectionist Temporal Classification
  • Sequence prediction
Slides (*.pdf)
Quiz 9
17 Wednesday
Oct 27
  • Connectionist Temporal Classification (CTC)
  • Sequence To Sequence Prediction
Slides (*.pdf)
Slides (No Animations) (*.pdf)
Labelling Unsegmented Sequence Data with Recurrent Neural Networks
18 Monday
Nov 1
  • Sequence To Sequence Methods
  • Attention
Slides (*.pdf)
Slides (No Animations) (*.pdf)
Quiz 10
19 Wednesday
Nov 3
  • Transformers and GNNs
Slides (*.pdf)
Video (MT)
Attention Is All You Need
A comprehensive Survey on Graph Neural Networks
20 Monday
Nov 8
  • Learning Representations, AutoEncoders
Slides (*.pdf)
Video (MT)
Quiz 11
21 Wednesday
Nov 10
  • Variational Auto Encoders, 1
Slides (*.pdf)
Video (MT)
Tutorial on VAEs (Doersch)
Autoencoding variational Bayes (Kingma)
22 Monday
Nov 15
  • Variational Auto Encoders, 2
Slides (*.pdf)
Video (MT)
Quiz 12
23 Wednesday
Nov 17
  • Generative Adversarial Networks, 1
Slides (*.pdf)
Video (MT)
24 Monday
Nov 22
  • Generative Adversarial Networks, 2
Slides (*.pdf)
Quiz 13
- Wednesday
Nov  24
  • No class
25 Monday
Nov 29
  • Hopfield Nets
Slides (*.pdf)
Quiz 14
26 Wednesday
Dec 1
  • Boltzmann Machines

Schedule of Recitations

Recitation Date Topics Materials Videos Instructor
0A Due: Aug. 30 Python & OOP Fundamentals Notebook (*.zip)

Video (YT): 1, 2

Sheila, Urvil
0B Due: Aug. 30 Fundamentals of NumPy Notebook (*.zip)

Video (YT): 1, 2, 3, 4, 5, 6, 7, 8

Rukayat, Yuxin, Zilin
0C Due: Aug. 30 PyTorch Tensor Fundamentals Notebook (*.zip)

Video (YT): 1, 2, 3

Manish, Ojas, Xiang
0D Due: Aug. 30 Dataset & DataLoaders Notebook + Slides (*.zip) Video (YT)
Diksha, Joseph
0E Due: Aug. 30 Introduction to Google Colab Notebook (*.zip) Video (YT)
David, Tianhao
0F Due: Aug. 30 AWS Fundamentals Handout (*.zip)

Video (YT): 1, 2, 3

0G Due: Aug. 30 Debugging, Monitoring Notebook (*.zip)

Video (YT): 1, 2

Clay, Chaoran
0H Due: Aug. 30 Remote Notebooks Notebook + Markdown (*.zip)

Video (YT): 1, 2, 3, 4, 5, 6, 7, 8, 9, 10

Mehar, Dijing
0I Due: Aug. 30 What to do if you're struggling Slides (*.zip) Video (YT)
Omisa, Yuxin
1 Sept 3 2021 Your first MLP Code Slides (*.pdf) Video (MT) Video (YT)
Abbey, Urvil
2 Sept 10, 2021 Optimizing the Networks, Ensembles Notebook + Slides (*.zip) Video (MT) Hao, Dijing
HW1 Bootcamp Sept 14, 2021 How to get started with HW1 Slides, Notebook Video (MT)
Fan, Manish
3 Sept 17, 2021 Debugging Neural Nets Slides+Notebook(*.zip) Video (YT)
Xiang, Sheila
4 Sept 24, 2021 Computing Derivatives Slides (*.zip) Video (MT)
Zilin, David
5 Oct 1, 2021 CNN: Basics & Backprop CNN Basics (*.zip),
CNN Backprop (*.pdf)
Video (YT)
Video (MT)
Omisa, Rukayat
6 Oct 8, 2021 CNNs: Classification & Verification Slides (*.pdf)
Notebook (*.ipynb)
Video (MT)
Ojas, Chaoran
HW2 Bootcamp Oct 9, 2021 How to get start with HW2 Slides(p1) (*.pptx)
Slides(p2) (*.pdf)
Notebook(p2) (*.ipynb)
Video (MT)
Dijing, Urvil, Diksha, Clay
7 Oct 15, 2021 Paper Writing Workshop Slides (*.zip) Video (MT)
David, Rukayat
8 Oct 22, 2021 RNN Basics Slides (*.pdf)
Code (*.zip)
Video (MT)
Ojas, Clay
HW3 Bootcamp Oct 27, 2021 How to get start with HW3 Slides(p1) (*.pdf)
Slides(p2) (*.pdf)
Notebook (*.ipynb)
Video (YT)
Zilin, Abbey, Ojas, Xiang, Sheila, Fan
9 Oct 29, 2021 CTC, Beam Search Slides (*.zip) Video (MT)
Omisa, Mehar
10 Nov 6 2021 Attention, MT, LAS Slides (*.zip) Video (YT)
HW4 Bootcamp Nov 7 2021 How to get start with HW4 Notebook (*.ipynb)
Video (YT)
Clay, Manish
11 Nov 12 2021 Autoencoders, VAEs Slides (*.pdf)
Video (MT)
Zhe, Dijing
12 Nov 19 2021 Generative Adversarial Networks (GANs) TBA TBA Xiang, Manish
13 Nov 26 2021 Graph Neural Networks TBA TBA Mehar, Diksha
14 Dec 3 2021 Hopfield nets, Boltzmann Machines, RBMs TBA TBA Diksha, Joseph

Assignments and Quizzes

∑ Ongoing, ∏ Upcoming

Assignment Release Date Due Date Related Materials / Links
HW0p1 Summer Break Sept 5th, 2021
11:59 PM EST
(see recitation 0s)
HW0p2 Summer Break Sept 5th, 2021
11:59 PM EST
(see recitation 0s)
HW1p1 Sept 9th, 2021 Sept 30th, 2021
11:59 PM EST
Autolab, Handout(*.zip)
HW1p2 Sept 9th, 2021 Early Submission: Sept 16th, 2021
11:59 PM EST
Autolab, Writeup (*.pdf)
Final: Sept 30th, 2021
11:59 PM EST
Project Proposal Canvas Submission Sept 30th, 2021
11:59 PM EST
HW2p1 Sept 30th, 2021 Oct 21st, 2021
11:59 PM EST
Autolab, Handout (*.zip)
HW1p2 Sept 30th, 2021 Early Submission: Oct 6th, 2021
11:59 PM EST
Face Classification: Autolab,
Face Verification: Autolab,
Writeup (*.pdf)
Final: Oct 21st, 2021
11:59 PM EST
Project Midterm Report - Nov 4th, 2021
11:59 PM EST
Canvas Submission
HW3p1 Oct 21st, 2021 Nov 11th, 2021
11:59 PM EST
Autolab, handout (*.zip), Writeup (*.pdf)
HW3p2 Oct 21st, 2021 Early Submission: Oct 28th, 2021
11:59 PM EST
Autolab, Kaggle, Writeup (*.pdf)
Final: Nov 11th, 2021
11:59 PM EST
HW4p1 Nov 4th, 2021 Nov 25th, 2021
11:59 PM EST
Writeup (*.pdf),
HW4p2 Nov 4th, 2021 Early Submission: Nov 11th, 2021
11:59 PM EST
Writeup (*.pdf)
Final: Nov 29th, 2021
11:59 PM EST
Final Project Video Presentation & Preiliminary Project Report Dec 1st, 2021
11:59 PM EST
Dec 5th, 2021
11:59 PM EST
Preliminary Report: Canvas Submission
Project Peer reviews Dec 6th, 2021
11:59 PM EST
Dec 8th, 2021
11:59 PM EST
Final Project Report Submission - Dec 9th, 2021
11:59 PM EST
Canvas Submission

Documentation and Tools


This is a selection of optional textbooks you may find useful

Deep Learning
Dive Into Deep Learning By Aston Zhang, Zachary C. Lipton, Mu Li, and Alexander J. Smola PDF, 2020
Deep Learning
Deep Learning By Ian Goodfellow, Yoshua Bengio, Aaron Courville Online book, 2017
Neural Networks and Deep Learning
Neural Networks and Deep Learning By Michael Nielsen Online book, 2016