11-785 Deep Learning

11-785 Introduction to Deep Learning

Spring 2025
Class Streaming Link

In-Person Venue: Giant Eagle Auditorium, Baker Hall (A51)

Active Deadlines and Bulletin

Assignment	Deadline	Description	Links
HW1P1	Early Submission: Friday, Jan 24 11:59 PM ET Final Submission: Friday, Feb 7 11:59 PM ET	Linear Layers, Activations, and Loss Functions	Autolab
HW1P2	Early Submission: Friday, Jan 24 11:59 PM ET Final Submission: Friday, Feb 7 11:59 PM ET	Time-Synchronous MFCC to Phoneme Mapping	Autolab
HW1P1 Bonus	Final Submission: Friday, Apr 25 11:59 PM ET	Adam, AdamW Optimizers and Dropout	Autolab
HW1P1 Autograd	Final Submission: Friday, Apr 25 11:59 PM ET	Automatic Differentiation Engine	Autolab
Project Gallery

The Course

“Deep Learning” systems, typified by deep neural networks, are increasingly taking over all the AI tasks, ranging from language understanding, speech and image recognition, to machine translation, planning, and even game playing and autonomous driving. As a result, expertise in deep learning is fast changing from an esoteric desirable to a mandatory prerequisite in many advanced academic settings, and a large advantage in the industrial job market.

In this course we will learn about the basics of deep neural networks, and their applications to various AI tasks. By the end of the course, it is expected that students will have significant familiarity with the subject, and be able to apply Deep Learning to a variety of tasks. They will also be positioned to understand much of the current literature on the topic and extend their knowledge through further study.

If you are only interested in the lectures, you can watch them on the YouTube channel.

Course Description from a Student's Perspective

The course is well rounded in terms of concepts. It helps us understand the fundamentals of Deep Learning. The course starts off gradually with MLPs and it progresses into the more complicated concepts such as attention and sequence-to-sequence models. We get a complete hands on with PyTorch which is very important to implement Deep Learning models. As a student, you will learn the tools required for building Deep Learning models. The homeworks usually have 2 components which is Autolab and Kaggle. The Kaggle components allow us to explore multiple architectures and understand how to fine-tune and continuously improve models. The task for all the homeworks were similar and it was interesting to learn how the same task can be solved using multiple Deep Learning approaches. Overall, at the end of this course you will be confident enough to build and tune Deep Learning models.

Prerequisites

We will be using Numpy and PyTorch in this class, so you will need to be able to program in python3.
You will need familiarity with basic calculus (differentiation, chain rule), linear algebra, and basic probability.

Units

Courses 11-785 and 11-685 are equivalent 12-unit graduate courses, and have a final project and HW5 respectively.
Course 11-485 is the undergraduate version worth 9 units, the only difference being that there is no final project or HW5.

Your Supporters

Instructors:

Bhiksha Raj : bhiksha@cs.cmu.edu
Rita Singh : rsingh@cs.cmu.edu

TAs:

Kateryna Shapovalenko: kshapova@andrew.cmu.edu
Miya Sylvester: nsylvest@andrew.cmu.edu
Alexander Moker: amoker@andrew.cmu.edu
Purusottam Samal: psamal@andrew.cmu.edu
Shravanth Srinivas: shravans@andrew.cmu.edu
Yuzhou Wang: yuzhouwa@andrew.cmu.edu
Massa Baali: mbaali@andrew.cmu.edu
Vedant Singh: vhsingh@andrew.cmu.edu
Sadrishya Agrawal: sadrisha@andrew.cmu.edu
Michael Kireeff: mkireeff@andrew.cmu.edu
Vishan Oberoi: voberoi@andrew.cmu.edu
Ishita Gupta: ishitag@andrew.cmu.edu
Shubham Kachroo: skachroo@andrew.cmu.edu
Shrey Jain: shreyj@andrew.cmu.edu
Floris Nzabakira: fnzabaki@andrew.cmu.edu
Christine Muthee: cmuthee@andrew.cmu.edu
Ahmed Issah: aissah@andrew.cmu.edu
Shubham Singh: shubham4@andrew.cmu.edu
Tanghang Elvis Tata: etanghan@andrew.cmu.edu
John Liu: johnliu@andrew.cmu.edu
Harshith Arun Kumar: harunkum@andrew.cmu.edu
Damilare Olatunji: dolatunj@andrew.cmu.edu
Brian Ebiyau: bebiyau@andrew.cmu.edu
Peter Wauyo: pwauyo@andrew.cmu.edu
Eman Ansar: eansar@andrew.cmu.edu

Acknowledgments

Wall of fame

Past TA Acknowledgments

Pittsburgh Schedule (Eastern Time)

Lecture: Monday and Wednesday, 8:00 a.m. - 9:20 a.m. - Good times :)

Recitation Labs: Friday, 8:00 a.m. - 9:20 a.m.

Office Hours: Please refer the below OH Calendar / Piazza for up-to-date information.

Homework Hackathon: During 'Homework Hackathons', students will be assisted with homework by the course staff. It is recommended to come as study groups.
Every Saturday

Location: TBD
Time: Saturday 2-5 PM EST

Event Calendar: The Google Calendar below contains all course events and deadlines for student's convenience. Please feel free to add this calendar to your Google Calendar by clicking on the plus (+) button on the bottom right corner of the calendar below. Any adhoc changes to the schedule will be reflected in this calendar first.

OH Calendar: The Google Calendar below contains the schedule for Office Hours. Please feel free to add this calendar to your Google Calendar by clicking on the plus (+) button on the bottom right corner of the calendar below. Any adhoc changes to the schedule will be reflected in this calendar first.

Syllabus

Policy
Breakdown
Score Assignment		Grading will be based on weekly quizzes (24%), homeworks (50%) and a course project (25%). Note that 1% of your grade is assigned to Attendance.
Quizzes
Quizzes		There will be weekly quizzes. We will retain your best 12 out of the remaining 14 quizzes. Quizzes will generally (but not always) be released on Friday and due 48 hours later. Quizzes are scored by the number of correct answers. Quizzes will be worth 24% of your overall score.
Assignments
Assignments		There will be five assignments in all, plus the Peer Review assignment during the last week of the semester. Assignments will include Autolab components, where you implement low-level operations, and a Kaggle component, where you compete with your colleagues over relevant DL tasks. Autolab components are scored according to the number of correctly completed parts. We will post performance cutoffs for HIGH (90%), MEDIUM (70%), LOW (50%), and VERY LOW (30%) for Kaggle competitions. Scores will be interpolated linearly between these cutoffs. Assignments will have a “preliminary submission deadline”, an “on-time submission deadline” and a “late-submission deadline.” Early submission deadline: You are required to make at least one submission to Kaggle by this deadline. People who miss this deadline will automatically lose 10% of subsequent marks they may get on the homework. This is intended to encourage students to begin working on their assignments early. On-time deadline: People who submit by this deadline are eligible for up to five bonus points. These points will be computed by interpolation between the A cutoff and the highest performance obtained for the HW. The highest performance will get 105. Late deadline: People who submit after the on-time deadline can still submit until the late deadline. There is a 10% penalty applied to your final score, for submitting late. Slack days: Everyone gets up to 10 slack days, which they can distribute across all their homework P2s only. Once you use up your slack days you will fall into the late-submission category by default. Slack days are accumulated over all parts of all homeworks. Kaggle scoring: We will use max(max(on-time score), max(slack-day score), .0.9max(late-submission score))* as your final score for the HW. If this happens to be a slack-days submission, slack days corresponding to the selected submission will be counted. Assignments carry 50% of your total score, with each of the four HWs being worth 12.5%. A fifth HW, HW5, will be released later in the course and will have the same weight as a course project. Please see Project section below for more details. Bonus HWs will count towards the score of the correlating HWp1 assignment number. (For example, Bonus1 points go towards HW1p1.) The Peer Review assignment is required of all students, 11-485/685/785. The task is for all students to review and grade 4-6 of the videos. It is to be completed over the last weekend, after classes finish (but before finals week). We will tell you which projects you have been assigned to review; each review should take around 15~20 minutes. Here is what we expect you to do for each review: Watch the video carefully. As you watch the video, jot down some notes/concerns/questions that you might have. Reference the initial report to clear up any confusion. You must post at least one comment to the corresponding Piazza post of your reviewee. This comment must be a meaningful question or concern that demonstrates you have understood the material. Finally, fill out the project review form carefully. More details will be shared over Piazza.
Project
Project		All students taking a graduate version of the course are required to do a course project. The project is worth 25% of your grade. These points are distributed as follows: 20% - Midterm Report; 35% - Project Video; 5% - Responding to comments on Piazza; 40% - Project report. Note that a Project is mandatory for 11-785 students. In the event of a catastrophe (remember Spring 2020), the Project may be substituted with HW5. 11-685 Students may choose to do a Project instead of HW5. Either your Project OR HW5 will be graded. Important information for project reports and video presentations (including midterm report rubric, final report rubric, video timeline, and video grading): Link.
Attendance
Attendance		If you are in section A you are expected to attend in-person lectures. We will track attendance. If you are in any of the other (out-of-timezone) sections, you must watch lectures live on zoom. Real-time viewing is mandatory unless you are in inconvenient time zones. Others are required to obtain specific permission to watch the pre-recorded lectures (on MediaServices). If viewed on MediaServices, the lectures of each week must be viewed before Monday 8AM of the following week (otherwise, it doesn’t count). At the end of the semester, we will select a random subset of lectures and tabulate attendance. If you have attended at least 70% of these (randomly chosen) lectures, you get the attendance point.
Final grade
Final grade		The end-of-term grade is curved. Your overall grade will depend on your performance relative to your classmates.
Pass/Fail
Pass/Fail		Students registered for pass/fail must complete all quizzes, HWs and if they are in the graduate course, the project. A grade equivalent to B- is required to pass the course.
Auditing
Auditing		Auditors are not required to complete the course project, but must complete all quizzes and homeworks. We encourage doing a course project regardless.
		End Policy

Study groups

We believe that effective collaboration can greatly enhance student learning. Thus, this course employs study groups for both quizzes and homework ablations. It is highly recommended that you join a study group; Check piazza for further updates.

Piazza: Discussion Board

Piazza is what we use for discussions. You should be automatically signed up if you're enrolled at the start of the semester. If not, please sign up here. Also, please follow the Piazza Etiquette when you use the piazza forum.

AutoLab: Software Engineering

AutoLab is what we use to test your understand of low-level concepts, such as engineering your own libraries, implementing important algorithms, and developing optimization methods from scratch.

Kaggle: Data Science

Kaggle is where we test your understanding and ability to extend neural network architectures discussed in lecture. Similar to how AutoLab shows scores, Kaggle also shows scores, so don't feel intimidated -- we're here to help. We work on hot AI topics, like speech recognition, face recognition, and neural machine translation.

MediaServices/YouTube: Lecture and Recitation Recordings

CMU students who are not in the live lectures should watch the uploaded lectures at MediaServices in order to get attendance credit. Links to individual videos will be posted as they are uploaded.

Our YouTube Channel is where non-CMU folks can view all lecture and recitation recordings. Videos marked “Old“ are not current, so please be aware of the video title.

Books and Other Resources

The course will not follow a specific book, but will draw from a number of sources. We list relevant books at the end of this page. We will also put up links to relevant reading material for each class. Students are expected to familiarize themselves with the material before the class. The readings will sometimes be arcane and difficult to understand; if so, do not worry, we will present simpler explanations in class.

You can also find a nice catalog of models that are current in the literature here. We expect that you will be in a position to interpret, if not fully understand many of the architectures on the wiki and the catalog by the end of the course.

Academic Integrity

You are expected to comply with the University Policy on Academic Integrity and Plagiarism.

You are allowed to talk with and work with other students on homework assignments.
You can share ideas but not code. You should submit your own code.

Your course instructor reserves the right to determine an appropriate penalty based on the violation of academic dishonesty that occurs. Violations of the university policy can result in severe penalties including failing this course and possible expulsion from Carnegie Mellon University. If you have any questions about this policy and any work you are doing in the course, please feel free to contact your instructor for help.

Class Notes

A book containing class notes is being developed in tandem with this course; check it out.

Schedule of Lectures

You can watch the recorded lectures on MediaServices.

Lecture	Date	Topics	Slides, Videos	Additional Materials	Quiz
0	Friday, Jan 03	Course Logistics Learning Objectives Grading Deadlines	Youtube		No Quiz
1	Monday, Jan 13	Introduction	Slides (PDF) MediaServices Youtube	The New Connectionism (1988) On Alan Turing's Anticipation of Connectionism McCullogh and Pitts paper Rosenblatt: The perceptron Bain: Mind and body Hebb: The Organization Of Behaviour	Quiz 1
2	Wednesday, Jan 15	Neural Nets As Universal Approximators	Slides (PDF) MediaServices Youtube	Shannon (1949) Boolean Circuits On the Bias-Variance Tradeoff	Quiz 1
3	Friday, Jan 17	Training Part I The Problem of Learning Empirical Risk Minimization	Slides (PDF) Youtube	Widrow and Lehr (1992) Adaline and Madaline	Quiz 2
-	Monday, Jan 20	No class (MLK day)
4	Wednesday, Jan 22	Training Part II Gradient Descent Training the Network Backpropagation		Widrow and Lehr (1992) Adaline and Madaline Convergence of perceptron algorithm Threshold Logic TC (Complexity) AC (Complexity)
5	Monday, Jan 27	Training Part III Backpropagation Calculus of Backpropagation		Werbos (1990) Rumelhart, Hinton and Williams (1986)	Quiz 3
6	Wednesday, Jan 29	Training Part IV Convergence issues Loss Surfaces Momentum		Backprop fails to separate, where perceptrons succeed, Brady et al. (1989) Why Momentum Really Works	Quiz 3
7	Monday, Feb 3	Training Part V Optimization Batch Size, SGD, Mini-batch, Second-order Methods		Momentum, Polyak (1964) Nestorov (1983) Derivatives and Influences	Quiz 4
8	Wednesday, Feb 5	Training Part VI Optimizers and Regularizers Choosing a Divergence (Loss) Function Batch Normalization Dropout		Derivatives and Influence Diagrams ADAGRAD, Duchi, Hazan and Singer (2011) Adam: A method for stochastic optimization, Kingma and Ba (2014)	Quiz 4
9	Monday, Feb 10	Shift Invariance Convolutional Neural Networks (CNNs) - Part I			Quiz 5
10	Wednesday, Feb 12	Convolutional Neural Networks (CNNs) - Part II Models of Vision and CNNs			Quiz 5
11	Monday, Feb 17	Convolutional Neural Networks (CNNs) - Part III Learning in CNNs		CNN Explainer	Quiz 6
12	Wednesday, Feb 19	Convolutional Neural Networks (CNNs) - Part IV Learning in CNNs Transpose Convolution CNN Stories			Quiz 6
13	Monday, Feb 24	Time Series and Recurrent Networks		Fahlman and Lebiere (1990) How to compute a derivative, extra help for HW3P1 (*.pptx)	Quiz 7
14	Wednesday, Feb 26	Stability and Memory, LSTMs			Quiz 7
-	Monday, Mar 3	No class (Spring break)
-	Wednesday, Mar 5	No class (Spring break)
15	Monday, Mar 10	Sequence Prediction Alignments and Decoding			Quiz 8
16	Wednesday, Mar 12	Sequence Prediction Connectionist Temporal Classification Blanks Beam Search			Quiz 8
17	Monday, Mar 17	Language Models Sequence To Sequence Predictions			Quiz 9
18	Wednesday, Mar 14	Attention models Transformers			Quiz 9
19	Monday, Mar 24	Transformers and Newer Architectures			Quiz 10
20	Wednesday, Mar 26	Large Language Models			Quiz 10
21	Monday, Mar 31	Representation and Autoencoders			Quiz 11
22	Wednesday, Apr 02	Variational Auto Encoders			Quiz 11
23	Monday, Apr 07	Diffusion			Quiz 12
24	Wednesday, Apr 09	Generative Adversarial Networks I and II			Quiz 12
25	Monday, Apr 14	Graph Neural Networks			Quiz 14
26	Wednesday, Apr 16	Hopfield Networks Boltzmann Machines			Quiz 14
27	Monday, Apr 21	Guest lecture / buffer
28	Wednesday, Apr 23	Guest lecture / buffer

Recitations/Labs and Bootcamps

Recitation	Date	Group	Topics	Materials	Youtube Videos	Instructor
0.1	Monday, Jan 02	Python Programming + Pytorch	Python + OOP Fundamentals	Python Fundamentals OOP Fundamentals	Python-1, Python-2, OOP-1, OOP-2	Eman Ansar, Yichen Xin, Carmel Prosper SAGBO
0.2			Numpy Fundamentals + JAX	Numpy Notebook Broadcasting Pitfalls & JAX Notebook	Numpy-1, Numpy-2, Numpy-3, Numpy-4, Broadcasting Pitfalls & Jax	Shubham Singh, Michael Kireeff
0.3			Notebooks and Conda Environments		Link	Vishan Tanghang, Elvis Tata
0.4			Pytorch	Pytorch Notebook	Link	Shravanth Srinivas, Vishan Oberoi
0.5		Computational Resources	Google Cloud Platform	VM Setup Script	GCP-1, GCP-2 (optional)	Alexander Moker, John Liu
0.6			Google Colab	Notebook	Link	Shubham Kachroo, Sadrishya Agrawal
0.7			AWS	AWS Notebook	Link	Tanghang Elvis Tata, Peter Wauyo
0.8			Kaggle	AWS Notebook	Link	Olatunji Damilare E., Floris Nzabakira
0.9			PSC		Link	Alexander Moker, Shrey Jain
0.10		Data Handling and Processing	Datasets & Dataloaders	Datasets P1 Notebook, Datasets P2 Notebook, Dataloader Notebook	Datasets-1, Datasets-2, Dataloader-1, Dataloader-2,	Vedant Singh, Christine Muthee
0.11		Data Handling and Processing	Data Preprocessing	Audio Preprocessing Notebook, Image Preprocessing Notebook	Data Preprocessing-1 (Audio), Data Preprocessing-2 (Image)	Vedant Singh, Eman Ansar
0.12		Debugging and Problem Solving	Debugging	Debugging Notebook (Part I) Debugging Notebook (Part III)	Debugging-1, Debugging-3	Khushali Daga, Romerik Lokossou
0.13		Debugging and Problem Solving	What to Do When Struggling		Part 1, Part 2	Christine Muthee, Shravanth Srinivas
0.14		HWs and Project Workflow Management	Workflow of HWs		Link	Alexander Moker, Syed Abdul Hannan
0.15			Wandb	WandB Notebook	Link	Floris Nzabakira, Ahmed Issah
0.16			Git		Link	Sadrishya Agrawal, Shubham Kachroo
0.17			Flow of the Project & How to Write a Report		Link	Olatunji Damilare E., Yuzhou Wang
0.18		Algorithmic Techniques	Losses	Losses P1 Notebook, Losses P2 Notebook	Losses-1, Losses-2	Massa Baali, Shrey Jain
0.19		Algorithmic Techniques	Block Processing	Block Processing Notebook	Link	Puru Samal, Eman Ansar
0.20		Model Logistics	Paper to Code	Notebook	Link	Massa Baali, Peter Wauyo
0.21			Pipeline		Link	Puru Samal
0.22			Distributed Training		Link	Ishita Gupta, Harshith Arun Kumar
0.23			Saving & Loading Model (Checkpointing)	Saving & Loading Notebook	Link	Ahmed Issah, John Liu
Lab 1		Saturday, Jan. 18th		Your First MLP	Colab Notebook	Link	Miya Sylvester, Vedant Singh
HW1 Bootcamp		Saturday, Jan. 18th	HW1P1, HW1P2	Slides	Shubham Singh, Shubham Kachroo, Shravanth Srinivas, Issah Ahmed, Vishan Oberoi	Link

Assignments

Assignment	Release Date (EST)	Due Date (EST)	Related Materials / Links
HW1P1	Friday, Jan 17 11:59 PM	Early Deadline: Friday, Jan 24 11:59 PM On-Time Deadline: Friday, Feb 7 11:59 PM	Autolab
HW1P2			Autolab
HW1P1 Bonus		Friday, Apr 25 11:59 PM	Autolab
HW1P1 Autograd		Friday, Apr 25 11:59 PM	Autolab
HW2P1	Friday, Feb 7 11:59 PM	Early Deadline: Friday, Feb 21 11:59 PM On-Time Deadline: Saturday, Mar 1 11:59 PM
HW2P2
HW2P1 Bonus		Friday, Apr 25 11:59 PM
HW2P1 Autograd		Friday, Apr 25 11:59 PM
HW3P1	Saturday, Mar 1 11:59 PM	Early Deadline: Friday, Mar 14 11:59 PM On-Time Deadline: Friday, Mar 28 11:59 PM
HW3P2	Saturday, Mar 1 11:59 PM
HW4P1	Friday, Mar 28 11:59 PM	Early Deadline: Friday, Apr 11 11:59 PM On-Time Deadline: Friday, Apr 25 11:59 PM
HW4P2	Friday, Mar 28 11:59 PM