Neural Networks and Deep Learning ECBM E4040 - Fall 2018


2018/10/03 New! Last announcement on the webpage. Course management and announcements have moved to courseworks.

Zoran Kostic, PhD

Associate Professor of Prof. Practice, Electrical Engineering Dept., Columbia University., instructor webpage, Office hours: Th. 11:30am-1pm, Shapiro 813, (for email communication - use subject: E4040 deep learning from student)

Course Assistants:

Shilin Hu, Zixiao Zhang, Yilin Lyu, Yong Yang, Zixi Huang and Zekun Gong(common email

OH: Mudd 1301 Time: Shilin & Yong: Tue 17:00 - 19:00; Zixiao & Yilin: Thu 13:00 - 15:00; Zixiao & Zekun: Fri 8:00 - 10:00

Course Description

The course covers theoretical underpinnings, architecture and performance, datasets, and applications of neural networks and deep learning. The course will use Python coding language, TensorFlow deep learning framework and Google Cloud computational platform with GPUs (recommended). The main resource for the course is the book entitled "Deep Learning" by Ian Goodfellow, Yoshua Bengio, and Aaron Courville, MIT Press,, 2016.

Students will contribute to the course by:

There will be 4 homeworks in total. The first, homework 0, will help you ascertain the programming know-how needed to solve the following homeworks. All homeworks will be graded. If you have difficulties with homework 0, please carefully evaluate whether you will have enough time to brush up your programming knowledge by the end of the change of program period. The course requires theoretical background in probability and statistics and linear algebra. You are strongly advised to drop the class if you cannot satisfactorily finish questions of homework 0 by the add/drop date, or if you judge that the first lecture description of prerequsite linear algebra and probability/statistics may be overly demanding.

Syllabus link

Grading policy

HW(40%), (one midterm) Exam(25%), Project(35%).

Late homeworks (assignments) Each student is entitled to 4 late days without penalty. For all homeworks together, a student can divide those four days in any fashion needed. Examples: (i) Homework 3 is late 4 days, in which case no other homework can be late for any amount of time; (ii) Homework 1 is late 1 day, homework 2 is late 2 days, in which case the student did not use all four late days. The unit of delay can not be divided into less than a full day (like hours). Requests for additional extensions will not be granted: if the budget of 4 days is blown, the student will be given 0 credit for homework(s) for which their submission is late.

Computational Environment and Prerequisites

1. Python Tutorial

There are many Python tutorials online for starters. Here we provide you with one tutorial link. For this course we recommend students to use python 3 grammar when finishing all programming assignments. Since on the cloud we only installed python 3, you may have problems if you code in python 2.7 on local machine and then want to run the same code on google cloud platform

2. Google Compute Engine (GCE) Setup on the Google Cloud Platform (GCP)

Students will be running deep learning code on the Google Compute Engine (GCE) virtual machine (VM) instances, with access to GPU resources. This tutorial teaches how to setup a VM instance using the VM image which instructors have created for students.

3. Local Environment Setup

For those who have a GPU-enabled computer or want to build a deep learning environment on the local machine/laptop, please see this guide on how to setup TensorFlow and PyTorch working environments.

4. Linux Introduction

The GCP VM instances which we use are Linux-based. If you haven't used Linux system before, this tutorial offers hands-on instruction to basic Linux commands that you will be using throughout the course.

5. TensorFlow Tutorial

TensorFlow is an open-source deep learning framework developed by Google. It is essential that you learn to program in TensorFlow for this course.

6. Git Commands

All of our homework distributions will be based on Bitbucket repositories, which can be manipulated by Git commands. If you don't know what Git is, it is strongly recommended that you read the first 3 chapters of this website.

7. FAQ

Solutions to some common problems

ECBM E4040 Neural Networks and Deep Learning, 2018.

Columbia University