Neural Networks and Deep Learning ECBM E4040 - Fall 2017

Zoran Kostic, PhD

Associate Professor of Prof. Practice, Electrical Engineering Dept., Columbia University., webpage, Office hours: Th. 11:30am-1pm, Shapiro 813, (for email communication - use subject: E4040 deep learning from student)

Course Assistants:

Tingyu Mao, Jingyu Qian, Pablo Vicente Juan, Abhishek Jindal, Wenyu Fu, Abhishek Bhatia, (common email

Course Description

The course covers theoretical underpinnings, architecture and performance, datasets, and applications of neural networks and deep learning. The course will use TensorFlow deep learning framework and Google Cloud computational platform with GPUs. The main resource for the course is the book entitled "Deep Learning" by Ian Goodfellow, Yoshua Bengio, and Aaron Courville, MIT Press,, 2016.

Students will contribute to the course by:

There will be 4 homeworks. The first, homework 0, will help you ascertain the programming know-how needed to solve the following homeworks. All homeworks will be graded. If you have difficulties with homework 0, please carefully evaluate whether you will have enough time to brush up your programming knowledge by the end of the change of program period. The course requires theoretical background in probability and statistics and linear algebra. You are strongly advised to drop the class if you cannot satisfactorily finish questions of homework #0 by the add/drop date, or if you judge that the first lecture description of prerequsite linear algebra and probability/statistics may be overly demanding.

Computational Environment and Prerequisites

1. Python Tutorial

There are many Python tutorials online for starters. Here we provide you with one tutorial link. Remember that for python 2.7 is used on google cloud platform. However, if you want to run the code on your local windows machine, then it should be python 3. Moreover, for this course we recommend students to use python 3 grammar when finishing all programming assignments.

2. Google Compute Engine (GCE) Setup on the Google Cloud Platform (GCP)

Students will be running deep learning code on the Google Compute Engine (GCE) virtual machine (VM) instances, with access to GPU resources. This tutorial teaches how to setup a VM instance using the VM image which instructors have created for students.

3. Local Environment Setup

For those who have a GPU-enabled computer and want to build a deep learning environment on the local machine/laptop, please see this guide on how to setup TensorFlow and PyTorch working environments.

4. Linux Introduction

The GCP VM instances which we use are Linux-based. If you haven't used Linux system before, this tutorial offers hands-on instruction to basic Linux commands that you will be using throughout the course.

5. TensorFlow Tutorial

TensorFlow is an open-source deep learning framework developed by Google. It is essential that you learn to program in TensorFlow for this course.

6. Git Commands

All of our homework distributions will be based on Bitbucket repositories, which can be manipulated by Git commands. If you don't know what Git is, it is strongly recommended that you read the first 3 chapters of this website.

7. FAQ

Solutions to some common problems

ECBM E4040 Neural Networks and Deep Learning, 2017.

Columbia University