Neural Networks and Deep Learning ECBM E4040 - Fall 2019


Announcements

2019/08/17 New! Webpage for 2019 class is under development.

Zoran Kostic, PhD

Associate Professor of Professional Practice, Electrical Engineering Dept., Columbia University.
zk2172@columbia.edu, instructor webpage, Office Hours: Th. 11:30am-1pm, Shapiro 813, (for email communication use subject "E4040 deep learning from student")

Course Assistants:

Deepak Ravishankar, Huixiang Zhuang, TODO add other TAs (common email e4040TAs@columbia.edu).

Office Hours: Mudd 1301; Time: Deepak Ravishankar: TBD; Huixiang Zhuang: TBD;


Course Description

The course covers theoretical underpinnings, architecture and performance, datasets, and applications of neural networks and deep learning. The course will use Python coding language, TensorFlow deep learning framework and Google Cloud computational platform with GPUs. The main resource for the course is the book entitled "Deep Learning" by Ian Goodfellow, Yoshua Bengio, and Aaron Courville, MIT Press, http://www.deeplearningbook.org, 2016.

Students will contribute to the course by:

TODO Decide and discuss assignment -1 pretest There will be 4 homeworks in total. The first, homework 0, will help you ascertain the programming know-how needed to solve the following homeworks. All homeworks will be graded. If you have difficulties with homework 0, please carefully evaluate whether you will have enough time to brush up your programming knowledge by the end of the change of program period. The course requires theoretical background in probability and statistics and linear algebra. You are strongly advised to drop the class if you cannot satisfactorily finish questions of homework 0 by the add/drop date, or if you judge that the first lecture description of prerequsite linear algebra and probability/statistics may be overly demanding.

(Tentative) 2019 Syllabus link


Grading policy

HW(40%), (one midterm) Exam(25%), Project(35%).

Late homeworks (assignments) Each student is entitled to 4 late days without penalty. For all homeworks together, a student can divide those four days in any fashion needed. Examples: (i) Homework 3 is late 4 days, in which case no other homework can be late for any amount of time; (ii) Homework 1 is late 1 day, homework 2 is late 2 days, in which case the student did not use all four late days. The unit of delay can not be divided into less than a full day (like hours). Requests for additional extensions will not be granted: if the budget of 4 days is blown, the student will be given 0 credit for homework(s) for which their submission is late.


Computational Environment(s) and Prerequisites

Introduction

Computational tools are essential for learning about, designing, and experimenting with deep learning models. At the highest level, deep learning developers use one of the deep learning frameworks to build and run models, which rely on a myriad of either generic or custom software libraries. Course E4040 will use a particular version of the Tensorflow framework. The underlying computing language is Python, and the course will rely on Python code/libraries and Jupyter Notebooks for developing and experimenting with code. The students will be asked to run the tools and deep learning models on their own (local) computers, as well as in the Google cloud. Versioning of tools is complex, and students need to follow the precise tool version instructions in order to have functional computing environments. The course relies on Conda and PIP for tool versioning.


Local Environment Setup -> link

Students will be asked to use their personal/local computers to run python code, jupyter notebooks, Tensorflow framework and deep learning models. This is possible both with computers which have graphics processing units (GPUs) and with computers which do not have GPUs, where the execution will be faster on computers with GPUs. The begining assignments in this course are runnable on single processor (non-GPU) computers. Later assignments and projects will require that students run them on machines with GPUs; in this case the assignments can be run on personal computers or on the Google Cloud GPU machines. We will provide coupons for Google cloud access. Clicking on this section leads to instructions how to install the required tools on personal computers.

Python Tutorial -> link

There are many online Python tutorials. This section is linked to one such tutorial. For majority of this course, we request that students use python 3.6 (compatible with the prescribed version of Tensorflow 1.13) when working on programming assignments. In the shared Google cloud image, we only installed Python 3.6 version. Therefore, you should strive to use exactly the same version on your personal/local computers.

Google Compute Engine (GCE) Setup on the Google Cloud Platform (GCP) -> link

For more complex assignments, students will be able to run deep learning code on Google Compute Engine (GCE) virtual machine (VM) instances, with access to GPU resources. The instructors will provide coupons (codes) which will enable students to use Google cloud resources for free. This tutorial provides instructions how to setup a VM instance using the VM image created for the E4040 course.

TensorFlow Tutorial -> link

TensorFlow is an open-source deep learning framework developed by Google. It is essential that you learn to program in TensorFlow for this course. Note that the 2019 version of the E4040 course uses version 1.13 of Tensorflow, whereas the most recent TensorFlow homepage may be refering to and using a more recent version (such as 2.0). For 2019 assignments, students have to use the prescribed versions of Tensorflow and Python. If students wish to experiment with other versions of tools, they are advised to create additional (conda) virtual environments to prevent conflicts with tool versions used for the assignments.

Linux Introduction -> link

The GCP VM instances which we use are Linux-based. If you haven't used Linux system before, this tutorial offers hands-on instructions to basic Linux commands that you will be using throughout the course.

Git Commands -> link

Your homework assignments will be based on Github classroom repositories, which can be manipulated by Git commands. If you don't know what Git is, it is strongly recommended that you read the first 3 chapters of this website. Additional instructions will be provided for use of assignment-related github directories.

Google Colab -> link

Colab provides a free cloud service based on Jupyter Notebooks. It supports free GPUs with popular deep learning libraries with an execution time limit. It is a good platform for developing and testing the deep learning code, which can avoid the issues of tool installation. However, it has limitations. We do not recommend that you do the assignments on Colab since everything except Jupyter Notebooks would be automatically deleted every 12 hours. Moreover, we request that you first set up your own local computing environment and GCE since both are a necessary skill for a practicing deep learning professional.

FAQ -> link

Solutions to some common problems


ECBM E4040 Neural Networks and Deep Learning, 2019.

Columbia University