Course Outline

The Official Course Outline now lives in ECOS.

ECOS is UNSW's Enterprise Course Outline Solution.

Contents

• Course Details
• Course Summary
• Student Learning Outcomes
• Textbook
• Assumed Knowledge
• Planned Topics
• Teaching Strategies
• Assessment
• Student Conduct
• Course Evaluation and Development

Course Details

Course Code	COMP9444
Course Title	Neural Networks and Deep Learning
Convenor	Alan Blair and Sonit Singh
Admin	Zhongsui Guo
Email	cs9444@cse.unsw.edu.au
Classes	Tuesday 9-11am and Thursday 11am-1pm in Mathews Theatre A (K-D23-201)
Consultations	TBA
Units of Credit	6
Ed Website	https://edstem.org/au/courses/18763/discussion/
WebCMS	https://webcms3.cse.unsw.edu.au/COMP9444/24T3/
Handbook Entry	http://www.handbook.unsw.edu.au/postgraduate/courses/current/COMP9444.html

Course Summary

This course aims to introduce students to the main topics and methods in the field of neural networks and deep learning, ranging from traditional neural network models to the latest research and applications of deep learning.

Topics chosen from: perceptrons, feedforward neural networks, backpropagation, deep convolutional networks, image processing; geometric analysis of trained networks; recurrent networks, language processing, semantic analysis, long short term memory; deep reinforcement learning; autoencoders, generative models, adversarial training; designing successful applications of neural networks; recent developments in neural networks and deep learning.

Student Learning Outcomes

After completing COMP9444, students should be able to

• discuss the social, intellectual, and neurobiological context of neural networks and deep learning;
• describe a variety of NN and DL techniques - including fully connected, convolutional and recurrent networks, deep reinforcement learning, generative models and adversarial training;
• analyse a problem and devise a suitable neural network solution;
• use a Python module or simulation package to implement neural networks for a range of tasks, including image and language processing, reinforcement learning, and unsupervised learning.

Textbook

The textbook and reference book for this course are:

Understanding Deep Learning (UDL)
By Simon J.D. Prince
MIT Press, 2023

https://udlbook.github.io/udlbook/
https://mitpress.mit.edu/9780262048644/understanding-deep-learning/

Deep Learning (DL)
By Ian Goodfellow, Yoshua Bengio and Aaron Courville
MIT Press, 2016

http://www.deeplearningbook.org
https://mitpress.mit.edu/books/deep-learning

Assumed Knowledge

The course will assume knowledge of the following mathematical topics:

• Linear Algebra (DL 2.1-2.8; UDL Appendix B)
• Probability (DL 3.1-3.14; UDL Appendix C)
• Calculus and Chain Rule (DL 6.5.2)

Students should study the relevant sections of the textbook (shown in brackets) and, if necessary, try to revise these topics on their own during the first few weeks of the course.

Planned Topics

The planned topics for this course are:

Week	Topic	Textbook
wk1 Tue	Neuroanatomy and Perceptrons	(DL 1.2, 9.10)
wk1 Wed	Multi Layer Networks and Backpropagation	(DL 4.3); (UDL Chapter 3, 4)
wk2 Tue	Probability, Generalisation and Overfitting	(DL 3.1-14, 5.1-6, 7.11-12)
wk2 Wed	PyTorch
wk3 Tue	Cross Entropy, Softmax, Weight Decay, Momentum	(DL 6.1-5); (UDL Chapter 5, 6, 9)
wk3 Wed	Hidden Unit Dynamics	(DL 8.2-3)
wk4 Tue	Convolutional Networks	(DL 7.9, 9.1-5); (UDL Chapter 10, 11)
wk4 Wed	Image Processing	(DL 7.4, 8.4, 8.7.1)
wk5 Tue	Recurrent Networks	(DL 10.2)
wk5 Wed	Long Short Term Memory	(DL 10.7, 10.10)
wk6	(Flexibility Week)
wk7 Tue	Word Vectors	(DL 12.4)
wk7 Wed	Language Processing	(DL 10.4); (UDL Chapter 12)
wk8 Tue	Reinforcement Learning	(DL 12.5.1.1); (UDL Chapter 19)
wk8 Wed	TD-Learning and Q-Learning
wk9 Tue	Policy Learning and Deep RL	(DL 18.1, 20.9)
wk9 Wed	Autoencoders and Adversarial Training	(DL 14.1-5, 20.10.3, 20.10.4); (UDL Chapter 14, 15, 17)
wk10 Tue	Generative Models	(UDL Chapter 18)
wk10 Wed	Deep Learning and Ethics, Review	(UDL Chapter 21)

The relevant sections of the textbook are shown in brackets.
The textbook may be supplemented with additional materials for some topics.

Teaching Strategies

The course will include Lectures (Weeks 1-5, 7-10); Tutorials (Weeks 1-5) and Project Mentoring sessions (Weeks 6-10).

Most of the course materials will be delivered via WebCMS and Ed . This includes course content in the form of text and images, online discussion forums, quizzes, coding exercises using Jupyter notebooks, as well as links to the Tutorial Questions.

Students are encouraged to read through the materials on Ed before each lecture. The lecture time will be used to summarize the material, discuss recent developments, and answer any questions about the topic.

Tutorials will be used in Weeks 1 to 5, to discuss worked examples and develop a deeper understanding of fundamental topics.

Project Mentoring Sessions will be used in Weeks 6 to 10, to assist with the Group Project.

You are expected to:

• review the course materials before and after each scheduled class
• attempt the Tutorial Questions ahead of time and be ready to ask questions in the tutorial
• complete relevant Quizzes, Coding Exercises, Assignment and Project
• discuss the material with your fellow students
• consider exploring topics of particular interest by writing and running your own programs
• ask questions and contribute to the discussion in the online forums

Assessment

The assessment for this course will be:

Labs + Assignment (including 3% for Lab Exercises in Weeks 1-3 + 17% for Assignment)	20%
Group Project (Weeks 4-10 with final presentation in Week 10)	35%
Exam	45%

Students are expected to form themselves into groups of 5 for the Group Project, by the end of Week 3. Each group will be assigned a Mentor. More details about group formation and mentoring will be provided at the beginning of Week 1.

The assignment will involve writing code in PyTorch. These are the versions of modules currently installed on the CSE lab machines. Please try to install equal or later versions on your own laptop.

python3	3.11.2
torch	1.13.0
numpy	1.24.2
sklearn	1.2.1

Student Conduct

The Student Code of Conduct sets out what the University expects from students as members of the UNSW community. As well as the learning, teaching and research environment, the University aims to provide an environment that enables students to achieve their full potential and to provide an experience consistent with the University's values and guiding principles. A condition of enrolment is that students inform themselves of the University's rules and policies affecting them, and conduct themselves accordingly.

In particular, students have the responsibility to observe standards of equity and respect in dealing with every member of the University community. This applies to all activities on UNSW premises and all external activities related to study and research, and includes behaviour in person as well as behaviour on social media, for example Facebook groups set up for the purpose of discussing UNSW courses or course work. Behaviour that is considered in breach of the Student Code Policy as discriminatory, sexually inappropriate, bullying, harassing, invading another's privacy or causing any person to fear for their personal safety is serious misconduct and can lead to severe penalties, including suspension or exclusion from UNSW.

If you have any concerns, you may raise them with your lecturer, or approach the School Ethics Officer, Grievance Officer, or one of the student representatives.

Plagiarism is defined as using the words or ideas of others and presenting them as your own. UNSW and CSE treat plagiarism as academic misconduct, which means that it carries penalties as severe as being excluded from further study at UNSW. with at UNSW:

Make sure that you read and understand the UNSW Policy on Plagiarism and Academic Integrity Ignorance is not accepted as an excuse for plagiarism. In particular, you are also responsible for ensuring that your assignment files are not accessible by anyone but you, by setting the correct permissions in your CSE directory and in any related code repository. Note also that plagiarism includes paying or asking another person to do a piece of work for you and then submitting it as your own work.

UNSW has an ongoing commitment to fostering a culture of learning informed by academic integrity. All UNSW staff and students have a responsibility to adhere to this principle of academic integrity. Plagiarism undermines academic integrity and is not tolerated at UNSW. Plagiarism at UNSW is defined as using the words or ideas of others and passing them off as your own.

Course Evaluation and Development

Every term, student feedback is requested in a survey using UNSW's myExperience online survey system where the feedback will be used to make improvements to the course. Students are also encouraged to provide informal feedback during the session, and to let course staff know of any problems as soon as they arise. Suggestions will be listened to openly, positively, constructively, and thankfully, and every reasonable effort will be made to address them.

Changes made in response to feedback from previous offerings include: migration of the course content to an online platform with expanded text, clickable references, coding exercises and online discussion forums; introduction of a group project, and, most recently, introduction of scheduled tutorials. We hope these changes will help to make this course a rewarding and enjoyable experience.