COMP3142 Notices

Don't worry if you cannot find many crashes on MJS (assignment 2)

Posted by Yuekang Li Tuesday 19 November 2024, 03:20:44 PM.

If you cannot find say >=10 crashes on mjs in 30 minutes, don't worry.

We will lower the performance requirement (penalty), and evaluate your fuzzer's performance according to the average performance of the submitted implementations. Based on the questions I have received so far, I think as far as you can implement the logic of the 6 features correctly, your implementation should be above average.

This performance comparison is just a reference for us to check the correctness of your implementation. Only if your implementation performs very badly (e.g., cannot find any crashes on mjs in 30 minutes), we will investigate. Otherwise, you are still marked according to the source code and descriptions you have submitted.

Meanwhile, if you are debugging why your implementation cannot find crashes, you could try this:

Use only "test_3.js" and "test_13.js" as the initial seeds for mjs, and observe.

If you can find more crashes or find crashes faster, then it might be the problem of seed prioritisation or power scheduling.

If this doesn't help to find more crashes or find crashes faster, then it might be the problem of the mutators.

Please Submit MyExperience Response

Posted by Yuekang Li Friday 15 November 2024, 01:38:07 PM.

Hi Class,

Please submit your MyExperience feedback if you haven’t done so.

This is the first term running this course. Your feedback and constructive suggestions would be very precious to make the course better!

By the way, I couldn’t make it for consultation today because I need to attend a workshop. Sorry for the late update.

And for next week, I’ll be overseas.

So I’ll provide online consultation sessions instead. I’ll send out new notices later. Please stay tuned!

Consultation Session on Thursday (14th Nov 2-4pm)

Posted by Yuekang Li Wednesday 13 November 2024, 02:25:01 PM.

Hi Class,

Yesterday's lecture is the last one for this term.

You can find a summary of all the course contents here: https://prezi.com/view/Cw3bRhTGJ8ucD4R3xAXT/

(You can get to the last page, zoom in/out, and drag the graph around to explore).

All the lecture recordings are also available on YouTube: https://www.youtube.com/playlist?list=PLneo1-0n40Q8oqdXtP9mC90KRkxlImdxK

This Thursday (tomorrow), instead of a lecture, I'll be offering a consultation session to answer questions related to assignment 2. You can find me in my office from 2-4 pm. I'm in 501G, K17. (If you don't have access to the lab on the fifth floor, which is always locked, please email me, and I'll fetch you.)

Quiz 2

Posted by Yuekang Li Monday 11 November 2024, 02:30:20 PM.

Quiz 2 will be available today at 10 pm. And the due date will be next Monday (18th Nov) 10 pm (similar to quiz 1).

The results of quiz 1 will be viewable this Friday (15th Nov) 10 pm.

The results of quiz 2 will be viewable next next Friday (29th Nov) 10 pm.

You can find the upcoming due dates on the top right corner of this course's webcms main page.

Utility for Checking the Correctness of Feature 1 of Assignment 2

Posted by Yuekang Li Friday 08 November 2024, 05:36:40 PM.

Hi Class,

It seems most of you have started implementing feature 1.

I created a utility for you to check if you have implemented feature 1 correctly:

https://github.com/ThePatrickStar/mini-lop/blob/main/seed_inspector.py

You can use it the same way as using "main.py", e.g. (in the docker):

python3 /mini-lop/seed_inspector.py -c /configs/mjs-config.toml

This tool can print out how many new edges are covered by each new seed you store in the "queue" folder.

If your implementation is correct, none of the new seed should cover 0 new edges.

An example output of the tool is like this:

I modified conf.py to implement this tool. You'll need to update your conf.py in order to use it.

Detailed changes are here:

https://github.com/ThePatrickStar/mini-lop/commit/c2e959b05003d238df8d4b25470df810312fc562

Assignment 2 Specification Update

Posted by Yuekang Li Thursday 07 November 2024, 05:28:52 PM.

Hi Class,

The specifications for Assignment 2 are updated.

The major change is the addition of the submission guideline to the last two pages of the document. Please go through it carefully.

I've also updated the source code of the default mini-lop implementation on Github and inside the docker. This is to address a potential rare issue of generating an infinite loop in the .js test input passed to mjs: https://edstem.org/au/courses/19161/discussion/2372638

This update is in the run_target function of the execution.py file . You can just replace the execution.py file with the newest version. This update shouldn't affect your existing implementations as the function name and parameters have not been changed.

We will do the marking of your assignment 2 following these rules:

• If a feature is not implemented:
  • If you have a very detailed (pseudo-code-like) description of that feature
    • you can earn up to 50% of the mark for that feature
• Else:
  
  • We will give you a full mark of that feature by default
    • We will review your explanation of the implementation and the actual code and look for things that don't make sense (or are not aligned with the natural language task descriptions in the specs). This review shall roughly follow the bullet points listed in the specs.
      • If such things are found, we will deduce your mark accordingly.

We will also run your code on our side. As long as your code coverage and crash detection are not >30-40% worse than our internal implementation in our environment, you will not receive an overall mark deduction. Otherwise, we will go through your code in detail and mark accordingly.

There's a walk-through of the assignment in today's lecture. Please watch the recording if you'd like to have more information. I'll upload a copy to YouTube as well later. Of course, you are always welcome to ask questions on our course forum. Just try to avoid doing it at the last minute.

Quiz1-Q4-Update

Posted by Yuekang Li Wednesday 30 October 2024, 12:28:28 PM.

Hi Class,

I made a mistake creating the options for Quiz-1 question 4. All the previous options are wrong.

Sorry about this and thank you for those who pointed this out!

I've updated the options of Q4 and please revise your answer. (only the last submission is marked)

Hint: You don't have to count things out manually. You can put the code into some text editor (e.g. Google doc) and use the search function to check for occurrences of operators and operands.

Quiz-1

Posted by Yuekang Li Monday 28 October 2024, 06:23:26 PM.

Hi class,

The questions for quiz 1 are here: https://webcms3.cse.unsw.edu.au/COMP3142/24T3/activities/quizzes/2274

They will become viewable to you after 10:00 pm today. The due date is 4th Nov 2025 10:00 pm (next Monday).

Please send Sarah and me an email if you need an extension.

You can submit multiple times and only the last try will be marked.

Assignment 2 Specification Released

Posted by Yuekang Li Sunday 27 October 2024, 07:15:59 PM.

Hi Class,

Hope you had a great weekend!

The specification for assignment 2 has been released: https://webcms3.cse.unsw.edu.au/COMP3142/24T3/resources/106784

Please have a look and play around with the docker image and code.

Note that some tasks relate to lecture content in weeks 8 and 9. I'll update the descriptions later, together with the questions/feedback on Ed.

The submission guideline for this assignment will be released later.

In this assignment, different features have different levels of difficulty for implementation. But this is intended. Features 2, 4, and 5 are relatively easier. So you can earn enough marks to pass. Feature 7 is a bonus feature and Features 5 and 6 provide optional bonuses as well. You can challenge yourself to earn higher marks (the total mark is capped at 60).

I told some of you that I planned to make assignment 2 something that can be used to attend Google's fuzzing competition. But it turned out that this could be too difficult for an assignment. However, if you want to attend this competition after trying out this assignment, please let me know. I do have some ideas that might help to earn awards this year.

Assignment-1 Mark Released

Posted by Yuekang Li Friday 25 October 2024, 03:24:23 PM.

Hi Class,

The mark for assignment 1 is released. You can check your mark with give .

We will release the specification for Assignment 2 this weekend. Please stay tuned.

Note: this course doesn't have a final exam. You can consider assignment 2 as a take-home coding exam though.

Practices for Flex Week and Updates for Assignment 2

Posted by Yuekang Li Friday 11 October 2024, 05:23:32 PM.

Hi Class,

I hope your term is going well!

This week, we’ve covered fuzzing mechanisms, focusing on American Fuzzy Lop (AFL). Now it’s time to put theory into practice by using grey-box fuzzers to find bugs! I've prepared some resources for you to explore during flex week: https://docs.google.com/presentation/d/1X-OHZjzspmG5R9cQONOCMBBENFzBYjp-tW3gU3Uuzhc/edit?usp=sharing

Additionally, I did a live demo on using AFL to fuzz llama.cpp during Thursday's lecture. Make sure to check it out!

Please try to practice with these materials, as Assignment 2 will require you to develop ensemble fuzzing strategies that orchestrate multiple fuzzers. Familiarity with each fuzzer is essential. If you have any questions—technical or otherwise—please don’t hesitate to reach out!

Regarding Assignment 2, it will be an individual assignment . This decision is based on two factors:

1. Since we don’t have tutorials, it can be challenging for some of you to find group partners.
2. Ensuring fair grading between one-person and multi-person groups is difficult (especially when it comes to peer evaluation).

Though working as individuals, I still highly encourage you to discuss your ideas and questions with others, including myself!

Clarification for Q1.1.1

Posted by Yuekang Li Saturday 05 October 2024, 07:15:11 PM.

Hi Class,

I made a mistake when answering this forum question days ago (updated my answer just now): https://edstem.org/au/courses/19161/discussion/2261402

To clarify, by "boundary value testing", we mean normal BVT. Normally, if we don't explicitly specify whether it's "robust" or "worst-case", then it's considered normal by default.

This is my bad. So you don't have to change your answers if your answer is made for RBVT. Answers for both normal BVT and RBVT will be considered correct when we do the marking.

Consultation Time and A Glance at Assignment 2

Posted by Yuekang Li Thursday 03 October 2024, 01:33:50 PM.

Hi Class,

We don't have tutorials/labs for this course but assignment 2 will involve more coding/development tasks.
So, let's have an additional consultation session every Friday (except for week 6), from 1 pm to 3 pm, starting from tomorrow. You're welcome to ask me any questions! But please try to avoid directly asking me whether your assignment answers are correct or not (we can discuss the assignment specifications though).
You can find me in my office @ k17 (CSE building) 501G.

To give you a glance at assignment 2 before consensus date: the main task will be developing a fuzzing tool (based on existing ones) that can attend the fuzzing competition organized by Google every year. The side tasks involve other stuff related to bug hunting (like analysing code complexity, reporting bugs etc.).

It was designed to be a team-based project but you are also allowed to do it solo. We will make sure it's doable for one person.

Just one more thing, the current group lead of Google's fuzzing team is an UNSW alumni who is based in Sydney. We may have a chance to invite him for a guest lecture in week 10. Stay tuned!

Link (the updated timetable on webcms):
https://webcms3.cse.unsw.edu.au/COMP3142/24T3/timetable

Assignment-1 Specification Updates

Posted by Yuekang Li Monday 30 September 2024, 04:34:08 PM.

Hi Class,

I hope you have already started working on assignment 1. Please don't do it in the last minute!

According to the feedback on Ed, some of the question descriptions are updated for clarity.
You can find out what's been updated here: https://draftable.com/compare/KZvXCqOdqyag

And the specification on webcms is the newest one.

Just a reminder: if you are on an ELP and you need extra time for this assignment, please send the course staffs an email 24 hours before the due date.

Assignment 1 Released

Posted by Yuekang Li Tuesday 24 September 2024, 06:00:05 PM.

Hi Class,

The specifications for assignment 1 have been released:
https://webcms3.cse.unsw.edu.au/COMP3142/24T3/resources/105983

Please note that the due date is 6th Oct 2024 at 10 pm Sydney time (AEDT) (same as the census date).

This is an INDIVIDUAL assignment. You can try to ask PiePie for help. But don't rely on it. It may not give you correct answers.

For ELP students, please send Sarah or me emails to request extensions.

Good luck and happy bug hunting!

COMP3142: Lecture Recordings and Livestream

Posted by Yuekang Li Tuesday 17 September 2024, 11:50:31 AM.

Good day class!

The lecture recordings are now available on this YouTube channel in case you don't like Echo360: https://www.youtube.com/playlist?list=PLneo1-0n40Q8oqdXtP9mC90KRkxlImdxK

The lecture livestream should have been set up (not tested though). You can access it on Echo360 via Moodle. Please let me know if there's any issue with the livestream.

For those who are newly enrolled or have skipped the first lecture, here are two important updates:

• Use this link to join the Ed Forum
• Our planned timetable for this term is here

Cheers!

Welcome to COMP3142 24T3

Posted by Yuekang Li Monday 09 September 2024, 09:25:44 AM.

Good day class!

I'm Yueakng Li, the lecturer in charge of COMP3142. Welcome to term 3 and this is the first time offering this course!

We will use webcms as the main website for this course. Moodle is only used to access course recordings. We will use edforum as the course forum. (I'll invite you to the forum later on.)

Wish you a fruitful term and see you tomorrow @ old main building G32 14:00.

Cheers,
Yuekang