There's no point sending me emails about LE grades. I am seriously busy (again) this week, so won't be able to fix them until
Week 2. Most of you are going to pass. All that fixing the assignment mark will do is increase your final mark by a few points.
The seemingly never-ending marking saga continues ...
"Final" marks went in today, and these are what you'll get tomorrow (Thu 20 May).
If you get an LE grade, it means I'm still looking at your assignment mark and will update the mark later. If you got 6.7 for Assignment 2 I re-tested your submission; if you still have 6.7, it's because you messed up something else (compilation error or too slooooow). You should have an LE grade until I get a chance to investigate.
If you get a WD grade, either you have a Supp (you know who you are) or I suspect that you have "over collaborated", and I'll be in touch for an explanation. I am still doing plagiarism checking, so even if you get a mark tomorrow, it might still turn into a "Please Explain" ... feeling guilty?
Tonight, I am having a night off after an exhausting round of marking, so don't expect an answer to any email until late tomorrow (I have meetings all day).
There's no need to send any more emails about scoring 6.7 for the Assignment 2 automarking.
I will try to find out what went wrong with these submissions tonight. You ahve supplied some useful evidence for what the problem might be.
I am extremely annoyed. I thought I'd fixed all the submissions that didn't follow the spec, and submitted files in a sub-directory. All of the 6.7's I have seen so far are like this. You should all feel guilty about wasting my time on fixing this.
Midnight update: I've run most of the 6.7's. I'll finish them in the morning. Some have compile errors and excessive time usage. I'll deal with them (and other TODOs) once I've submitted the bulk of the marks.
You should be able to see the testing/automarking output by going to
and clicking on [Collect Submission]
If you want to play with the testing data, it's available as
P.S. Just because "It works on my machine" doesn't mean it'll work on the CSE servers. You should have checked by running it on our servers before submitting.
Thanks for all the email about "My ass2 mark is not right". It was entirely auto-marked by a script that I spent a week tweaking and tweaking and tweaking. Clearly I haven't got it right yet ... except for all the people who got full marks ... funny that ...
Note that there were two parts to this assignment (a) getting the right answers to queries, (b) doing some filtering. It's easy to get all the answers right if you simply read all of the data, but the goal here was to use the signatures to reduce the number of data pages read. My script measures this by using the query stats that you produced. In some cases, you should have returned exact numbers (e.g. there are 1000 tuples and 1000 tuple signatures and you scan 1000 signatures to work out which data pages might contain matching tuples). Other stats were "flexible" depending on how you constructed your signatures; the script also allowed for this, but maybe some people did way better than what I was expecting and were flagged as being incorrect. I'll certainly fix those.
Note also that we cut off building the databases after 60 seconds (which is already very generous for building databases of the size being tested ... many people built the largest database correctly in under 10 seconds). If you think that your system would have worked fine if only I'd given an extra few seconds to build the database, let me know.
I will shortly be putting up "collectable" output from the auto-marking. It has two major parts: (a) a trace of the testing, (b) a trace of the auto-marking script. Please read those before querying the mark. If you have a query, tell me what you think the script got wrong.
I'll re-run the scripts tonight if any clear patterns emerge. I can enter the final marks into Astra tomorrow )Wed 19 May) morning to make it to the results that will be released on Thursday. After that, final marks can be adjusted if I think there is justification for doing it.
I will also have a look at the ass1 marks that people have queried. There is no chance to finish this before tomorrow. If there's a query on one of your assignment marks that I can't resolve before then, your mark will end being "LE". Don't send me email saying "What does LE mean?". It means "Late Entry" i.e. your mark was not ready to be submitted on time. I told people who already queried assignment marks that their mark would be "withheld" ... this manifests itself via an "LE".
Speaking of plagiarism ... I'll be running "over collaboration" checks on the two assignments and the exam. If you're feeling guilty, send me a confession email and we can arrange a "plea deal".
Don't expect assignment marks until
Tuesday (May 18). You can query them after that if you need to.
Marks can be adjusted even after they're "finalised" next week.
Marks need to be "finalised" by May 17 (but can be updated after that if needed).
We'll mark the exam by May 14 and finish the assignment marking by May 10.
Inevitably, people have queries about marks. Send me email, with a clear description of the problem, and I'll add it to my TODO list, which I'll try to work my way through before May 17. Don't expect an instant fix to your mark, but remind me if nothing has changed by May 14.
In your email please put your zID. This saves me a couple of minutes trying to work out what your zID is, based on just your name.
Submit your answers now. Anything submitted after 8pm AEST will not be marked.
I warned you what would happen if you all left your submissions to the last minute. You can have a leeway of a couple of minutes to allow for the inevitable give/webcms slowdown, but I won't be marking any submitted after 8:10pm. You may as well stop submitting now.
Q2 and Q3 has a typo in the line ...
Type your answer to this question into the file called bnl.c
Clearly, it should be the file mentioned in the give command on the next line (and everywhere else).
I've fixed the online version.
If you happen to be poking around the files in the exam paper directory, you might see Q09.html.
This is a remnant of an old exam, and should be ignored.
I just removed it from the online version. Unfortunately, everyone who downloaded exam-paper,zip will still have a copy. Feel free to delete it.
The Final Exam starts at 2pm AEST TODAY ... last submissions accepted at 8pm AEST.
Remember the fit-to-sit rule: if you take the exam today, you won't get a Supp.
If you are unable to take the exam today, then you must put in a Special Consideration request ASAP via MyUNSW. You will need documentation.
The starting point for the exam is the "Instructions" page which you can access as
This will give you access to the Exam Paper itself.
If Webcms3 is naughty again (i.e. goes slow), then you can access the Instructions via the second link above, and you can access the lecture slides via:
And you can access pretty much everything else via
Some hints for the exam ...
Give as you go ... submit answers to questions once you have completed them; don't leave all submissions until the end.
Don't get bogged down ... if you're stuck on a question (e.g. you've spent more than 50 mins on it), go on to another question and come back to the "stuck" one later.
Programs that don't compile or that are (trivial mods) to the supplied code are worth zero marks. Otherwise, marks are based on the number of tests you pass.
Sorry, guys. I have far too much work at the moment (writing school report for accreditation, writing the exam, marking assignments).
Realistically, I may have time to respond to the occasional forum post, but nothing beyond that. If anyone points out an error in a solution, I'll fix that.
Good luck for the Exam on Monday.
The Course Outline said that the Supp would be held in the week May 9-13. Apart from this not being a week, it falls in the middle of the real exam period.
The Supp Exam will be held in T2's O-week ( May 24-28 ) . We don't have a specific date yet, but if you are awarded a Supp, we will contact you closer to the date with details.
Despite what I thought were reasonably clear instructions about how to build and submit the ass2.tar file for Assignment 2, here's what we got
If you're one of the following (5144842, 5194012, 5238306, 5239010), please make a TAR files as instructed and send it to me as an email attachment. If you can't manage to make a TAR file, use zip , but call the file ass2.zip .
Out of the TAR files, 6 had compilation errors, which suggest that you didn't compile with the -Wall option.
And I haven't got to the point of finding out who forgot to submit files that they need to make their code work.
If you all submitted as per instructions, marking would be a lot simpler/quicker.
I was hoping that Ass1 marks would be done by now. Seems not. We may need to bring in extra markers to get it done.
I want both Ass1 and Ass2 marks to be available by the end of this week.
So far, 25% of students have filled out the course evaluation on MyExperience, which is good, but ...
It would be great to push it beyond 33%, so if you haven't filled out the survey, please spend 10 minutes to evaluate the course ( https://unsw.bluera.com/unsw/ )
P.S. Some other Engineering Schools have over 40% response rate.
The slides for the last part of the Thursday lecture, where I talk about the exam, are at the top of the Videos and Slides page (under "All Weeks").
The slides for Oliver Tan's lectures are under "Week 10" > "Online Sessions"
Today 2-4 ... https://unsw.zoom.us/j/84426315784
Oliver Tan will tell us about how Cockroach DB is web-scale
Online session 2-4 ... https://unsw.zoom.us/j/89169783336
I'll talk a little, but most of the session is Oliver Tan talking about DBMS trends.
Find out what's hot in DBs in the Bay Area.
The systems support people sent us a list of people who they believe lost files last Thursday/Friday in the Great Home Directory Wipe-out.
These are the student IDs of the affected people:
5082514 5092597 5141959 5144842 5165035 5173587 5188650 5191527 5203199 5218457 5222381 5226987 5230245 5230272 5232091 5233467 5234037 5237012 5237481 5237709 5240946 5241257 5241763 5242063 5242233 5243151 5244460 5255375 5282222 5282736 5284122 5286846 5290495 5291376 5300831 5310085 5324608 5346267
We have given everyone on this list a 2 day extension, assuming that they lost a day recovering their files and then had to restore lost code.
If you lost files and your ID us not on this list, let us know. And don't try to claim you lost all your files if you didn't. We can check.
Of course, if you submitted before Thursday, this extension is irrelevant to you.
The ongoing issue causing files to be removed on CSE has now been resolved. There is no longer a chance that your files will be removed. BUT if you used vlab yesterday or today there is a possibility that, unfortunately, all files have been removed from your home directory.
Anyone who has used vlab over the last two days should login and check if they are effected.
If you are effected and don't have a local backup of your files the best best is to use the tkrestore command to get a backup of your files from Wednesday night. Instruction here .
If you are missing your assignment 2 and have used give please email the class account and we can send your your most recent submission.
If you are having issues with tkrestore please email email@example.com for assistance. Note that assistance might not be provided until Monday as there are a number of students and staff effected.
We should be able to get a list of effected students, an extension to assignment 2 will be provided to those effected.
Based on the file deletion mess from yesterday, I think you would be very wise to set up your home computer so that you can work locally during the exam. You will still need web access to CSE to look at the questions, and download the work files, but apart from that, the only time you need to access the CSE servers is to submit your answers.
If you don't have a C compiler and standard POSIX libraries on your home machine, it might be worth investigating how to get them. Maybe not this week, but soon. You could check whether you have a usable C programming environment by trying to get the Assignment 2 code to run locally.
There is an ongoing issue with vlab being unable to connect to the CSE file servers, this means that vlab can look like your files have been deleted.
We have no reason to believe that any files are actually missing. They are just inaccessible.
Turns out they are actually missing.
Three options to recover
https://unsw.zoom.us/j/84782045263 ... running from 2-4 ... i.e. starting soon
Ooops ... I just realised that I had the wrong permissions on the Week 9 page ... now fixed, so you can at least see the intended videos for this week ... some of which are there already ... the rest are coming
A shorter than usual session today: 3:00 - 4:00 ... https://unsw.zoom.us/j/87089453194
The assignment 2 testing page is now available.
There's no script that you can run and have it say "OK, OK, OK, ..." because you'll probably all have slightly different numbers in the Query Stats . If you're close to what they say, and if you get the correct matching tuples, that's ok.
If you get better Query Stats (e.g. less false matches and still correct results) than the ones supplied, let me know.
This was all run on my Mac, so the timing results might be different to what you observe on the CSE machines.
Eeek! Not giving you a (standard) method for codeword generation *and* not being precise about rounding up or rounding down numbers of bits may have been a mistake ...
Not totally, but makes supplying sample test cases problematic.
Edit : I just remembered that I put some codeword generation code in the spec.
I have run a few of your submissions (please get rid of debugging output before you submit!), and just about all of them have different values for data pages read, etc. They all return the correct tuples, which, after all, is the main point, but it's going to be impossible to come up with a set of standard query stats.
I'll put up some testing info later today, but please don't panic if you don't get exactly the same numbers for the query stats, e.g. stats for some random query
# sig pages read: 1 # signatures read: 5 # data pages read: 9 # tuples examined: 956 # false match pages: 0
Some of them should be consistent, but others could be different depending on how you generated your codewords. I'll say more about this in the testing page available later today
As long as you produce the correct answer tuples, you're most of the way there.
Regular time-slot again ... 2-4 ... https://unsw.zoom.us/j/87071641199
A few people have been asking about how long insert takes. Here's how I did it:
jas@bock soln % ./create R simc 100000 5 100 jas@bock soln % ./gendata 90000 5 123456 13 > R.input jas@bock soln % time ./insert R < R.input 10.30s user 9.76s system 98% cpu 20.362 total
If you try it using the same parameters and it's substantially slower ...
The online session starts from 2.30 today ... https://unsw.zoom.us/j/88486037589
I changed the wording of Q2 a little, to (hopefully) clarify. If the new wording would change your answer, you can re-submit.
https://unsw.zoom.us/j/89603261463 .... starting real soon
As usual, 2-4 ... https://unsw.zoom.us/j/88513568125
Will talk about assignment 2.
Looks like I messed up the interface between create.c and stats.c (and maybe other *.c files).
I'll fix this tomorrow (Sunday).
Fixed. The problem was newRelation() in create.c having the args in the wrong order. If you just want to download a new version of create.c , you can use the command:
unzip /web/cs9315/21T1/assignments/ass2/ass2.zip create.c
Please let me know of any other issues ASAP. But check that they're not already on the forum before asking.
Just realised that I hadn't made these visible. Now they are. I'll put up the CATC slides over the weekend.
The spec for Assignment 2 is now available.
If you want to look at it while the CSE servers are down, I'd suggest "Save as PDF" on your local machine for the spec, and download ass2.zip onto your local machine.
As always, there will be glitches, unclear bits, etc. Let me know asap so that I can fix them.
Someone pointed out that we’re technically not supposed to have assessments in Week 6, so we’ll do Quiz 3 in Week 7, and Quiz 4 in Week 8.
Due to scheduled maintenance on the power supply to K17, the CSE machine room will be shut down from 7pm Fri 26 March until at least 8pm on Sat 27 March. This means: no Webcms3, no Grieg, no VLab, etc. Essentially, no CSE.
I'll try to get Assignment 2 ready before then, so that you can download the spec and code onto your home machine if you want to work on it on Saturday. We won't be using PostgreSQL for this assignment; all you'll need is a C compiler to work on it.
I have meetings until 2pm and then have to do some admin work. If you've left your assignment to the last minute and now urgently require my help ... you'll need to wait until after 3pm. Send email to cs9315@cse and maybe Dylan will be able to help you.
Grieg is a powerful server, but not infinitely powerful. 500 or more PostgreSQL servers might render it unusable. Please run only one PostgreSQL server and one psql process at a time.
I have seen one person running 30 psql' s. I suspect that this is because they used Control-Z to "end" their psql process. What Control-Z actually does is suspend the process, but leaves it sitting there. If you start another psql you now have two psql 's, etc. etc.
Similarly, you should shut down you PostgreSQL server whenever you finish a session working on your database assignment. Leaving it running after you've logged off Grieg does not help to keep the load on Grieg down.
https://unsw.zoom.us/j/84704579750 ... 2pm - 4pm ... nd-hashing, nd-trees
The assignment 1 test script will now run tests on the following:
input (correct intset input)
error (incorrect intset input)
There is also now a time limit of 1 second per query. This should be plenty of time. marking will use a slightly longer time limit.
If you are running into timeouts while debugging you can use:
$ ./run_tests.py --timeout=0
to disable this.
https://unsw.zoom.us/j/83132909905 ... 2pm - 4pm .. Slides are available
If you get this error during Assignment 1, it's most likely because your code caused a memory error (e.g. reference through a NULL/invalid pointer).
To fix this, you'll first need to work out where the problem is occurring. To work that out, I'd suggest adding some diagnostic printf 's into the code to show the value of whatever pointers you're using at various stages during the execution of whatever function was running when the error occurred (often intset_in() or intset_out()).
Quick reminder: this weekend is Census weekend. If you remain in a course, that you really want to drop, after this weekend, you end up paying for it. After that, the only way to get your money back is via the tedious process of Fee Remission, and that's not guaranteed to succeed.
So, if you're serious about dropping (this course or others), do it now. If dropping COMP9315, send me an email telling me that you dropped.
You officially drop the course via MyUNSW. Telling me has no official status. It's just for my records.
I forgot to set late submissions for Quiz 2, and people who tried to submit after the 9pm deadline could not submit. Even more unfortunate because the Quiz 2 deadline was 9pm rather than midnight like Quiz 1, and people were assuming that the deadline would remain at 11:59pm Friday.
So, to stop us having to deal with email submissions of Quiz answers, we've extended the Quiz 2 deadline to Sun 14 March at 9pm . Submit before then if your Friday submission failed (i.e. was after 9pm).
We will be using 9pm deadlines for Quizzes from now on because of a request that we do not have deadlines outside "normal" class hours (up to 9pm Mon-Fri).
https://unsw.zoom.us/j/85302782302 ... 2pm - 4pm
Testing resources for assignment 1 are now available here .
There are currently tests for input, the ? operator, and the # operator, with tets for other operators to follow later this week.
Please report any problems or bugs on the forums.
https://unsw.zoom.us/j/87515489621 ... 2pm - 4pm
If you're trying to install your PostgreSQL server on Grieg and are getting "Disk quota exceeded" messages, please let me know (via email to firstname.lastname@example.org). I can't fix this myself, so I need to request the Systems Support people to do some magic.
If you have put other stuff under /srvr/ YourZID /, then you might consider cleaning that up first. You can check how much stuff you have under your /srvr/ YourZID / directory via the commands
grieg % cd /srvr/YourZID/ grieg % du -sh *
When I do this, with a fully-compiled PostgreSQL server and one small database, I see, among other things
4.0K env 74M pgsql 224M postgresql-12.5
If you see something similar, but are still getting "Disk quota exceeded", let me know via email.
Great! Zoom put the (useless) band of users right across the middle of the screen in the video. This is *not* what it looked like while I was recording.
This is impossible to edit out, and I'm not going to re-make 2 hours of video. I am annoyed.
One more annoying thing like this, and I'm finished with Zoom.
Maybe I'll try Google Meet next week. Is there anyone who can't use Google Meet?
I've tried a suggestion from one of you for a recurrent meeting ...
Let's see if this link works next week ...
The way I've been structuring Videos and Slides pages, whenever you ask a query in the Comments, the heading in the email is just something like "Files, Buffers, Pages" and there are 5 or 6 slide sets under that heading. Sometimes it's not clear which slide you're asking about and it takes a while to track it down.
When you post a question on a specific slide, please make sure that your post contains enough info to unambiguously identify where your question is coming from.
E.g. "One slide 5 in PostgreSQL Buffer Manager, why is ..."
Still haven't worked out recurrent events in Zoom ...
There has been a small update to the Operations on intSets section of the Assignment 1 Spec.
For the exact details please read the
Assignment 1 - Adding a Set Data Type to PostgreSQL
Is now available for your viewing pleasure.
The due date is Friday 19 March, 11:59pm (the end of week05).
Testing resources will be available soon.
The spec might have minor updates and clarifications added but the task will not change in any major way.
A quick reminder for the 150 people who haven't yet done Quiz 1. It's due before midnight tonight.
Accessible via https://unsw.zoom.us/j/83562198595
Sometime, I'll work out how to set up recurring events, so we use the same URL week after week.
If you're looking for the slides, code and videos for the online sessions, in Webcms3 go to
Videos and Slides > Week X > Online Sessions
We'll use Zoom again ... https://unsw.zoom.us/j/88943978525
This time I'll (try to) mute everybody. Ask questions via the chat.
Ignore this message if you're still enrolled in COMP9315 ...
Everyone who sent me email saying they've dropped the course has been moved to "Ex Student" in Webcms3. I want to check, for anyone who's dropped the course and told me ...
Thanks for the feedback
It seems that the answer to the above questions is "Yes". This is a Webcms3 bug which I will fix sometime.
OK ... we'll try Zoom for today's (Thu 18 Feb) online session ...
John Shepherd is inviting you to a scheduled Zoom meeting.
Topic: COMP9315 Online Session
Time: Feb 18, 2021 02:00 PM Canberra, Melbourne, Sydney
Join from PC, Mac, Linux, iOS or Android: https://unsw.zoom.us/j/82195443853?pwd=YmJpMFl2VGc2ZG1QeEhtMThDSDAvQT09 Password: 883235
Webcms3 has its own database to record enrolments; this database is different from the official MyUNSW enrolment database.
Webcms3 tracks enrolments (via daily downloads from MyUNSW), but is conservative about removing students from its enrolment list. Students are initially moved to "unofficial", but don't lose access to course material unless flagged as having dropped the course by the course convenor.
If you receive this email and you are no longer enrolled in COMP9315, please let me know and I'll flag you as "dropped" in the Webcms3 database.
Much as we all love working from video, the definitive PostgreSQL installation instructions are in the Prac Exercise. If the video differs from the Prac, follow the Prac.
Also, if you're installing at home (on Mac or Linux), copy the env file and change the value of PG_HOME to reflect the top level directory where you installed PostgreSQL. This directory can be anywhere (e.g. on my laptop it's in /Users/jas/pgsql). And, of course, don't forget to source env before you start work with PostgreSQL.
Welcome to COMP9315 21T1. You and 400 other lucky, lucky students are about to be introduced to the wonders of the PostgreSQL database engine (and relational DBMS engines in general).
We're still COVID-affected** so no "lectures" in the traditional sense. All the material will be made available in bite-sized topic-based Videos and Course Notes. In the timeslots scheduled for lectures, I'll be running Online Sessions where you can ask questions and I'll solve problems (if no questions).
I'll be posting a welcome video soon, and will then get started on the topic-based videos.
** Reading this again, it sounds like I have COVID. I don't.