• Project 2 FAQs

    Posted by Xin Cao Saturday 16 September 2017, 12:14:35 AM.

    Dear All,

    1. You will NOT lose marks due to the precision problem. In the project specification, "double precision" means that you should use "double" to store the distances.

    2. Please delete the intermediate folders and files generated during the iterations. Someone may meet the "Wrong FS" problem. You can try the following method:

    Configuration conf = new Configuration();

    FileSystem fs = FileSystem.get(new URI("hdfs://localhost:9000"),conf);

    Then, you can use fs to delete the folders.

    3. "\t" display problem. The same character "\t" may be displayed differently in your output file. That is caused by your text editor. Your data format should be correct, if you only use "\t" as the separator.

    Remember that the submission deadline is this Sunday.



  • Today's lecture is canceled

    Posted by Xin Cao Monday 11 September 2017, 08:45:38 AM.

    Dear All,

    I am very sorry for the late notice.

    I am not able to talk due to the severe cough, and thus I have to cancel today's lecture. I thought I would get better today, and I didn't expect that my cough could last for so long.

    Chapter 8 - streaming data mining will be introduced next week. This week's lab is not affected. Please try to attend because it is very relevant to your third project.

    My apology again. Thanks for your understanding.

    Kind regards,


  • Project 2 Submission Deadline Extended

    Posted by Xin Cao Wednesday 06 September 2017, 05:52:03 PM.

    Hi All,

    The submission deadline of project 2 is extended to 09:59:59 pm on 17 Sep 2017. You have one more week to work on it.

    More sample input and output will be provided later this week.

    Consequently, the third project will be released on next Friday.

    Kind regards,


  • Project 2 Released

    Posted by Xin Cao Saturday 26 August 2017, 07:32:09 PM.

    Dear All,

    Project 2 is released now. You have two weeks to do this project.

    The solutions to the problems in Lab3 are published as well. Please feel free to contact me if you have any questions.



  • Virtual Machine Image Download

    Posted by Xin Cao Wednesday 26 July 2017, 09:33:06 PM.

    Dear All,

    The image can be downloaded at:

    1. Download and Install VirtualBox

    2. Download the zip file and uncompress it, and rename the file "xubuntu-disk.vmdk" as "xubuntu-disk2.vmdk"

    3. Open VirtualBox, File->Import Applicance

    4. Browse the image folder, select the "*.ovf" file

    5. The image will be imported to your computer, which may take 10 minutes

    comp9313 is used as both username and password. The hadoop installation path is the same as in the virtual machine on lab computers.

    Hadoop MapReduce and Eclipse+plugin have been installed and configured.

    The video recording of the first lecture is still not available now. I've contacted the IT service center. Hopefully there is no problem...


Back to top

COMP9313 17s2 (Big Data Management) is powered by WebCMS3
CRICOS Provider No. 00098G