COMP9444 Neural Networks and Deep Learning
Term 3, 2020

Exercises 6: Word Vectors

1. Consider the sentence

   "two flowers grew tall on two tall towers"

   1. Write the co-occurrence matrix X for this sentence, using a 4-word context window (i.e. two context words on either side of the central word)
   2. Use torch.svd() to compute the singular value decompositon of this matrix X = USV^T
   3. Extract a word representation from the first two columns of U and use matplotlib to plot the words on a 2-dimensional graph.