COMP9444 Neural Networks and Deep Learning
Term 2, 2020

Exercises 1: Perceptrons

1. Perceptron Learning
   1. Construct by hand a Perceptron which correctly classifies the following data; use your knowledge of plane geometry to choose appropriate values for the weights w₀, w₁ and w₂.

      Training Example	x1	x2	Class
      a.	0	1	-1
      b.	2	0	-1
      c.	1	1	+1

   2. Demonstrate the Perceptron Learning Algorithm on the above data, using a learning rate of 1.0 and initial weight values of

      w₀ = -1.5
      w₁ =   0
      w₂ =   2

      In your answer, you should clearly indicate the new weight values at the end of each training step. Here, the first three steps have been done for you. You need to fill in the rest of the table.

   Iteration	w0	w1	w2	Training Example	x1	x2	Class	s=w0+w1x1+w2x2	Action
   1	-1.5	0	2	a.	0	1	-	+0.5	Subtract
   2	-2.5	0	1	b.	2	0	-	-2.5	None
   3	-2.5	0	1	c.	1	1	+	-1.5	Add

2. XOR Network

   Construct by hand a Neural Network (or Multi-Layer Perceptron) that computes the XOR function of two inputs. Make sure the connections, weights and biases of your network are clearly visible.

   Challenge: Can you construct a Neural Network to compute XOR which has only one hidden unit, but also includes shortcut connections from the two inputs directly to the (one) output.
   Hint: start with a network that computes the inclusive OR, and then try to think of how it could be modified.

3. Computing any Logical Function with a 2-layer Network

   Assuming False=0 and True=1, explain how each of the following could be constructed:

   1. Perceptron to compute the OR function of m inputs
   2. Perceptron to compute the AND function of n inputs
   3. 2-layer Neural Network to compute the function (A ∨ B) ∧ (¬ B ∨ C ∨ ¬ D) ∧ (D ∨ ¬ E)
   4. 2-Layer Neural Network to compute any (given) logical expression, assuming it is written in Conjunctive Normal Form.

   Hint: in each case, first decide on the input-to-output or input-to-hidden weights, then determine the bias.

   ---