In this short practical, we recall the background on PyTorch needed for this course.

Matrices with PyTorch

  1. Import torch and print the current version. Make sure that your version of PyTorch is at least 2.1.2.

  2. Data in PyTorch are called tensors and can be of different dimensions. They can be scalars (0-dimension) to $n$-dimensions matrices. Create a few tensors of different dimensions.

  3. Create the following Numpy array and create the corresponding tensor using either torch.tensor or torch.from_numpy. What is the difference?

    np.array([[[1, 2], [3, 4]], [[5, 6], [7, 8]]])

  4. Create a 2x3 matrix. Change its change to 3x2 using reshape or view. What is the difference?

  5. Multiply the 2x3 matrix (of question 4) with its transposed. What is the size of the resulting matrix?

Dealing with gradients

  1. Create 4 vector tensors of size 1 called y, x_1, w_1, and b with values 1.0, 1.1, 2.2, and 0.0 respectively.
  2. Create the tensor z equal to $x_1 * w_1 + b$.
  3. Create the tensor a equal to the activation, i.e. $a = \sigma(z)$, where $\sigma$ is the sigmoid function.
  4. Import torch.nn.functional and use binary_cross_entropy to calculate the loss (between a and y). Explain how the value was obtained.
  5. Now, use requires_grad to declare w_1 and b as trainable variables.
  6. Import grad from torch.autograd. Use it to calculate the gradient of w_1 and b with respect to the loss. What is the use of the parameter retain_graph?
  7. Print w_1.grad and b.grad, what is the result?
  8. Use backward() on the loss to compute the gradient. Repeat step 7.