Sign In

PyTorch Basics: Tensors & Gradients

Part 1 of "Pytorch: Zero to GANs"


  1. PyTorch Basics: Tensors & Gradients
  2. Linear Regression & Gradient Descent
  3. Image Classfication using Logistic Regression
  4. Training Deep Neural Networks on a GPU
  5. Coming soon.. (CNNs, RNNs, GANs etc.)


System setup



Here's what you need to do to get started:



pip install jovian --upgrade
$ jovian clone <notebook_id>



$ ls 01-pytorch-basics
01-pytorch-basics.ipynb  environment.yml
$ cd 01-pytorch-basics
$ jovian install


  1. We can activate the virtual environment by running
$ conda activate 01-pytorch-basics


$ jupyter notebook

jupyter notebook


We begin by importing PyTorch:

In [1]:
import torch



In [2]:
# Number
t1 = torch.tensor(4.)


In [3]:

Let's try creating slightly more complex tensors:

In [4]:
# Vector
In [5]:
# Matrix
tensor([[ 5.,  6.],
        [ 7.,  8.],
        [ 9., 10.]])
In [6]:
# 3-dimensional array
t4 = torch.tensor([
    [[11, 12, 13], 
     [13, 14, 15]], 
    [[15, 16, 17], 
     [17, 18, 19.]]])
         [13., 14., 15.]],

        [[15., 16., 17.],


In [7]:
In [8]:
In [9]:
torch.Size([3, 2])
In [10]:
torch.Size([2, 2, 3])

Tensor operations and gradients


In [11]:
# Create tensors.
x = torch.tensor(3.)
w = torch.tensor(4., requires_grad=True)
b = torch.tensor(5., requires_grad=True)


Let's create a new tensor y by combining these tensors:

In [12]:
# Arithmetic operations
y = w * x + b
tensor(17., grad_fn=<AddBackward0>)


In [13]:
# Compute derivatives


In [14]:
# Display gradients
print('dy/dx:', x.grad)
print('dy/dw:', w.grad)
print('dy/db:', b.grad)



Interoperability with Numpy


  • Matplotlib for plotting and visualization
  • OpenCV for image and video processing
  • Pandas for file I/O and data analysis


Here's how we create an array in Numpy:

In [15]:
import numpy as np

x = np.array([[1, 2], [3, 4.]])
array([[1., 2.],
       [3., 4.]])

We can convert a Numpy array to a PyTorch tensor using torch.from_numpy.

In [16]:
# Convert the numpy array to a torch tensor.
y = torch.tensor(x)
tensor([[1., 2.],
        [3., 4.]], dtype=torch.float64)


In [17]:
x.dtype, y.dtype
(dtype('float64'), torch.float64)


In [18]:
# Convert a torch tensor to a numpy array
z = y.numpy()
array([[1., 2.],
       [3., 4.]])


Commit and upload the notebook


In [1]:
pip install jovian --upgrade

Collecting jovian Downloading (42kB) 100% |████████████████████████████████| 51kB 1.1MB/s ta 0:00:011 Collecting pyyaml (from jovian) Downloading (265kB) 100% |████████████████████████████████| 266kB 14.4MB/s a 0:00:011 Requirement already satisfied, skipping upgrade: uuid in /srv/conda/envs/notebook/lib/python3.7/site-packages (from jovian) (1.30) Requirement already satisfied, skipping upgrade: requests in /srv/conda/envs/notebook/lib/python3.7/site-packages (from jovian) (2.21.0) Requirement already satisfied, skipping upgrade: idna<2.9,>=2.5 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from requests->jovian) (2.8) Requirement already satisfied, skipping upgrade: chardet<3.1.0,>=3.0.2 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from requests->jovian) (3.0.4) Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from requests->jovian) (2019.3.9) Requirement already satisfied, skipping upgrade: urllib3<1.25,>=1.21.1 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from requests->jovian) (1.24.1) Building wheels for collected packages: pyyaml Building wheel for pyyaml ( ... done Stored in directory: /home/jovyan/.cache/pip/wheels/d9/45/dd/65f0b38450c47cf7e5312883deb97d065e030c5cca0a365030 Successfully built pyyaml Installing collected packages: pyyaml, jovian Found existing installation: jovian 0.1.53 Uninstalling jovian-0.1.53: Successfully uninstalled jovian-0.1.53 Successfully installed jovian-0.1.89 pyyaml-5.1.2 Note: you may need to restart the kernel to use updated packages.
In [1]:
import jovian
In [ ]:
[jovian] Saving notebook..


commenting on jovian

Further Reading










  1. PyTorch Tutorial for Deep Learning Researchers by Yunjey Choi:

  2. FastAI development notebooks by Jeremy Howard:

In [ ]:
In [ ]: