PyTorch¶
- supportlevel
A
- pagelastupdated
2020-05-15
- maintainer
PyTorch is a commonly used Python package for deep learning.
Basic usage¶
First, check the tutorials up to and including GPU computing.
If you plan on using NVIDIA’s containers to run your model, please check the page about NVIDIA’s singularity containers.
The basic way to use is via the Python in the anaconda
module.
If you’re not using Tensorflow as well, you can pick either -tf1
-
or -tf2
-version. If you’re using Tensorflow as well, please check our
Tensorflow page.
Don’t load any additional CUDA modules, anaconda
includes everything.
If you use GPUs, you need --constraint='kepler|pascal|volta'
in
order to select a GPU new enough to run tensorflow. (Note that as we
get newer cards, this will need further updating).
Simple PyTorch model¶
Let’s run the MNIST example from PyTorch’s tutorials:
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
self.conv1 = nn.Conv2d(1, 20, 5, 1)
self.conv2 = nn.Conv2d(20, 50, 5, 1)
self.fc1 = nn.Linear(4*4*50, 500)
self.fc2 = nn.Linear(500, 10)
def forward(self, x):
x = F.relu(self.conv1(x))
x = F.max_pool2d(x, 2, 2)
x = F.relu(self.conv2(x))
x = F.max_pool2d(x, 2, 2)
x = x.view(-1, 4*4*50)
x = F.relu(self.fc1(x))
x = self.fc2(x)
return F.log_softmax(x, dim=1)
The full code for the example is in
tensorflow_mnist.py
.
One can run this example with srun
:
wget https://raw.githubusercontent.com/AaltoSciComp/scicomp-docs/master/triton/examples/pytorch/pytorch_mnist.py
module load anaconda
srun --time=00:15:00 --gres=gpu:1 python pytorch_mnist.py
or with sbatch
by submitting
pytorch_mnist.sh
:
#!/bin/bash
#SBATCH --gres=gpu:1
#SBATCH --time=00:15:00
module load anaconda
python pytorch_mnist.py
The Python-script will download the MNIST dataset to data
folder.
Running simple PyTorch model with NVIDIA’s containers¶
Let’s run the MNIST example from PyTorch’s tutorials:
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
self.conv1 = nn.Conv2d(1, 20, 5, 1)
self.conv2 = nn.Conv2d(20, 50, 5, 1)
self.fc1 = nn.Linear(4*4*50, 500)
self.fc2 = nn.Linear(500, 10)
def forward(self, x):
x = F.relu(self.conv1(x))
x = F.max_pool2d(x, 2, 2)
x = F.relu(self.conv2(x))
x = F.max_pool2d(x, 2, 2)
x = x.view(-1, 4*4*50)
x = F.relu(self.fc1(x))
x = self.fc2(x)
return F.log_softmax(x, dim=1)
The full code for the example is in
pytorch_mnist.py
.
One can run this example with srun
:
wget https://raw.githubusercontent.com/AaltoSciComp/scicomp-docs/master/triton/examples/pytorch/pytorch_mnist.py
module load nvidia-pytorch/20.02-py3
srun --time=00:15:00 --gres=gpu:1 singularity_wrapper exec python pytorch_mnist.py
or with sbatch
by submitting
pytorch_singularity_mnist.sh
:
#!/bin/bash
#SBATCH --gres=gpu:1
#SBATCH --time=00:15:00
module load nvidia-pytorch/20.02-py3
singularity_wrapper exec python pytorch_mnist.py
The Python-script will download the MNIST dataset to data
folder.
Common problems¶
Random CUDA errors: don’t load any other CUDA modules, only
anaconda
. Anaconda includes the necessary libraries in compatible versions.