Contents Menu Expand Light mode Dark mode Auto light/dark mode
HPC Helper 0.0.1 documentation
HPC Helper 0.0.1 documentation

SSH basics

  • Automatic SSH

Remote VS Code

  • VS Code

Submit Interactive Jobs

  • Use GPU on interactive session
  • List of partitions
  • Start an interactive session
  • Loading Anaconda
  • Run a python file
  • Two GPUs

Submit Batch Jobs

  • Job submission with GPU: Running batch jobs
  • Run bash file

MIG partition Issues

  • Interactive GPU best practices
  • See only allocated GPUs in nvidia-smi
  • Problem with MIG partition

Jupyter Lab port forward on local network

  • Remote Jupyter Lab

Jupyter Lab port forward from the Login node of Sunbird

  • Jupyter lab on Sunbird using port forwarding
  • Installing Jupyter lab
  • Tinkering configuration file
  • Port forwarding
  • Running Jupyter lab on login node
  • Wrapping ssh command in Bash file

Jupyter Lab port forward from a compute node of Sunbird

  • Jupyter lab server on a GPU node
  • Job Script
  • Node number
  • What if someone is already using port 8888 on Sunbird or your local PC?
  • Running Jupyter lab on local PC usign ssh tunnel
  • Some screenshots

Remote VNC session

  • Running GUIs on Sunbird via VNC server
  • Quick VNC server on sunbird

Swansea University VPN Tunnel

  • Setting up the VPN
  • Setting up the VPN software
  • Setting up the host (Ubuntu only)
  • Setting up for SSH
  • Setting up the RDP server
  • Enable connection after logout from the host
  • Getting the Local IP
  • Using SSH
  • SSH
  • Adding an alias for the HOST PC
  • Passwordless login
  • RDP connection

Basic commands

  • Docker containers
  • What is a Container? What is Docker?
  • Virtualisation
  • Potential applications
  • Docker command line
  • Install using the repository
  • Copy paste
  • Docker is working correctly ?
  • Getting help
  • Docker Command Line Interface (CLI) syntax
  • Running containers interactively
  • Installing Ubuntu
  • Summary
  • Cleaning up containers
  • Finding Containers on Docker Hub
  • Exploring Container image versions
  • Docker Hub is a repository
  • Create a new Docker image
  • Share your new container image on Docker hub

Advanced commands

  • Creating more complex container image
  • Using scripts and files from outside the container
  • Including your scripts and data within a container image
  • If the files are on a GITHUB repo
  • More fancy Dockerfile options
  • Add the sum.py script to the PATH so you can run it directly

Port Mapping of Jupyter server

  • Jupyter lab inside Docker container
  • Run a Docker image as container interactively
  • Running the Jupyter-lab server
  • Opening Jupyter-lab in browser
  • Some screenshots

Singularity Basics

  • Singularity: software carpentry
  • Images and containers
  • Pulling image
  • Running an image
  • Singularity’s image cache
  • Running specific command with a Singularity container
  • Singularity container: interactive shell

Volume Binding

  • Files and directories within a Singularity container
  • Binding additional host system directories to the container

Basic Builds

  • Using Docker images with Singularity
  • Running pulled image

NVIDIA Modulus Singularity container (Failed)

  • Nvidia Modulus 22.03.1 Docker images on Sunbird

Interactive GPU session

  • Setup Nvidia Modulus v22.03 on Sunbird using interactive GPU session
  • Installation
  • Running an interactive GPU session

Multi-GPU

  • Parallel Processing using Multi-GPU Configurations

With PySDF

  • Nvidia Modulus with tesselated geometry module on Sunbird
  • Installing Python 3.8
  • Installing Pytorch
  • Installing Modulus dependencies
  • Installing Modulus
  • Trying an example
  • Running Nvidia Modulus Docker image
  • Final testing

v22.03 Summary

  • Efficiently train model

Constructing the Apptainer script

  • Getting the base image ready to use
  • Getting the Modulus source code
  • The base image
  • Getting the base image ready
  • Converting the recipe
  • Build the image
  • Testing the image

Overiding Defaults of Apptainer

  • Problems with Apptainer
  • Running the Modulus Apptainer image
  • Using the GPU
  • Downloading Modulus examples
  • Testing the Modulus Apptainer image
  • Apptainer supports guest pull from NVIDIA NGC

Multi-GPU trainign using mpirun

  • Running jobs using multiple GPUs
  • Problem with mpirun (SKIP IF NEEDED)
  • Use mpirun
  • Best Practice

The Apptainer script

  • Building an Apptainer image of Modulus v22.09
  • Getting the examples using the SSH key
  • Port forwarding the Jupyter-lab
  • Checking if the port is already occupied
  • A robust way to execute jobs
Back to top
Copyright © 2022, Prakhar Sharma
Made with Sphinx and @pradyunsg's Furo