HPC OnDemand User Guide

HPC OnDemand is a way for users to run interactive jobs on the Charlotte HPC Research Cluster. Start computing immediately. A simple interface makes HPC OnDemand easy to learn and use. This includes:

• Easy file management
• Slurm job management and monitoring
• Launch Desktop applications such as RStudio, MATLAB, and Jupyter Lab/Notebook
• Work in a Graphical Linux Desktop
• Additionally, you can use the command-line by launching an SSH session onto one of our submit hosts

HPC OnDemand is based on an open-source web portal called Open OnDemand, developed by researchers at the Ohio Supercomputer Center and supported by contributors around the world. Want to learn more about the Open OnDemand Project?

• Visit their website: https://www.openondemand.org/
• Find them on GitHub: https://github.com/OSC/ondemand
• See this research talk from May 2024: https://www.youtube.com/watch?v=hMkdkxroGoA

Connecting to HPC OnDemand

If you haven’t gotten a Research Computing account yet, please visit our “Get Started” page for more information. Once you’ve established your account, navigate to:

• https://ondemand.charlotte.edu

Enter your Ninernet username and password, and go through the DUO 2FA prompts. If successful, you should now be at the HPC OnDemand Dashboard:

Navigating around the HPC OnDemand Portal

As you can see from the dashboard above, you are greeted with the Message of the Day, along with several shortcuts to popular applications:

• Starlight Shell Access: An SSH session onto one of our login nodes / submit hosts
• Jupyter: Launch a Jupyter Lab job onto the cluster
• R Studio: Launch an R Studio job onto the cluster
• Visual Studio Code: Launch a VSCode job onto the cluster

There is also a menu across the top of the Dashboard, for quick access to:

• Files: Shortcuts to your home and scratch directories, as well as links to /projects and /vast/temp
• Jobs: View the “Active Jobs” on the cluster, as well as access the “Job Composer”
• Clusters: Another shortcut to shell access, as well as links to get overall system status, or specific partition status
• Interactive Apps: A list of available GUI applications that you can launch on the cluster. (*the list of applications varies on user permissions)

File Management

Quickly access and manage your files and directories. Under the “Files” menu, you have 2 menu shortcuts directly to your 2 personal directories: Home and Scratch. There are 2 additional menu shortcuts to the root of Projects and the root of VAST Temp. From there, you should be able to navigate to any of the Project or VAST Temp volumes that you have been granted access to. The file management interface looks like your typical web-based file manager, with options to upload or download files, create new files or directories, copy/move/delete existing files, and even open a terminal for quick and convenient command line access:

While using the HPC OnDemand Portal is quick and convenient for file management, it is best suited for smaller uploads and downloads (i.e., less than 2GB in size). For the best experience, please keep the following things in mind:

• Downloading files: When downloading files from your cluster storage, the portal interface will be inaccessible for the duration of your download. So for very large downloads, I would encourage you to still use one of the other data transfer methods: FileZilla, WinSCP, rsync, Globus, etc.
• Uploading files: When uploading files, the browser tab that you are doing the upload in must be left untouched, or else your upload may be interrupted. However, you can continue to work in the portal while an upload is occurring, so long as you do so in another browser tab. I recommend that you open another browser tab to the OnDemand Dashboard prior to starting your upload, then switching back to it to continue working after you’ve initiated the upload.

Monitoring System, Partition, and Job Statuses

HPC OnDemand gives you several ways to check the status of the cluster. Here are a few of those ways:

• Job Status: ( Menu – Jobs -> Active Jobs )
  
  This gives you a listing of not only “active” jobs, but all jobs on the cluster (active, pending, recently completed, etc). It is like a graphical version of “squeue”. You can choose between “Your Jobs” and “All Jobs” by changing the pulldown in the upper right-hand corner. Underneath that you can use the filter field to narrow down the job listings. Useful filters, for example, are: queue (partition) name, job status, job name, etc.
• Cluster Status: ( Menu – Clusters -> System Status )
  
  To see an overall status of cluster resources, choose this option. This gives you information like the availability of nodes, processors, and GPUs, and also shows how many jobs are running and queued. Please keep in mind that these numbers include ALL resources on the cluster, including compute nodes in partitions that you might not have access to.
• Partition Status: ( Menu – Clusters -> Partition Status )
  
  If you are only interested in seeing the status of a specific partition, choose this option. You will only be presented with a drop-down list of partitions that you have permissions to submit to; choose one that you are interested in. If you choose a GPU partition, it will also give you GPUs available.

Job Composer

The Job Composer is helpful tool for new users to learn about batch scheduling compute jobs on an HPC cluster. You can find the Job Composer in the “Jobs” menu; it will launch in a new browser tab when chosen.

You can quickly access the growing list of example job templates we have by choosing the “Templates” menu at the top of the page. Select one of the templates you would like to submit to the cluster, then click the “Create New Job” button on the right-hand side of the page:

Whichever template you chose will now show up in your list of available jobs that you can submit to the cluster. You can scroll down and look at the contents of the submit script, and you can even edit it by clicking the “Open Editor” button at the bottom of the page (the editor will open in a new browser tab). If you made changes, make sure you save them and close the editor browser tab to return to your job list. When you’re satisfied with your job, make sure the job you want to submit is highlighted in your list of jobs, then click the green submit button:

Your job should be queued and will eventually run! If you chose one of our templates, the jobs run very fast once they get scheduled. You may have to refresh the browser tab to see the latest status of the job. Once it is marked as “Completed”, you can look at the results (scroll down and look at the right-hand column for “Folder Contents” in the “Job Details” box.

Interactive Apps

Coming Soon!