Getting Started on the Cluster

Faculty and their research students can request access to the cluster by contacting Mike Conner [connerms], Linux System Administrator, Information Technology Services. Access is granted through the Login, or Master node. The Master node can be used to manage files, prepare jobs, and submit jobs to a scheduler. Jobs are submitted to the scheduler along with a request for resources to run the jobs. Jobs are placed in a queue and are run when the resources become available. Policies configured in the scheduler also try to balance priorities, reduce time waiting in the queue, and maximize the use of the resources.

To prepare jobs for the scheduler, users write a shell script that sets all necessary variables and contains all commands to be run. This file is submitted to the scheduler along with the request for resources.

Moab Scheduler

Compute jobs on the cluster are managed by a scheduler: Moab. To run jobs on the cluster you'll need to prepare your jobs then submit them to the scheduler along with some information about the resources that are needed to run the job.

Jobs can be sumitted to the scheduler using the msub command. msub can be used with arguments to set parameters directly, or can be used with a script that sets parameters and defines the job to be executed.

These are common parameters passed to Moab when submitting a job: - Initial Working Directory: (-d) The directory the job should execute in - Resource list: (-l) Defines the resources needed by the job. Often defined resources are: - nodes and processors per node (nodes=N:ppn=N) - physical memory per task (pmem=N) - walltime (walltime=NN:NN:NN) - Output Path: (-o) The path for the output file of the job - Error Path: (-e) The path for the error file of the job - Name: (-N) A name for the job

A complete list of parameters that Moab accepts is available in the Moab documentation.

As an example, consider this command. It will print the date, wait 5 seconds, then print the date again.

$ date; sleep 5; date;

To execute this job on the cluster, we would turn this command in to a very simple script:

File date.sub:

#!/bin/bash
date
sleep 5
date

Once the script is created it can submitted to the scheduler. But we also need to tell the scheduler what resources are needed and how long the job will likely take. This information can be passed to Moab directly in the msub command or it can be placed in the script itself.

The job can be submitted to the scheduler adding flags to the msub command to request one node, one core, and specify a duration (i.e. walltime) of 6 seconds.

$ msub -l nodes=1:ppn=1,walltime=00:06 date.sub

To place this request in the script itself, we modify the script to include an additional line with the resources request:

File date.sub:

#!/bin/bash
#PBS -l nodes=1:ppn=1,walltime=02:00

date
sleep 5
date

Then the job is submitted with msub:

$ msub date.sub

The msub command will return a number for your job, the JOBID. Now that the job is in the queue, you can check the status of the job using the showq command or the checkjob command:

$ showq

-or-

$ checkjob JOBID

Refer to the Moab Documentation for complete list of paramaters that can be fed to the scheduler.