IU-BigRed2-Mason

IU BigRed2

On BigRed2, we are running GenesisII BES service which interacts with BigRed2 compute resources to run grid users jobs. Users can run either batch jobs or MPI jobs on BigRed2.

To submit bacth jobs on BigRed2, use this path in grid name space

/resources/xsede.org/bigred2.iu.xsede.org/bes-containers/pbs-sudo-bigred2

To submit MPI jobs (openMPI) on BigRed2, use this path in grid name space

/resources/xsede.org/bigred2.iu.xsede.org/bes-containers/pbs-sudo-mpi-bigred2
Make sure your MPI code was compiled on Bigred2.

IU Mason

On BigRed2, we are running Unicore6 BES service which interacts with Mason compute resources to run grid users jobs. Users can run batch jobs Mason.

To submit bacth jobs on Mason, use this path in grid name space

/resources/xsede.org/mason.iu.xsede.org/bes-containers/mason-u6-bes

Job submission via BES

To submit jobs to BES resource

  1. User should have installed client on your desktop/laptop.
  2. Login to grid with your xsede credentials and create job JSDL file. For more information on how to create JSDL file, click here
  3. Use grid run command to run your job
ex. grid run /resources/xsede.org/bigred2.iu.xsede.org/bes-containers/pbs-sudo-bigred2 --jsdl=local:<path-to-jsdl-file>

Job Submission via Grid Queue

You can also submit jobs to grid queue, submitting to queues is recommended since it allows better resource allocation and job handling.

We recommend using Campus bridging queue located /resources/xsede.org/queues/CBQueue in the grid name space.

For more information on how to submit jobs to queue, click here


GenesisII client on Mason

GFFS at XSEDE is currently available on the Mason cluster and can be loaded using the standard Modules environment manager commands:

login1$ module load genesis-ii

Use your XSEDE User Portal (XUP) username/password to authenticate with the GFFS GIU servers and the XSEDE grid. Here's a command line example on the Mason cluster:

login1$ grid grid:\$> xsedeLogin --username=<XUP-username> Password for : Replacing client tool identity with credentials for "CN=......, O=National Center for Supercomputing Applications, C=US". grid:\$>


Data sharing on IU Mason

To share your Mason data with XSEDE grid users

1. Send email to bret@iu.edu asking to give rwx (read, write and execute) permissions to xsede-user (your xsede portal-id) on Mason GenesisII LighWeightExportPortType. Due to center policies, Bret Hammond at IU maintains the GFFS container and only he can give permissions to xsede-user to create exports on that container.

2. Run set_acls.sh script (located in ~grimshaw/set_acls.sh) allowing the user that runs GenesisII container access to your data

step1. bash /N/u/xd-agrimshaw/set_acls.sh gffs-user /N/u/xd-agrimshaw/Mason rx
The above command gives user 'gffs-user' permissions to read user 'grimshaw''s home directory on Mason. gffs-user is the user running GenesisII GFFS container on Mason. When user shares data, it it important to give atleast read access to all the directories upto (and including) the directory to be shared.
step2. bash /N/u/xd-agrimshaw/set_acls.sh -R gffs-user /home1/01264/grimshaw/shared-dir rwx
The above command give user 'gffs-user' permissions to read/write/execute recursively on user 'grimshaw''s shared-dir on Mason

3. Install xsede grid client on your desktop/laptop. If you have already installed the client, skip to Step3.

4. On your grid client, login to xsede grid using your xsede credentials

ex. grid xsedeLogin --username=<xsede-portal-id>

5. Run grid export create command to share your Mason data on grid

ex. grid export --create /resources/xsede.org/maosn.iu.xsede.org/containers/mason-gffs local:/N/u/xd-agrimshaw/Mason/shared-dir /home/xsede.org/grimshaw/mason-shared-dir
This command creates a export in grid namespace, /home/xsede.org/grimshaw/mason-shared-dir.

6. To share your export with other xsede users, give appropriate permissions to access the shared directory

ex. grid chmod /home/xsede.org/grimshaw/mason-shared-dir +rwx /users/xsede.org/vana
Here, xsede grid user 'grimshaw' shared /home/xsede.org/grimshaw/mason-shared-dir with xsede grid user 'vana'. Now xsede user 'vana' will be able to directly read/write/execute files in user 'grimshaw''s Mason home directory /N/u/xd-agrimshaw/Mason/shared-dir.