Sharing Resources

There are many resource types that can be shared, file system resources, storage resources, relational databases, compute clusters, and running jobs. We will keep our focus here on sharing data, specifically files and directories. For all of the below we will assume the GFFS client is already installed and that we have linked the GFFS into our Unix home directory at $HOME/XSEDE. We will further assume that our user Sarah has a directory in the GFFS at /home/Sarah. Given where the GFFS is mounted, the Unix path to that directory is $HOME/XSEE/home/Sarah. We will also assume below unless otherwise noted that our current working directory is $HOME/XSEE/home/Sarah.

Before we get to sharing local data resources, lets’ look first how to create a file or directory “somewhere in the GFFS”. To create a file or directory in the GFFS is simple. For example,

     mkdir test
     echo “This is a test” >> test/newfile

creates a new file in the newly created “test” directory. Once created both the directory and the file are available throughout the GFFS subject to access control.

However, where is the data actually stored? The short answer is that the “test” directory will be created in the same place where the current working directory is located. Similarly, “newfile” will be placed in the same location as the “test” directory.

So, where is that? A bit of background here is useful. The GFFS uses a standards-based Web Services model. Most Web Services (including those that implement the GFFS) execute inside of a program called a Web Services container. A container is a program that accepts Web Services connections (in our case https connections), parses the request, and calls the appropriate function to handle the request. Web Services containers are often written in Java, and execute on Windows, Linux, or MacOS machines like any other application. The difference is that they listen for http/https connections and respond to them.

In the GFFS, files and directories are stored in different GFFS Web Service containers (just “containers” from here on.) There are GFFS containers at the NSF service providers, and there are containers wherever someone wants to share a resource. Therefore, the first step to sharing a resource is to install the GFFS container. The installation process is very similar to installing the client if one chooses to use only the default options. It can be more complicated if, for example, your resource is behind a NAT or firewall. The GFFS container requires no special permissions or privilege, though it is recommend that it be started as its own user with minimum privilege.

To get back to our earlier question, “where is the data actually stored?” The new directory and the new file will be placed on the same container as Sarah’s GFFS home directory. In a moment, we will see how to over-ride this. For now, however, we will assume that is where they will be placed.

Given that the new items will be placed on the same GFFS container as Sarah’s home directory, exactly where will they be placed? There are two possibilities1. Either Sarah’s home directory is being stored a container in the containers own databases and storage space (in other words, the container is acting as a storage service), or her home directory is an export, in other words it is being stored in a local file system somewhere. Below we briefly examine each of these options. Keep in mind that either way a GFFS container is providing access to the data.

Return to GFFS page


1 There are as many possibilities as there are different implementations of the RNS and ByteIO specifications. We are using the most typical implementations in the GFFS as of this writing.