Sharing Resources

Main.SharingResources History

Hide minor edits - Show changes to output

November 11, 2011, at 04:09 PM by 128.143.137.203 -
Changed line 2 from:
As discussed above there are many resource types that can be shared, file system resources, storage resources, relational databases, compute clusters, and running jobs. We will keep our focus here on sharing data, specifically files and directories. For all of the below we will assume the GFFS client is already installed and that we have linked the GFFS into our Unix home directory at $HOME/XSEDE. We will further assume that our user Sarah has a directory in the GFFS at /home/Sarah. Given where the GFFS is mounted, the Unix path to that directory is $HOME/XSEE/home/Sarah. We will also assume below unless otherwise noted that our current working directory is $HOME/XSEE/home/Sarah.
to:
There are many resource types that can be shared, file system resources, storage resources, relational databases, compute clusters, and running jobs. We will keep our focus here on sharing data, specifically files and directories. For all of the below we will assume the GFFS client is already installed and that we have linked the GFFS into our Unix home directory at $HOME/XSEDE. We will further assume that our user Sarah has a directory in the GFFS at /home/Sarah. Given where the GFFS is mounted, the Unix path to that directory is $HOME/XSEE/home/Sarah. We will also assume below unless otherwise noted that our current working directory is $HOME/XSEE/home/Sarah.
November 11, 2011, at 03:19 PM by 128.143.137.203 -
Changed lines 18-19 from:
Given that the new items will be placed on the same GFFS container as Sarah’s home directory, exactly where will they be placed? There are two possibilities'^3^'. Either Sarah’s home directory is being stored a container in the containers own databases and storage space (in other words, the container is acting as a storage service), or her home directory is an export, in other words it is being stored in a local file system somewhere. Below we briefly examine each of these options. Keep in mind that either way a GFFS container is providing access to the data.
to:
Given that the new items will be placed on the same GFFS container as Sarah’s home directory, exactly where will they be placed? There are two possibilities'^1^'. Either Sarah’s home directory is being stored a container in the containers own databases and storage space (in other words, the container is acting as a storage service), or her home directory is an export, in other words it is being stored in a local file system somewhere. Below we briefly examine each of these options. Keep in mind that either way a GFFS container is providing access to the data.
Changed line 24 from:
'^3^' There are as many possibilities as there are different implementations of the RNS and ByteIO specifications. We are using the most typical implementations in the GFFS as of this writing.
to:
'^1^' There are as many possibilities as there are different implementations of the RNS and ByteIO specifications. We are using the most typical implementations in the GFFS as of this writing.
November 11, 2011, at 02:58 PM by 128.143.137.203 -
Changed line 22 from:
Footnote
to:
'''Footnote'''
November 11, 2011, at 02:57 PM by 128.143.137.203 -
Changed lines 22-24 from:
Footnote
to:
Footnote

'^3^' There are as many possibilities as there are different implementations of the RNS and ByteIO specifications. We are using the most typical implementations in the GFFS as of this writing.
November 11, 2011, at 02:56 PM by 128.143.137.203 -
Changed line 20 from:
[GFFS|Return to GFFS page]]
to:
[[GFFS|Return to GFFS page]]
November 11, 2011, at 02:55 PM by 128.143.137.203 -
Changed lines 19-22 from:
[GFFS|Return to GFFS page]]
to:
[GFFS|Return to GFFS page]]

Footnote
November 11, 2011, at 02:55 PM by 128.143.137.203 -
Changed lines 18-19 from:
Given that the new items will be placed on the same GFFS container as Sarah’s home directory, exactly where will they be placed? There are two possibilities'^3^'. Either Sarah’s home directory is being stored a container in the containers own databases and storage space (in other words, the container is acting as a storage service), or her home directory is an export, in other words it is being stored in a local file system somewhere. Below we briefly examine each of these options. Keep in mind that either way a GFFS container is providing access to the data.
to:
Given that the new items will be placed on the same GFFS container as Sarah’s home directory, exactly where will they be placed? There are two possibilities'^3^'. Either Sarah’s home directory is being stored a container in the containers own databases and storage space (in other words, the container is acting as a storage service), or her home directory is an export, in other words it is being stored in a local file system somewhere. Below we briefly examine each of these options. Keep in mind that either way a GFFS container is providing access to the data.
[GFFS|Return to GFFS page]]
November 11, 2011, at 02:54 PM by 128.143.137.203 -
November 11, 2011, at 02:54 PM by 128.143.137.203 -
Added lines 1-18:
(:Title Sharing Resources:)
As discussed above there are many resource types that can be shared, file system resources, storage resources, relational databases, compute clusters, and running jobs. We will keep our focus here on sharing data, specifically files and directories. For all of the below we will assume the GFFS client is already installed and that we have linked the GFFS into our Unix home directory at $HOME/XSEDE. We will further assume that our user Sarah has a directory in the GFFS at /home/Sarah. Given where the GFFS is mounted, the Unix path to that directory is $HOME/XSEE/home/Sarah. We will also assume below unless otherwise noted that our current working directory is $HOME/XSEE/home/Sarah.

Before we get to sharing local data resources, lets’ look first how to create a file or directory “somewhere in the GFFS”. To create a file or directory in the GFFS is simple. For example,
mkdir test
echo “This is a test” >> test/newfile

creates a new file in the newly created “test” directory. Once created both the directory and the file are available throughout the GFFS subject to access control.

However, where is the data actually stored? The short answer is that the “test” directory will be created in the same place where the current working directory is located. Similarly, “newfile” will be placed in the same location as the “test” directory.

So, where is that? A bit of background here is useful. The GFFS uses a standards-based Web Services model. Most Web Services (including those that implement the GFFS) execute inside of a program called a Web Services container. A container is a program that accepts Web Services connections (in our case https connections), parses the request, and calls the appropriate function to handle the request. Web Services containers are often written in Java, and execute on Windows, Linux, or MacOS machines like any other application. The difference is that they listen for http/https connections and respond to them.

In the GFFS, files and directories are stored in different GFFS Web Service containers (just “containers” from here on.) There are GFFS containers at the NSF service providers, and there are containers wherever someone wants to share a resource. Therefore, the first step to sharing a resource is to install the GFFS container. The installation process is very similar to installing the client if one chooses to use only the default options. It can be more complicated if, for example, your resource is behind a NAT or firewall. The GFFS container requires no special permissions or privilege, though it is recommend that it be started as its own user with minimum privilege.

To get back to our earlier question, “where is the data actually stored?” The new directory and the new file will be placed on the same container as Sarah’s GFFS home directory. In a moment, we will see how to over-ride this. For now, however, we will assume that is where they will be placed.

Given that the new items will be placed on the same GFFS container as Sarah’s home directory, exactly where will they be placed? There are two possibilities'^3^'. Either Sarah’s home directory is being stored a container in the containers own databases and storage space (in other words, the container is acting as a storage service), or her home directory is an export, in other words it is being stored in a local file system somewhere. Below we briefly examine each of these options. Keep in mind that either way a GFFS container is providing access to the data.