Using AFS At Glasgow

Local AFS Cell (/afs/phas.gla.ac.uk)

File System Layout

Beneath /afs/phas.gla.ac.uk are the following directories:

• backup : Snap shot backups for user home directories and other files. The snap shot is done each night at 1am.
• data : Large store area, not backed up.
• group : Group based storage area, backed up.
• project :
• system : Common programs installed system wide.
• user : User home directories are stored under this directory. The initial quota on home directories is 50GB. Nightly snap shot backups and long term backups.

Backups

Overnight backups of user home areas are taken at 1am each night. These backups are user accessible and can be found in /afs/phas.gla.ac.uk/backup/user/_letter_/_username_. Longer term backups are also done - nightly snapshots of user home directories are retained for 7 days except that the snapshots done on a Saturday are retained for two months and the snapshots done on the first Saturday of each month are retained for six months. These longer term backups are not user accessible please contact SystemAdministrators to request a file to be recovered.

Cronjobs

Normal cronjobs can not write to the afs file system and can only read the publicly accessible parts of the file system. Cronjobs which require access to the afs file system can be created using the kcrontab command.

Batch system

The currrent batch system (acess from the machine ppepbs.physics.gla.ac.uk) will not be able to access the local afs cell. A new batch system has been setup which can access the afs file system and which is accessible from any linux desktop machine that has been moved onto the new system.

To use the new system simply use the normal qsub and qstat commands from any PPE linux desktop that has been moved over the the new system.

Home web pages

User's home web pages are served from the directory public_html in their home directory. The web server will use the public_html directory in the afs file system of an account moved over to the afs in preference to the public_html directory in the old account. Until files are moved from the old public_html directory to the new directory a user's web pages will be inaccessible. It is important to move the files under the public_html directory in the old NFS file system to the new public_html directory and not to move the public_html directory itself due to afs file permissions. For example to do this use the command:

mv /home/_username_/public_html/* /afs/phas.gla.ac.uk/user/_letter_/_username_/public_html/

Substituting _letter_ and _username_ as required.

External access (linux)

To access the phas.gla.ac.uk afs cell from an external linux machine requires installing the openafs client and editing a couple of config files all of which will require root access. Packages exist for all the major linux distributions just use the distributions software installer. If that fails try the openafs web site: https://www.openafs.org

After installing the openafs edit the file /etc/krb5.conf and add:

PHAS.GLA.AC.UK = {
  default_domain = phas.gla.ac.uk
  kdc = kdc2.phas.gla.ac.uk:88
  kdc = kdc3.phas.gla.ac.uk:88
  kdc = kdc1.phas.gla.ac.uk:88
  admin_server = kdc1.phas.gla.ac.uk
 }

To the realms section and add

allow_weak_crypto = true

to the libdefaults section. Or use this example krb5.conf file which also has the kerberos setting for cern and fermi lab.

The edit the CellServDB file which can usually be found at /usr/vice/etc/CellServDB or /etc/openafs/CellServDB and add:

>phas.gla.ac.uk         #Univeristy of Glasgow Physics And Astronomy
194.36.1.27                     #afsdb3.phas.gla.ac.uk
194.36.1.19                     #afsdb1.phas.gla.ac.uk
194.36.1.33                     #afsdb2.phas.gla.ac.uk

either at the top or bottom of the file. Restart the openafs client and then try to get an kerberos ticket and an afs token:

Changing username as appropriate. If both commands are successful then try to access the afs cell:

Hints, Tips And Issues

Common AFS commands

• fs lq : List the size and amount used of the volume of the current working directory.
• fs listacl : List the acl (access control list) for the current working directory.
• fs setacl dir user/group permissions : Add to the acl of a directory.

AFS ACLs (Protecting Data)

AFS file systems use directory based ACLs to determine the access permissions for a given file. The following is taken from the afs documentation (http://docs.openafs.org/AdminGuide/ch15s02.html):

• The permissions on a directory's ACL apply to all of the files in the directory. When you move a file to a different directory, you effectively change the access permissions that apply to it to those on its new directory's ACL. Changing a directory's ACL changes the protection on all the files in it.
• When you create a subdirectory, its initial ACL is created as a copy of its parent directory's ACL. You can then change the subdirectory's ACL independently. However, the parent directory's ACL continues to control access to the subdirectory in the following way: the parent directory's ACL must grant the l (lookup) permission to a user (or a group the user belongs to) in order for the user to access the subdirectory at all.

By default the following directories are created in a user's afs home area:

• private - access only for the user and the system administrator, this is also the default for any new directories.
• public - public (world wide) access for anyone.
• public_html - public (world wide) access for anyone, user web pages belong here.
• public_ppe - readable to the ppe group only.

To see the acl list on a directory use the command fs listacl dir .

To allow a user or group user read access to a directory dir :

fs setacl dir user read

To give write permissions:

fs setacl dir user write

And to remove all permissions for a user or group:

fs setacl dir user none

System administrator access is required to all directories to allow backups to be taken.

Recursive setting of ACLs

A helper script to recursively set an ACL is available on all the scientific linux desktop machines. Usage is similar to the standard fs setacl command:

fs-recursive-setacl dir user acl

Accessing the CERN afs cell (/afs/cern.ch)

Getting a kerberos ticket for another kerberos realm using kinit will overwrite the existing kerberos tickets file destroying any local Glasgow ticket. To get around this tickets for different kerberos realms have to be written to different files. A couple of helper scripts have been written to made this straight forward: kinit-cern, klist-cern, kdestroy-cern, kinit-fnal, klist-fnal, kdestroy-fnal.

They are used in the same way as the standard kinit, klist and kdestroy. For example:

$ kinit-cern <lxplus user name>@CERN.CH

rsyncing files to afs

rsync will copy files to an afs file system in the same way as any other file system. However if directories in the source file system have the stick bit set rsync will complain that it 'failed to set permissions'. A directory with the sticky bit set will have a 's' in the permissions bit field eg:

drwxrwsr-x  4 ppemgr ppemgr   45 Jan 14  2009 sl44

These errors can be safely ignored but they may obscure other errors when rsyncing. To remove stick bits before rsyncing use chmod:

chmod a-s _dir_

replacing _dir_ as required. To recursively remove sticky bits use the -R option.

-- AndrewPickford - 2009-10-14