AFS in PPE

PPE storage provision is currently under review, and the following is subject to change.

Introduction

PPE uses AFS (the Andrew File System), and specifically the OpenAFS distribution, as a way to share some home areas and data directories between different machines, both internally and externally.

This page provides specific information about the AFS infrastructure within PPE. For a general introduction to AFS and information about how to use it, please refer to the OpenAFS User Guide.

Directory Layout

The local AFS cell (phas.gla.ac.uk, mounted as /afs/phas.gla.ac.uk) contains the following directories:

Refer to the section on back-ups for a list of those directories which are backed-up, and details of the retention schedule.

Cron jobs

Normal cron jobs cannot write to the AFS file system, and will only be able to read publicly-accessible files. You can create a cron job with full access to AFS using the kcrontab command.

AFS on Mac OS X

OpenAFS clients for recent versions of Mac OS X can be obtained from Auristor:

https://www.auristor.com/openafs/client-installer/

This package should include all the necessary components, but you may find that you need to use the debug version to get it working.

External Kerberos Access (Linux / Mac OS X)

You can configure Kerberos on a Linux or Mac OS X machine to enable password-less log-in to remote machines once a Kerberos ticket has been created. To do this, open /etc/krb5.conf (/Library/Preferences/edu.mit.Kerberos under OS X) in a text editor, and add the following to the realms section:

PHAS.GLA.AC.UK = {
  default_domain = phas.gla.ac.uk
  kdc = kdc2.phas.gla.ac.uk:88
  kdc = kdc1.phas.gla.ac.uk:88
  admin_server = kdc1.phas.gla.ac.uk
 }

In the same file, add the following to the libdefaults section:

Then add/edit to the libdefaults section:

allow_weak_crypto = true
default_realm = PHAS.GLA.AC.UK
dns_lookup_realm = false
dns_lookup_kdc = false
ticket_lifetime = 25h
renew_lifetime = 672h
forwardable = true
proxiable = true

To use Kerberos with SSH, open /etc/ssh/ssh_config (/etc/ssh_config on OS X) and check the following values are set:

GSSAPIAuthentication yes
GSSAPIDelegateCredentials yes
GSSAPIKeyExchange yes

You will need administrative rights to make any changes to these files.

To use a Kerberos ticket to log-in to ppelx:

$ kinit <USERNAME>@PHAS.GLA.AC.UK
$ ssh <USERNAME>@ppelx.physics.gla.ac.uk

External AFS Access (Linux / Mac OS X)

Access to the phas.gla.ac.uk AFS cell from a non-PPE machine requires that you first configure Kerberos as described above, then install the appropriate OpenAFS client.

After installing OpenAFS, open /etc/krb5.conf (/Library/Preferences/edu.mit.Kerberos under OS X) in a text editor, and add the following to the domain_realm section:

 .phas.gla.ac.uk = PHAS.GLA.AC.UK
 .physics.gla.ac.uk = PHAS.GLA.AC.UK
 .ppe.gla.ac.uk = PHAS.GLA.AC.UK

You will then need to edit your CellServDB file, which can usually be found in either /usr/vice/etc/CellServDB or /etc/openafs/CellServDB (/var/db/openafs/etc/CellServDB under OS X), adding the following lines:

>phas.gla.ac.uk         #Univeristy of Glasgow Physics And Astronomy
194.36.1.27                     #afsdb3.phas.gla.ac.uk
194.36.1.19                     #afsdb1.phas.gla.ac.uk
194.36.1.33                     #afsdb2.phas.gla.ac.uk

Finally, edit the ThisCell file which can be found in the same directory, replacing its contents with:

phas.gla.ac.uk

Restart the OpenAFS client either by restarting the service, or by rebooting the machine.

To create a Kerberos ticket and obtain an AFS token, use commands similar to the following:

$ kinit <USERNAME>@PHAS.GLA.AC.UK
$ aklog

Hints, Tips and Issues

Common AFS commands

AFS Access Control Lists (ACLs)

AFS uses directory-based ACLs to determine the permissions for the files contained within. Changing the permissions on a directory changes the permissions for all the files it contains, while moving a file from one directory to another may change its permissions. Subdirectories inherit the permissions on their parent directory when created, but may be configured independently thereafter. A detailed description of these permissions can be found in the OpenAFS User Guide.

PPE home areas contain the following directories by default:

Accessing the CERN AFS cell

Access to the CERN AFS cell (cern.ch, mounted as /afs/cern.ch) requires you to obtain a ticket for a different Kerberos realm. When obtaining this ticket, it is important that it be written to a different file from default, otherwise it will overwrite your PPE ticket and prevent access to your local files. To simplify this, a number of helper scripts have been provided: kinit-cern, klist-cern, kdestroy-cern, ssh-cern, kinit-fnal, klist-fnal, kdestroy-fnal, and ssh-fnal. These commands work similarly to the standard kinit, klist, kdestroy and ssh commands.

Using rsync with AFS

rsync will raise a permissions error when attempting to copy files which have the sticky bit set. Such errors can be safely ignored.

Automatic token renewal

AFS tokens can be automatically renewed for up to 30 days after log-in. This requires that a small script be configured to run automatically.

Users of the Bash shell should add the following to the start of their .bash_profile:

if [ -e /bin/ps ] && [ -e /bin/grep ]
then
    kproc=`/bin/ps x -u ${USER} | /bin/grep krenew | /bin/grep ${KRB5CCNAME}`
    if [ "${kproc}" == "" ] && [ -e /usr/bin/krenew ]
    then
        /usr/bin/krenew -K 60 -t -k ${KRB5CCNAME} &
    fi
fi