AFS on Mac OS X

OpenAFS clients for recent versions of Mac OS X can be obtained from Auristor:

https://www.auristor.com/openafs/client-installer/

This package should include all the necessary components, but you may find that you need to use the debug version to get it working.

AFS in PPE

PPE storage provision is currently under review, and the following is subject to change.

Introduction

PPE uses AFS (the Andrew File System), and specifically the OpenAFS distribution, as a way to share some home areas and data directories between different machines, both internally and externally.

This page provides specific information about the AFS infrastructure within PPE. For a general introduction to AFS and information about how to use it, please refer to the OpenAFS User Guide.

AFS Access Control Lists (ACLs)

Accessing the CERN AFS cell

Using rsync with AFS

Automatic token renewal

AFS at Glasgow

Directory Layout

Cron jobs

External Kerberos Access (Linux / Mac OS X)

In the same file, add the following to the libdefaults section:

$ kinit <USERNAME>@PHAS.GLA.AC.UK
$ ssh <USERNAME>@ppelx.physics.gla.ac.uk

External AFS Access (Linux / Mac OS X)

Restart the OpenAFS client either by restarting the service, or by rebooting the machine.

$ kinit <USERNAME>@PHAS.GLA.AC.UK
$ aklog

Hints, Tips and Issues

Automatic Token Renewal

To automatically renew an AFS token for up to 30 days after logging in requires either a small addition the shell login script for ssh logins or for desktop logins a small script to be auto run. Each login change is independent of the others and all can be safely used at the same time.

ssh login

For bash shell users add the following to the top of .bash_profile:

if [ -e /bin/ps ] && [ -e /bin/grep ]
then
    kproc=`/bin/ps x -u ${USER} | /bin/grep krenew | /bin/grep ${KRB5CCNAME}`
    if [ "${kproc}" == "" ] && [ -e /usr/bin/krenew ]
    then
        /usr/bin/krenew -K 60 -t -k ${KRB5CCNAME} &
    fi
fi

Or for csh/tcsh users add the following to the top of .cshrc or .tcshrc:

if ( -e /bin/ps && -e /bin/grep ) then
    setenv kproc `/bin/ps x -u ${USER} | /bin/grep krenew | /bin/grep ${KRB5CCNAME}`
    if ( "${kproc}" == "" && -e /usr/bin/krenew ) then
        /usr/bin/krenew -K 60 -t -k ${KRB5CCNAME} &
    endif
endif

KDE desktop

Gnome Desktop

External Kerberos access (linux/mac)

GSSAPIAuthentication yes
GSSAPICleanupCredentials yes 
GSSAPIKeyExchange yes

Then try to get an kerberos ticket and log into ppelx:

Changing username as appropriate.

External afs access (linux/mac)

To access the phas.gla.ac.uk afs cell from an external linux machine requires first setting up kerberos access as described above and then installing the openafs client and editing a couple of config files all of which will require root access. Packages exist for all the major linux distributions just use the distributions software installer. If that fails try the openafs web site: https://www.openafs.org . For OS X the packages can be downloaded from http://www.openafs.org/macos.html

After installing the openafs edit the file /etc/krb5.conf or for OS X edit /Library/Preferences/edu.mit.Kerberos and add/edit to the realms section:

$ ssh-cern <lxplus user name>@lxplus.cern.ch

 .phas.gla.ac.uk = PHAS.GLA.AC.UK
 .physics.gla.ac.uk = PHAS.GLA.AC.UK
 .ppe.gla.ac.uk = PHAS.GLA.AC.UK

Or use this example krb5.conf file which also has the kerberos setting for cern and fermi lab.

External access (linux/mac)

Next edit the ThisCell file (/usr/vice/etc/ThisCell or /etc/openafs/ThisCell on linux or /var/db/openafs/etc/ThisCell for OS X) replacing the contents with

phas.gla.ac.uk

Restart the openafs client and then try to get an kerberos ticket and an afs token:

External access (linux)

To access the phas.gla.ac.uk afs cell from an external linux machine requires installing the openafs client and editing a couple of config files all of which will require root access. Packages exist for all the major linux distributions just use the distributions software installer. If that fails try the openafs web site: https://www.openafs.org

After installing the openafs edit the file /etc/krb5.conf and add:

PHAS.GLA.AC.UK = {
  default_domain = phas.gla.ac.uk
  kdc = kdc2.phas.gla.ac.uk:88
  kdc = kdc3.phas.gla.ac.uk:88
  kdc = kdc1.phas.gla.ac.uk:88
  admin_server = kdc1.phas.gla.ac.uk
 }

To the realms section and add

allow_weak_crypto = true

to the libdefaults section. Or use this example krb5.conf file which also has the kerberos setting for cern and fermi lab.

The edit the CellServDB file which can usually be found at /usr/vice/etc/CellServDB or /etc/openafs/CellServDB and add:

>phas.gla.ac.uk         #Univeristy of Glasgow Physics And Astronomy
194.36.1.27                     #afsdb3.phas.gla.ac.uk
194.36.1.19                     #afsdb1.phas.gla.ac.uk
194.36.1.33                     #afsdb2.phas.gla.ac.uk

either at the top or bottom of the file. Restart the openafs client and then try to get an kerberos ticket and an afs token:

Changing username as appropriate. If both commands are successful then try to access the afs cell:

They are used in the same way as the standard kinit, klist and kdestroy. For example:

To allow a user or group user read access to a directory dir :

Recursive setting of ACLs

A helper script to recursively set an ACL is available on all the scientific linux desktop machines. Usage is similar to the standard fs setacl command:

fs-recursive-setacl dir user acl

Hints, Tips And Issues

Common AFS commands

AFS ACLs (Protecting Data)

Accessing the CERN afs cell (/afs/cern.ch)

rsyncing files to afs

rsync will copy files to an afs file system in the same way as any other file system. However if directories in the source file system have the stick bit set rsync will complain that it 'failed to set permissions'. A directory with the sticky bit set will have a 's' in the permissions bit field eg:

drwxrwsr-x  4 ppemgr ppemgr   45 Jan 14  2009 sl44

These errors can be safely ignored but they may obscure other errors when rsyncing. To remove stick bits before rsyncing use chmod:

chmod a-s _dir_

replacing _dir_ as required. To recursively remove sticky bits use the -R option.

fs setacl dir user read

To give write permissions:

fs setacl dir user write

And to remove all permissions for a user or group:

fs setacl dir user none

Backups

Overnight backups of user home areas are taken at 1am each night. These backups are user accessible and can be found in /afs/phas.gla.ac.uk/backup/user/_letter_/_username_. Longer term backups are also done - nightly snapshots of user home directories are retained for 7 days except that the snapshots done on a Saturday are retained for two months and the snapshots done on the first Saturday of each month are retained for six months. These longer term backups are not user accessible please contact SystemAdministrators to request a file to be recovered.

• system : Common programs installed system wide.

Cronjobs

Normal cronjobs can not write to the afs file system and can only read the publicly accessible parts of the file system. Cronjobs which require access to the afs file system can be created using the kcrontab command.

Batch system

The currrent batch system (acess from the machine ppepbs.physics.gla.ac.uk) will not be able to access the local afs cell. A new batch system has been setup which can access the afs file system and which is accessible from any linux desktop machine that has been moved onto the new system.

To use the new system simply use the normal qsub and qstat commands from any PPE linux desktop that has been moved over the the new system.

Home web pages

User's home web pages are served from the directory public_html in their home directory. The web server will use the public_html directory in the afs file system of an account moved over to the afs in preference to the public_html directory in the old account. Until files are moved from the old public_html directory to the new a user's web pages will be inaccessible. It is important to move the files under the public_html directory in the old NFS file system to the new public_html directory and not to move the public_html directory itself due to afs file permissions. For example to do this use the command:

mv /home/_username_/public_html/* /afs/phas.gla.ac.uk/user/_letter_/_username_/public_html/

Substituting _letter_ and _username_ as required.

Common AFS commands

• fs lq : List the size and amount used of the volume of the current working directory.
• fs listacl : List the acl (access control list) for the current working directory.
• fs setacl dir user/group permissions : Add to the acl of a directory.

AFS ACLs (Protecting Data)

AFS file systems use directory based ACLs to determine the access permissions for a given file. The following is taken from the afs documentation (http://docs.openafs.org/AdminGuide/ch15s02.html):

• The permissions on a directory's ACL apply to all of the files in the directory. When you move a file to a different directory, you effectively change the access permissions that apply to it to those on its new directory's ACL. Changing a directory's ACL changes the protection on all the files in it.
• When you create a subdirectory, its initial ACL is created as a copy of its parent directory's ACL. You can then change the subdirectory's ACL independently. However, the parent directory's ACL continues to control access to the subdirectory in the following way: the parent directory's ACL must grant the l (lookup) permission to a user (or a group the user belongs to) in order for the user to access the subdirectory at all.

By default the following directories are created in a user's afs home area:

• private - access only for the user and the system administrator, this is also the default for any new directories.
• public - public (world wide) access for anyone.
• public_html - public (world wide) access for anyone, user web pages belong here.
• public_ppe - readable to the ppe group only.

To see the acl list on a directory use the command fs listacl dir . The allow the ppe group read access to a directory fs setacl dir ppe rl. Replacing dir as appropriate.

System administrator access is required to all directories to allow backups to be taken.

Command AFS commands

• fs lq : List the size and amount used of the volume of the current working directory.
• fs listacl : List the acl (access control list) for the current working directory.
• fs setacl dir user/group permissions : Add to the acl of a directory.

Local AFS Cell (/afs/phas.gla.ac.uk)

File System Layout

• backup: Snap shot backups for user home directories and other files. The snap shot is done each night at 1am.
• data: Large store area, not backed up.
• group: Group based storage area, backed up.
• project:
• system:
• user: User home directories are stored under this directory. The initial quota on home directories is 50GB. Nightly snap shot backups and long term backups.

Accessing the CERN afs cell (/afs/cern.ch)

To get an afs token to use with the cern afs cell (/afs/cern.ch) from any ppe linux machine do:

Accessing the CERN afs cell

To get an afs token to use with the cern afs cell (/afs/cern.ch) from any ppe linux machine do:

$ kinit
$ aklog

If your cern lxplus user name is different from your Glasgow ppe user name then it needs to be given as a parameter to kinit:

$ kinit <lxplus user name>

Using AFS At Glasgow

-- AndrewPickford - 2009-10-14