Difference: FileStorage (1 vs. 12)

Revision 122019-01-07 - GordonStewart

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Storage

Line: 8 to 8
 
  • Home areas (/home in Linux), which are mostly backed-up.
  • Data areas (/data and /nfs in Linux), which due to the sheer volume of data are generally not backed-up.
Deleted:
<
<
  • Scratch space on certain machines (/pcdisk/<HOSTNAME>), which are not backed-up.
  It is your responsibility to ensure that your important files are protected, and so you should take particular note of which areas are backed-up.

Revision 112018-07-31 - GordonStewart

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Storage

Line: 7 to 7
 Various file storage areas are available to PPE group members, with different areas intended for, or suited to, different uses. These include:

  • Home areas (/home in Linux), which are mostly backed-up.
Changed:
<
<
  • Data areas (/data in Linux), which due to the sheer volume of data are generally not backed-up.
>
>
  • Data areas (/data and /nfs in Linux), which due to the sheer volume of data are generally not backed-up.
 
  • Scratch space on certain machines (/pcdisk/<HOSTNAME>), which are not backed-up.

It is your responsibility to ensure that your important files are protected, and so you should take particular note of which areas are backed-up.

Linux

Changed:
<
<
The majority of PPE Linux machines are configured to mount home and data areas automatically. Home areas are located in /home, and data areas in /data.
>
>
The majority of PPE Linux machines are configured to mount home and data areas automatically. Home areas are located in /home, and data areas in /data and /nfs.
  If you want to access storage from a Linux machine which has not been configured to mount these storage areas automatically, you can use CIFS for which you will require administrator rights. The command to mount a directory takes the following form:
Changed:
<
<
$ mount -t cifs cifs://ppesmb<PATH> <MOUNT_PATH> -o user=guphysics/<USERNAME>
>
>
$ mount -t cifs cifs://uberwald.ppe.gla.ac.uk<PATH> <MOUNT_PATH> -o user=guphysics/<USERNAME>
  For example, if user rabbit wished to mount the /data/carrots directory locally as /mnt/yummy, the following command would be needed:
Changed:
<
<
$ mount -t cifs cifs://ppesmb/data/carrots /mnt/yummy -o user=guphysics/rabbit
>
>
$ mount -t cifs cifs://uberwald.ppe.gla.ac.uk/data/carrots /mnt/yummy -o user=guphysics/rabbit
 

Microsoft Windows

Changed:
<
<
Windows users can access home and data areas using Samba. You can connect to the Samba server using the alias ppesmb, for instance by entering \\ppesmb in the navigation bar in Windows Explorer. Most PPE desktops are bound to the GUPHYSICS domain, and so you will not need to enter any credentials. If you are using a laptop or one of the few desktops which is not part of the domain, you will need to enter your GUPHYSICS credentials to authenticate.
>
>
Windows users can access home and data areas using Samba. You can connect to the Samba server using the alias uberwald.ppe.gla.ac.uk, for instance by entering \\uberwald.ppe.gla.ac.uk in the navigation bar in Windows Explorer. Most PPE desktops are bound to the GUPHYSICS domain, and so you will not need to enter any credentials. If you are using a laptop or one of the few desktops which is not part of the domain, you will need to enter your GUPHYSICS credentials to authenticate.
 

Mac OS X

Line: 34 to 34
  connect_to_server.png
Changed:
<
<
Enter smb://ppesmb as the Server Address and click Connect.
>
>
Enter smb://uberwald.ppe.gla.ac.uk as the Server Address and click Connect.
  ppesmb.png

Revision 102017-08-11 - GordonStewart

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Storage

Line: 16 to 16
  The majority of PPE Linux machines are configured to mount home and data areas automatically. Home areas are located in /home, and data areas in /data.
Deleted:
<
<
Most Linux machines are provided with world-writable scratch space located in /pcdisk/<HOSTNAME>. All users have permission to read and write files in this location. This scratch space is contained on the machine's local disk as opposed to being mounted over the network, and as such may be used to improve the performance of heavily I/O-dependent programs. You should transfer any important files out of this space after your program has completed, and should also clean up any temporary files.
 If you want to access storage from a Linux machine which has not been configured to mount these storage areas automatically, you can use CIFS for which you will require administrator rights. The command to mount a directory takes the following form:

$ mount -t cifs cifs://ppesmb<PATH> <MOUNT_PATH> -o user=guphysics/<USERNAME>

Changed:
<
<
For example, if user rabbit wished to mount the /data/carrots directory locally as /mnt, the following command would be needed:
>
>
For example, if user rabbit wished to mount the /data/carrots directory locally as /mnt/yummy, the following command would be needed:
 
Changed:
<
<
$ mount -t cifs cifs://ppesmb/data/carrots /mnt -o user=guphysics/rabbit
>
>
$ mount -t cifs cifs://ppesmb/data/carrots /mnt/yummy -o user=guphysics/rabbit
 

Microsoft Windows

Revision 92016-04-21 - GordonStewart

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Storage

Line: 14 to 14
 

Linux

Changed:
<
<
Linux machines are configured to mount home and data areas automatically. Home areas are located in /home, and data areas in /data.
>
>
The majority of PPE Linux machines are configured to mount home and data areas automatically. Home areas are located in /home, and data areas in /data.
 
Added:
>
>
Most Linux machines are provided with world-writable scratch space located in /pcdisk/<HOSTNAME>. All users have permission to read and write files in this location. This scratch space is contained on the machine's local disk as opposed to being mounted over the network, and as such may be used to improve the performance of heavily I/O-dependent programs. You should transfer any important files out of this space after your program has completed, and should also clean up any temporary files.
 
Added:
>
>
If you want to access storage from a Linux machine which has not been configured to mount these storage areas automatically, you can use CIFS for which you will require administrator rights. The command to mount a directory takes the following form:
 
Added:
>
>
$ mount -t cifs cifs://ppesmb<PATH> <MOUNT_PATH> -o user=guphysics/<USERNAME>
 
Added:
>
>
For example, if user rabbit wished to mount the /data/carrots directory locally as /mnt, the following command would be needed:
 
Added:
>
>
$ mount -t cifs cifs://ppesmb/data/carrots /mnt -o user=guphysics/rabbit
 
Added:
>
>

Microsoft Windows

 
Changed:
<
<

Local Scratch Disk

>
>
Windows users can access home and data areas using Samba. You can connect to the Samba server using the alias ppesmb, for instance by entering \\ppesmb in the navigation bar in Windows Explorer. Most PPE desktops are bound to the GUPHYSICS domain, and so you will not need to enter any credentials. If you are using a laptop or one of the few desktops which is not part of the domain, you will need to enter your GUPHYSICS credentials to authenticate.
 
Changed:
<
<
After logging into a LINUX desktop the scratch space available can be found at /pcdisk/ppepcXX where XX is the number of the LINUX desktop. This directory is world writable such that any PPE user can create files within it. Since this scratch space is directly mounted to the machine i.e. not over NFS it can be used to speed up heavily I/O dependent programs. Please remember to copy any important data off this space once such a process has completed. (At present this scratch space is un-managed and therefore will not be cleaned up automatically.)
>
>

Mac OS X

 
Changed:
<
<

From Windows

Windows users should access their HomeAreas and DataAreas using Samba. Users should connect to the Samba server via the alias ppesmb. Users can connect to the Samba server either by navigating to it via the guphysics domain or via clicking on Start->Run and then typing \\ppesmb. Laptop users should use their guphysics username and password to authenticate.

* Users logging into a desktop which is bound to the guphysics domain will find that their ppesmb home area is automatically mounted as the U: drive.

* When browsing the \\ppesmb folder DataAreas can be more clearly identified by switching the folder to the details view. To do this open the \\ppesmb folder following the instructions given above. Then right click within the folder and select View->Details. The DataAreas can then be identified by the Comments field.

From OS X

Start finder, then from the menu bar select Go then Connect to Server.

>
>
Start Finder, then Go | Connect to Server....
  connect_to_server.png
Changed:
<
<
Enter smb://ppesmb for the server address in the diolog box and click Connect.
>
>
Enter smb://ppesmb as the Server Address and click Connect.
  ppesmb.png
Changed:
<
<
Enter your GUPHYSICS username and password if requested. From the final dialog box select the samba share you wish to access.

ppemgr02.png

From Linux Laptop

>
>
If requested, enter your GUPHYSICS username and password.
 
Changed:
<
<
To mount an external drive on Linux you will need root or sudo permissions. When you have these, use CIFS*:
>
>
Select the Samba share that you wish to access.
 
Changed:
<
<
mount -t cifs SERVER_NAME LOCAL_MOUNT_DIRECTORY -o user=USER_NAME

So, for example, if your name is Napoleon Solo and you have a GUPHYSICS account and you wished to mount the detdev01 directory in the //ppesmb Samba file system to the local /mnt directory, you would use:

mount -t cifs //ppesmb/data/detdev01 /mnt -o user=guphysics/nsolo

Napoleon would then be prompted for his (GUPHYSICS) password before mounting took place. The password can also be passed as an additional comma-seperated argument (pass=PSSWRD), or inside a file along with the user name (credentials=FILE_NAME).

N.B. The user name must include the GUPHYSICS domain. Also, make sure the directory you are mounting to exists.

For more details and some useful CIFS documentation, please see here.

* sudo users should preface the command with sudo and enter their Linux password.

-- AndrewPickford - 25 Jan 2009

>
>
ppemgr02.png
 
META FILEATTACHMENT attachment="connect_to_server.png" attr="" comment="" date="1280914253" name="connect_to_server.png" path="connect_to_server.png" size="60937" stream="connect_to_server.png" tmpFilename="/usr/tmp/CGItemp12385" user="AndrewPickford" version="1"
META FILEATTACHMENT attachment="ppesmb.png" attr="" comment="" date="1280914273" name="ppesmb.png" path="ppesmb.png" size="38025" stream="ppesmb.png" tmpFilename="/usr/tmp/CGItemp12603" user="AndrewPickford" version="1"

Revision 82016-04-21 - GordonStewart

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
Changed:
<
<
>
>

Storage

PPE storage provision is currently under review, and the following is subject to change.

Various file storage areas are available to PPE group members, with different areas intended for, or suited to, different uses. These include:

  • Home areas (/home in Linux), which are mostly backed-up.
  • Data areas (/data in Linux), which due to the sheer volume of data are generally not backed-up.
  • Scratch space on certain machines (/pcdisk/<HOSTNAME>), which are not backed-up.

It is your responsibility to ensure that your important files are protected, and so you should take particular note of which areas are backed-up.

Linux

Linux machines are configured to mount home and data areas automatically. Home areas are located in /home, and data areas in /data.

 
Deleted:
<
<

File Storage

 
Deleted:
<
<
  • Storage areas for user files are provided via NFS for LINUX users and via SAMBA for Windows users.
  • Data files should be stored within these storage areas rather than on a local machine.
  • Users should be aware of which areas are backed up.
 
Deleted:
<
<

From LINUX

 
Deleted:
<
<
  • HomeAreas are available via /home/
  • DataAreas are available via /data/
  • Scratch areas on the local disc are available via /pcdisk/ppepcXX
 
Deleted:
<
<
After logging into a LINUX system either from the console of a LINUX desktop or via ssh a user will start their session within their home area. Other areas NFS areas are auto mounted and so may not be visible via a df command. A list of the current data areas can be found using the ypcat command.
 
Deleted:
<
<
Listing the available data areas
$ ypcat -k auto.data
 

Local Scratch Disk

Revision 72013-04-24 - KennyWraight

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
Line: 49 to 49
 

From Linux Laptop

Changed:
<
<
To mount an external drive on Linux you will need root permissions. When you have these, use CIFS:
>
>
To mount an external drive on Linux you will need root or sudo permissions. When you have these, use CIFS*:
  mount -t cifs SERVER_NAME LOCAL_MOUNT_DIRECTORY -o user=USER_NAME
Changed:
<
<
So, for example, if your name is Napoleon Solo and you have a guphysics account and you wished to mount the detdev01 directory in the //ppesmb Samba file system to the local /mnt directory, you would use:
>
>
So, for example, if your name is Napoleon Solo and you have a GUPHYSICS account and you wished to mount the detdev01 directory in the //ppesmb Samba file system to the local /mnt directory, you would use:
  mount -t cifs //ppesmb/data/detdev01 /mnt -o user=guphysics/nsolo
Changed:
<
<
Napoleon would then be prompted for his password before mounting took place. The password can also be passed as an additional comma-seperated argument ("pass") or inside a file, along with the user name ("credentials").
>
>
Napoleon would then be prompted for his (GUPHYSICS) password before mounting took place. The password can also be passed as an additional comma-seperated argument (pass=PSSWRD), or inside a file along with the user name (credentials=FILE_NAME).
 
Changed:
<
<
N.B. The user name must include the guphysics domain. Make sure the mount directory exists.
>
>
N.B. The user name must include the GUPHYSICS domain. Also, make sure the directory you are mounting to exists.
 
Changed:
<
<
For some more details and some useful CIFS documentation, please see here.
>
>
For more details and some useful CIFS documentation, please see here.

* sudo users should preface the command with sudo and enter their Linux password.

  -- AndrewPickford - 25 Jan 2009

Revision 62013-04-23 - KennyWraight

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
Line: 49 to 49
 

From Linux Laptop

Changed:
<
<
To mount an external drive on Linux using CIFS use:
>
>
To mount an external drive on Linux you will need root permissions. When you have these, use CIFS:
  mount -t cifs SERVER_NAME LOCAL_MOUNT_DIRECTORY -o user=USER_NAME

Revision 52013-04-22 - KennyWraight

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
Line: 49 to 49
 

From Linux Laptop

Added:
>
>
To mount an external drive on Linux using CIFS use:

mount -t cifs SERVER_NAME LOCAL_MOUNT_DIRECTORY -o user=USER_NAME

So, for example, if your name is Napoleon Solo and you have a guphysics account and you wished to mount the detdev01 directory in the //ppesmb Samba file system to the local /mnt directory, you would use:

mount -t cifs //ppesmb/data/detdev01 /mnt -o user=guphysics/nsolo

Napoleon would then be prompted for his password before mounting took place. The password can also be passed as an additional comma-seperated argument ("pass") or inside a file, along with the user name ("credentials").

N.B. The user name must include the guphysics domain. Make sure the mount directory exists.

For some more details and some useful CIFS documentation, please see here.

 -- AndrewPickford - 25 Jan 2009

META FILEATTACHMENT attachment="connect_to_server.png" attr="" comment="" date="1280914253" name="connect_to_server.png" path="connect_to_server.png" size="60937" stream="connect_to_server.png" tmpFilename="/usr/tmp/CGItemp12385" user="AndrewPickford" version="1"

Revision 42013-04-22 - AndrewPickford

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
Added:
>
>
 

File Storage

  • Storage areas for user files are provided via NFS for LINUX users and via SAMBA for Windows users.
Line: 45 to 47
  ppemgr02.png
Added:
>
>

From Linux Laptop

 -- AndrewPickford - 25 Jan 2009

META FILEATTACHMENT attachment="connect_to_server.png" attr="" comment="" date="1280914253" name="connect_to_server.png" path="connect_to_server.png" size="60937" stream="connect_to_server.png" tmpFilename="/usr/tmp/CGItemp12385" user="AndrewPickford" version="1"

Revision 32010-08-04 - AndrewPickford

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

File Storage

Line: 33 to 33
 

From OS X

Changed:
<
<
Start finder, then from the menu bar select Go then Connect to Server. Enter smb://ppesmb for the server address in the diolog box and click Connect. Enter your GUPHYSICS username and password if requested. From the final dialog box select the samba share you wish to access.
>
>
Start finder, then from the menu bar select Go then Connect to Server.

connect_to_server.png

Enter smb://ppesmb for the server address in the diolog box and click Connect.

ppesmb.png

Enter your GUPHYSICS username and password if requested. From the final dialog box select the samba share you wish to access.

ppemgr02.png

  -- AndrewPickford - 25 Jan 2009 \ No newline at end of file
Added:
>
>
META FILEATTACHMENT attachment="connect_to_server.png" attr="" comment="" date="1280914253" name="connect_to_server.png" path="connect_to_server.png" size="60937" stream="connect_to_server.png" tmpFilename="/usr/tmp/CGItemp12385" user="AndrewPickford" version="1"
META FILEATTACHMENT attachment="ppesmb.png" attr="" comment="" date="1280914273" name="ppesmb.png" path="ppesmb.png" size="38025" stream="ppesmb.png" tmpFilename="/usr/tmp/CGItemp12603" user="AndrewPickford" version="1"
META FILEATTACHMENT attachment="ppemgr02.png" attr="" comment="" date="1280914285" name="ppemgr02.png" path="ppemgr02.png" size="135484" stream="ppemgr02.png" tmpFilename="/usr/tmp/CGItemp12698" user="AndrewPickford" version="1"

Revision 22010-07-27 - AndrewPickford

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

File Storage

Line: 31 to 31
  * When browsing the \\ppesmb folder DataAreas can be more clearly identified by switching the folder to the details view. To do this open the \\ppesmb folder following the instructions given above. Then right click within the folder and select View->Details. The DataAreas can then be identified by the Comments field.
Added:
>
>

From OS X

Start finder, then from the menu bar select Go then Connect to Server. Enter smb://ppesmb for the server address in the diolog box and click Connect. Enter your GUPHYSICS username and password if requested. From the final dialog box select the samba share you wish to access.

 -- AndrewPickford - 25 Jan 2009

Revision 12009-01-25 - AndrewPickford

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="WebHome"

File Storage

  • Storage areas for user files are provided via NFS for LINUX users and via SAMBA for Windows users.
  • Data files should be stored within these storage areas rather than on a local machine.
  • Users should be aware of which areas are backed up.

From LINUX

  • HomeAreas are available via /home/
  • DataAreas are available via /data/
  • Scratch areas on the local disc are available via /pcdisk/ppepcXX

After logging into a LINUX system either from the console of a LINUX desktop or via ssh a user will start their session within their home area. Other areas NFS areas are auto mounted and so may not be visible via a df command. A list of the current data areas can be found using the ypcat command.

Listing the available data areas
$ ypcat -k auto.data

Local Scratch Disk

After logging into a LINUX desktop the scratch space available can be found at /pcdisk/ppepcXX where XX is the number of the LINUX desktop. This directory is world writable such that any PPE user can create files within it. Since this scratch space is directly mounted to the machine i.e. not over NFS it can be used to speed up heavily I/O dependent programs. Please remember to copy any important data off this space once such a process has completed. (At present this scratch space is un-managed and therefore will not be cleaned up automatically.)

From Windows

Windows users should access their HomeAreas and DataAreas using Samba. Users should connect to the Samba server via the alias ppesmb. Users can connect to the Samba server either by navigating to it via the guphysics domain or via clicking on Start->Run and then typing \\ppesmb. Laptop users should use their guphysics username and password to authenticate.

* Users logging into a desktop which is bound to the guphysics domain will find that their ppesmb home area is automatically mounted as the U: drive.

* When browsing the \\ppesmb folder DataAreas can be more clearly identified by switching the folder to the details view. To do this open the \\ppesmb folder following the instructions given above. Then right click within the folder and select View->Details. The DataAreas can then be identified by the Comments field.

-- AndrewPickford - 25 Jan 2009

 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback