the HP XC. So, for example, if the HP XC system interconnect is based on a Quadrics® QsNet

II® switch, then the SFS will serve files over ports on that switch. The file operations are able to proceed at the full bandwidth of the HP XC system interconnect because these operations are implemented directly over the low-level communications libraries. Further optimizations of file I/O can be achieved at the application level using special file system commands – implemented as ioctls – which allow a program to interrogate the attributes of the file system, modify the stripe size and other attributes of new (zero-length) files, and so on. Some of these optimizations are implicit in the HP-MPI I/O library, which implements the MPI-2 file I/O standard.

File System Layout

In an HP XC system, the basic file system layout is the same as that of the Red Hat Advanced Server 3.0 Linux file system.

The HP XC file system is structured to separate cluster-specific files, base operating system files, and user-installed software files. This allows for flexibility and ease of potential upgrades of the system software and keeps software from conflicting with user installed software. Files are segregated into the following types and locations:

•Software specific to HP XC is located in /opt/hptc•HP XC configuration data is located in /opt/hptc/etc•Clusterwide directory structure (file system) is located in /hptc_clusterBe aware of the following information about the HP XC file system layout:

•Open source software that by default would be installed under the /usr/local directory is instead installed in the /opt/hptc directory.

•Software installed in the /opt/hptc directory is not intended to be updated by users.

•Software packages are installed in directories under the /opt/hptc directory under their own names. The exception to this is third-party software, which usually goes in /opt/r.

•Four directories under the /opt/hptc directory contain symbolic links to files included in the packages:

—/opt/hptc/bin—/opt/hptc/sbin—/opt/hptc/lib—/opt/hptc/man

Each package directory should have a directory corresponding to each of these directories in which every file has a symbolic link created in the /opt/hptc/ directory.

1.1.7 System Interconnect Network

The HP XC system interconnect provides high-speed connectivity for parallel applications. The system interconnect network provides a high-speed communications path used primarily for user file service and for communications within user applications that are distributed among nodes of the system. The system interconnect network is a private network within the HP XC. Typically, every node in the HP XC is connected to the system interconnect.

Table 1-2indicates the types of system interconnects that are used on HP XC systems.

Table 1-2 HP XC System Interconnects

 

CP3000

CP4000

CP6000

Quadrics QSNet II®

 

X

X

 

 

 

 

Myrinet®

X

X

 

 

 

 

 

Gigabit Ethernet®

X

X

X

 

 

 

 

InfiniBand®

X

X

X

 

 

 

 

22 Overview of the User Environment