HP XC System 3.x Software 116

configuring local disk, 96 core availability, 38

D

DDT, 53 debugger

TotalView, 53 debugging

DDT, 53 gdb, 53 idb, 53 pgdbg, 53 TotalView, 53

debugging options setting, 39

debugging parallel applications, 53 debugging serial applications, 53 determining LSF execution host, 75 developing applications, 37 developing libraries, 43 developing parallel applications, 40 developing serial applications, 39

E

examining core availability, 38 external scheduler, 45

F

fault tolerance SLURM, 66

file system local disk, 96

Fortran, 26

building parallel applications, 41

G

gdb, 53 GNU, 26

modulefile, 32 GNU C/C++

building parallel applications, 41 GNU make

building parallel applications, 42, 93

H

head node, 19

historical information of jobs, 82 HP XC system (see system)

HP XC user commands (see commands) HP-MPI, 26, 40, 48, 59

compiler utilities, 42 using srun with, 64

I

I/O performance considerations, 96 idb, 53

Intel compilers, 37

Intel Fortran (see Fortran)

Intel Trace Collector/Analyzer, 59–60

interrupting jobs, 38 intranode communication, 97

J

job

examining status, 81

getting information about, 80 history, 82

interruption, 38 killing, 65 monitoring, 64 sending a signal to, 65 submission, 45

submission from non-HP XC host, 79 job accounting, 65

job allocation information obtaining, 80

job manager, 68 job scheduler, 68 JOBID translation, 83

L

launching jobs srun, 63

libraries, 26

building parallel applications, 42 library development, 43

Linux manpages manpages, 30

local disk configuring, 96

login node, 37 login procedure, 27

LSF HPC documentation, 15

lsf partition, 75 obtaining information, 76

LSF-HPC, 67–68bhist command, 83 bhosts command, 75 bjobs command, 81, 83 bqueues command, 76 bsub command, 77, 83

determining available system resources, 75 differences from standard LSF, 69 displaying host load information, 76 examining system queues, 76

execution host, 76

getting job information, 80 getting job status, 81 interaction with SLURM, 73 job control, 74

job preemption, 75 job startup, 74 job status, 84 JOBID, 83

lshosts command, 75 lsid command, 75 lsload command, 76

116 Index

Contents

Version Page Table of Contents 3 Configuring Your Environment with Modulefiles 4 Developing Applications 5 Submitting Jobs 6 Debugging Applications 7 Tuning Applications 8 Using SLURM 9 Using LSF 10 Advanced Topics List of Figures Page List of Tables Page List of Examples Page About This Document Intended Audience Document Organization HP XC Information For More Information Supplementary Information man Related Information Related Linux Web Sites •http://www.redhat.com •http://www.linux.org/docs/index.html •http://www.linuxheadquarters.com •http://linuxvirtualserver.org Typographic Conventions HP Encourages Your Comments 1 Overview of the User Environment System Architecture Table 1-1Determining the Node Platform processor vendor_id Node Specialization head node of the HP XC system client nodes such as logging into the system or running jobs File System implemented as ioctls File System Layout •HP XC-specificsoftware is located in /opt/hptc Determining System Configuration Information User Environment Application Development Environment Run-TimeEnvironment Load Sharing Facility (LSF-HPC) Standard LSF How LSF-HPCand SLURM Interact -srun JOB_STARTER Components, Tools, Compilers, Libraries, and Debuggers 2 Using the System Logging In to the System Overview of Launching and Managing Jobs Introduction Environment (page 24) JobID How and SLURM Launch and Manage a Job (page 73) Performing Other Common User Tasks Getting System Help and Information 3 Configuring Your Environment with Modulefiles Overview of Modules Supplied Modulefiles Modulefiles Automatically Loaded on the System Viewing Available Modulefiles Viewing Loaded Modulefiles Loading a Modulefile Unloading a Modulefile Modulefile Conflicts Creating a Modulefile Viewing Modulefile-SpecificHelp Page 4 Developing Applications Application Development Environment Overview Compilers Examining Nodes and Partitions Before Running Jobs Interrupting a Job Setting Debugging Options Developing Serial Applications Developing Parallel Applications Pthreads $ mpicc object1.o ... -pthread -omyapp.exe Quadrics SHMEM $ gcc -oshping shping.c -lshmem -lelan MPI Library Building Parallel Applications Compiling and Linking Non-MPIApplications Compiling and Linking HP-MPIApplications Developing Libraries <linkcommand> <32-bit> -L/opt/mypackage/lib/i686 -lmystuff <linkcommand> <64-bit> -L/opt/mypackage/lib/x86_64 -lmystuff 5 Submitting Jobs Overview of Job Submission Submitting a Serial Job Using Standard LSF Submitting a Serial Job Using LSF-HPC $ bsub -Ihostname Submitting a Serial Job Through SLURM only $ cc hw_hostname.c -ohw_hostname $ ./hw_hostname $ srun ./hw_hostname $ srun -n4 ./hw_hostname Submitting a Non-MPIParallel Job Submitting a Parallel Job That Uses the HP-MPIMessage Passing Interface Submitting a Batch Job or Job Script $ cat myscript.sh #!/bin/sh $ bsub -I -n4 myscript.sh $ bsub -n4 -ext"SLURM[nodes=4]" -I ./myscript.sh Running Preexecution Programs /opt/hptc/bin/srun -N1 my_pre_exec my_pre_exec 6 Debugging Applications Debugging Serial Applications Debugging Parallel Applications SSH and TotalView Setting Up TotalView module load mpimodule load totalview Using TotalView with SLURM $ srun -Nx -A Using TotalView with LSF-HPC $ bsub -nx -ext"SLURM[nodes=x]" \ -Is /usr/bin/xterm Setting TotalView Preferences $ totalview %C %R -n"%B/tvdsvr -working_directory%D -callback%L -set_pw%P -verbosity%V %F Debugging Running Applications $ mpirun -srun -n2Psimple Exiting TotalView $ squeue $ scancel --user username Page 7 Tuning Applications Using the Intel Trace Collector and Intel Trace Analyzer Running a Program – Intel Trace Collector and HP-MPI Visualizing Data – Intel Trace Analyzer and HP-MPI Page Page 8 Using SLURM Introduction to SLURM SLURM Utilities Launching Jobs with the srun Command Monitoring Jobs with the squeue Command Terminating Jobs with the scancel Command Getting System Information with the sinfo Command Job Accounting Fault Tolerance Security 9 Using LSF Using Standard LSF on an HP XC System Using LSF-HPC •Introduction to LSF-HPCin the HP XC Environment (page 68) •Determining the LSF Execution Host (page 75) Introduction to LSF-HPCin the HP XC Environment Overview of LSF-HPC •Request contiguous nodes •Execute only one task per node •Request nodes with specific features Differences Between LSF-HPCand Standard LSF "Notes About Using in the HP XC Environment By LSF standards, the HP XC system is a single host. Therefore, all LSF LSF-SLURM n15 UNKNOWN server 2007M $ ssh n15 lshosts HP XCCompute Node Resource Support $ bsub -n10 -Isrun hostname $ bsub -n10 -ext"SLURM[nodes=10]" -Isrun hostname $ bsub -n10 -ext"SLURM[nodes=10;exclude=n16]" -Isrun hostname $ bsub -n10 -ext"SLURM[constraint=dualcore]" -Isrun hostname $ bsub -n10 -ext "SLURM[nodelist=n[1-10]]"srun hostname How LSF-HPCand SLURM Launch and Manage a Job User $ bsub -n4 -ext"SLURM[nodes=4]" -ooutput.out ./myscript Job Startup and Job Control Preemption The following example shows the output from the lshosts command: lsfhost.loc SLINUX6 Itanium2 3456M Submitting Jobs Chapter 5. Submitting Jobs The basic synopsis of the bsub command is: bsub [bsub-options]jobname jobname [job-options] jobname about running jobs. Refer to "Submitting a Batch Job or Job Script for information about running scripts bsub -n num-procs [bsub-options]srun [srun-options] jobname [job-arguments] "Submitting a Non-MPIParallel Job Job <70> is submitted to default queue <normal>. <<Waiting for dispatch <<Starting on lsfhost.localdomain>> n6 Example 9-2 shows one way to submit a parallel job to run one task per node Job <71> is submitted to default queue <normal>. <<Waiting for dispatch Job <72> is submitted to default queue <normal Job Allocation Information for a Running Job Job Allocation Information for a Finished Job Example 9-5Using the bjobs Command (Long Output) Job <24>, User <msmith>,Project <default>,Status <RUN date and time stamp: Submitted from host <n16>, CWD <$HOME Viewing the Historical Information for a Job 9-6Using the bhist Command (Short Output) Translating SLURM and LSF-HPCJOBIDs $ bsub -o%J.out -n8 sleep $ bjobs -l99 | grep slurm $ bhist -l99 | grep slurm $ scontrol show job | grep Name Working Interactively Within an LSF-HPCAllocation $ bsub -I -n4 -ext"SLURM[nodes=4]" /bin/bash <<Starting on lsfhost.localdomain <<Waiting for dispatch $ bjobs -l124 | grep slurm $ unset SLURM_JOBID Section (page 48) Example 9-8Launching an Interactive MPI Job Example 9-9Launching an Interactive MPI Job on All Cores in the Allocation This example assumes 2 cores per node LSF-HPCEquivalents of SLURM srun Options Table 9-2 LSF-HPCEquivalents of SLURM srun Options Page Page Page 10 Advanced Topics Enabling Remote Execution with OpenSSH Running an X Terminal Session from a Remote Node $ hostname $ host mymachine Step 2. Logging in to HP XC System Step 3. Running an X terminal Session Using SLURM $ srun -N1xterm -display14.26.206.134:0.0 Step 4. Running an X terminal Session Using LSF-HPC Using the GNU Parallel Make Capability $ cd subdir; srun -n1 -N1$(MAKE) -j4 $ make PREFIX=’srun –n1 –N1 MAKE_J='-j4 Local Disks on Compute Nodes I/O Performance Considerations Communication Between Nodes Page Appendix A Examples Building and Running a Serial Application Launching a Serial Interactive Shell Through LSF-HPC Running LSF-HPCJobs with a SLURM Allocation Request Launching a Parallel Interactive Shell Through LSF-HPC Note the output from the bjobs command: Examine the the running job's information: $ bhist -l124 Submitting a Simple Job Script with LSF-HPC Submitting an Interactive Job with LSF-HPC Submitting an HP-MPIJob with LSF-HPC View the running job: $ bjobs -l1009 View the finished job: $ bhist -l1009 Using a Resource Requirements String in an LSF-HPCCommand Page Glossary FCFS to the queue first-come See FCFS first-served Linux Virtual See LVS Server load file single command Network See NIS Information Services NIS SMP power) available per unit of space ssh network standard LSF Page Index