Index | HP XC System 3.x Software instruction

configuring local disk, 96 core availability, 38

D

DDT, 53 debugger

TotalView, 53 debugging

DDT, 53 gdb, 53 idb, 53 pgdbg, 53 TotalView, 53

debugging options setting, 39

debugging parallel applications, 53 debugging serial applications, 53 determining LSF execution host, 75 developing applications, 37 developing libraries, 43 developing parallel applications, 40 developing serial applications, 39

E

examining core availability, 38 external scheduler, 45

F

fault tolerance SLURM, 66

file system local disk, 96

Fortran, 26

building parallel applications, 41

G

gdb, 53 GNU, 26

modulefile, 32 GNU C/C++

building parallel applications, 41 GNU make

building parallel applications, 42, 93

H

head node, 19

historical information of jobs, 82 HP XC system (see system)

HP XC user commands (see commands) HP-MPI, 26, 40, 48, 59

compiler utilities, 42 using srun with, 64

I

I/O performance considerations, 96 idb, 53

Intel compilers, 37

Intel Fortran (see Fortran)

Intel Trace Collector/Analyzer, 59–60

interrupting jobs, 38 intranode communication, 97

J

job

examining status, 81

getting information about, 80 history, 82

interruption, 38 killing, 65 monitoring, 64 sending a signal to, 65 submission, 45

submission from non-HP XC host, 79 job accounting, 65

job allocation information obtaining, 80

job manager, 68 job scheduler, 68 JOBID translation, 83

L

launching jobs srun, 63

libraries, 26

building parallel applications, 42 library development, 43

Linux manpages manpages, 30

local disk configuring, 96

login node, 37 login procedure, 27

LSF HPC documentation, 15

lsf partition, 75 obtaining information, 76

LSF-HPC, 67–68bhist command, 83 bhosts command, 75 bjobs command, 81, 83 bqueues command, 76 bsub command, 77, 83

determining available system resources, 75 differences from standard LSF, 69 displaying host load information, 76 examining system queues, 76

execution host, 76

getting job information, 80 getting job status, 81 interaction with SLURM, 73 job control, 74

job preemption, 75 job startup, 74 job status, 84 JOBID, 83

lshosts command, 75 lsid command, 75 lsload command, 76

116 Index

Image 116

HP XC System 3.x Software manual Index

Contents

HP XC System Software Users Guide Page Table of Contents Submitting Jobs Configuring Your Environment with ModulefilesDeveloping Applications Tuning Applications Using SlurmUsing LSF Debugging Applications Glossary 109 Index 115 Advanced TopicsExamples List of Figures Page List of Tables Determining the Node PlatformPage List of Examples Submitting a Job ScriptPage About This Document Intended AudienceDocument Organization This document is organized as follows HP XC Information Supplementary Information $ man lsfcommandnameFor More Information Related Information Manpages$ man discover $ man 8 discover $ man -k keyword Related Linux Web Sites Related MPI Web SitesRelated Compiler Web Sites Additional Publications Typographic Conventions HP Encourages Your CommentsEnvironment Variable User input Overview of the User Environment System ArchitectureHP XC System Software Operating System Node Specialization Storage and I/O File System SAN StorageLocal Storage File System Layout Network Address Translation NAT Determining System Configuration InformationSystem Interconnect Network Modules CommandsUser Environment Application Development Environment Run-Time EnvironmentParallel Applications Serial Applications How LSF-HPC and Slurm Interact Load Sharing Facility LSF-HPCStandard LSF Mpirun command Components, Tools, Compilers, Libraries, and Debuggers Using the System LVS Login RoutingUsing the Secure Shell to Log Logging In to the System Getting Information About Resources IntroductionGetting Information About Queues Performing Other Common User Tasks Getting System Help and Information $ man sinfo Configuring Your Environment with Modulefiles Overview of Modules Supplied Modulefiles Modulefiles Automatically Loaded on the System Viewing Available ModulefilesViewing Loaded Modulefiles Loading a Modulefile Automatically Loading a Modulefile at Login Unloading a ModulefileModulefile Conflicts Loading a Modulefile for the Current Session Creating a Modulefile Viewing Modulefile-Specific Help$ module load modules $ man modulefile $ module help totalviewPage Compilers Developing ApplicationsApplication Development Environment Overview Examining Nodes and Partitions Before Running Jobs Interrupting a JobMPI Compiler Partition Avail Timelimit Nodes State Nodelist Setting Debugging Options Developing Serial ApplicationsSerial Application Build Environment Building Serial Applications Developing Parallel Applications Parallel Application Build EnvironmentModulefiles OpenMP Pthreads Quadrics ShmemMPI Library Intel Fortran and C/C++Compilers Building Parallel Applications Examples of Compiling and Linking HP-MPI Applications Developing LibrariesDesigning Libraries for the CP4000 Platform To build a 64-bit application, you might enter Linkcommand 32-bit -L/opt/mypackage/lib/i686 -lmystuffLinkcommand 64-bit -L/opt/mypackage/lib/x8664 -lmystuff ExtSLURMslurm-arguments Submitting JobsOverview of Job Submission Submitting a Serial Job Using Standard LSF Submitting a Serial Job Using LSF-HPCSubmitting a Serial Job with the LSF bsub Command $ bsub hostname Submitting a Serial Job Through Slurm only Submitting a Non-MPI Parallel Job $ bsub -n4 -I srun hostnameBsub -nnum-procsbsub-optionsmpijob Mpirun mpirun--options-srunsrun-optionsmpi-jobname Bsub -nnum-procs bsub-optionsscript-name Submitting a Batch Job or Job Script$ bsub -n4 -I mpirun -srun ./helloworld $ cat myscript.sh #!/bin/sh Srun hostname mpirun -srun hellompi$ bsub -I -n4 Myscript.sh $ bsub -n4 -ext SLURMnodes=4 -I ./myscript.sh $ cat ./envscript.sh #!/bin/sh name=`hostname` Running Preexecution Programs$ bsub -n4 -I ./myscript.sh Opt/hptc/bin/srun Mypreexec Debugging Applications Debugging Serial ApplicationsDebugging Parallel Applications TotalView Setting Up TotalView Using TotalView with SlurmSSH and TotalView Module load mpimodule load totalview Debugging an Application Using TotalView with LSF-HPCSetting TotalView Preferences Sourcefile initfdte.f was not found, using assembler mode Debugging Running ApplicationsDirectories in File ⇒ Search Path $ mpirun -srun -n2 Psimple $ squeue $ scancel --user usernameExiting TotalView Page Building a Program Intel Trace Collector and HP-MPI Tuning ApplicationsUsing the Intel Trace Collector and Intel Trace Analyzer Running a Program Intel Trace Collector and HP-MPI Visualizing Data Intel Trace Analyzer and HP-MPILibs Cldflags Using the Intel Trace Collector and Intel Trace Analyzer Page Using Slurm Launching Jobs with the srun CommandSrun Squeue Scancel Sinfo Scontrol Introduction to Slurm Monitoring Jobs with the squeue Command Using the srun Command with HP-MPIUsing the srun Command with LSF-HPC Srun Roles and Modes Job Accounting Terminating Jobs with the scancel CommandGetting System Information with the sinfo Command # chmod a+r /hptccluster/slurm/job/jobacct.log Fault ToleranceSecurity Using LSF-HPC Using LSFUsing Standard LSF on an HP XC System Introduction to LSF-HPC in the HP XC Environment Overview of LSF-HPC Differences Between LSF-HPC and Standard LSF HostnameResources Hostname Status JL/U MAX Njobs RUN Ssusp Ususp RSV Unknown Unknown Job Terminology$ ssh n15 lshosts HP XCCompute Node Resource Support SLURMnodelist =nodelist if specified $ bsub -n 10 -I srun hostname $ bsub -n 10 -ext SLURMnodes=10 -I srun hostname$ bsub -n 10 -ext SLURMnodes=10exclude=n16 -I srun hostname $ bsub -n 10 -ext SLURMconstraint=dualcore -I srun hostname How LSF-HPC and Slurm Launch and Manage a Job $ bsub -n4 -ext SLURMnodes=4 -o output.out ./myscript Job Startup and Job Control #!/bin/sh hostname srun hostname Mpirun -srun ./hellompi Determining the LSF Execution Host Determining Available LSF-HPC System ResourcesGetting the Status of LSF-HPC Getting Information About LSF Execution Host Node Getting Host Load Information Examining LSF-HPC System QueuesGetting Information About the lsf Partition SLINUX6 $ sinfo -p lsf -lNe Summary of the LSF bsub Command Format$ sinfo -p lsf LSF-SLURM External Scheduler For information about running scriptsBsub -n num-procs-ext SLURMslurm-arguments \ Bsub-options srun srun-optionsjobname job-options Submitting a Job from a Non-HP XC Host Starting on lsfhost.localdomain n6Waiting for dispatch ... Starting on lsfhost.localdomain n1 Type=SLINUX64 Getting Information About Jobs Getting Job Allocation InformationSlurmid=slurmjobidncpus=slurmnprocsslurmalloc=nodelist $ bjobs -l Examining the Status of a Job $ bhist -lTime stamp $ bjobs Viewing the Historical Information for a Job $ bhistSummary of time in seconds spent Various States Jobid User Jobname Pend Psusp RUN Ususp Ssusp Unkwn Total Translating Slurm and LSF-HPC JOBIDs Working Interactively Within an LSF-HPC Allocation $ bsub -I -n4 -ext SLURMnodes=4 /bin/bash$ bjobs -l 124 grep slurm $ srun --jobid=150 hostname $ unset Slurmjobid Alternatively, you can use the following$ export SLURMJOBID=150 $ export SLURMNPROCS=4 $ unset Slurmjobid $ unset Slurmnprocs Job 125 is submitted to the default queue normal LSF-HPC Equivalents of Slurm srun Options$ srun --jobid=250 uptime $ bsub -n4 -ext SLURMnodes=4 -o %J.out sleep Bsub -iinputfile Mpi=mpitype Quit-on-interrupt Page Advanced Topics Enabling Remote Execution with OpenSSHRunning an X Terminal Session from a Remote Node Determining IP Address of Your Local Machine Logging in to HP XC System Running an X terminal Session Using SlurmRunning an X terminal Session Using LSF-HPC Using the GNU Parallel Make Capability $ bsub -n4 -Ip srun -n1 xterm -display$ srun -n4 hostname n46 $ srun -n2 hostname n46 $ cd subdir srun -n1 -N1 $MAKE -j4 Example Procedure $ make PREFIX=’srun -n1 -N1 MAKEJ=-j4 Local Disks on Compute Nodes Performance ConsiderationsModified Makefile is invoked as follows $ make PREFIX=srun -n1 -N1 MAKEJ=-j4 Communication Between Nodes Shared File ViewPrivate File View Fp = fopen myfile, a+Page Appendix a Examples Building and Running a Serial ApplicationLaunching a Serial Interactive Shell Through LSF-HPC Examine the LSF execution host information Running LSF-HPC Jobs with a Slurm Allocation Request Launching a Parallel Interactive Shell Through LSF-HPC Example 2. Four cores on Two Specific NodesR15s r1m r15m It tmp swp mem LoadSched LoadStop SLURMnodes=2 124 Lsfad Examine the the running jobs information$ hostname n16 $ srun hostname n5 $ bjobs Submitting a Simple Job Script with LSF-HPC Examine the the finished jobs informationShow the environment Display the script Run some commands from the pseudo-terminal Submitting an Interactive Job with LSF-HPCSubmit the job Show the job allocation Submitting an HP-MPI Job with LSF-HPC Exit the pseudo-terminalView the interactive jobs View the finished jobs Lsfhost.localdomai View the running jobView the finished job $ bsub -n 8 -R ALPHA5 SLINUX64 \ -ext SLURMnodes=4-4 myjob Using a Resource Requirements String in an LSF-HPC CommandStates by date and time 108 Glossary To the queue First-come See Fcfs First-served Global storageAs local storage Are not appropriate for replication Single command Login requests and directs them to a node with a login roleLinux Virtual See LVS Server Load file LSF master host Network See NIS Information Services Remotely. PXE booting is configured at the Bios levelNotably to install and remove software packages Slurm backup Symmetric See SMP Multiprocessing Power available per unit of spaceSsh 114 Index Index PGI Utilities, 63 Slurm commands