$ ovp -l | HP XC System 4.x Software instruction

NOTE: Except for the network_stress and network_bidirectional tests, these tests only apply to systems that install LSF incorporated with SLURM. The network_stress and network_bidirectional tests also function under Standard LSF.

You can list the available tests with the ovp -lcommand:

$ ovp -l

Test list for perf_health: cpu_usage memory_usage

cpu memory network_stress network_bidirectional network_unidirectional

By default, the ovp command reports if the nodes passed or failed the given test. Use the ovp --verboseoption to display additional information.

The results of the test are written to a file in your home directory. The file name has the form ovp_node_date[rx].log where node is the node from which the command was launched and date is a date stamp in the form mmddyy. Subsequent test runs of the test are identified with an r (for run) and a number; this prevents a log file from overwriting a previous one. The --chdir=option enables you to designate a directory different from your home directory as the execution directory.

The following is an example of the CPU usage test:

$ ovp --verify=perf_health/cpu_usage

XC CLUSTER VERIFICATION PROCEDURE date time

Verify perf_health:

Testing cpu_usage ...

+++ PASSED +++

This verification has completed successfully.

A total of 1 test was run.

Details of this verification have been recorded in:

HOME_DIRECTORY/ovp_n16_mmddyy.log

The following example runs the same test but with the --verboseoption to show additional output.

$ ovp --verbose --verify=perf_health/cpu_usage

XC CLUSTER VERIFICATION PROCEDURE date time

7.2 Running Performance Health Tests

71

Image 71

HP XC System 4.x Software manual You can list the available tests with the ovp -lcommand, $ ovp -l

Contents

HP XC System Software XC User Guide Page Table of Contents Compiling and Linking Serial Applications Configuring Your Environment with Modulefiles Srun Roles Srun Modes Debugging Applications 115 107 131 125 11-1 List of Figures10-1 List of Tables List of Examples Typographic Conventions About This DocumentIntended Audience New and Changed Information in This Edition HP XC and Related HP Products Information HP XC System Software Users Guide Related Information Https//computing.llnl.gov/linux/slurm/documentation.html Http//systemimager.org Http//www-unix.mcs.anl.gov/mpi Manpages $ man -k keyword HP Encourages Your Comments$ man discover $ man 8 discover Page Page /proc/cpuinfo file is dynamic Only to the administrator of the HP XC systemNode Specialization Determining the Node Platform Local Storage Storage and I/OFile System SAN Storage HP XC System Interconnects System Interconnect NetworkFile System Layout 1 LVS Determining System Configuration InformationUser Environment Network Address Translation NAT Parallel Applications CommandsApplication Development Environment Load Sharing Facility LSF Run-Time EnvironmentSerial Applications Slurm Nodes for the job Requested by the HP-MPI mpirun commandHow LSF and Slurm Interact HP-MPI Components, Tools, Compilers, Libraries, and Debuggers Page Logging In to the System Using the SystemLVS Login Routing Using the Secure Shell to Log $ bqueues IntroductionGetting Information About Queues Getting Information About Resources Performing Other Common User Tasks $ man sinfo Getting System Help and InformationDetermining the LSF Cluster Name and the LSF Execution Host $ lsid Overview of Modules Configuring Your Environment with Modulefiles Supplied Modulefiles Supplied Modulefiles HP-MPI Loading a Modulefile Modulefiles Automatically Loaded on the SystemViewing Available Modulefiles Viewing Loaded Modulefiles Modulefile Conflicts Unloading a ModulefileViewing Modulefile-Specific Help $ module load modules $ man modulefile Creating a Modulefile Application Development Environment Overview Developing Applications MPI Compiler Compiler CommandsCompilers Developing Serial Applications Setting Debugging OptionsExamining Nodes and Partitions Before Running Jobs Interrupting a Job Parallel Application Build Environment Developing Parallel ApplicationsSerial Application Build Environment Building Serial Applications Pthreads ModulefilesHP-MPI OpenMP GNU C and C++ Compilers MPI LibraryIntel Fortran and C/C++Compilers PGI Fortran and C/C++ Compilers Building Parallel Applications Designing Libraries for the CP4000 Platform To compile and link a C application using the mpicc commandDeveloping Libraries To build a 64-bit application, you might enter Linkcommand 32-bit -L/opt/mypackage/lib/i686 -lmystuffLinkcommand 64-bit -L/opt/mypackage/lib/x8664 -lmystuff Developing Applications Overview of Job Submission Submitting a Serial Job Using LSFSubmitting a Serial Job with the LSF bsub Command Submitting Jobs $ bsub -I srun hostname Example 5-2 Submitting a Serial Job Using LSFSubmitting a Serial Job Through Slurm Only Example 5-1 Submitting a Job from the Standard Input To submit a parallel job Following is the command line used to compile this programSubmitting a Parallel Job Submitting a Non-MPI Parallel Job $ bsub -n4 -I srun hostname Example 5-5 Submitting a Non-MPI Parallel Job $ bsub -n4 -I mpirun -srun ./helloworld Example 5-7 Submitting an MPI Job Nodelist=list of nodes Arguments for the Slurm External Scheduler $ bsub -n 10 -ext SLURMnodelist=n1-10 srun hostname $ bqueues -l dualcore grep Slurm Submitting a Batch Job or Job ScriptExample 5-14 Submitting a Job Script $ bsub -n 10 -ext SLURMconstraint=dualcore -I srun hostname $ bsub -n4 -I ./myscript.sh $ bsub -n4 -ext SLURMnodes=4 -I ./myscript.sh Submitting Multiple MPI Jobs Across the Same Set of Nodes Using a Script to Submit Multiple JobsUsing a Makefile to Submit Multiple Jobs Use the squeue command to acquire information on the jobs $ tail 113.out Following command line makes the program and executes it$ cat mymake $ cat 117.out $ bsub -R type=SLINUX64 -n4 -I srun hostname Submitting a Job from a Host Other Than an HP XC HostRunning Preexecution Programs Type=SLINUX64Page TotalView Debugging ApplicationsDebugging Serial Applications Debugging Parallel Applications Module load mpi module load totalview Setting Up TotalViewDebugging with TotalView SSH and TotalView $ srun -Nx-A $ mpirun -tv -srun application Using TotalView with SlurmUsing TotalView with LSF Setting TotalView Preferences $ mpirun -tv -srun -n2 ./Psimple Debugging an Application Run the application $ scancel --user usernameDebugging Running Applications Exiting TotalViewPage Xtools Utilities Monitoring Node Activity Where Running Performance Health Tests $ ovp --verbose --verify=perfhealth/cpuusage You can list the available tests with the ovp -lcommand$ ovp -l $ ovp --verify=perfhealth/cpuusage HOMEDIRECTORY/ovpn16mmddyy.log HOMEDIRECTORY/ovpn16mmddyyr1.log Page Building a Program Intel Trace Collector and HP-MPI Tuning ApplicationsUsing the Intel Trace Collector and Intel Trace Analyzer Example 8-2 C Example Running the vtjacobic Example Program Running a Program Intel Trace Collector and HP-MPIExample 8-1 The vtjacobic Example Program For more information, see the following Web site HP-MPI and the Intel Trace Collector Installation KitIntel Trace Collector and Analyzer with HP-MPI on HP XC Following is a Fortran example called vtjacobif Running a Program # bsub -n4 -I mpirun.mpich -np 2 ./vtjacobic Running a Program Across Nodes Using LSFVisualizing Data Intel Trace Analyzer and HP-MPI Page Slurm Utilities Using SlurmLaunching Jobs with the srun Command Introduction to Slurm Srun Roles and Modes Monitoring Jobs with the squeue CommandUsing the srun Command with HP-MPI Using the srun Command with LSF Example 9-5cancels all pending jobs Terminating Jobs with the scancel CommandGetting System Information with the sinfo Command $ sinfo -R Fault ToleranceJob Accounting Security Information for LSF Using LSF $ squeue --jobs $SLURMJOBID Overview of LSF Integrated with Slurm Example 10-1 Examples of LSF Job Launch Lsfadmin@n16 ~$ bsub -n4 -I srun hostname Differences Between LSF and LSF Integrated with Slurm Lshosts and lsload commands display for each of these items $ ssh n15 lshosts Job TerminologyConnection back to the terminal from which the job was Submitted. This job may run immediately, or it may run Batch system scheduling policies Batch jobs are submitted with the srun -bcommand. ByDefault, the output is written to Are submitted with the bsub -Icommand Job Startup and Job Control Using LSF Integrated with Slurm in the HP XC EnvironmentUseful Commands Submitting Jobs LSF with Slurm Job Launch Exit Codes LSF-SLURM External SchedulerHow LSF and Slurm Launch and Manage a Job How LSF and Slurm Launch and Manage a Job User logs in to login node n16 Determining Available System Resources Determining the LSF Execution Host Getting Information About the LSF Execution Host Node Following example shows the output from the lshosts commandExamining System Core Status Getting Information About the lsf Partition Getting Information About JobsGetting Host Load Information Examining System Queues Than what the job requests Getting Job Allocation InformationThis allocation string has the following values LSF job with Slurm allocated resources Example 10-4 Job Allocation Information for a Finished Job Example 10-5 Using the bjobs Command Short OutputExamining the Status of a Job Viewing the Historical Information for a Job Example 10-6 Using the bjobs Command Long OutputExample 10-7 Using the bhist Command Short Output Output Provided by the bhist Command Translating Slurm and LSF JOBIDs Example 10-8 Using the bhist Command Long OutputUse the bjobs command to view the Slurm Jobid $ bjobs -l 124 grep slurm Working Interactively Within an Allocation$ sacct -j $ bsub -I -n4 -ext SLURMnodes=4 /bin/bash Alternatively, you can use the following Example 10-9 Launching an Interactive MPI JobThis example assumes 2 cores per node $ unset Slurmjobid $ unset Slurmnprocs LSF Equivalents of Slurm srun Options3describes the srun options and lists their LSF equivalents LSF Equivalents of Slurm srun Options Requests a specific list of nodes. Slurm Suppress informational message Determining IP Address of Your Local Machine Advanced TopicsEnabling Remote Execution with OpenSSH Running an X Terminal Session from a Remote Node Logging in to HP XC System Running an X terminal Session Using SlurmRunning an X terminal Session Using LSF Monitors display server address Using the GNU Parallel Make CapabilityOptions used in this command are Allocate 4 cores Run the job on 1 core $ cd subdir srun -n1 -N1 $MAKE -j4 $ cd subdir srun $MAKE $ make PREFIX=’srun -n1 -N1 MAKEJ=-j4 Example Procedure $ make PREFIX=srun -n1 -N1 MAKEJ=-j4 Local Disks on Compute NodesModified Makefile is invoked as follows Shared File View Using Mpich on the HP XC System11.5 I/O Performance Considerations Communication Between Nodes Bsub command launches the wrapper script Using Mpich with Slurm AllocationUsing Mpich with LSF Allocation Examine the LSF execution host information ExamplesBuilding and Running a Serial Application Launching a Serial Interactive Shell Through LSF Running LSF Jobs with a Slurm Allocation Request Example 2. Four Cores on Two Specific Nodes Launching a Parallel Interactive Shell Through LSF $ hostname n16 $ srun hostname n5 $ bjobs Examine the running jobs information Display the script Submitting a Simple Job Script with LSFExamine the finished jobs information Show the environment Show the job allocation Submitting an Interactive Job with LSFRun the job Submit the job View the interactive jobs Run some commands from the pseudo-terminalShow the Slurm job ID Exit the pseudo-terminal View the running job Submitting an HP-MPI Job with LSFView the node state $ bsub -n 8 -R ALPHA5 SLINUX64 \ -ext SLURMnodes=4-4 myjob Using a Resource Requirements String in an LSF CommandView the finished job 124 Network Availability set Glossary Fcfs Ipmi LVS PXE SVA Index Index LVS PGI 135