HP XC System 4.x Software manuals

135 pages1.41 Mb

1Version
3Table of Contents
- About This Document
- 1 Overview of the User Environment
- 2 Using the System
- 43 Configuring Your Environment with Modulefiles
- 4 Developing Applications
- 5 Submitting Jobs
- 56 Debugging Applications
- 7 Monitoring Node Activity
- 8 Tuning Applications
- 9 Using SLURM
- 610 Using LSF
- 11 Advanced Topics
- A Examples
- 7Glossary
- Index
8List of Figures
9List of Tables
10List of Examples
11About This Document
- Intended Audience
- New and Changed Information in This Edition
- Typographic Conventions
- 12HP XC and Related HP Products Information
- 13Related Information
  - 14•http://www.platform.com
  - —Administering Platform LSF
  - —Administration Primer
  - —Platform LSF Reference
  - —Quick Reference Card
  - —Running Jobs with Platform LSF
  - lsf_diff
  - —HP XC System Software Administration Guide
  - —HP XC System Software User's Guide
  - •https://computing.llnl.gov/linux/slurm/documentation.html
  - •http://www.nagios.org
  - •http://oss.oetiker.ch/rrdtool
  - •http://supermon.sourceforge.net
  - •http://www.llnl.gov/linux/pdsh
  - pdsh
  - 15•http://www.balabit.com/products/syslog_ng
  - syslog-ng
  - syslog
  - •http://systemimager.org
  - •http://linuxvirtualserver.org
  - •http://www.macrovision.com
  - •http://sourceforge.net/projects/modules
  - module
  - •http://dev.mysql.com
  - MySQL Reference Manual
  - third-party
  - •http://www.redhat.com
  - •http://www.linux.org/docs/index.html
  - bash
  - •http://www.linuxheadquarters.com
  - •http://www.gnu.org
- 16Manpages
- 17HP Encourages Your Comments
191.1.1 HP XC System Software
201.1.4 Node Specialization
211.1.5 Storage and I/O
1.1.6 File System
221.1.7 System Interconnect Network
1.1.8 Network Address Translation (NAT)
231.2 Determining System Configuration Information
1.3 User Environment
- 1.3.1 LVS
- 1.3.2 Modules
- 1.3.3 Commands
241.4 Application Development Environment
- 1.4.1 Parallel Applications
- 1.4.2 Serial Applications
251.5 Run-TimeEnvironment
- 1.5.1 SLURM
- 1.5.2 Load Sharing Facility (LSF)
- 1.5.3 Standard LSF
- 261.5.4 How LSF and SLURM Interact
- 1.5.5 HP-MPI
271.6 Components, Tools, Compilers, Libraries, and Debuggers
292 Using the System
- 2.1Logging In to the System
  - 2.1.1 LVS Login Routing
  - 2.1.2 Using the Secure Shell to Log In
- 2.2 Overview of Launching and Managing Jobs
  - 302.2.1 Introduction
  - 2.2.2 Getting Information About Queues
  - 2.2.3 Getting Information About Resources
  - 2.2.4Getting Information About System Partitions
  - 2.2.5 Launching Jobs
  - 2.2.6 Getting Information About Your Jobs
  - 2.2.7 Stopping and Suspending Jobs
  - 2.2.8Resuming Suspended Jobs
- 312.3 Performing Other Common User Tasks
  - 2.3.1 Determining the LSF Cluster Name and the LSF Execution Host
- 322.4 Getting System Help and Information
333 Configuring Your Environment with Modulefiles
- 3.1Overview of Modules
- 343.2 Supplied Modulefiles
  - 35Table 3-1Supplied Modulefiles (continued)
  - “Viewing Available Modulefiles”
  - “Viewing Loaded Modulefiles”
- 363.3 Modulefiles Automatically Loaded on the System
- 3.4 Viewing Available Modulefiles
- 3.5 Viewing Loaded Modulefiles
- 3.6 Loading a Modulefile
  - 3.6.1 Loading a Modulefile for the Current Session
  - 3.6.2 Automatically Loading a Modulefile at Login
- 373.7 Unloading a Modulefile
- 3.8Viewing Modulefile-SpecificHelp
- 3.9 Modulefile Conflicts
- 383.10 Creating a Modulefile
394 Developing Applications
- 4.1Application Development Environment Overview
- 404.2 Compilers
  - 4.2.1 MPI Compiler
- 414.3 Examining Nodes and Partitions Before Running Jobs
- 4.4 Interrupting a Job
- 4.5 Setting Debugging Options
- 4.6 Developing Serial Applications
  - 4.6.1 Serial Application Build Environment
  - 4.6.2 Building Serial Applications
- 424.7 Developing Parallel Applications
  - 4.7.1Parallel Application Build Environment
    - 434.7.1.1 Modulefiles
    - 4.7.1.2 HP-MPI
    - 4.7.1.3 OpenMP
    - 4.7.1.4 Pthreads
    - -pthread
    - GNU -pthread
    - GNU
    - Intel -pthread
    - Intel
    - PGI -lpgthread
    - PGI
    - For example:
    - 4.7.1.5 Quadrics SHMEM
    - 44shmem.h
    - 4.7.1.6 MPI Library
    - MPI: A Message Passing Interface Standard
    - mpi.h
    - mpif.h
    - mpic++
    - mpif77
    - mpif90
    - PMPI_xxx()
    - mpi
    - 4.7.1.7 Intel Fortran and C/C++Compilers
    - 4.7.1.8 PGI Fortran and C/C++ Compilers
    - 4.7.1.9 GNU C and C++ Compilers
    - 4.7.1.10 Pathscale Compilers
    - http://www.pathscale.com/ekopath.html
    - 4.7.1.11 GNU Parallel Make
    - “Using the GNU Parallel Make Capability”
  - 454.7.2 Building Parallel Applications
- 464.8 Developing Libraries
  - 4.8.1 Designing Libraries for the CP4000 Platform
495 Submitting Jobs
- 5.1Overview of Job Submission
- 5.2 Submitting a Serial Job Using LSF
  - 5.2.1 Submitting a Serial Job with the LSF bsub Command
  - 505.2.2 Submitting a Serial Job Through SLURM Only
- 515.3 Submitting a Parallel Job
  - 5.3.1Submitting a Non-MPIParallel Job
  - 525.3.2 Submitting a Parallel Job That Uses the HP-MPIMessage Passing Interface
  - 535.3.3 Submitting a Parallel Job Using the SLURM External Scheduler
- 565.4 Submitting a Batch Job or Job Script
  - 57Example 5-16Submitting a Batch Job Script That Uses a Subset of the Allocation
  - Example 5-17Submitting a Batch job Script That Uses the srun --overcommitOption
  - SLURM Reference Manual
- 585.5 Submitting Multiple MPI Jobs Across the Same Set of Nodes
  - 5.5.1 Using a Script to Submit Multiple Jobs
  - 5.5.2 Using a Makefile to Submit Multiple Jobs
    - 59mymake
    - run1
    - run2
    - The following command line makes the program and executes it:
    - 601This line attempts to submit a program that does not exist
- 615.6 Submitting a Job from a Host Other Than an HP XC Host
- 5.7 Running Preexecution Programs
636 Debugging Applications
- 6.1Debugging Serial Applications
- 6.2 Debugging Parallel Applications
  - TotalView
  - DDT
  - 646.2.1 Debugging with TotalView
    - 656.2.1.3 Using TotalView with SLURM
    - These commands allocate
    - nodes and run TotalView to debug the program named
    - 6.2.1.4 Using TotalView with LSF
    - You may need to verify the full path name of the xterm and mpirun commands:
    - -tv
    - 6.2.1.5 Setting TotalView Preferences
    - Follow these steps to set the TotalView preferences:
    - 1.Invoke the TotalView debugger: $ totalview
    - TotalView's main control window (called the TotalView Root Window) appears
    - 2.Select Preferences from the File pull-downmenu. A Preferences window opens
    - 3.Select (that is, click on) the Launch Strings tab in the Preferences window
    - Ensure that the
    - button is selected in the
    - tab
    - In the
    - Command:
    - Defaults
    - 6.Select the Bulk Launch tab in the Preferences window
    - Make sure that Enable debug server bulk launch is not selected
    - Make sure that
    - selected
    - 8.Exit TotalView by selecting Exit from the File pull-downmenu
    - 666.2.1.6 Debugging an Application
    - This section describes how to use TotalView to debug an application
    - Compile the application to be debugged. For example:
    - Use the -g option to enable debugging information
    - 2.Run the application in TotalView:
    - The TotalView main control window, called the
    - TotalView root window
    - 4.The TotalView process window opens
    - opens
    - Step
    - Set the search path as follows:
    - a.Select the File pull-downmenu of the TotalView process window
    - b.Select Search Path from the list that appears
    - •Current working directory
    - •Directories in File ⇒ Search Path
    - Directories in
    - •Directories specified in your PATH environment variable
    - Yes
    - Select OK to close this pop-upwindow. You can safely ignore this warning
    - At Location
    - Action Point
    - b.Enter the name of the location where you want to set a breakpoint
    - c.Select OK
    - 9.Select the Go button to run the application and go to the breakpoint
    - Help
    - 676.2.1.7 Debugging Running Applications
    - “Debugging an Application”
    - Compile a
    - long-running
    - application as in
    - 2.Run the application:
    - Start TotalView:
    - Unattached
    - The TotalView Process Window appears, displaying information on the
    - Attached
    - Double-click
    - 7.Now you should be able set breakpoints to debug the application
    - 6.2.1.8Exiting TotalView
    - If it is still there, use the following command to remove all of your jobs:
697 Monitoring Node Activity
- 7.1The Xtools Utilities
- 707.2 Running Performance Health Tests
  - 72n11
  - n12
  - n14
  - n15
  - --keep
758 Tuning Applications
- 8.1 Using the Intel Trace Collector and Intel Trace Analyzer
  - 8.1.1Building a Program — Intel Trace Collector and HP-MPI
  - 768.1.2 Running a Program – Intel Trace Collector and HP-MPI
- 778.2 The Intel Trace Collector and Analyzer with HP-MPIon HP XC
  - 8.2.1 Installation Kit
  - 8.2.2 HP-MPIand the Intel Trace Collector
    - 78Running a Program
    - The following is a C example called vtjacobic:
    - The following is a Fortran example called vtjacobif:
- 798.3 Visualizing Data – Intel Trace Analyzer and HP-MPI
819 Using SLURM
- 9.1Introduction to SLURM
- 9.2 SLURM Utilities
- 9.3 Launching Jobs with the srun Command
  - 9.3.1 The srun Roles and Modes
  - 9.3.2 Using the srun Command with HP-MPI
  - 9.3.3 Using the srun Command with LSF
- 829.4 Monitoring Jobs with the squeue Command
- 839.5 Terminating Jobs with the scancel Command
- 9.6 Getting System Information with the sinfo Command
- 849.7 Job Accounting
- 9.8 Fault Tolerance
- 9.9 Security
8510 Using LSF
- 10.1Information for LSF
- 8610.2Overview of LSF Integrated with SLURM
  - 87describes the relationship between the
  - and the LSF JOBID
  - Example 10-1Examples of LSF Job Launch
- 8810.3 Differences Between LSF and LSF Integrated with SLURM
- 8910.4Job Terminology
  - 90batch system scheduling policies
  - Batch job submissions typically provide instructions on I/O
  - management, such as files from which to read input and
  - filenames to collect output
  - batch jobs are submitted with the srun -b command. By
  - default, the output is written to
  - batch job was launched
  - Use Ctrl-Cat any time to terminate the job
  - Interactive batch job
  - A job submitted to LSF or SLURM that maintains I/O
  - connections with the terminal from which the job was
  - scheduling policies, so it may pause before starting. After
  - can provide input if the job allows it
  - By default, SLURM jobs are interactive. Interactive LSF jobs
  - are submitted with the bsub -I command
  - Serial job
  - A job that requests only one slot and does not specify any of
  - the following constraints:
  - mincpus
  - nodes
  - Serial jobs are allocated a single CPU on a shared node with
  - minimal capacities that satisfies other allocation criteria. LSF
  - CPU per job. Parallel jobs and serial jobs cannot run on the
  - same node
  - Pseudo-paralleljob
  - A job that requests only one slot but specifies any of these
  - constraints:
  - nodes=1
  - mincpus
  - Pseudo-paralleljobs are allocated one node for their exclusive use
  - SLURM
  - nodes
- 9110.5 Using LSF Integrated with SLURM in the HP XC Environment
  - 10.5.1 Useful Commands
  - 10.5.2Job Startup and Job Control
  - 10.5.3 Preemption
- 10.6 Submitting Jobs
- 9210.7 LSF-SLURMExternal Scheduler
- 10.8 How LSF and SLURM Launch and Manage a Job
  - 93Figure 10-1How LSF and SLURM Launch and Manage a Job
  - User
  - 1.A user logs in to login node n16
  - The user executes the following LSF
  - command on login node
  - n16
  - -n4
  - -ext
  - "SLURM[nodes=4]
  - myscript
  - LSF schedules the job and monitors the state of the resources
  - jobID
- 9410.9 Determining the LSF Execution Host
- 10.10 Determining Available System Resources
  - 9510.10.1 Examining System Core Status
  - 10.10.2Getting Information About the LSF Execution Host Node
  - 10.10.3 Getting Host Load Information
  - 10.10.4 Examining System Queues
  - 10.10.5 Getting Information About the lsf Partition
- 9610.11 Getting Information About Jobs
- 10010.12 Translating SLURM and LSF JOBIDs
- 10110.13Working Interactively Within an Allocation
  - 102Section 5.3.2 (page 52)
  - Example 10-9Launching an Interactive MPI Job
  - Example 10-10 uses the -n8 option to launch on all cores in the allocation
  - Example 10-10Launching an Interactive MPI Job on All Cores in the Allocation
  - This example assumes 2 cores per node
  - Alternatively, you can use the following:
  - ssh X forwarding
- 10310.14 LSF Equivalents of SLURM srun Options
  - 104Table 10-3LSF Equivalents of SLURM srun Options (continued)
10711 Advanced Topics
- 11.1Enabling Remote Execution with OpenSSH
- 11.2 Running an X Terminal Session from a Remote Node
  - Step 1. Determining IP Address of Your Local Machine
  - 108Step 2. Logging in to HP XC System
  - Step 3. Running an X terminal Session Using SLURM
  - Step 4. Running an X terminal Session Using LSF
- 10911.3 Using the GNU Parallel Make Capability
  - 110One way is to prefix the actual compilation line in the rule with an
  - command. So, instead of executing
  - it would execute
  - . With concurrency, multiple command nodes would have multiple
  - commands instead of multiple
  - commands. For projects that recursively run
  - -N1
  - $(VARIABLE)
  - PREFIX
  - smg98
  - http://www.llnl.gov/asci/applications/SMG98README.html
  - The following parallel make approaches are described:
  - “Example Procedure 1”
  - Makefile_type1
  - “Example Procedure 2”
  - Makefile_type2
  - “Example Procedure 3”
  - Makefile_type3
  - The original Makefile is shown below:
  - 11111.3.1 Example Procedure
  - 11.3.2 Example Procedure
  - 11.3.3 Example Procedure
- 11211.4 Local Disks on Compute Nodes
- 11311.5 I/O Performance Considerations
  - 11.5.1Shared File View
  - 11.5.2 Private File View
- 11.6 Communication Between Nodes
- 11.7 Using MPICH on the HP XC System
  - 11411.7.1 Using MPICH with SLURM Allocation
  - 11.7.2Using MPICH with LSF Allocation
115A Examples
- A.1 Building and Running a Serial Application
- A.2 Launching a Serial Interactive Shell Through LSF
  - Examine the LSF execution host information:
  - Examine the partition information:
  - Examine the local host information:
  - Examine the job information:
  - Run the LSF bsub -Is command to launch the interactive shell:
  - Note the output when hostname is again issued:
  - Note the FROM_HOST when bjobs is again issued:
  - Note the output when srun hostname is issued:
  - Exit from the shell:
- 116A.3 Running LSF Jobs with a SLURM Allocation Request
  - A.3.1 Example 1. Two Cores on Any Two Nodes
  - A.3.2 Example 2. Four Cores on Two Specific Nodes
- 117A.4 Launching a Parallel Interactive Shell Through LSF
  - 118Note the output from the bjobs command:
  - Examine the running job's information:
  - Examine the finished job's information:
- 119A.5 Submitting a Simple Job Script with LSF
  - Show the environment:
  - Display the script:
  - Run the job:
- 120A.6 Submitting an Interactive Job with LSF
  - Submit the job:
  - Show the job allocation:
  - 121Show the SLURM job ID:
  - Run some commands from the pseudo-terminal:
  - Exit the pseudo-terminal:
  - View the interactive jobs:
  - View the finished jobs:
  - View the node state:
- 122A.7 Submitting an HP-MPIJob with LSF
  - View the running job:
  - View the finished job:
- 123A.8 Using a Resource Requirements String in an LSF Command
125Glossary
131Index

2 pages71.37 Kb