Manuals / Q-Logic / Computer Equipment / Switch

Q-Logic IB6054601-00 D manual Introduction Documentation and Technical Support

Models: IB6054601-00 D

1 122

Download 122 pages 48.66 Kb

Page 17

Q

1 – Introduction Documentation and Technical Support

■Readme file

The Troubleshooting Appendix for installation, InfiniPath and OpenFabrics administration, and MPI issues is located in the InfiniPath User Guide.

Visit the QLogic support Web site for documentation and the latest software updates.

http://www.qlogic.com

IB6054601-00 D

1-7

Image 17

Q-Logic IB6054601-00 D manual Introduction Documentation and Technical Support

Contents

InfiniPath User Guide InfiniPath User Guide Version Index Page Table of Contents Section Using InfiniPath MPI Appendix a Benchmark Programs InfiniPath User Guide Version InfiniPath Software Structure Appendix D Recommended ReadingInfiniPath User Guide Version IB6054601-00 D How this Guide is Organized Who Should Read this GuideOverview SwitchesInteroperability Appendix E Glossary of technical termsPathScale-QLogic Adapter Model Numbers What’s New in this ReleaseInfiniPath/OpenFabrics Supported Distributions and Kernels Supported Distributions and KernelsSoftware Components Documentation and Technical Support Conventions Used in this DocumentThis Guide uses these typographical conventions InfiniPath product documentation includesIntroduction Documentation and Technical Support Page Introduction Installed LayoutDocumentation can be found Memory FootprintMPI include files are Adapter Required Component Optional Memory Footprint Comment Memory Footprint, 331 MB per Node Configuration and Startup Bios SettingsInfiniPath Driver Filesystem InfiniPath Driver Software ConfigurationInfiniPath Driver Startup Layered Ethernet Driver Subnet Management AgentCreate or edit the following file as root Ipathether Configuration on Fedora and RHEL4$ ifconfig -a $ ls /sys/class/net ONBOOT=YES Ipathether Configuration on Suse 9.3, Sles 9, and SlesGuid can also be returned by running Determine the MAC address that will be usedIf this file does not exist, skip to Step # lsmod grep -q ipathether modprobe ipathetherAdd the following lines to the file Configuring the IPoIB Network Interface OpenFabrics Configuration and Startup# ifup eth2 # ifconfig eth2 # ifconfig ib0 10.1.17.3 netmask 0xffffff00OpenSM To enable the driver, use the command as root Further Information on Configuring and Loading DriversStarting and Stopping the InfiniPath Software You can stop it again like this# /sbin/chkconfig --list opensmd grep -w on # chkconfig infinipath off# /etc/init.d/infinipath start stop restart $ /sbin/lsmod grep ipathetherSoftware Status Configuring ssh and sshd Using shosts.equivChange the file to mode 600 when finished editing Process Limitation with ssh Remove Unneeded ServicesPerformance and Management Tips # killall -HUP sshdDisable Powersaving Features For Suse 9.3 and 10.0 run this command as root Balanced Processor PowerSDP Module Parameters for Best Performance CPU AffinityHomogeneous Nodes Hyper-ThreadingRun the shell script ipathcontrol as follows IpathbughelperFor non-QLogic RPMs, it will look like Option -qwill query and -qawill query allNon-QLogic built Will produce output like this Command strings can also be used. Here is a sampleCustomer Acceptance Utility Displays help messages giving defined usage Page Getting Started with MPI InfiniPath MPIOther MPI Implementations There is more information on the mpihosts file in section An Example C Program$ cp /usr/share/mpich/examples/basic $ mpicc -o cpi cpi.cRun it with Examples Using Other Languages$ mpif77 -o fpi3 fpi3.f $ mpirun -np 2 -m mpihosts ./fpi3Run it Configuring MPI Programs for InfiniPath MPIInfiniPath MPI Details Configuring for ssh Using ssh-agentThis tells ssh that your key pair should let you Edit ~/.ssh/config so that it reads like thisShows how to compile a program Gets verbose output of all the commands in the scriptCompiling and Linking Provides helpTo use gcc for compiling and linking C++ programs use To Use Another CompilerTo use gcc for compiling and linking Fortran77 programs use To use PGI for Fortran90/Fortran95 programs, useCompiler and Linker Variables Cross-compilation Issues$ mpif90 -f90=ifort $ mpif95 -f95=ifort $ mpif90 -f90=pgf90 -show pi3f90.f90 -o pi3f90Running MPI Programs Mpihosts File Environment for Node Programs Console I/O in MPI Programs$ mpirun -np n -m mpihosts -rcfile mpirunrc program Environment for Multiple Versions of InfiniPath or MPINumber of processes to spawn Multiprocessor NodesCreate up to specified number of processes per node $ mpirun -np n -m mpihosts -ppn p program-namePrint mpirun help message. Default Off Print MPI version. Default OffStartup script for setting environment on nodes Default Run each process in an xterm window. Default OffSets the working directory for the node program Disable-mpi-progress-check# modprobe rdmacm # modprobe rdmaucm Using Other MPI ImplementationsMPI Over uDAPL $ mpdboot -f hostsfile Using MPDMPD Description File I/O in MPIInfiniPath MPI and Hybrid MPI/OpenMP Applications MPI-IO with RomioDebugging MPI Programs MPI ErrorsUsing Debuggers InfiniPath MPI Limitations No ports available on /dev/ipath Benchmark 1 Measuring MPI Latency Between Two Nodes Benchmark Programs$ mpirun -np 2 -ppn 1 -m mpihosts osulatency Benchmark 2 Measuring MPI Bandwidth Between Two NodesBenchmark 3 Messaging Rate Microbenchmarks $ mpirun -np 8 ./mpimultibw $ mpirun -np 4 -ppn 1 -m mpihosts mpilatency 100 Benchmark 4 Measuring MPI Latency in Host RingsMight produce output like this Page Allocating Resources Batch Queuing Script$ batchmpirun -np n mympiprogram Generating the mpihosts FileSimple Process Management Clean Termination of MPI Processes# /sbin/fuser -v /dev/ipath # lsof /dev/ipath# ulimit -l Lock Enough Memory on Nodes When Using Slurm# /sbin/fuser -k /dev/ipath Mechanical and Electrical Considerations Troubleshooting InfiniPath Adapter InstallationSome HTX Motherboards May Need 2 or More CPUs in Use Table C-1. LED Link and Data Indicators$ ipathpkttest -B Mtrr Mapping and Write CombiningIncorrect Mtrr Mapping Issue with SuperMicro H8DCE-HTe and QHT7040 Change Setting for Mapping MemoryIncorrect Mtrr Mapping Causes Unexpected Low Bandwidth To check your bandwidth tryInstall Warning with RHEL4U2 Software Installation IssuesMpirun Installation Requires 32-bit Support OpenFabrics DependenciesRemove the InfiniPath kernel components with the command Installing Newer Drivers from Other DistributionsCheck your distribution for the exact RPM name $ rpm -e infinipath-kernel --nodepsKernel and Initialization Issues Installing for Your DistributionReload all modules by using this command as root These are used for SLES, SUSE, and FedoraPcimsiquirk Kernel Needs CONFIGPCIMSI=yInfiniPath Interrupts Not Working Driver Load Fails Due to Unsupported KernelNormal output will like similar to this $ grep ibipath /proc/interrupts$ grep -i acpi /proc/cmdline OpenFabrics Load Errors If ibipath Driver Load FailsInfiniPath ibipath Initialization Failure MPI Job Failures Due to Initialization ProblemsLoad and Configure IPoIB Before Loading SDP OpenFabrics IssuesSystem Administration Troubleshooting Stop OpenSM Before Stopping/Restarting InfiniPathMvapich Performance Issues Performance IssuesInfiniPath MPI Troubleshooting Broken Intermediate Link$ mpirun-ipath-ssh -np 2 -ppn 1 -m ~/tmp/idev osulatency $ mpirunCompiler Can’t Find Include, Module or Library Files Compiler/Linker Mismatch$ export MPICHCC=gcc $ mpicc mpiworld.c $ mpicc myprogram.cSpecifying the Run-time Library Path Compiling on Development Nodes$ mpicc myprogram.c -I/path/to/devel/include $ mpif90 myprogramf90.f90 -I/path/to/devel/include# /etc/ldconfig Run Time Errors With Different MPI ImplementationsTroubleshooting InfiniPath MPI Troubleshooting HP-MPI mpirun and executable used together Using MPI.mod FilesWhen compiling, use descriptive names for the object files Extending MPI Modules Troubleshooting InfiniPath MPI Troubleshooting $ mpirun -m ~/tmp/sm -np 2 -mpilatency 1000 Error Messages Generated by mpirunMessages from the InfiniPath Library Troubleshooting InfiniPath MPI Troubleshooting MPI Messages $ mpirun -np 2 -m ~/tmp/q mpilatency 100 Following indicates an unknown hostThere is no route to a valid host $ mpirun -np 2 -m ~/tmp/q mpilatency 100000 There is no route to any hostOne program on one node died $ mpirun -np ~/tmp/q -q 60 mpilatency 1000000 Driver and Link Error Messages Reported by MPI ProgramsMPI Stats Useful Programs and Files for Debugging These two commands perform the same functionsCheck Cluster Homogeneity with ipathcheckout Restarting InfiniPathShell script that gathers status Summary of Useful Programs and FilesModprobe Example contents areIbstatus Ipathcheckout Ipathcontrol Ipathbug-helper Option-qwill query and --qawill query all To check the contents of an RPM, use these commands# lsmod egrep ’ipathibrdmafindex’ $ mpirun -np 2 -m /tmp/id1 -d0x101 mpilatency 1Table C-3. statusstr File Example contents for QLogic-built drivers Command strings can also be used. Its format is as followsBooks for Learning MPI Programming References for MPIReference and Source for Slurm InfiniBand OpenFabricsRocks ClustersGlossary GID LID Mtrr SDP Glossary IB6054601-00 D Index OpenFabrics Configuration Configuration of on Suse and Sles