Preliminary | Q-Logic OFED, IB0056101-00 G.02 specs

QLogic HCA and QLogic OFED Software Install Guide QLogic OFED Version 1.4

Notes

S

Page viii

Preliminary

IB0056101-00 G.02

Image 8

Q-Logic OFED, IB0056101-00 G.02 manual Preliminary

Contents

QLogic HCA and QLogic Ofed Software Install Guide Document Revision History Preliminary Preliminary Removed phrase and Transmission Control Moved table InfiniPath and OpenFabrics Ipathether Configuration Setup GenerAtes Error Removed note OpenFabrics programs 32-bit Preliminary Table of Contents Software Installation Installation Verification and Additional Settings Write Combining List of Figures Preliminary How this Guide is Organized IntroductionWho Should Read this Guide Overview Interoperability Conventions Used in this GuideTypographical Conventions Meaning Documentation Product documentation includes Contact Information Support Headquarters Preliminary Feature Overview Features Other Changes Support QLogic Adapter Model Numbers Description Software Components „ Ofed SRP Step-by-Step Installation Checklist Hardware Installation Software Installation Preliminary Preliminary Hardware Installation Hardware Installation RequirementsHardware Adapter Models and Related Platforms Form Factors Run ipath control Product Number Description Cabling and SwitchesQLogic InfiniBand Cables Configuring the Bios Safety with Electricity Verify the Package ContentsList of the Package Contents Unpacking Information Hardware Installation Unpacking Information QLogic QLE7280 with IBA7220 Asic Unpacking the QLogic Adapter Asic Dual Adapter Installation PCIe Slot in a Typical Motherboard QLogic PCIe HCA Assembly with Riser Card Hardware Installation for QHT7140 with HTX Riser Assembled PCIe HCA with Riser HTX Slot QLogic QHT7140 Adapter with Riser Card Assembled QHT7140 with Riser Hardware Installation for the QHT7140 Without an HTX Riser Switch Configuration and Monitoring Cabling the Adapter to the InfiniBand Switch Completing the Installation Software Installation Cluster SetupTypes of Nodes in a Cluster Environment Supported Linux Distributions Distribution Identifiers InfiniPath/OpenFabrics Supported Distributions and KernelsDistribution InfiniPath/OpenFabrics Supported Kernels Distribution Identifiers Setting Up Your Environment Compiler SupportRhel5 Sles10 Choose the Appropriate Download Files Required OS PackagesOS Distribution Required Packages Specific Component Requirements Package Description Installation Documentation Available Packages for QLogic Ofed 1.4 Release Install InfiniPath and OpenFab Install QLogicIB-Basic with the Installer Tool Kit on You will see a screen similar to this Ofed SDP Next screen contains these choices Then type n to proceed to the next screen # ./INSTALL --32bit Configure Ofed IP over IB IPV4 addresses now? n Install -Clists all the available components, which include Install OptionsCommand Meaning Additional component names allowed for -Eand -Doptions About rpm Installation Using rpm to Install InfiniPath and OpenFabrics Become root, then Next, install all non-prefixed RPMs RPM Organization Install QLogic Ofed User-level Software with the rpm Command Finally, install the prefixed RPMs in /usr/mpi/qlogic Software Installation Install QLogic Ofed Using Rocks Install Frontend and Compute Nodes Rocks Installation on an Existing Frontend Node Install QLogic Ofed Using a Platform OCS Kit Install FastFabric Software CD/ISO ImageThen use the following command for each node Use the following command to rebuild the entire cluster Install Additional Software Installed LayoutInstalling Lustre MPI include files are Removing Software Packages Uninstalling Using the Installer ToolUninstalling InfiniPath and OpenFabrics RPMs Qlc is a part of all the InfiniPath package names Uninstalling Software with Rocks or Platform OCSDowngrading RPMs Preliminary InfiniPath and OpenFabrics Driver Overview Configuring Drivers and ServicesOpenFabrics Drivers and Services Configuration and Startup Configuring the IPoIB Network Interface To verify the configuration, typeOutput from this command will be similar to this Type Command to disable it on reboot is OpenSM Etc/init.d/opensmd You can stop opensmd again like thisTo use Ofed SRP, follow these steps Output, look for lines similar to these Guid Configuring and Administering the Vnic Interface # ibqlgcvnicquery Preliminary Create the Vnic interfaces using the configuration file Etc/infiniband/qlgcvnic.cfg RX CSUM=TRUE HEARTBEAT=100 Preliminary To restart the QLogic Vnic driver, run the following command MPI over uDAPL This information is collected from Other Configuration Changing the MTU Size On every node, type the following command as root Managing the InfiniPath Driver Etc/infiniband/openib.conf Configure InfiniPath Driver State To enable the driver, use the command as rootStart, Stop or Restart InfiniPath Device files are Unloading the Driver/Modules Manually Further Information on Configuring and Loading DriversSequence of commands to restart the driver are as follows Preliminary Adapter and Other Settings Installation Verification and Additional SettingsLED Link and Data Indicators Hostname1 Hostname2 Customer Acceptance UtilityThis will also show link speed Ipath checkout Options Preliminary Installation Troubleshooting Hardware IssuesNode Spontaneously Reboots Some HTX Motherboards May Need Two or More CPUs in Use Bios Settings Software Installation IssuesEnable Advanced Configuration and Power Interface Acpi Issue with Supermicro H8DCE-HTe and QHT7040 # yum install kernel-devel Mpirun Installation Requires 32-bit Support Resolving Conflicts Configuration Issues Lockable Memory Error on Initial Installation of InfiniPathOutdated ipathether Configuration Setup Generates Error Check your distribution for the exact RPM name Eth2 error fetching interface information Device not found Verify Write Combining is Working Write CombiningIntroduction Mtrr Mapping and Write Combining Edit Bios Settings to Fix Mtrr IssuesPAT and Write Combining Use the ipathmtrr Script to Fix Mtrr Issues Preliminary Configuration File Name Description Configuration FilesTable C-1. Configuration Files Also installed in /etc/infiniband Sample and Template Files DescriptionWith the Ofed documentation, or Qlgcvnictools subdirectory Different Nodes May Use Different RPMs Package DescriptionsPackage Names with the QLogicIB-Basic Download InfiniPath and OpenFabrics RPMs InfiniPath RPM Version Numbers and IdentifiersOpenFabrics RPM Names Documentation RPMs InfiniPath RPMsTable D-1. Documentation/RPMs Table D-2. InfiniPath/RPMs OpenFabrics RPMs Table D-3. InfiniPath-Devel/RPMsTable D-4. InfiniPath-MPI/RPMs Table D-5. OpenFabrics/RPMs Dapl-xxx.x8664.rpm Dapl-utils-xxx.x8664.rpmIbsim-xxx.x8664.rpm Ibutils-xxx.x8664.rpm Libibverbs-xxx.x8664.rpm Libibverbs-utils-xxx.x8664.rpmLibipathverbs-xxx.x86 64.rpm Librdmacm-xxx.x86 64.rpm Table D-6. OpenFabrics-Devel/RPMs Table D-7. OpenSM/RPM Other HCAs Table D-8. OpenSM-Devel/RPMTable D-9. Other HCAs/RPMs Table D-10. Other HCAs-Devel/RPMs Other MPIs Table D-11. OtherMPIs/RPMs Openmpigcc-xxx.x8664.rpm Openmpigccqlc-xxx.yyy.x8664.rpmOpenmpiintelqlc-xxx.yyy.x8664.rpm a Openmpipathscaleqlc-xxx.yyy.x8664.rpm Preliminary Form factors for HCAs Hardware installation overview Index Module Hardware requirementsHTX motherboards may required two or more CPUs A-1 Mpi-selector 5-15,5-18 VNIC, configuration Skip=LIST 7-3SoftwarePage Preliminary