Q-Logic IB0056101-00 G.02, OFED specs

Image 123

Q-Logic IB0056101-00 G.02, OFED manual

Contents

QLogic HCA and QLogic Ofed Software Install Guide Document Revision History Preliminary Preliminary Removed phrase and Transmission Control Ipathether Configuration Setup Gener Ates ErrorMoved table InfiniPath and OpenFabrics Removed note OpenFabrics programs 32-bit Preliminary Table of Contents Software Installation Installation Verification and Additional Settings Write Combining List of Figures Preliminary Introduction Who Should Read this GuideHow this Guide is Organized Overview Meaning InteroperabilityConventions Used in this Guide Typographical Conventions Product documentation includes Documentation Support Headquarters Contact Information Preliminary Features Feature Overview Support Other Changes Description QLogic Adapter Model Numbers „ Ofed SRP Software Components Hardware Installation Step-by-Step Installation Checklist Software Installation Preliminary Preliminary Adapter Models and Related Platforms Hardware InstallationHardware Installation Requirements Hardware Run ipath control Form Factors Cabling and Switches QLogic InfiniBand CablesProduct Number Description Configuring the Bios Unpacking Information Safety with ElectricityVerify the Package Contents List of the Package Contents Hardware Installation Unpacking Information QLogic QLE7280 with IBA7220 Asic Asic Unpacking the QLogic Adapter Dual Adapter Installation PCIe Slot in a Typical Motherboard QLogic PCIe HCA Assembly with Riser Card Assembled PCIe HCA with Riser Hardware Installation for QHT7140 with HTX Riser HTX Slot QLogic QHT7140 Adapter with Riser Card Assembled QHT7140 with Riser Hardware Installation for the QHT7140 Without an HTX Riser Cabling the Adapter to the InfiniBand Switch Switch Configuration and Monitoring Completing the Installation Supported Linux Distributions Software InstallationCluster Setup Types of Nodes in a Cluster Environment Distribution Identifiers Distribution IdentifiersInfiniPath/OpenFabrics Supported Distributions and Kernels Distribution InfiniPath/OpenFabrics Supported Kernels Sles10 Setting Up Your EnvironmentCompiler Support Rhel5 Specific Component Requirements Choose the Appropriate Download FilesRequired OS Packages OS Distribution Required Packages Available Packages for QLogic Ofed 1.4 Release Package Description Installation Documentation Install InfiniPath and OpenFab Kit on Install QLogicIB-Basic with the Installer Tool You will see a screen similar to this Ofed SDP Then type n to proceed to the next screen Next screen contains these choices Configure Ofed IP over IB IPV4 addresses now? n # ./INSTALL --32bit Additional component names allowed for -Eand -Doptions Install -Clists all the available components, which includeInstall Options Command Meaning About rpm Installation Using rpm to Install InfiniPath and OpenFabrics Become root, then RPM Organization Next, install all non-prefixed RPMs Install QLogic Ofed User-level Software with the rpm Command Finally, install the prefixed RPMs in /usr/mpi/qlogic Software Installation Install Frontend and Compute Nodes Install QLogic Ofed Using Rocks Rocks Installation on an Existing Frontend Node Use the following command to rebuild the entire cluster Install QLogic Ofed Using a Platform OCS KitInstall FastFabric Software CD/ISO Image Then use the following command for each node MPI include files are Install Additional SoftwareInstalled Layout Installing Lustre Uninstalling Using the Installer Tool Uninstalling InfiniPath and OpenFabrics RPMsRemoving Software Packages Uninstalling Software with Rocks or Platform OCS Downgrading RPMsQlc is a part of all the InfiniPath package names Preliminary Configuring Drivers and Services OpenFabrics Drivers and Services Configuration and StartupInfiniPath and OpenFabrics Driver Overview Type Configuring the IPoIB Network InterfaceTo verify the configuration, type Output from this command will be similar to this OpenSM Command to disable it on reboot is You can stop opensmd again like this To use Ofed SRP, follow these stepsEtc/init.d/opensmd Guid Output, look for lines similar to these Configuring and Administering the Vnic Interface # ibqlgcvnicquery Preliminary Etc/infiniband/qlgcvnic.cfg Create the Vnic interfaces using the configuration file RX CSUM=TRUE HEARTBEAT=100 Preliminary To restart the QLogic Vnic driver, run the following command This information is collected from MPI over uDAPL On every node, type the following command as root Other Configuration Changing the MTU Size Etc/infiniband/openib.conf Managing the InfiniPath Driver Device files are Configure InfiniPath Driver StateTo enable the driver, use the command as root Start, Stop or Restart InfiniPath Further Information on Configuring and Loading Drivers Sequence of commands to restart the driver are as followsUnloading the Driver/Modules Manually Preliminary Installation Verification and Additional Settings LED Link and Data IndicatorsAdapter and Other Settings Customer Acceptance Utility This will also show link speedHostname1 Hostname2 Ipath checkout Options Preliminary Some HTX Motherboards May Need Two or More CPUs in Use Installation TroubleshootingHardware Issues Node Spontaneously Reboots Issue with Supermicro H8DCE-HTe and QHT7040 Bios SettingsSoftware Installation Issues Enable Advanced Configuration and Power Interface Acpi # yum install kernel-devel Resolving Conflicts Mpirun Installation Requires 32-bit Support Check your distribution for the exact RPM name Configuration IssuesLockable Memory Error on Initial Installation of InfiniPath Outdated ipathether Configuration Setup Generates Error Eth2 error fetching interface information Device not found Write Combining IntroductionVerify Write Combining is Working Edit Bios Settings to Fix Mtrr Issues PAT and Write CombiningMtrr Mapping and Write Combining Use the ipathmtrr Script to Fix Mtrr Issues Preliminary Configuration Files Table C-1. Configuration FilesConfiguration File Name Description Qlgcvnictools subdirectory Also installed in /etc/infinibandSample and Template Files Description With the Ofed documentation, or Package Descriptions Package Names with the QLogicIB-Basic DownloadDifferent Nodes May Use Different RPMs InfiniPath RPM Version Numbers and Identifiers OpenFabrics RPM NamesInfiniPath and OpenFabrics RPMs Table D-2. InfiniPath/RPMs Documentation RPMsInfiniPath RPMs Table D-1. Documentation/RPMs Table D-5. OpenFabrics/RPMs OpenFabrics RPMsTable D-3. InfiniPath-Devel/RPMs Table D-4. InfiniPath-MPI/RPMs Ibutils-xxx.x8664.rpm Dapl-xxx.x8664.rpmDapl-utils-xxx.x8664.rpm Ibsim-xxx.x8664.rpm Librdmacm-xxx.x86 64.rpm Libibverbs-xxx.x8664.rpmLibibverbs-utils-xxx.x8664.rpm Libipathverbs-xxx.x86 64.rpm Table D-6. OpenFabrics-Devel/RPMs Table D-7. OpenSM/RPM Table D-10. Other HCAs-Devel/RPMs Other HCAsTable D-8. OpenSM-Devel/RPM Table D-9. Other HCAs/RPMs Table D-11. OtherMPIs/RPMs Other MPIs Openmpipathscaleqlc-xxx.yyy.x8664.rpm Openmpigcc-xxx.x8664.rpmOpenmpigccqlc-xxx.yyy.x8664.rpm Openmpiintelqlc-xxx.yyy.x8664.rpm a Preliminary Index Form factors for HCAs Hardware installation overview Hardware requirements HTX motherboards may required two or more CPUs A-1Module Mpi-selector 5-15,5-18 Skip=LIST 7-3Software VNIC, configurationPage Preliminary