Q-Logic IB0056101-00 G.02, OFED

Draft

Introduction

Contact Information

1-6 Preliminary IB0056101-00 G.02

Notes

Contents

Main Document Revision History Page Page web site for updated information on supported compilers. This information is not on the web site. ates Error RPMs to Use for Each Node in a Cluster from Software Installation section to Appendix Page Page Table of Contents 1 Introduction 2 Feature Overview 3 Step-by-Step Installation Checklist 4 Hardware Installation 5 Software Installation 6 Configuring Drivers and Services 7 Installation Verification and Additional Settings A Installation Troubleshooting B Write Combining C Configuration Files D Package Descriptions List of Figures List of Tables Page 1 Who Should Read this Guide How this Guide is Organized Overview Interoperability Conventions Used in this Guide Documentation Contact Information Page 2 Features Other Changes Continued Support Page Software Components 3 Installation Checklist Hardware Installation Software Installation Page Page 4 Hardware Installation Requirements Hardware Form Factors Cabling and Switches Optical Fibre Option Configuring the BIOS Safety with Electricity Unpacking Information Verify the Package Contents List of the Package Contents Page Figure 4-1. QLogic QLE7280 with IBA7220 ASIC Figure 4-2. QLogic QLE7140 Card with Riser, Top View Unpacking the QLogic Adapter Hardware Installation Hardware Installation for QLE7240, QLE7280, or QLE7140 with PCI Express Riser Dual Adapter Installation Installation Steps Page Page Hardware Installation for QHT7140 with HTX Riser Page Page Hardware Installation for QLE7240, QLE7280, and QLE7140 Without a PCI Express Riser Hardware Installation for the QHT7140 Without an HTX Riser Switch Configuration and Monitoring Cabling the Adapter to the InfiniBand Switch Completing the Installation 5 Cluster Setup Types of Nodes in a Cluster Environment Supported Linux Distributions Distribution Identifiers NOTE: Support for RHEL4 U4 and SLES 10.0 has been removed. Table 5-2. Distribution Identifiers Distribution Identifier Used On Compiler Support Setting Up Your Environment Choose the Appropriate Download Files Page Page Install QLogicIB-Basic with the Installer Tool Page Page The next screen contains these choices: Page Page About rpm Installation Using rpm to Install InfiniPath and OpenFabrics Page RPM Organization Install QLogic OFED User-level Software with the rpm Command Page Rebuilding or Reinstalling the kernel-ib Driver with rpm After a Kernel Upgrade Install QLogic OFED Using Rocks Install Frontend and Compute Nodes Rocks Installation on an Existing Frontend Node Install QLogic OFED Using a Platform OCS Kit Install FastFabric Software CD/ISO Image Install Additional Software Installing Lustre Installed Layout Removing Software Packages Uninstalling Using the Installer Tool Uninstalling InfiniPath and OpenFabrics RPMs Uninstalling Software with Rocks or Platform OCS Downgrading RPMs Page 6 Services InfiniPath and OpenFabrics Driver Overview OpenFabrics Drivers and Services Configuration and Startup Configuring the IPoIB Network Interface OpenSM SRP Using QLogic SRP Using OFED SRP In the output, look for lines similar to these: 4. Find the result that corresponds to the target you want, and echo it into the Configuring and Administering the VNIC Interface Configuring Drivers and Services OpenFabrics Drivers and Services Configuration and Startup IB0056101-00 G.02 Preliminary 6-7 #ib_qlgc_vnic_query Configuring Drivers and Services OpenFabrics Drivers and Services Configuration and Startup 6-8 Preliminary IB0056101-00 G.02 Page Page Page Page MPI over uDAPL Other Configuration: Changing the MTU Size Managing the InfiniPath Driver Configure InfiniPath Driver State Start, Stop or Restart InfiniPath Unloading the Driver/Modules Manually Further Information on Configuring and Loading Drivers Page 7 Additional Settings LED Link and Data Indicators Adapter and Other Settings Customer Acceptance Utility Page Page A Troubleshooting Hardware Issues Node Spontaneously Reboots Some HTX Motherboards May Need Two or More CPUs in Use BIOS Settings Enable Advanced Configuration and Power Interface (ACPI) Issue with Supermicro H8DCE-HTe and QHT7040 Software Installation Issues Missing Kernel RPM Errors Page Resolving Conflicts openmpi_gcc Fails to Install Because of Dependency on gfortran (RHEL 4) mpirun Installation Requires 32-bit Support Lockable Memory Error on Initial Installation of InfiniPath Configuration Issues ibsrpdm Command Hangs When Two HCAs are Installed but Only Unit 1 is Connected to the Switch Outdated ipath_ether Configuration Setup Generates Error Page B Introduction Verify Write Combining is Working PAT and Write Combining MTRR Mapping and Write Combining Edit BIOS Settings to Fix MTRR Issues Use the ipath_mtrr Script to Fix MTRR Issues Page C Page D Package Names with the QLogicIB-Basic Download Different Nodes May Use Different RPMs InfiniPath RPM Version Numbers and Identifiers OpenFabrics RPM Names InfiniPath and OpenFabrics RPMs Documentation RPMs InfiniPath RPMs Table D-1. Documentation/RPMs RPM Name Front End Compute Development Table D-2. InfiniPath/RPMs RPM Name Front End Compute Development OpenFabrics RPMs Page Page The development RPMs in TableD-6 are only needed on the nodes where OFED programs are compiled. Table D-6. OpenFabrics-Devel/RPMs Table D-7. OpenSM/RPM Table D-6. OpenFabrics-Devel/RPMs (Continued) Other HCAs Other MPIs Table D-11. OtherMPIs/RPMs Table D-10. Other HCAs-Devel/RPMs (Continued) Table D-11. OtherMPIs/RPMs (Continued) Page Index A B C D I K L M N O P Q S T U V W Page Preliminary