Q-Logic IB0056101-00 G manual Dual Adapter Installation, Installation Steps

Page 33

A

4 – Hardware Installation Hardware Installation

Hardware Installation

This section contains hardware installation instructions for the QLE7240, QLE7280, QLE7140, QHT7040, and QHT7140.

Hardware Installation for QLE7240, QLE7280, or QLE7140 with PCI Express Riser

Installation for the QLE7240, QLE7280, and QLE7140 is similar. The following instructions are for the QLE7140, but can be used for any of these three adapters.

Most installations will be in 1U and 2U chassis, using a PCIe right angle riser card. This results in an installation of the adapter that is parallel to the motherboard. This type of installation is described first. Installation in a 3U chassis is described in “Hardware Installation for the QHT7140 Without an HTX Riser” on page 4-16.

Installing the QLogic QLE7140 in a 1U or 2U chassis requires a PCIe right angle riser card.

A taller riser card can be used if necessary. The QLE7140 can connect to any of the standard compatible PCI Express riser cards.

Dual Adapter Installation

If you have a motherboard with dual PCIe slots, dual adapters can be installed. The adapters must match. For example, on a motherboard with two x16 slots, dual QLE7280 adapters can be installed, but not a QLE7240 adapter and a QLE7280 adapter. Check the design of your motherboard to see how riser cards can be used.

Follow the instructions in “Installation Steps” on page 4-9.

See the Using MPI section in the QLogic HCA and InfiniPath Software User Guide for information on using the IPATH_UNIT environment variable to control which HCA to use.

Installation Steps

To install the QLogic adapter with a PCIe riser card:

1.The BIOS should already be configured properly by the motherboard manufacturer. However, if any additional BIOS configuration is required, it will usually need to be done before installing the QLogic adapter. See “Configuring the BIOS” on page 4-4.

2.Shut down the power supply to the system into which you will install the QLogic adapter.

3.Take precautions to avoid electrostatic damage (ESD) to the cards by properly grounding yourself or touching the metal chassis to discharge static electricity before handling the cards.

IB0056101-00 G

4-9

Image 33
Contents QLogic HCA and InfiniPath→ Software Install Guide Document Revision History QLogic HCA and InfiniPath→ Software Install Guide VersionTable of Contents Software Installation Installation Troubleshooting Configuration Files RPM Descriptions List of Figures Page How this Guide is Organized Who Should Read this GuideOverview Conventions Used in this Guide InteroperabilityTypographical Conventions Convention MeaningDocumentation Support Headquarters Contact InformationIntroduction Contact Information IB0056101-00 G What’s New in this Release QLogic Adapter Model NumbersDescription QLogic Model Description New FeaturesSupported Distributions and Kernels InfiniPath/OpenFabrics Supported Distributions KernelsDistribution InfiniPath/OpenFabrics Supported Compiler Support Software ComponentsDistribution InfiniPath/OpenFabrics Supported Kernels „ Ofed SRP Feature Overview Software Components Hardware Installation Step-by-Step Installation ChecklistInstalling Software Using Rocks on Software InstallationStep-by-Step Installation Checklist Software Installation Page Hardware Hardware Installation RequirementsAdapter Models and Related Platforms QLogic Platform Plugs IntoForm Factors Cabling and Switches QLogic InfiniBand CablesProduct Number Description Optical Fibre Option Configuring the BiosVerify the Package Contents Safety with ElectricityList of the Package Contents Unpacking InformationHardware Installation Unpacking Information QLogic QLE7280 with IBA7220 Asic Asic Unpacking the QLogic AdapterInstallation Steps Dual Adapter InstallationPCIe Slot in a Typical Motherboard QLogic PCIe HCA Assembly with Riser Card Assembled PCIe HCA with Riser Hardware Installation for QHT7140 with HTX RiserHTX Slot QLogic QHT7140 Adapter with Riser Card Assembled QHT7140 with Riser Hardware Installation for the QHT7140 Without an HTX Riser Cabling the Adapter to the InfiniBand Switch Switch Configuration and MonitoringCompleting the Installation Types of Nodes in a Cluster Environment Cluster SetupSetting Up Your Environment Supported Linux DistributionsInfiniPath/OpenFabrics Supported Distributions and Kernels Choosing the Appropriate Download Files Available PackagesPackage Description Comments Unpacking the InfiniPath tar File Libibverbs-2.2-xxx.11.yyy.x8664.rpm Check for Missing Files or RPMsDistribution Used On Installing the InfiniPath and OpenFabrics RPMsRhel4 Sles10Choosing the RPMs to Install Rpms to InstallIf you want to Then install Preparing Using rpm to Install InfiniPath and OpenFabricsInstalled Layout Starting the InfiniPath Service InfiniPath and OpenFabrics Driver Overview Configuring the InfiniPath DriversConfiguring the ibipath Driver Configuring the ipathether Network InterfaceIpathether Configuration on Red Hat # QLogic Interconnect Ethernet DEVICE=eth2 BOOTPROTO=static Etc/sysconfig/network-scripts/ifcfg-eth200117504e011 Ipathether Configuration on SlesEtc/sysconfig/network/ifcfg-eth2 Etc/sysconfig/hardware/hwcfg-eth-id-$MACEtc/udev/rules.d/30-netpersistentnames.rules MODULE=ipathether STARTMODE=autoOpenFabrics Drivers and Services Configuration and Startup Output from this command will be similar to this Configuring the IPoIB Network InterfaceConfiguring and Administering the Vnic Interface Etc/sysconfig/infinipath. Change this line to00066a0130000001 Etc/infiniband/qlgcvnic.cfg Service 1 1000066a00000101 InfiniNIC.InfiniConSys.Data01# HEARTBEAT=100 NAME=eioc1 IOCGUID=0x66A0137FFFFE7This example uses DGID, Iocguid and Iocstring # /etc/init.d/qlgcvnic stop # /etc/init.d/infinipath start # /etc/init.d/qlgcvnic startOpenSM # ibqlgcvnicinfoEtc/init.d/opensmd 21000001ff040bf6 Using Ofed SRP# mkdir /mnt/targetname # mount /dev/sde1 /mnt/targetname # ibsrpdm -cOther Configuration Changing the MTU Size Starting and Stopping the InfiniPath SoftwareMPI over uDAPL To check the configuration state, use the command Rebuilding or Reinstalling Drivers After a Kernel Upgrade LED Link and Data Indicators Further Information on Configuring and Loading DriversLED Link and Data Indicators LED States IndicationOff Adapter SettingsHostname1 Hostname2 Customer Acceptance UtilityIpathcheckout Options Command MeaningUninstalling InfiniPath and OpenFabrics RPMs Additional Installation InstructionsUninstalling Ofed 1.3 Software Removing Software PackagesInstalling QLogic MPI in an Alternate Location Installing Lustre/etc/redhat-release file or the /etc/SuSE-release file Installing on an Unsupported DistributionUsr/src/infinipath/drivers/build-guards.sh Managing and Installing Software Using RocksInstalling Rocks and InfiniPath RPMs Use the following contents Software Installation Additional Installation Instructions Home/install/contrib/4.2.1/x8664/RPMS Further Information on Rocks and kickstart Page Bios Settings Hardware IssuesNode Spontaneously Reboots Some HTX Motherboards May Need Two or More CPUs in UseEdit Bios Settings to Fix Mtrr Issues Mtrr Mapping and Write CombiningMtrr Mapping Discrete Software Installation Issues Issue with Supermicro H8DCE-HTe and QHT7040Use the ipathmtrr Script to Fix Mtrr Issues OpenFabrics Dependencies Version Number Conflict with opensm-*on RHEL5 SystemsOpenFabrics Library Dependencies Rpm no packages given for eraseMissing Kernel RPM Errors Mpirun Installation Requires 32-bit Support Resolving ConflictsLibgcc-3.4.3-9.EL4.i386.rpm Ifup on ipathether on Sles 10 Reports unknown device Lockable Memory Error on Initial Installation of InfiniPathGlibc-2.3.4-2.i686.rpm or glibc-2.3.4-2.i386.rpm Eth3 device QLogic Corp Unknown device 0010 revPage Configuration File Name Description Table B-1. Configuration FilesSample and Template Files Description Different Nodes May Use Different RPMs InfiniPath and OpenFabrics RPMsRPM Version Numbers and Identifiers Infinipath-2.2-33597.832rhel4qlc.x8664.rpmDocumentation and InfiniPath RPMs RPM OrganizationTable C-1. InfiniPath Documentation/RPMs RPM Name Front End Compute DevelopmentTable C-2. InfiniPath/RPMs Table C-3. InfiniPath-Devel/RPMsContains ipathcheckout Ipathbug-helpera Table C-4. InfiniPath-MPI/RPMs OpenFabrics RPMsTable C-5. OpenFabrics Documentation/RPMs RPM Name CommentsIb-bonding-2.2-xxx.090.yyy.x8664.rpm Dapl-utils-2.2-xxx.207.yyy.x8664.rpmIbsim-2.2-xxx.04.yyy.x8664.rpm Ibutils-2.2-xxx.12.yyy.x8664.rpmLibibverbs-utils-2.2-xxx.111.yyy.x8664.rpm Libibverbs-2.2-xxx.111.yyy.x8664.rpmLibipathverbs-2.2-xxx.11.yyy.x8664.rpm Librdmacm-2.2-xxx.106.yyy.x8664.rpmTable C-7. OpenFabrics-Devel/RPMs Table C-9. Other HCAs/RPMs Table C-8. OpenSM/RPMsTable C-11. OtherMPIs/RPMs Table C-10. Other HCAs-Devel/RPMsRPM Name Front Comput EndRPM Name Front Comput Developme End Openmpipathscale-2.2-xxx.125.yy Openmpiintel-2.2-xxx.125.yyy.x8Openmpipgi-2.2-xxx.125.yyy.x86 Qlogic-mpi-regisRPM Descriptions OpenFabrics RPMs IB0056101-00 G Index Help Openssh Openssh-server 5-3 Optical media converters No packages given for erase warning message A-4 Verbose Page Page Camberley Surrey, GU16 7ER UK +44 0 1276 804