Q-Logic IB0056101-00 G Cabling and Switches, QLogic InfiniBand Cables, Product Number Description

Page 27

A

4 – Hardware Installation Hardware Installation Requirements

Cabling and Switches

The cable installation uses a standard InfiniBand (IB) 4X cable. Any InfiniBand cable that has been qualified by the vendor should work. For SDR, the longest passive copper IB cable that QLogic has currently qualified is 20 meters. For DDR-capable adapters and switches, the DDR-capable passive copper cables cannot be longer than 10 meters. Active cables can eliminate some of the cable length restrictions.

InfiniBand switches are available through QLogic.

NOTE:

If you are using the QLE7240 or QLE7280 and want to use DDR mode, then DDR-capable switches must be used.

The copper cables listed in Table 4-2are available from QLogic:

Table 4-2. QLogic InfiniBand Cables

Product Number	Description

7104-1M-Cable	4x-4x cable—1 meter

7104-2M-Cable	4x-4x cable—2 meters

7104-3M-Cable	4x-4x cable—3 meters

7104-4M-Cable	4x-4x cable—4 meters

7104-5M-Cable	4x-4x cable—5 meters

7104-6M-Cable	4x-4x cable—6 meters

7104-7M-Cable	4x-4x cable—7 meters

7104-8M-Cable	4x-4x cable—8 meters

7104-9M-Cable	4x-4x Cable—9 meters

7104-10M-Cable	4x-4x cable—10 meters

7104-12M-Cable	4x-4x cable—12 meters (SDR only)

7104-14M-Cable	4x-4x cable—14 meters (SDR only)

7104-16M-Cable	4x-4x cable—16 meters (SDR only)

7104-18M-Cable	4x-4x cable—18 meters (SDR only)

For cabling instructions, see “Cabling the Adapter to the InfiniBand Switch” on page 4-17.

IB0056101-00 G

4-3

Image 27

Contents QLogic HCA and InfiniPath→ Software Install Guide Document Revision History QLogic HCA and InfiniPath→ Software Install Guide VersionTable of Contents Software Installation Installation Troubleshooting Configuration Files RPM Descriptions List of Figures Page How this Guide is Organized Who Should Read this GuideOverview Convention Meaning InteroperabilityConventions Used in this Guide Typographical ConventionsDocumentation Support Headquarters Contact InformationIntroduction Contact Information IB0056101-00 G What’s New in this Release QLogic Adapter Model NumbersDescription QLogic Model Description New FeaturesSupported Distributions and Kernels InfiniPath/OpenFabrics Supported Distributions KernelsDistribution InfiniPath/OpenFabrics Supported Compiler Support Software ComponentsDistribution InfiniPath/OpenFabrics Supported Kernels „ Ofed SRP Feature Overview Software Components Hardware Installation Step-by-Step Installation ChecklistInstalling Software Using Rocks on Software InstallationStep-by-Step Installation Checklist Software Installation Page QLogic Platform Plugs Into Hardware Installation RequirementsHardware Adapter Models and Related PlatformsForm Factors Cabling and Switches QLogic InfiniBand CablesProduct Number Description Optical Fibre Option Configuring the BiosUnpacking Information Safety with ElectricityVerify the Package Contents List of the Package ContentsHardware Installation Unpacking Information QLogic QLE7280 with IBA7220 Asic Asic Unpacking the QLogic AdapterInstallation Steps Dual Adapter InstallationPCIe Slot in a Typical Motherboard QLogic PCIe HCA Assembly with Riser Card Assembled PCIe HCA with Riser Hardware Installation for QHT7140 with HTX RiserHTX Slot QLogic QHT7140 Adapter with Riser Card Assembled QHT7140 with Riser Hardware Installation for the QHT7140 Without an HTX Riser Cabling the Adapter to the InfiniBand Switch Switch Configuration and MonitoringCompleting the Installation Types of Nodes in a Cluster Environment Cluster SetupSetting Up Your Environment Supported Linux DistributionsInfiniPath/OpenFabrics Supported Distributions and Kernels Choosing the Appropriate Download Files Available PackagesPackage Description Comments Unpacking the InfiniPath tar File Libibverbs-2.2-xxx.11.yyy.x8664.rpm Check for Missing Files or RPMsSles10 Installing the InfiniPath and OpenFabrics RPMsDistribution Used On Rhel4Choosing the RPMs to Install Rpms to InstallIf you want to Then install Preparing Using rpm to Install InfiniPath and OpenFabricsInstalled Layout Starting the InfiniPath Service InfiniPath and OpenFabrics Driver Overview Configuring the InfiniPath DriversConfiguring the ibipath Driver Configuring the ipathether Network InterfaceIpathether Configuration on Red Hat # QLogic Interconnect Ethernet DEVICE=eth2 BOOTPROTO=static Etc/sysconfig/network-scripts/ifcfg-eth200117504e011 Ipathether Configuration on SlesMODULE=ipathether STARTMODE=auto Etc/sysconfig/hardware/hwcfg-eth-id-$MACEtc/sysconfig/network/ifcfg-eth2 Etc/udev/rules.d/30-netpersistentnames.rulesOpenFabrics Drivers and Services Configuration and Startup Output from this command will be similar to this Configuring the IPoIB Network InterfaceConfiguring and Administering the Vnic Interface Etc/sysconfig/infinipath. Change this line to00066a0130000001 Etc/infiniband/qlgcvnic.cfg Service 1 1000066a00000101 InfiniNIC.InfiniConSys.Data01# HEARTBEAT=100 NAME=eioc1 IOCGUID=0x66A0137FFFFE7This example uses DGID, Iocguid and Iocstring # /etc/init.d/qlgcvnic stop # /etc/init.d/infinipath start # /etc/init.d/qlgcvnic startOpenSM # ibqlgcvnicinfoEtc/init.d/opensmd 21000001ff040bf6 Using Ofed SRP# mkdir /mnt/targetname # mount /dev/sde1 /mnt/targetname # ibsrpdm -cOther Configuration Changing the MTU Size Starting and Stopping the InfiniPath SoftwareMPI over uDAPL To check the configuration state, use the command Rebuilding or Reinstalling Drivers After a Kernel Upgrade LED States Indication Further Information on Configuring and Loading DriversLED Link and Data Indicators LED Link and Data IndicatorsOff Adapter SettingsHostname1 Hostname2 Customer Acceptance UtilityIpathcheckout Options Command MeaningRemoving Software Packages Additional Installation InstructionsUninstalling InfiniPath and OpenFabrics RPMs Uninstalling Ofed 1.3 SoftwareInstalling QLogic MPI in an Alternate Location Installing Lustre/etc/redhat-release file or the /etc/SuSE-release file Installing on an Unsupported DistributionUsr/src/infinipath/drivers/build-guards.sh Managing and Installing Software Using RocksInstalling Rocks and InfiniPath RPMs Use the following contents Software Installation Additional Installation Instructions Home/install/contrib/4.2.1/x8664/RPMS Further Information on Rocks and kickstart Page Some HTX Motherboards May Need Two or More CPUs in Use Hardware IssuesBios Settings Node Spontaneously RebootsEdit Bios Settings to Fix Mtrr Issues Mtrr Mapping and Write CombiningMtrr Mapping Discrete Software Installation Issues Issue with Supermicro H8DCE-HTe and QHT7040Use the ipathmtrr Script to Fix Mtrr Issues Rpm no packages given for erase Version Number Conflict with opensm-*on RHEL5 SystemsOpenFabrics Dependencies OpenFabrics Library DependenciesMissing Kernel RPM Errors Mpirun Installation Requires 32-bit Support Resolving ConflictsLibgcc-3.4.3-9.EL4.i386.rpm Eth3 device QLogic Corp Unknown device 0010 rev Lockable Memory Error on Initial Installation of InfiniPathIfup on ipathether on Sles 10 Reports unknown device Glibc-2.3.4-2.i686.rpm or glibc-2.3.4-2.i386.rpmPage Configuration File Name Description Table B-1. Configuration FilesSample and Template Files Description Infinipath-2.2-33597.832rhel4qlc.x8664.rpm InfiniPath and OpenFabrics RPMsDifferent Nodes May Use Different RPMs RPM Version Numbers and IdentifiersRPM Name Front End Compute Development RPM OrganizationDocumentation and InfiniPath RPMs Table C-1. InfiniPath Documentation/RPMsTable C-2. InfiniPath/RPMs Table C-3. InfiniPath-Devel/RPMsContains ipathcheckout Ipathbug-helpera RPM Name Comments OpenFabrics RPMsTable C-4. InfiniPath-MPI/RPMs Table C-5. OpenFabrics Documentation/RPMsIbutils-2.2-xxx.12.yyy.x8664.rpm Dapl-utils-2.2-xxx.207.yyy.x8664.rpmIb-bonding-2.2-xxx.090.yyy.x8664.rpm Ibsim-2.2-xxx.04.yyy.x8664.rpmLibrdmacm-2.2-xxx.106.yyy.x8664.rpm Libibverbs-2.2-xxx.111.yyy.x8664.rpmLibibverbs-utils-2.2-xxx.111.yyy.x8664.rpm Libipathverbs-2.2-xxx.11.yyy.x8664.rpmTable C-7. OpenFabrics-Devel/RPMs Table C-9. Other HCAs/RPMs Table C-8. OpenSM/RPMsEnd Table C-10. Other HCAs-Devel/RPMsTable C-11. OtherMPIs/RPMs RPM Name Front ComputRPM Name Front Comput Developme End Qlogic-mpi-regis Openmpiintel-2.2-xxx.125.yyy.x8Openmpipathscale-2.2-xxx.125.yy Openmpipgi-2.2-xxx.125.yyy.x86RPM Descriptions OpenFabrics RPMs IB0056101-00 G Index Help Openssh Openssh-server 5-3 Optical media converters No packages given for erase warning message A-4 Verbose Page Page Camberley Surrey, GU16 7ER UK +44 0 1276 804