Q-Logic IB0056101-00 G guide

D	Aliso Viejo, CA 92656 949.389.6000 www.qlogic.com
Corporate Headquarters QLogic Corporation 26650 Aliso Viejo Parkway	Aliso Viejo, CA 92656 949.389.6000 www.qlogic.com
Europe Headquarters QLogic (UK) LTD. Quatro House Lyon Way, Frimley	Camberley Surrey, GU16 7ER UK +44 (0) 1276 804 670

© 2006–2008 QLogic Corporation. Specifications are subject to change without notice. All rights reserved worldwide. QLA, QLogic, SANsurfer, the QLogic logo, InfiniPath, SilverStorm, and EKOPath are trademarks or registered trademarks of QLogic Corporation. AMD Opteron is a trademark of Advanced Microdevices Inc. BladeCenter and IBM are registered trademarks of International Business Machines Corporation. DataDirect Networks is a trademark of DataDirect Networks, Inc. EMCORE is a trademark of EMCORE Corporation. HTX is a trademark of the HyperTransport Technology Consortium. IBM and BladeCenter are registered trademarks of International Business Machines Corporation. InfiniBand is a trademark and service mark of the InfiniBand Trade Association. Intel is a registered trademark of Intel Corporation. Linux is a registered trademark of Linus Torvalds. LSI Logic and Engenio are trademarks or registered trade- marks of LSI Logic Corporation. Lustre is a registered trademark of Cluster File Systems, Inc. Mellanox is a registered trademark and ConnectX is a trademark of Mellanox Technologies, Inc. PathScale is a trademark of PathScale LLC. PCI Express is a registered trademark of PCI-SIG Corporation. Red Hat and Enterprise Linux are registered trademarks of Red Hat, Inc. Super- micro is a registered trademark of Super Micro Computer Inc. SUSE is a registered trademark of Novell Inc. Zarlink is a trademark of Zarlink Semiconductor Inc. All other brand and product names are trademarks or registered trademarks of their respective owners. Information supplied by QLogic Corporation is believed to be accurate and reliable. QLogic Corporation assumes no responsibility for any errors in this brochure. QLogic Corporation reserves the right, without notice, to make changes in product design or specifications.

Image 114

Q-Logic IB0056101-00 G manual Camberley Surrey, GU16 7ER UK +44 0 1276 804

Contents

QLogic HCA and InfiniPath→ Software Install Guide QLogic HCA and InfiniPath→ Software Install Guide Version Document Revision History Table of Contents Software Installation Installation Troubleshooting Configuration Files RPM Descriptions List of Figures Page Who Should Read this Guide How this Guide is Organized Overview Typographical Conventions InteroperabilityConventions Used in this Guide Convention Meaning Documentation Contact Information Support Headquarters Introduction Contact Information IB0056101-00 G What’s New in this Release QLogic Adapter Model NumbersDescription New Features QLogic Model Description Supported Distributions and Kernels InfiniPath/OpenFabrics Supported Distributions KernelsDistribution InfiniPath/OpenFabrics Supported Compiler Support Software ComponentsDistribution InfiniPath/OpenFabrics Supported Kernels „ Ofed SRP Feature Overview Software Components Step-by-Step Installation Checklist Hardware Installation Software Installation Installing Software Using Rocks on Step-by-Step Installation Checklist Software Installation Page Adapter Models and Related Platforms Hardware Installation RequirementsHardware QLogic Platform Plugs Into Form Factors Cabling and Switches QLogic InfiniBand CablesProduct Number Description Configuring the Bios Optical Fibre Option List of the Package Contents Safety with ElectricityVerify the Package Contents Unpacking Information Hardware Installation Unpacking Information QLogic QLE7280 with IBA7220 Asic Unpacking the QLogic Adapter Asic Dual Adapter Installation Installation Steps PCIe Slot in a Typical Motherboard QLogic PCIe HCA Assembly with Riser Card Hardware Installation for QHT7140 with HTX Riser Assembled PCIe HCA with Riser HTX Slot QLogic QHT7140 Adapter with Riser Card Assembled QHT7140 with Riser Hardware Installation for the QHT7140 Without an HTX Riser Switch Configuration and Monitoring Cabling the Adapter to the InfiniBand Switch Completing the Installation Cluster Setup Types of Nodes in a Cluster Environment Setting Up Your Environment Supported Linux DistributionsInfiniPath/OpenFabrics Supported Distributions and Kernels Choosing the Appropriate Download Files Available PackagesPackage Description Comments Unpacking the InfiniPath tar File Check for Missing Files or RPMs Libibverbs-2.2-xxx.11.yyy.x8664.rpm Rhel4 Installing the InfiniPath and OpenFabrics RPMsDistribution Used On Sles10 Choosing the RPMs to Install Rpms to InstallIf you want to Then install Using rpm to Install InfiniPath and OpenFabrics Preparing Installed Layout Starting the InfiniPath Service Configuring the InfiniPath Drivers InfiniPath and OpenFabrics Driver Overview Configuring the ibipath Driver Configuring the ipathether Network InterfaceIpathether Configuration on Red Hat Etc/sysconfig/network-scripts/ifcfg-eth2 # QLogic Interconnect Ethernet DEVICE=eth2 BOOTPROTO=static Ipathether Configuration on Sles 00117504e011 Etc/udev/rules.d/30-netpersistentnames.rules Etc/sysconfig/hardware/hwcfg-eth-id-$MACEtc/sysconfig/network/ifcfg-eth2 MODULE=ipathether STARTMODE=auto OpenFabrics Drivers and Services Configuration and Startup Configuring the IPoIB Network Interface Output from this command will be similar to this Configuring and Administering the Vnic Interface Etc/sysconfig/infinipath. Change this line to00066a0130000001 Service 1 1000066a00000101 InfiniNIC.InfiniConSys.Data01 Etc/infiniband/qlgcvnic.cfg NAME=eioc1 IOCGUID=0x66A0137FFFFE7 # HEARTBEAT=100 This example uses DGID, Iocguid and Iocstring # /etc/init.d/infinipath start # /etc/init.d/qlgcvnic start # /etc/init.d/qlgcvnic stop OpenSM # ibqlgcvnicinfoEtc/init.d/opensmd Using Ofed SRP 21000001ff040bf6 # ibsrpdm -c # mkdir /mnt/targetname # mount /dev/sde1 /mnt/targetname Other Configuration Changing the MTU Size Starting and Stopping the InfiniPath SoftwareMPI over uDAPL To check the configuration state, use the command Rebuilding or Reinstalling Drivers After a Kernel Upgrade LED Link and Data Indicators Further Information on Configuring and Loading DriversLED Link and Data Indicators LED States Indication Adapter Settings Off Customer Acceptance Utility Hostname1 Hostname2 Command Meaning Ipathcheckout Options Uninstalling Ofed 1.3 Software Additional Installation InstructionsUninstalling InfiniPath and OpenFabrics RPMs Removing Software Packages Installing Lustre Installing QLogic MPI in an Alternate Location Installing on an Unsupported Distribution /etc/redhat-release file or the /etc/SuSE-release file Managing and Installing Software Using Rocks Usr/src/infinipath/drivers/build-guards.sh Installing Rocks and InfiniPath RPMs Use the following contents Software Installation Additional Installation Instructions Home/install/contrib/4.2.1/x8664/RPMS Further Information on Rocks and kickstart Page Node Spontaneously Reboots Hardware IssuesBios Settings Some HTX Motherboards May Need Two or More CPUs in Use Edit Bios Settings to Fix Mtrr Issues Mtrr Mapping and Write CombiningMtrr Mapping Discrete Software Installation Issues Issue with Supermicro H8DCE-HTe and QHT7040Use the ipathmtrr Script to Fix Mtrr Issues OpenFabrics Library Dependencies Version Number Conflict with opensm-*on RHEL5 SystemsOpenFabrics Dependencies Rpm no packages given for erase Missing Kernel RPM Errors Mpirun Installation Requires 32-bit Support Resolving ConflictsLibgcc-3.4.3-9.EL4.i386.rpm Glibc-2.3.4-2.i686.rpm or glibc-2.3.4-2.i386.rpm Lockable Memory Error on Initial Installation of InfiniPathIfup on ipathether on Sles 10 Reports unknown device Eth3 device QLogic Corp Unknown device 0010 revPage Table B-1. Configuration Files Configuration File Name Description Sample and Template Files Description RPM Version Numbers and Identifiers InfiniPath and OpenFabrics RPMsDifferent Nodes May Use Different RPMs Infinipath-2.2-33597.832rhel4qlc.x8664.rpm Table C-1. InfiniPath Documentation/RPMs RPM OrganizationDocumentation and InfiniPath RPMs RPM Name Front End Compute Development Table C-2. InfiniPath/RPMs Table C-3. InfiniPath-Devel/RPMsContains ipathcheckout Ipathbug-helpera Table C-5. OpenFabrics Documentation/RPMs OpenFabrics RPMsTable C-4. InfiniPath-MPI/RPMs RPM Name Comments Ibsim-2.2-xxx.04.yyy.x8664.rpm Dapl-utils-2.2-xxx.207.yyy.x8664.rpmIb-bonding-2.2-xxx.090.yyy.x8664.rpm Ibutils-2.2-xxx.12.yyy.x8664.rpm Libipathverbs-2.2-xxx.11.yyy.x8664.rpm Libibverbs-2.2-xxx.111.yyy.x8664.rpmLibibverbs-utils-2.2-xxx.111.yyy.x8664.rpm Librdmacm-2.2-xxx.106.yyy.x8664.rpm Table C-7. OpenFabrics-Devel/RPMs Table C-8. OpenSM/RPMs Table C-9. Other HCAs/RPMs RPM Name Front Comput Table C-10. Other HCAs-Devel/RPMsTable C-11. OtherMPIs/RPMs End RPM Name Front Comput Developme End Openmpipgi-2.2-xxx.125.yyy.x86 Openmpiintel-2.2-xxx.125.yyy.x8Openmpipathscale-2.2-xxx.125.yy Qlogic-mpi-regis RPM Descriptions OpenFabrics RPMs IB0056101-00 G Index Help Openssh Openssh-server 5-3 Optical media converters No packages given for erase warning message A-4 Verbose Page Page Camberley Surrey, GU16 7ER UK +44 0 1276 804