Q-Logic IB0056101-00 G manual Configuring the ibipath Driver, Ipathether Configuration on Red Hat

Page 54

5 – Software Installation Configuring the InfiniPath Drivers

S

Configuring the ib_ipath Driver

The ib_ipath module provides both low-level InfiniPath support and management functions for OpenFabrics protocols. The ib_ipath driver has several configuration variables that set reserved buffers for the software, define events to create trace records, and set the debug level.The startup script for ib_ipath is installed automatically as part of the software installation, and normally does not need to be changed.

The primary configuration file for the InfiniPath drivers ib_ipath and ipath_ether, and other modules and associated daemons, is:

/etc/sysconfig/infinipath

Normally, this configuration file is set up correctly at installation and the drivers are loaded automatically during system boot once the RPMs have been installed. If you are upgrading, your existing configuration files will not be overwritten.

The device files are:

/dev/ipath

/dev/ipath0, /dev/ipath1, ...

The numbered device files allow access to a specific InfiniPath unit.

See the ib_ipath man page for more details.

Configuring the ipath_ether Network Interface

These instructions are for enabling TCP/IP networking over the InfiniPath link. To enable IPoIB networking, see “Configuring the IPoIB Network Interface” on page 5-17.

You must create a network device configuration file for the layered Ethernet device on the QLogic adapter. This configuration file will resemble the configuration files for the other Ethernet devices on the nodes.

Two slightly different procedures are given in the following sections for the ipath configuration; one for Fedora/RHEL (see “ipath_ether Configuration on Red Hat” on page 5-12) and one for SLES 10 (see “ipath_ether Configuration on SLES” on page 5-14).

Many of the entries that are used in the configuration directions are explained in the file sysconfig.txt, located in the following directory:

/usr/share/doc/initscripts-*/sysconfig.txt

ipath_ether Configuration on Red Hat

The following procedure will cause the ipath_ether network interfaces to be automatically configured the next time you reboot the system. These instructions are for the Fedora 6, Red Hat Enterprise Linux 4 (RHEL4), and RHEL5 distributions.

5-12

IB0056101-00 G

Image 54
Contents QLogic HCA and InfiniPath→ Software Install Guide QLogic HCA and InfiniPath→ Software Install Guide Version Document Revision HistoryTable of Contents Software Installation Installation Troubleshooting Configuration Files RPM Descriptions List of Figures Page Who Should Read this Guide How this Guide is OrganizedOverview Typographical Conventions InteroperabilityConventions Used in this Guide Convention MeaningDocumentation Contact Information Support HeadquartersIntroduction Contact Information IB0056101-00 G What’s New in this Release QLogic Adapter Model NumbersDescription New Features QLogic Model DescriptionSupported Distributions and Kernels InfiniPath/OpenFabrics Supported Distributions KernelsDistribution InfiniPath/OpenFabrics Supported Compiler Support Software ComponentsDistribution InfiniPath/OpenFabrics Supported Kernels „ Ofed SRP Feature Overview Software Components Step-by-Step Installation Checklist Hardware InstallationSoftware Installation Installing Software Using Rocks onStep-by-Step Installation Checklist Software Installation Page Adapter Models and Related Platforms Hardware Installation RequirementsHardware QLogic Platform Plugs IntoForm Factors Cabling and Switches QLogic InfiniBand CablesProduct Number Description Configuring the Bios Optical Fibre OptionList of the Package Contents Safety with ElectricityVerify the Package Contents Unpacking InformationHardware Installation Unpacking Information QLogic QLE7280 with IBA7220 Asic Unpacking the QLogic Adapter AsicDual Adapter Installation Installation StepsPCIe Slot in a Typical Motherboard QLogic PCIe HCA Assembly with Riser Card Hardware Installation for QHT7140 with HTX Riser Assembled PCIe HCA with RiserHTX Slot QLogic QHT7140 Adapter with Riser Card Assembled QHT7140 with Riser Hardware Installation for the QHT7140 Without an HTX Riser Switch Configuration and Monitoring Cabling the Adapter to the InfiniBand SwitchCompleting the Installation Cluster Setup Types of Nodes in a Cluster EnvironmentSetting Up Your Environment Supported Linux DistributionsInfiniPath/OpenFabrics Supported Distributions and Kernels Choosing the Appropriate Download Files Available PackagesPackage Description Comments Unpacking the InfiniPath tar File Check for Missing Files or RPMs Libibverbs-2.2-xxx.11.yyy.x8664.rpmRhel4 Installing the InfiniPath and OpenFabrics RPMsDistribution Used On Sles10Choosing the RPMs to Install Rpms to InstallIf you want to Then install Using rpm to Install InfiniPath and OpenFabrics PreparingInstalled Layout Starting the InfiniPath Service Configuring the InfiniPath Drivers InfiniPath and OpenFabrics Driver OverviewConfiguring the ibipath Driver Configuring the ipathether Network InterfaceIpathether Configuration on Red Hat Etc/sysconfig/network-scripts/ifcfg-eth2 # QLogic Interconnect Ethernet DEVICE=eth2 BOOTPROTO=staticIpathether Configuration on Sles 00117504e011Etc/udev/rules.d/30-netpersistentnames.rules Etc/sysconfig/hardware/hwcfg-eth-id-$MACEtc/sysconfig/network/ifcfg-eth2 MODULE=ipathether STARTMODE=autoOpenFabrics Drivers and Services Configuration and Startup Configuring the IPoIB Network Interface Output from this command will be similar to thisConfiguring and Administering the Vnic Interface Etc/sysconfig/infinipath. Change this line to00066a0130000001 Service 1 1000066a00000101 InfiniNIC.InfiniConSys.Data01 Etc/infiniband/qlgcvnic.cfgNAME=eioc1 IOCGUID=0x66A0137FFFFE7 # HEARTBEAT=100This example uses DGID, Iocguid and Iocstring # /etc/init.d/infinipath start # /etc/init.d/qlgcvnic start # /etc/init.d/qlgcvnic stopOpenSM # ibqlgcvnicinfoEtc/init.d/opensmd Using Ofed SRP 21000001ff040bf6# ibsrpdm -c # mkdir /mnt/targetname # mount /dev/sde1 /mnt/targetnameOther Configuration Changing the MTU Size Starting and Stopping the InfiniPath SoftwareMPI over uDAPL To check the configuration state, use the command Rebuilding or Reinstalling Drivers After a Kernel Upgrade LED Link and Data Indicators Further Information on Configuring and Loading DriversLED Link and Data Indicators LED States IndicationAdapter Settings OffCustomer Acceptance Utility Hostname1 Hostname2Command Meaning Ipathcheckout OptionsUninstalling Ofed 1.3 Software Additional Installation InstructionsUninstalling InfiniPath and OpenFabrics RPMs Removing Software PackagesInstalling Lustre Installing QLogic MPI in an Alternate LocationInstalling on an Unsupported Distribution /etc/redhat-release file or the /etc/SuSE-release fileManaging and Installing Software Using Rocks Usr/src/infinipath/drivers/build-guards.shInstalling Rocks and InfiniPath RPMs Use the following contents Software Installation Additional Installation Instructions Home/install/contrib/4.2.1/x8664/RPMS Further Information on Rocks and kickstart Page Node Spontaneously Reboots Hardware IssuesBios Settings Some HTX Motherboards May Need Two or More CPUs in UseEdit Bios Settings to Fix Mtrr Issues Mtrr Mapping and Write CombiningMtrr Mapping Discrete Software Installation Issues Issue with Supermicro H8DCE-HTe and QHT7040Use the ipathmtrr Script to Fix Mtrr Issues OpenFabrics Library Dependencies Version Number Conflict with opensm-*on RHEL5 SystemsOpenFabrics Dependencies Rpm no packages given for eraseMissing Kernel RPM Errors Mpirun Installation Requires 32-bit Support Resolving ConflictsLibgcc-3.4.3-9.EL4.i386.rpm Glibc-2.3.4-2.i686.rpm or glibc-2.3.4-2.i386.rpm Lockable Memory Error on Initial Installation of InfiniPathIfup on ipathether on Sles 10 Reports unknown device Eth3 device QLogic Corp Unknown device 0010 revPage Table B-1. Configuration Files Configuration File Name DescriptionSample and Template Files Description RPM Version Numbers and Identifiers InfiniPath and OpenFabrics RPMsDifferent Nodes May Use Different RPMs Infinipath-2.2-33597.832rhel4qlc.x8664.rpmTable C-1. InfiniPath Documentation/RPMs RPM OrganizationDocumentation and InfiniPath RPMs RPM Name Front End Compute DevelopmentTable C-2. InfiniPath/RPMs Table C-3. InfiniPath-Devel/RPMsContains ipathcheckout Ipathbug-helpera Table C-5. OpenFabrics Documentation/RPMs OpenFabrics RPMsTable C-4. InfiniPath-MPI/RPMs RPM Name CommentsIbsim-2.2-xxx.04.yyy.x8664.rpm Dapl-utils-2.2-xxx.207.yyy.x8664.rpmIb-bonding-2.2-xxx.090.yyy.x8664.rpm Ibutils-2.2-xxx.12.yyy.x8664.rpmLibipathverbs-2.2-xxx.11.yyy.x8664.rpm Libibverbs-2.2-xxx.111.yyy.x8664.rpmLibibverbs-utils-2.2-xxx.111.yyy.x8664.rpm Librdmacm-2.2-xxx.106.yyy.x8664.rpmTable C-7. OpenFabrics-Devel/RPMs Table C-8. OpenSM/RPMs Table C-9. Other HCAs/RPMsRPM Name Front Comput Table C-10. Other HCAs-Devel/RPMsTable C-11. OtherMPIs/RPMs EndRPM Name Front Comput Developme End Openmpipgi-2.2-xxx.125.yyy.x86 Openmpiintel-2.2-xxx.125.yyy.x8Openmpipathscale-2.2-xxx.125.yy Qlogic-mpi-regisRPM Descriptions OpenFabrics RPMs IB0056101-00 G Index Help Openssh Openssh-server 5-3 Optical media converters No packages given for erase warning message A-4 Verbose Page Page Camberley Surrey, GU16 7ER UK +44 0 1276 804