HP UX 11i v2 Networking Software manual

InfiniBand Transport Services

The InfiniBand transport-service types include:

•Reliable Connection

•Unreliable Connection

•Reliable Datagram

•Unreliable Datagram

HP only supports Reliable Connected and Unreliable Datagram transports in its fabric product.

HP Fabric Subnet Manager

The HP Fabric Subnet Manager (SM) configures and maintains fabric operations. There can be multiple Subnet Managers, but only one master. HP supports the use of an SM on the switch.

About the HP Fabric Clustering System Software Suite

The HP Fabric Software Suite supports the following upper-layer protocols: MPI and IPoIB. Oracle 10gRAC is also supported.

MPI

MPI stands for Message-Passing Interface, an industry standard for developing portable message-passing applications. HP MPI provides developers of technical applications with an application programming interface (API) and software library to support parallel, message-passing applications that are efficient, portable and flexible.

For more information on HP MPI, consult http://www.hp.com/go/mpi.

IPoIB

For more information on IPoIB, consult http://www.ietf.org/html.charters/ipoib-charter.html.

IPoIB allows IP applications to utilize the HP Fabric. IPoIB is configured like a normal Ethernet interface. During the installation process, IPoIB interfaces are automatically added to the network configuration. These interfaces correspond to the ports on the HCA.

Oracle 10gRAC

Oracle Corporation's clustered database.

For more information on 10gRAC, consult http://www.oracle.com and http://www.hp.com/go/oracle.

SKGXP

Middleware layer used by Oracle 10gRAC.

Itutil

The HP-UX Interconnect Transport utility, itutil, is a CLI (command line interface) to administer and manage HP-UX RDMA infrastructure and HP Fabric.

IT-API

The Interconnect Transport API, IT-API, is an industry standard API for RDMA-capable transports including InfiniBand.

For more information on IT-API, consult http://www.opengroup.org/icsc.

20 Introduction to Technology

Image 20

HP UX 11i v2 Networking Software manual About the HP Fabric Clustering System Software Suite

Contents

HP Fabric Clustering System HP-UX Administrators Guide Copyright 2004-2008 Hewlett-Packard Development Company, L.P Table of Contents Configuration Interfaces Switch Administration and Management 109 129 131 152 154155 156 227 217223 Page HyperTerminal Login List of FiguresPage Default User Names, Passwords and Privileges Description of Access LevelsList of Tables Show ib Keyword Descriptions Show logging Command Syntax Description Show trace Command Syntax DescriptionsShow user Command Syntax Descriptions IP Commands About This Document Intended AudienceNew and Changed Documentation in This Edition Publishing History What’s in This Document Typographical Conventions Related Documents HP Encourages Your Comments Related DocumentsPage Understanding InfiniBand Introduction to TechnologyUnderstanding the Fabric Clustering System About the HP Fabric Clustering System Software Suite User-Space Rdma Library Kernel Rdma SubsystemInteraction with HyperFabric Driver About the HP Fabric Clustering System Software SuitePage Hardware Overview Hardware ComponentsHost Channel Adapters HP offers the following new Host Channel Adapters HCA Hardware Overview AB286C Host Channel Adapter Connector View AB286C Host Channel Adapter Side View AB286A Host Channel Adapter Connector View AB286A Host Channel Adapter Side View Figureigure Hardware Components Switch Other Product Elements Hardware Overview Installation Planning Preliminary ConsiderationsApplications Preliminary Considerations Application Availability Installation PlanningSite Set-up Enter your product name Click the double arrow Installing HP Fabric Clustering SystemHP Fabric Clustering System Installation Prerequisites Installing HP Fabric Clustering System Install HP Fabric Clustering System AdaptersInstalling the AH304A Host Channel Adapter Secure the card and reassemble the system Installing the 410533-B21 Host Channel AdapterInstalling the AD313A Host Channel Adapter Installing the AB286C Host Channel Adapter Inserting the AD313A Host Channel Adapter On-Line Addition and Replacement Operations OL Inserting the AB286C Host Channel Adapter This opens the Software Selection window Install HP Fabric Clustering System SoftwareWhere devicename is the name assigned to the CD-ROM drive Install HP Fabric Clustering System Switches T-25 Torx screwdrivers Second person for the installationRack Mount Preparation IB subsystem IB Rdma IPoIB IB device driver ibt Installing the Switch Fixed and Sliding Rails Attach Rail to Switch Securely attach all rails to the rack Remove Support Bracket if installedRemove Shipping Sleeve Attach Cable Guides & Cables Attach Cable Guides & CablesCable Routing and Bend Limits Cable Installation Minimum Bend Radius Attach Cables Switch Setup Connect Cables to SwitchSwitch or Adapter Attach to other HP Fabric devices Setting up the Switch Enter configure to enter the global-configuration modeSet the default gateway address. This address is an example Enable the management port Internet Protocol over InfiniBand IPoIB Syntax and Example nwmgr -S ipoibPage Configuration Parameters ConfigurationSupported Features Non-Supported Features Supported ConfigurationsSample Configurations using the AB286A/C or AD313A HCA Configurations up to 24 End Nodes Configurations with More Than 24 End Nodes Sample Configurations using the AB286A/C or AD313A HCA 128 Node 50% non-Blocking Configuration Administration and Management Following is a listing of the major topics in this chapter Administration and Management HP-UX Host Administration and ManagementUsing Itutil Summary of Itutil Command Options Itutil Administrative CommandsItutil Command Options Following syntax must be used with itutil command options Itutil Management Commands Syntax itutil Syntax Example itutil Output Example Syntax Example itutil -i ib0 Output Example Syntax Example itutil -t IB Output Example Syntax Example itutil -s-i ib0 Output Example Administration and Management Syntax Example itutil -c Output Example Displaying Connectivity Information Using ItutilSyntax Example itutil -r Output Example Example itutil -T Using NetTL Viewing the Itutil Manpage Syntax nettl -llog class -esubsystem IPoIB Administration and Management Ifconfig CommandIB Tracing IPoIB Tracing Lanadmin Commands Disable the IPoIB Interface To disable the IPoIB interfaceDisplaying IPoIB Interface Link-Level Information Level information for a given ipoibppa Syntax Example lanadmin -g Output Example Syntax Example lanadmin -a Output ExampleSyntax Example lanadmin -s Output Example Syntax lanadmin -x-hipoibppa Syntax Example lanadmin -x -h Introduction Serviceguard and IPoIBLanscan Command Configuring IPoIB Interfaces for Serviceguard Clusters Using Serviceguard and IPoverIB Switch Administration and Management Switch Administration and Management Using the CLI CLI OverviewStarting a CLI Session Privilege Level Administrator Roles Entering CLI Modes Using Command Completion Exiting CLI Modes Command Abbreviation Quick Help Command-Line Editing Command before returning to user-execute mode Exiting the CLI SessionSpecifying the Card/Port Key Stroke Shortcuts Card/Port pairs RangesIndicates ports 2 through 4 on card Lists Advanced Switch Setup Configuring the System HostnameTesting Network Connectivity Enter the global-configuration mode Setting User Levels and Passwords Managing the SwitchLogging Onto the System Logging On Through the CLI Management Methods Managing Through the CLI Configuration, Image, and Log Files Configuration, Image, and Log File Overview Understanding the Upgrade Process Image Upgrade Procedure SummaryUpgrading Image Files Set-Up the Hardware Connection Enter the privileged-execute mode Check the Image Version Copy/Download the Image Optional View the images by using the dir image command Install a New Image Specify a New Boot Image Show the new system -image Deleting System Images and Image Files Reboot the SystemTroubleshooting an Image Upgrade Listing Configuration, Image, and Log Files Viewing Configuration FilesFile Management Viewing Log Files Saving Configuration Files Saving for System RebootSaving the Backup Configuration Specifying the Configuration to Use at System Reboot Arguments to the copy command are described below Saving and Copying FilesExample Show the new system image Deleting Configuration, Image, and Log Files Downloading Files to the SystemColon Tslog. The .cfg extension is optional Deleting Configuration Files Determine the configuration files in memoryDeleting Log Files Display the log files that are in memory Deleting Image Files Determine the currently installed system-imagesVerify that you had successfully removed the log file HP Fabric Switch devices use the following log format Managing Log FilesUnderstanding the Log Format Uploading Log Files Administering the System Switching User Identity Sending Messages to Individual Users Notifying UsersBroadcasting Messages to Users Understanding Usernames and Passwords Default User Names, Passwords and PrivilegesSetting or Changing a Password Creating User Accounts Displaying User Information Adding New Users Deleting a User Account Community Strings User Account Configuration CommandsUser Account Administrative Commands Following table displays the different access-levels Using DNS ServicesSetting Administrative Roles Setting the System Clock Setting the NTP Servers to Maintain the System Clock Rebooting the SystemSetting Time Through the CLI Save your configuration Reboot the System Through the CLI 108 Diagnosing Problems Monitoring and TroubleshootingDiagnosing Problems HP Fabric Switch Cluster Connection LEDs HP Fabric Switch System StatusIB Port Status LED Indicators LED Color, Behavior, and Meaning Front System Status LED Indicator Rear System Status LED Indicator Power Supply Troubleshooting Power supply/ Fan Status LED Indicator Ethernet Port Status LED Indicators Ethernet Management Port LEDsHP Fabric HCA Connection LEDs Bottom Top Green- over Green Off Flickering Been Top Established None HCA is not HCA Connection LED Colors and MeaningsLogical Solid Indicates a Determining Whether the HCA or Cable is Faulty Determining if the Switch is Faulty Next Steps Monitoring and Troubleshooting the HP-UX HostHP-UX Host Troubleshooting Procedure Monitoring and Troubleshooting the HP-UX Host Syntax example itutil Output example Monitoring and Troubleshooting Syntax example itutil -c fe802c9018a08a11 ib0 Sample nettl Log Messages Description Port failure Monitoring and Troubleshooting IPoIB Nettl Sample Log OutputSample IB Logs Following are examples of nettl log output Sample IPoIB Logs PDU in Trace NetTL Linkloop CommandKnown Problems Monitoring and Troubleshooting the Switch Health MonitoringIP Filter Product CLI can also be used to monitor the system About Tracing About Logging and TracingAbout HP Fabric Events Flow Troubleshooting the HP Fabric NetworkPing Setting Trace Levels Enabling TracingDisabling Tracing Verify Link Speed Replacing a Power Supply Module Replacing Individual Components Specifications Physical and Environmental SpecificationsHCA Physical Specifications AB286C HCA Environmental Specifications AB286C 130 Switch Command Line Interface Using the Documentation Show Commands DefaultsRelated Commands Examples Show arp IB Defaults There are no defaults for this commandShow arp ethernet Show authentication Examples To display the authentication methodShow backplane Examples To display the InfiniBand ARP table Table B-3 show boot-config Command Field Descriptions Show boot-configDefaults This command has no defaults Show card Syntax Description This command has the following argumentsShow card Defaults show card defaults to show card all Image data for internal configurationOper status Card Table B-6 Show card-inventory Command Syntax Descriptions Table B-7 show card-inventory Command Field DescriptionsShow card-inventory Synopsis Show clock Show configRelated Commands clock Related Commands copy Table B-8 show fan Command Field DescriptionsShow fan Syntax show fan Show ib sm configuration Show hostShow ib Table B-9 show ib Keyword Descriptions Table B-10 Show ib Command Syntax Descriptions Table B-11 Show ib Command Field DescriptionsUser-execute and privileged-execute modes InfiniBand read-only user Show ib sm neighbor Table B-12 show ib sm multicast Command Syntax DescriptionsShow ib sm multicast Show ib sm node subnet-prefix Table B-13 show ib sm neighbor Command Field DescriptionsArguments associated with this command are described below Node guid Use with the all keyword Class-versionType An unknown type Show ib sm partition Show ib sm port Table B-16 Show ib sm port Command Syntax DescriptionsTable B-17 show ib sm port Command Field Descriptions Value may be No state change1x or 12x Received by this port. There is no default value Transmitted by this port. There is no default valuePackets received by this port. There is no default value Packets transmitted by this port. There is no default value Local-phy-error Errors and the buffers are not immediately reclaimed Show ib sm service Table B-18 Show ib sm service Command Syntax Descriptions Table B-20 show ib sm switch Command Field Descriptions Show ib sm switchTable B-19 show ib sm switch Command Syntax Descriptions Partition enforcement is not supported by the switch Life-time-valuePort-state-change Lid-per-port Show ib-agent channel-adapter Table B-22 Show ib-agent summary Command Field Descriptions Show ib-agent summaryUsage Guidelines form Slot System chassis slot in which the device resides Type Show ib-agent switch Following example displays a summary of all the SMA nodesAgent Lid Decimal-base LID of this port Swguid Show ib-agent switch linear-frd-infoDisplay the attributes of a single switch Show ib-agent switch all mcast-info lid Show ib-agent switch all node-infoSwguid Guid of a specific InfiniBand switch Type, GUIDs, and capabilities Defaults This command has no defaults Usage Guidelines None Examples Show ib-agent switch all pkey-infoShow ib-agent switch port-info Show ib-agent switch sl-vl-map Node-guid Bit Guid of this node In-ib-portOut-ib-port SL0 to VL mapping Show ib-agent switch switch-info Table B-30 Show interface ib Command Syntax Descriptions Show interface ibSyntax is described in the table below Table B-31 Show interface ib Output Descriptions Link-trap Show interface ib sm Usage Guidelines None Show interface ib sm statistics Show interface mgmt-ethernet Show interface mgmt-ib Show interface mgmt-serial Show ip Show logging Table B-33 show ip Command Syntax DescriptionsShow location Table B-34 show logging Command Syntax Description Show ntpSyntax show ntp End Show power-supply Related Commands ntpShow running-status Usage Guidelines Examples Show sensor Show system-services Show snmp Show trace Table B-36 show trace Command Syntax DescriptionsShow terminal Table B-37 Show user Command Syntax Descriptions Show userUsername Specify the name of a specific user Shows all users in the user database Show version Arp ib IP CommandsTable B-38 IP Commands Default ip ip-over-ib-mtu Clear ib arp-cacheTable B-39 Auto-negotiate Syntax Description Enter the IP address of the target host Defaults The default ip-over-ib-mtu is Defaults The default domain name is an empty stringIp domain-name Ip ip-over-ib-mtu Defaults The default ip-over-ib-mtu unit is Table B-42 Command Syntax DescriptionDefaults The default is an empty string Ip name-server-one Table B-43 Command Syntax Description Table B-44 Command Syntax DescriptionIp name-server-two Ip route InfiniBand Commands Table B-45 InfiniBand CommandsIb sm subnet-prefix Arp ib Table B-46 ib sm subnet-prefix Syntax Description Table B-47 Acceptable PKey Values Following example removes a specified subnet managerFollowing example creates a Partition, and adds Sweep-interval is 10 seconds Ib-agent switch Response-timeout is 2,000 microsecondsIb-agent channel-adapter Syntax is described in the following table Config-if-ib submode Table B-50 interface ib Command Syntax DescriptionInterface ib Link-trap Commands syntax is described in the table below Defaults The default is no link-trapDefaults The default interface name is card#/port# Name Administrative Commands Commands stoats is described in the table belowTable B-53 Shutdown Command Syntax Description Table B-54 Administrative Commands Ftp-server enable Use to enable or disable ftp-services Logging Configures active logConfigures a radius-server. See radius-server Reboots the chassis. See reload Boot-config Table B-56 boot-config KeywordsAction Table B-55 action Keywords Broadcast Table B-57 boot-config ArgumentsTable B-58 Broadcast Command Arguments Configure Table B-59 clock Command Argument DescriptionsCard Clock Syntax Table B-60 copy Command Syntax Descriptions Table B-61 Copy Command Argument DescriptionsSystem upon boot-up Copy Arguments related to the delete command are described below Table B-62 Delete Command Argument DescriptionsDelete Syntax delete fsfile Related Commands card Table B-63 Dir Command Argument DescriptionsTo display installed system-images and image files Dir Disable EnableSyntax disable Syntax enable Exit Ftp-server enableExec Gateway Privileged-execute modes Defaults The default gateway IP address is Related Commands NoneHelp History Install Synopsis The hostname command assigns a name to the systemTable B-69 hostname Command Argument Description Hostname Interface Table B-70 install Command Argument DescriptionsTable B-71 interface Command Keywords Interface mgmt-ethernet Example Related Commands ip Interface mgmt-ibSyntax ipip mask No ipip mask Location Table B-72 Location Syntax Descriptions Login Synopsis The logout command ends the current CLI sessionLogging Logout Related Commands exit Related Commands dirMore Ntp Ping Radius-serverSyntax ping dest Table B-73 Radius-server Syntax Descriptions Synopsis The reload command reboots the chassis ReloadSyntax reload Enabling/Disabling the Ethernet Management Port Snmp-server No snmp-server Table B-74 Snmp-server Command Keyword Descriptions Table B-75 snmp-server Command Argument DescriptionsCommand Modes Privilege Level Usage Guidelines Defaults Telnet access is enabled by default Defaults The default is 24 lines per screen Defaults The default time-out interval is 15 minutesTerminal length Terminal time-out Syntax type type Table B-76 type Command Argument Type Username Syntax who Table B-77 Default User AccountsWho Write DefaultsThis command has no defaults 216 How to Use Windows HyperTerminal Attaching through Windows HyperTerminalHyperTerminal Prerequisite Loading HyperTerminal Configuring the HyperTerminal Connection How to Use Windows HyperTerminal Figure C-3 Connect To Dialog Box Verifying the HyperTerminal Connection Figure C-4 COM Port Properties Dialog Box Figure C-5 HyperTerminal Login 222 Glossary 223 Ietf Glossary Snmp 225 226 227 Ib-rw, 104 setting levels, 104 unrestricted-rwIndex CLI Index 229 See also HCAs hostname 231 Index See also ports Snmp 235 Changing, 104 user access levels changingAbout, 104 adding, 101 creating, 100 user information Index