Using Serviceguard and IPoverIB | HP UX 11i v2 Networking Software

NOTE: Please refer to the Serviceguard Configuration Guide for complete details of the supported SGeRAC configurations.

NOTE: Oracle RAC uses IPoIB for cache fusion traffic.

NOTE: IPV6 failover is not supported with ServiceGuard. IPoIB can run over IPV6, but the InfiniBand switch does not support IPV6 address failover.

Using Serviceguard and IPoverIB

Serviceguard actively monitors IPoIB interfaces configured in the cluster and upon detecting failure of the primary IPoIB interface(s), provides local IP address failover to any available standby IPoIB interfaces.

Configuring IPoIB Interfaces for Serviceguard Clusters

Configuration of IPoIB interfaces in a Serviceguard cluster is not different from configuration of any other LAN interfaces. The following diagrams show the supported configurations of IPoIB as cluster heartbeat:

Supported Configuration #1: Single IB HCA, Single Serviceguard Heartbeat:

Supported Configuration #2: Multiple IB HCAs, Single Serviceguard Heartbeat:

Supported Configuration #3: Multiple IB HCAs, Multiple Serviceguard Heartbeats:

72 Administration and Management

Image 72

HP UX 11i v2 Networking Software Configuring IPoIB Interfaces for Serviceguard Clusters, Using Serviceguard and IPoverIB

Contents

HP Fabric Clustering System HP-UX Administrators Guide Copyright 2004-2008 Hewlett-Packard Development Company, L.P Table of Contents Configuration Interfaces Switch Administration and Management 109 129 131 152 154155 156 217 223227 Page HyperTerminal Login List of FiguresPage Default User Names, Passwords and Privileges Description of Access LevelsList of Tables Show ib Keyword Descriptions Show logging Command Syntax Description Show trace Command Syntax DescriptionsShow user Command Syntax Descriptions IP Commands About This Document Intended AudienceNew and Changed Documentation in This Edition Publishing History What’s in This Document Typographical Conventions Related Documents HP Encourages Your Comments Related DocumentsPage Introduction to Technology Understanding the Fabric Clustering SystemUnderstanding InfiniBand About the HP Fabric Clustering System Software Suite User-Space Rdma Library Kernel Rdma SubsystemInteraction with HyperFabric Driver About the HP Fabric Clustering System Software SuitePage Hardware Overview Hardware ComponentsHost Channel Adapters HP offers the following new Host Channel Adapters HCA Hardware Overview AB286C Host Channel Adapter Connector View AB286C Host Channel Adapter Side View AB286A Host Channel Adapter Connector View AB286A Host Channel Adapter Side View Figureigure Hardware Components Switch Other Product Elements Hardware Overview Installation Planning Preliminary ConsiderationsApplications Preliminary Considerations Installation Planning Site Set-upApplication Availability Installing HP Fabric Clustering System HP Fabric Clustering System Installation PrerequisitesEnter your product name Click the double arrow Install HP Fabric Clustering System Adapters Installing the AH304A Host Channel AdapterInstalling HP Fabric Clustering System Installing the 410533-B21 Host Channel Adapter Installing the AD313A Host Channel AdapterSecure the card and reassemble the system Installing the AB286C Host Channel Adapter Inserting the AD313A Host Channel Adapter On-Line Addition and Replacement Operations OL Inserting the AB286C Host Channel Adapter Install HP Fabric Clustering System Software Where devicename is the name assigned to the CD-ROM driveThis opens the Software Selection window Install HP Fabric Clustering System Switches T-25 Torx screwdrivers Second person for the installationRack Mount Preparation IB subsystem IB Rdma IPoIB IB device driver ibt Installing the Switch Fixed and Sliding Rails Attach Rail to Switch Remove Support Bracket if installed Remove Shipping SleeveSecurely attach all rails to the rack Attach Cable Guides & Cables Cable Routing and Bend LimitsAttach Cable Guides & Cables Cable Installation Minimum Bend Radius Attach Cables Switch Setup Connect Cables to SwitchSwitch or Adapter Attach to other HP Fabric devices Setting up the Switch Enter configure to enter the global-configuration modeSet the default gateway address. This address is an example Enable the management port Internet Protocol over InfiniBand IPoIB Syntax and Example nwmgr -S ipoibPage Configuration Supported FeaturesConfiguration Parameters Non-Supported Features Supported ConfigurationsSample Configurations using the AB286A/C or AD313A HCA Configurations up to 24 End Nodes Configurations with More Than 24 End Nodes Sample Configurations using the AB286A/C or AD313A HCA 128 Node 50% non-Blocking Configuration Administration and Management Following is a listing of the major topics in this chapter HP-UX Host Administration and Management Using ItutilAdministration and Management Summary of Itutil Command Options Itutil Administrative CommandsItutil Command Options Following syntax must be used with itutil command options Itutil Management Commands Syntax itutil Syntax Example itutil Output Example Syntax Example itutil -i ib0 Output Example Syntax Example itutil -t IB Output Example Syntax Example itutil -s-i ib0 Output Example Administration and Management Displaying Connectivity Information Using Itutil Syntax Example itutil -r Output ExampleSyntax Example itutil -c Output Example Example itutil -T Using NetTL Viewing the Itutil Manpage Syntax nettl -llog class -esubsystem IPoIB Administration and Management Ifconfig CommandIB Tracing IPoIB Tracing Lanadmin Commands Disable the IPoIB Interface To disable the IPoIB interfaceDisplaying IPoIB Interface Link-Level Information Level information for a given ipoibppa Syntax Example lanadmin -a Output Example Syntax Example lanadmin -s Output ExampleSyntax Example lanadmin -g Output Example Syntax lanadmin -x-hipoibppa Syntax Example lanadmin -x -h Serviceguard and IPoIB Lanscan CommandIntroduction Configuring IPoIB Interfaces for Serviceguard Clusters Using Serviceguard and IPoverIB Switch Administration and Management Switch Administration and Management Using the CLI CLI OverviewStarting a CLI Session Privilege Level Administrator Roles Entering CLI Modes Using Command Completion Exiting CLI Modes Command Abbreviation Quick Help Command-Line Editing Command before returning to user-execute mode Exiting the CLI SessionSpecifying the Card/Port Key Stroke Shortcuts Card/Port pairs RangesIndicates ports 2 through 4 on card Lists Advanced Switch Setup Configuring the System HostnameTesting Network Connectivity Enter the global-configuration mode Setting User Levels and Passwords Managing the SwitchLogging Onto the System Logging On Through the CLI Management Methods Managing Through the CLI Configuration, Image, and Log Files Configuration, Image, and Log File Overview Understanding the Upgrade Process Image Upgrade Procedure SummaryUpgrading Image Files Set-Up the Hardware Connection Enter the privileged-execute mode Check the Image Version Copy/Download the Image Optional View the images by using the dir image command Install a New Image Specify a New Boot Image Show the new system -image Reboot the System Troubleshooting an Image UpgradeDeleting System Images and Image Files Listing Configuration, Image, and Log Files Viewing Configuration FilesFile Management Viewing Log Files Saving Configuration Files Saving for System RebootSaving the Backup Configuration Specifying the Configuration to Use at System Reboot Arguments to the copy command are described below Saving and Copying FilesExample Show the new system image Deleting Configuration, Image, and Log Files Downloading Files to the SystemColon Tslog. The .cfg extension is optional Deleting Configuration Files Determine the configuration files in memoryDeleting Log Files Display the log files that are in memory Determine the currently installed system-images Verify that you had successfully removed the log fileDeleting Image Files HP Fabric Switch devices use the following log format Managing Log FilesUnderstanding the Log Format Uploading Log Files Administering the System Switching User Identity Notifying Users Broadcasting Messages to UsersSending Messages to Individual Users Understanding Usernames and Passwords Default User Names, Passwords and PrivilegesSetting or Changing a Password Creating User Accounts Displaying User Information Adding New Users Deleting a User Account User Account Configuration Commands User Account Administrative CommandsCommunity Strings Using DNS Services Setting Administrative RolesFollowing table displays the different access-levels Setting the System Clock Setting the NTP Servers to Maintain the System Clock Rebooting the SystemSetting Time Through the CLI Save your configuration Reboot the System Through the CLI 108 Monitoring and Troubleshooting Diagnosing ProblemsDiagnosing Problems HP Fabric Switch Cluster Connection LEDs HP Fabric Switch System StatusIB Port Status LED Indicators LED Color, Behavior, and Meaning Front System Status LED Indicator Rear System Status LED Indicator Power Supply Troubleshooting Power supply/ Fan Status LED Indicator Ethernet Port Status LED Indicators Ethernet Management Port LEDsHP Fabric HCA Connection LEDs Bottom Top Green- over Green Off Flickering HCA Connection LED Colors and Meanings Logical Solid Indicates aBeen Top Established None HCA is not Determining Whether the HCA or Cable is Faulty Determining if the Switch is Faulty Monitoring and Troubleshooting the HP-UX Host HP-UX Host Troubleshooting ProcedureNext Steps Monitoring and Troubleshooting the HP-UX Host Syntax example itutil Output example Monitoring and Troubleshooting Syntax example itutil -c fe802c9018a08a11 ib0 Sample nettl Log Messages Description Port failure Monitoring and Troubleshooting IPoIB Nettl Sample Log OutputSample IB Logs Following are examples of nettl log output Sample IPoIB Logs PDU in Trace Linkloop Command Known ProblemsNetTL Monitoring and Troubleshooting the Switch Health MonitoringIP Filter Product CLI can also be used to monitor the system About Logging and Tracing About HP Fabric EventsAbout Tracing Troubleshooting the HP Fabric Network PingFlow Setting Trace Levels Enabling TracingDisabling Tracing Verify Link Speed Replacing a Power Supply Module Replacing Individual Components Specifications Physical and Environmental SpecificationsHCA Physical Specifications AB286C HCA Environmental Specifications AB286C 130 Switch Command Line Interface Using the Documentation Show Commands DefaultsRelated Commands Examples Defaults There are no defaults for this command Show arp ethernetShow arp IB Show authentication Examples To display the authentication methodShow backplane Examples To display the InfiniBand ARP table Show boot-config Defaults This command has no defaultsTable B-3 show boot-config Command Field Descriptions This command has the following arguments Show cardShow card Syntax Description Defaults show card defaults to show card all Image data for internal configurationOper status Card Table B-6 Show card-inventory Command Syntax Descriptions Table B-7 show card-inventory Command Field DescriptionsShow card-inventory Synopsis Show config Related Commands clockShow clock Related Commands copy Table B-8 show fan Command Field DescriptionsShow fan Syntax show fan Show ib sm configuration Show hostShow ib Table B-9 show ib Keyword Descriptions Table B-10 Show ib Command Syntax Descriptions Table B-11 Show ib Command Field DescriptionsUser-execute and privileged-execute modes InfiniBand read-only user Table B-12 show ib sm multicast Command Syntax Descriptions Show ib sm multicastShow ib sm neighbor Table B-13 show ib sm neighbor Command Field Descriptions Arguments associated with this command are described belowShow ib sm node subnet-prefix Node guid Use with the all keyword Class-versionType An unknown type Show ib sm partition Table B-16 Show ib sm port Command Syntax Descriptions Table B-17 show ib sm port Command Field DescriptionsShow ib sm port Value may be No state change1x or 12x Received by this port. There is no default value Transmitted by this port. There is no default valuePackets received by this port. There is no default value Packets transmitted by this port. There is no default value Local-phy-error Errors and the buffers are not immediately reclaimed Show ib sm service Table B-18 Show ib sm service Command Syntax Descriptions Show ib sm switch Table B-19 show ib sm switch Command Syntax DescriptionsTable B-20 show ib sm switch Command Field Descriptions Partition enforcement is not supported by the switch Life-time-valuePort-state-change Lid-per-port Show ib-agent channel-adapter Table B-22 Show ib-agent summary Command Field Descriptions Show ib-agent summaryUsage Guidelines form Slot System chassis slot in which the device resides Type Show ib-agent switch Following example displays a summary of all the SMA nodesAgent Lid Decimal-base LID of this port Show ib-agent switch linear-frd-info Display the attributes of a single switchSwguid Show ib-agent switch all mcast-info lid Show ib-agent switch all node-infoSwguid Guid of a specific InfiniBand switch Type, GUIDs, and capabilities Defaults This command has no defaults Show ib-agent switch all pkey-info Show ib-agent switch port-infoUsage Guidelines None Examples Show ib-agent switch sl-vl-map Node-guid Bit Guid of this node In-ib-portOut-ib-port SL0 to VL mapping Show ib-agent switch switch-info Table B-30 Show interface ib Command Syntax Descriptions Show interface ibSyntax is described in the table below Table B-31 Show interface ib Output Descriptions Link-trap Show interface ib sm Usage Guidelines None Show interface ib sm statistics Show interface mgmt-ethernet Show interface mgmt-ib Show interface mgmt-serial Show ip Table B-33 show ip Command Syntax Descriptions Show locationShow logging Table B-34 show logging Command Syntax Description Show ntpSyntax show ntp End Show power-supply Related Commands ntpShow running-status Usage Guidelines Examples Show sensor Show system-services Show snmp Table B-36 show trace Command Syntax Descriptions Show terminalShow trace Table B-37 Show user Command Syntax Descriptions Show userUsername Specify the name of a specific user Shows all users in the user database Show version IP Commands Table B-38 IP CommandsArp ib Default ip ip-over-ib-mtu Clear ib arp-cacheTable B-39 Auto-negotiate Syntax Description Enter the IP address of the target host Defaults The default ip-over-ib-mtu is Defaults The default domain name is an empty stringIp domain-name Ip ip-over-ib-mtu Defaults The default ip-over-ib-mtu unit is Table B-42 Command Syntax DescriptionDefaults The default is an empty string Ip name-server-one Table B-43 Command Syntax Description Table B-44 Command Syntax DescriptionIp name-server-two Ip route InfiniBand Commands Table B-45 InfiniBand CommandsIb sm subnet-prefix Arp ib Table B-46 ib sm subnet-prefix Syntax Description Table B-47 Acceptable PKey Values Following example removes a specified subnet managerFollowing example creates a Partition, and adds Sweep-interval is 10 seconds Ib-agent switch Response-timeout is 2,000 microsecondsIb-agent channel-adapter Syntax is described in the following table Config-if-ib submode Table B-50 interface ib Command Syntax DescriptionInterface ib Link-trap Commands syntax is described in the table below Defaults The default is no link-trapDefaults The default interface name is card#/port# Name Administrative Commands Commands stoats is described in the table belowTable B-53 Shutdown Command Syntax Description Table B-54 Administrative Commands Ftp-server enable Use to enable or disable ftp-services Logging Configures active logConfigures a radius-server. See radius-server Reboots the chassis. See reload Boot-config Table B-56 boot-config KeywordsAction Table B-55 action Keywords Table B-57 boot-config Arguments Table B-58 Broadcast Command ArgumentsBroadcast Configure Table B-59 clock Command Argument DescriptionsCard Clock Syntax Table B-60 copy Command Syntax Descriptions Table B-61 Copy Command Argument DescriptionsSystem upon boot-up Copy Arguments related to the delete command are described below Table B-62 Delete Command Argument DescriptionsDelete Syntax delete fsfile Related Commands card Table B-63 Dir Command Argument DescriptionsTo display installed system-images and image files Dir Disable EnableSyntax disable Syntax enable Ftp-server enable ExecExit Gateway Privileged-execute modes Defaults The default gateway IP address is Related Commands NoneHelp History Install Synopsis The hostname command assigns a name to the systemTable B-69 hostname Command Argument Description Hostname Table B-70 install Command Argument Descriptions Table B-71 interface Command KeywordsInterface Interface mgmt-ethernet Example Related Commands ip Interface mgmt-ibSyntax ipip mask No ipip mask Location Table B-72 Location Syntax Descriptions Login Synopsis The logout command ends the current CLI sessionLogging Logout Related Commands exit Related Commands dirMore Ntp Ping Radius-serverSyntax ping dest Table B-73 Radius-server Syntax Descriptions Synopsis The reload command reboots the chassis ReloadSyntax reload Enabling/Disabling the Ethernet Management Port Snmp-server No snmp-server Table B-74 Snmp-server Command Keyword Descriptions Table B-75 snmp-server Command Argument DescriptionsCommand Modes Privilege Level Usage Guidelines Defaults Telnet access is enabled by default Defaults The default is 24 lines per screen Defaults The default time-out interval is 15 minutesTerminal length Terminal time-out Syntax type type Table B-76 type Command Argument Type Username Table B-77 Default User Accounts WhoSyntax who Defaults This command has no defaultsWrite 216 How to Use Windows HyperTerminal Attaching through Windows HyperTerminalHyperTerminal Prerequisite Loading HyperTerminal Configuring the HyperTerminal Connection How to Use Windows HyperTerminal Figure C-3 Connect To Dialog Box Verifying the HyperTerminal Connection Figure C-4 COM Port Properties Dialog Box Figure C-5 HyperTerminal Login 222 Glossary 223 Ietf Glossary Snmp 225 226 Ib-rw, 104 setting levels, 104 unrestricted-rw Index227 CLI Index 229 See also HCAs hostname 231 Index See also ports Snmp Changing, 104 user access levels changing About, 104 adding, 101 creating, 100 user information235 Index