HP-UXIntegrity Server Networking
Page
Table of Contents
About This Document
1 Introduction to Technology
2 Hardware Overview
3 Installation Planning
5 Configuration
6 Administration and Management
Using Command Completion
Specifying the Card/Port
Testing Network Connectivity
Logging Onto the System
Logging On Through SNMP
7 Monitoring and Troubleshooting
A Specifications
B Switch Command Line Interface
link-trap
C How to Use Windows HyperTerminal
Glossary
Index
Page
List of Figures
Page
List of Tables
B-38
B-39
B-40
B-41
B-42
About This Document
Typographical Conventions
Related Documents
HP Encourages Your Comments
Page
1 Introduction to Technology
HP Fabric Clustering System
What is InfiniBand
InfiniBand Advantages
InfiniBand Capabilities
InfiniBand Transport Services
HP Fabric Subnet Manager
MPI
IPoIB
Oracle 10gRAC
User-SpaceRDMA Library
HCA
Kernel RDMA Subsystem
Interaction with HyperFabric Driver
Page
2 Hardware Overview
Host Channel Adapters
•AH304A PCIe adapter with 2 (two) 4X fabric copper interfaces
•AD313A PCIe adapter with 2 (two) 4X fabric copper interfaces
•AB286C PCI-X133Mhz adapter with 2 (two) 4X fabric copper interfaces
Page
Page
Page
Page
Host
Adapter
View)
Figureigure
Page
Switch
Other Product Elements
Page
3 Installation Planning
Application Availability
4 Installing HP Fabric Clustering System
Installing the AH304A Host Channel Adapter
Installing the 410533-B21Host Channel Adapter
Installing the AD313A Host Channel Adapter
Installing the AB286C Host Channel Adapter
On-LineAddition and Replacement Operations (OL*)
Replacing an HP Fabric Clustering System HCA (HP-UX11i v2 only)
Install HP Fabric Clustering System Software
1.Log on to the system as root
The drivers, utilities and manpages for the software are on the
•Go to http://www.software.hp.com
Go to
Rack Mount Preparation
Installing the Switch
Page
Remove Shipping Sleeve
Remove Support Bracket (if installed)
Cable Routing and Bend Limits
Attach Cables
Switch or Adapter
Attach to other HP Fabric devices:
Connect Cables to Switch
Setting up the Switch
Internet Protocol over InfiniBand (IPoIB)
Page
5 Configuration
Configurations up to 24 End Nodes
Configurations with More Than 24 End Nodes
Page
6 Administration and Management
Using Itutil
Summary of Itutil Command Options
Itutil Syntax
Itutil Administrative Commands
Itutil Management Commands
Description: Without any options set, the
command displays the basic details of all HP
Fabric-configured
host interfaces
Syntax: itutil
Restricting Options to One Interface
Syntax: itutil -i [interface_name]
Syntax Example: itutil -iib0
Description: This parameter only displays
Syntax: itutil -t [transport_type]
Syntax Example: itutil -tIB
Displaying Statistics
Syntax: itutil -s [-iinterface name|-ttransport type]
Page
Displaying Connectivity Information Using Itutil
NOTE: This is a partial view of the itutil -ccommandoutput
Displaying the List of
End-Nodes
Syntax: itutil -T [-iinterface name|-ttransport type]
Example: itutil -T
Viewing the Itutil Manpage
Using NetTL
Syntax: nettl -l [log class] -e[subsystem]
[log class]
Syntax Example: nettl -li w e -eIPOIB
Stopping Logging and Tracing
Syntax: nettl -sp
IPoIB Administration and Management
nwmgr Utility
ifconfig Command
lanadmin Commands
Syntax: lanadmin -xall_if_info [any_ipoib_ppa]
Syntax Example: lanadmin -xall_if_info
Displaying IPoIB Interface MAC Address
Syntax: lanadmin -a [ipoib_ppa]
Syntax Example: lanadmin -a9000
lanadmin Command Help
•Brief Help
•Extended Help
Brief Help
Syntax: lanadmin -x -h [ipoib_ppa]
Serviceguard and IPoIB
Introduction
Configuring IPoIB Interfaces for Serviceguard Clusters
Identify all IPoIB ports that are configured for the application on each node
Configure IP addresses for these interfaces using
/etc/rc.config.d/netconf
Verify the
Switch Administration and Management
Host Administration and Management” (page 56)
This section addresses the following topics:
•“CLI Overview”
•“Using the CLI”
CLI Overview
Using the CLI
Starting A CLI Session
Entering CLI Modes
Exiting CLI Modes
Using Command Completion
Command Abbreviation
Quick Help
Command-LineEditing
Exiting the CLI Session
Specifying the Card/Port
Page
Advanced Switch Setup
Configuring the System Hostname
Testing Network Connectivity
Setting User Levels and Passwords
Logging Onto the System
Managing the Switch
Page
Configuration, Image, and Log Files
Configuration, Image, and Log File Overview
Upgrading Image Files
•Out-of-Bandconnection
•In-Bandconnection
2. Check the Image Version
Refer to “Deleting Images Through the CLI ” (page 90)
1.Enter the privileged-executemode
3. Copy/Download the Image
There are two ways that the software can be downloaded from the CLI:
•Through an FTP server ((page 87))
•Through a TFTP server ((page 87))
Download From an FTP Server
The “remote system” must be the IP address of the TFTP server. Syntax:
HP-IB#copy tftp://<remote-system>/<filepath><filesystem>:<filename
3.(Optional) View the images by using the dir image command
4.Install a New Image
To install an image file:
•In the privileged-executemode, enter install image:file
HP-IB#install image:image-file-name
The name of the image file will change after it is installed
oper-status
boot-config
6. Reboot the System
1.Once the system is configured to use the new image, reboot the system
yes
copy
“Reboot the System Through the CLI” (page 107)
File Management
Listing Configuration, Image, and Log Files
Viewing Configuration Files
Viewing Log Files
Saving Configuration Files
Saving for System Reboot
Saving the Backup Configuration
Specifying the Configuration to Use at System Reboot
Saving and Copying Files
Downloading Files to the System
Deleting Configuration, Image, and Log Files
Deleting Configuration Files
To remove a configuration file:
1.Log in as the unrestricted user
3.Determine the configuration files in memory
delete config: file
Page
Managing Log Files
Understanding the Log Format
Uploading Log Files
Administering the System
Switching User Identity
Notifying Users
Creating User Accounts
Enter the username command and the
5.Repeat step 4 to change additional usernames and passwords
6.Exit the global-configurationmode
7.Use the show user command to verify changes
Displaying User Information
4.Create the user account and user password
The user name and password are alphanumeric strings up to 34 characters each
ib-admin
ib-passwd
5.Assign an SNMP community string to the new user account
username
HP-IB(config)#username ib-adminno
User Account Configuration Commands
Use the following commands and keywords to administer User Accounts:
Table 6-7User Account Administrative Commands
Using DNS Services
Setting Administrative Roles
Setting the System Clock
Setting the NTP Servers to Maintain the System Clock
Rebooting the System
Page
Page
7 Monitoring and Troubleshooting
LED Color, Behavior, and Meaning
HP Fabric Switch Cluster Connection LEDs
HP Fabric Switch System Status
Table 7-2Front System Status LED Indicator
Table 7-3Rear System Status LED Indicator
Power Supply Troubleshooting
Ethernet Management Port LEDs
HP Fabric HCA Connection LEDs
Figure 7-3HP Fabric HCA Interface LEDs
physical link status
logical link status
HCA LED location
HCA Connection LED Colors and Meanings
Determining Whether the HCA or Cable is Faulty
Determining if the Switch is Faulty
Next Steps
HP-UXHost Troubleshooting Procedure
Syntax example: itutil -T
5.Check communication path between nodes
Syntax example: itutil -lfe80::2:c901:8a0:8a11 ib0
Syntax example:
6.Check adapter transport and performance statistics
7.Check connection details
Syntax example: itutil -cfe80::2:c901:8a0:8a11 ib0
8.Check nettl logs
Sample nettl Log Messages
Description: Port failure
Description: Firmware download failure
Description:
Monitoring and Troubleshooting IPoIB
nettl Sample Log Output
Sample IPoIB Logs
Viewing Trace Packets in Raw Format
Syntax: netfmt [file_name]
Syntax Example: netfmt ipoib.TRC000
Viewing Trace Packets in Nice Format
Linkloop Command
Known Problems
Health Monitoring
About Logging and Tracing
About HP Fabric Events
About Tracing
Troubleshooting the HP Fabric Network
Ping
Setting Trace Levels
Replacing Individual Components
Replacing a Power Supply Module
A Specifications
HCA Physical Specifications (AB286C)
HCA Environmental Specifications (AB286C)
HP Fabric HCA Electrical Specifications (AB286C)
Page
B Switch Command Line Interface
Synopsis
Syntax
Command Modes
Privilege Level
Examples
Defaults
Related Commands
show arp ethernet
show arp IB
show authentication
show backplane
show boot-config
show card
Table B-5show card Command Field Descriptions (continued)
To display the configuration and status information for the card:
Defaults: “show card” defaults to “show card all”
Related Commands: “boot-config”(page 190) “card” (page 192)
show card-inventory
show clock
show config
show fan
show host
show ib
show ib sm configuration
subnet prefix |
]} The arguments associated with this command are described below:
Table B-10
show ib Command Syntax Descriptions
Command Modes:
show ib sm multicast
show ib sm neighbor
show ib sm node subnet-prefix
Table B-14show ib sm node subnet-prefixCommand Syntax Descriptions (continued)
The fields in the show ib sm node output are described in the table below
Table B-15show ib sm node subnet-prefixCommand Field Descriptions
show ib sm partition
show ib sm port
Table B-17show ib sm port Command Field Descriptions (continued)
Page
HP-IB#show ib sm port subnet-prefixfe:80:00:00:00:00:00:00
Subnet Management Ports
mkey : 00:00:00:00:00:00:00:00 gid-prefix :fe:80:00:00:00:00:00:00
lid : 14 master-sm-lid :1
cap-mask :00:10:02:48 diag-code :00:00
show ib sm service
show ib sm switch
Table B-20show ib sm switch Command Field Descriptions (continued)
Subnet Management Switches
subnet-prefix :fe:80:00:00:00:00:00:00 node-guid :00:05:ad:00:00:00:13:81
linear-fdb-cap :49152
random-fdb-cap :0 mcast-fdb-cap :1024 linear-fdb-top :1024
show ib-agent channel-adapter
show ib-agentsummary
show ib-agentswitch
show ib-agentswitch linear-frd-info
show ib-agentswitch all mcast-infolid
show ib-agentswitch all node-info
Table B-27show ib-agentswitch all node-infoCommand Field Descriptions
HP-IB#show ib-agentswitch all node-info
guid : 00:05:ad:00:00:00:13:7f
type : switch lid :
port-guid :00:05:ad:00:00:00:13:7f partition-cap :1
show ib-agentswitch all pkey-info
show ib-agentswitch port-info
show ib-agentswitch sl-vl-map
show ib-agentswitch switch-info
show interface ib
Table B-31show interface ib Output Descriptions (continued)
HP-IB#show interface ib 15/1
InfiniBand Interface Info
port : 15/1
name : 15/1
show interface ib sm
show interface ib sm statistics
show interface mgmt-ethernet
show interface mgmt-ib
show interface mgmt-serial
show ip
show location
show logging
show ntp
show power-supply
show running-status
show sensor
show snmp
show system-services
show terminal
show trace
show user
show version
arp ib
clear ib arp-cache
default ip ip-over-ib-mtu
ip domain-name
ip ip-over-ib-mtu
ip name-server-one
ip name-server-two
ip route
ib sm subnet-prefix
ib sm
key
no ib sm
The syntax and arguments are described in the table below:
Table B-46ib sm subnet-prefixSyntax Description
Regarding Partitions:
NOTE: HP recommends that the default partition be used for the cluster
Table B-47Acceptable P_Key Values
The following example removes a specified subnet manager
The following example creates a Partition, and adds
ib-agent channel-adapter
ib-agentswitch
interface ib
link-trap
name
Administrative Commands
This chapter lists and describes Administrative commands in alphabetical order
Table B-54Administrative Commands
Table B-54Administrative Commands (continued)
action
boot-config
broadcast
card
clock
configure
copy
delete
dir
disable
enable
exec
exit
ftp-serverenable
gateway
help
history
hostname
install
interface
interface mgmt-ethernet
interface mgmt-ib
location
login
logging
logout
more
ntp
ping
radius-server
reload
snmp-server
telnet
terminal length
terminal time-out
trace
type
username
who
write
Page
C How to Use Windows HyperTerminal
HyperTerminal Prerequisite
Loading HyperTerminal
Configuring the HyperTerminal Connection
Figure C-3Connect To Dialog Box
Verifying the HyperTerminal Connection
Figure C-5HyperTerminal Login
Page
Glossary
IETF
http://www.ietf.org/overview.html
IOC
I/O interface of the TCA is implemented by an I/O Controller
datagrams across networks
Multiple subnet managers per subnet are supported
SMP
Subnet Management Packet (SMP) on the fabric
SNMP
Target Channel
Page
Index
Page
Page
Page
Page
Page
(see also ports)
ports)
radius-servercommand, 189
RARP (Reverse Address Resolution Protocol), definition
RDMA (Remote Direct Memory Access), 19, 224 overview
show ib-agentswitch linear-frd-info
show interface ib command, 132, 163–167 show interface ib sm command, 165–166
show interface ib sm statistics command, 166–167 show interface ib sma
show interface mgmt-ethernet
show interface mgmt-ethernetcommand, 167 show interface mgmt-ib