HP UX 11i v2 Networking Software guide

The following example yields a 50% non-blocking configuration for 128 nodes using (11) 24-port switches.

Figure 5-3 128 Node 50% non-Blocking Configuration

(3) Spline-level Switches

Connect to 16	(8) Leaf-level Switches	Connect to 16
Nodes per Switch	(8) Leaf-level Switches	Nodes per Switch

Total (11) 24-Port Switches, 640 Gbps full duplicate fabric bandwidth

Configuration results in 128 Nodes

The following example yields a fully non-blocking configuration for 132 nodes using (17) 24-port switches.

Figure 5-4 132 Node Fully non-Blocking Configuration

(6) Spline-level Switches

Connect to 12	(11) Leaf-level Switches	Connect to 12
Nodes per Switch		Nodes per Switch

Total (17) 24-Port Switches, 1280 Gbps full duplicate fabric bandwidth

Configuration results in 132 Nodes

54 Configuration

Page 54

Image 54

HP UX 11i v2 Networking Software manual Node 50% non-Blocking Configuration

Contents

HP Fabric Clustering System HP-UX Administrators Guide Copyright 2004-2008 Hewlett-Packard Development Company, L.P Table of Contents Configuration Interfaces Switch Administration and Management 109 129 131 155 152154 156 217 223227 Page HyperTerminal Login List of FiguresPage List of Tables Default User Names, Passwords and PrivilegesDescription of Access Levels Show ib Keyword Descriptions Show user Command Syntax Descriptions Show logging Command Syntax DescriptionShow trace Command Syntax Descriptions IP Commands New and Changed Documentation in This Edition About This DocumentIntended Audience Publishing History What’s in This Document Typographical Conventions Related Documents HP Encourages Your Comments Related DocumentsPage Introduction to Technology Understanding the Fabric Clustering SystemUnderstanding InfiniBand About the HP Fabric Clustering System Software Suite Interaction with HyperFabric Driver User-Space Rdma LibraryKernel Rdma Subsystem About the HP Fabric Clustering System Software SuitePage Host Channel Adapters Hardware OverviewHardware Components HP offers the following new Host Channel Adapters HCA Hardware Overview AB286C Host Channel Adapter Connector View AB286C Host Channel Adapter Side View AB286A Host Channel Adapter Connector View AB286A Host Channel Adapter Side View Figureigure Hardware Components Switch Other Product Elements Hardware Overview Applications Installation PlanningPreliminary Considerations Preliminary Considerations Installation Planning Site Set-upApplication Availability Installing HP Fabric Clustering System HP Fabric Clustering System Installation PrerequisitesEnter your product name Click the double arrow Install HP Fabric Clustering System Adapters Installing the AH304A Host Channel AdapterInstalling HP Fabric Clustering System Installing the 410533-B21 Host Channel Adapter Installing the AD313A Host Channel AdapterSecure the card and reassemble the system Installing the AB286C Host Channel Adapter Inserting the AD313A Host Channel Adapter On-Line Addition and Replacement Operations OL Inserting the AB286C Host Channel Adapter Install HP Fabric Clustering System Software Where devicename is the name assigned to the CD-ROM driveThis opens the Software Selection window Rack Mount Preparation Install HP Fabric Clustering System SwitchesT-25 Torx screwdrivers Second person for the installation IB subsystem IB Rdma IPoIB IB device driver ibt Installing the Switch Fixed and Sliding Rails Attach Rail to Switch Remove Support Bracket if installed Remove Shipping SleeveSecurely attach all rails to the rack Attach Cable Guides & Cables Cable Routing and Bend LimitsAttach Cable Guides & Cables Cable Installation Minimum Bend Radius Attach Cables Switch or Adapter Switch SetupConnect Cables to Switch Attach to other HP Fabric devices Set the default gateway address. This address is an example Setting up the SwitchEnter configure to enter the global-configuration mode Enable the management port Internet Protocol over InfiniBand IPoIB Syntax and Example nwmgr -S ipoibPage Configuration Supported FeaturesConfiguration Parameters Sample Configurations using the AB286A/C or AD313A HCA Non-Supported FeaturesSupported Configurations Configurations up to 24 End Nodes Configurations with More Than 24 End Nodes Sample Configurations using the AB286A/C or AD313A HCA 128 Node 50% non-Blocking Configuration Administration and Management Following is a listing of the major topics in this chapter HP-UX Host Administration and Management Using ItutilAdministration and Management Itutil Command Options Summary of Itutil Command OptionsItutil Administrative Commands Following syntax must be used with itutil command options Itutil Management Commands Syntax itutil Syntax Example itutil Output Example Syntax Example itutil -i ib0 Output Example Syntax Example itutil -t IB Output Example Syntax Example itutil -s-i ib0 Output Example Administration and Management Displaying Connectivity Information Using Itutil Syntax Example itutil -r Output ExampleSyntax Example itutil -c Output Example Example itutil -T Using NetTL Viewing the Itutil Manpage Syntax nettl -llog class -esubsystem IB Tracing IPoIB Administration and ManagementIfconfig Command IPoIB Tracing Displaying IPoIB Interface Link-Level Information Lanadmin CommandsDisable the IPoIB Interface To disable the IPoIB interface Level information for a given ipoibppa Syntax Example lanadmin -a Output Example Syntax Example lanadmin -s Output ExampleSyntax Example lanadmin -g Output Example Syntax lanadmin -x-hipoibppa Syntax Example lanadmin -x -h Serviceguard and IPoIB Lanscan CommandIntroduction Configuring IPoIB Interfaces for Serviceguard Clusters Using Serviceguard and IPoverIB Switch Administration and Management Switch Administration and Management Starting a CLI Session Using the CLICLI Overview Privilege Level Administrator Roles Entering CLI Modes Using Command Completion Exiting CLI Modes Command Abbreviation Quick Help Command-Line Editing Specifying the Card/Port Command before returning to user-execute modeExiting the CLI Session Key Stroke Shortcuts Indicates ports 2 through 4 on card Card/Port pairsRanges Lists Testing Network Connectivity Advanced Switch SetupConfiguring the System Hostname Enter the global-configuration mode Logging Onto the System Setting User Levels and PasswordsManaging the Switch Logging On Through the CLI Management Methods Managing Through the CLI Configuration, Image, and Log Files Configuration, Image, and Log File Overview Upgrading Image Files Understanding the Upgrade ProcessImage Upgrade Procedure Summary Set-Up the Hardware Connection Enter the privileged-execute mode Check the Image Version Copy/Download the Image Optional View the images by using the dir image command Install a New Image Specify a New Boot Image Show the new system -image Reboot the System Troubleshooting an Image UpgradeDeleting System Images and Image Files File Management Listing Configuration, Image, and Log FilesViewing Configuration Files Viewing Log Files Saving the Backup Configuration Saving Configuration FilesSaving for System Reboot Specifying the Configuration to Use at System Reboot Example Arguments to the copy command are described belowSaving and Copying Files Show the new system image Colon Deleting Configuration, Image, and Log FilesDownloading Files to the System Tslog. The .cfg extension is optional Deleting Log Files Deleting Configuration FilesDetermine the configuration files in memory Display the log files that are in memory Determine the currently installed system-images Verify that you had successfully removed the log fileDeleting Image Files Understanding the Log Format HP Fabric Switch devices use the following log formatManaging Log Files Uploading Log Files Administering the System Switching User Identity Notifying Users Broadcasting Messages to UsersSending Messages to Individual Users Setting or Changing a Password Understanding Usernames and PasswordsDefault User Names, Passwords and Privileges Creating User Accounts Displaying User Information Adding New Users Deleting a User Account User Account Configuration Commands User Account Administrative CommandsCommunity Strings Using DNS Services Setting Administrative RolesFollowing table displays the different access-levels Setting the System Clock Setting Time Through the CLI Setting the NTP Servers to Maintain the System ClockRebooting the System Save your configuration Reboot the System Through the CLI 108 Monitoring and Troubleshooting Diagnosing ProblemsDiagnosing Problems IB Port Status LED Indicators HP Fabric Switch Cluster Connection LEDsHP Fabric Switch System Status LED Color, Behavior, and Meaning Front System Status LED Indicator Rear System Status LED Indicator Power Supply Troubleshooting Power supply/ Fan Status LED Indicator HP Fabric HCA Connection LEDs Ethernet Port Status LED IndicatorsEthernet Management Port LEDs Bottom Top Green- over Green Off Flickering HCA Connection LED Colors and Meanings Logical Solid Indicates aBeen Top Established None HCA is not Determining Whether the HCA or Cable is Faulty Determining if the Switch is Faulty Monitoring and Troubleshooting the HP-UX Host HP-UX Host Troubleshooting ProcedureNext Steps Monitoring and Troubleshooting the HP-UX Host Syntax example itutil Output example Monitoring and Troubleshooting Syntax example itutil -c fe802c9018a08a11 ib0 Sample nettl Log Messages Description Port failure Sample IB Logs Monitoring and Troubleshooting IPoIBNettl Sample Log Output Following are examples of nettl log output Sample IPoIB Logs PDU in Trace Linkloop Command Known ProblemsNetTL IP Filter Product Monitoring and Troubleshooting the SwitchHealth Monitoring CLI can also be used to monitor the system About Logging and Tracing About HP Fabric EventsAbout Tracing Troubleshooting the HP Fabric Network PingFlow Disabling Tracing Setting Trace LevelsEnabling Tracing Verify Link Speed Replacing a Power Supply Module Replacing Individual Components HCA Physical Specifications AB286C SpecificationsPhysical and Environmental Specifications HCA Environmental Specifications AB286C 130 Switch Command Line Interface Using the Documentation Related Commands Show CommandsDefaults Examples Defaults There are no defaults for this command Show arp ethernetShow arp IB Show backplane Show authenticationExamples To display the authentication method Examples To display the InfiniBand ARP table Show boot-config Defaults This command has no defaultsTable B-3 show boot-config Command Field Descriptions This command has the following arguments Show cardShow card Syntax Description Oper status Defaults show card defaults to show card allImage data for internal configuration Card Show card-inventory Table B-6 Show card-inventory Command Syntax DescriptionsTable B-7 show card-inventory Command Field Descriptions Synopsis Show config Related Commands clockShow clock Show fan Related Commands copyTable B-8 show fan Command Field Descriptions Syntax show fan Show ib Show ib sm configurationShow host Table B-9 show ib Keyword Descriptions User-execute and privileged-execute modes Table B-10 Show ib Command Syntax DescriptionsTable B-11 Show ib Command Field Descriptions InfiniBand read-only user Table B-12 show ib sm multicast Command Syntax Descriptions Show ib sm multicastShow ib sm neighbor Table B-13 show ib sm neighbor Command Field Descriptions Arguments associated with this command are described belowShow ib sm node subnet-prefix Type Node guid Use with the all keywordClass-version An unknown type Show ib sm partition Table B-16 Show ib sm port Command Syntax Descriptions Table B-17 show ib sm port Command Field DescriptionsShow ib sm port 1x or Value may beNo state change 12x Packets received by this port. There is no default value Received by this port. There is no default valueTransmitted by this port. There is no default value Packets transmitted by this port. There is no default value Local-phy-error Errors and the buffers are not immediately reclaimed Show ib sm service Table B-18 Show ib sm service Command Syntax Descriptions Show ib sm switch Table B-19 show ib sm switch Command Syntax DescriptionsTable B-20 show ib sm switch Command Field Descriptions Port-state-change Partition enforcement is not supported by the switchLife-time-value Lid-per-port Show ib-agent channel-adapter Usage Guidelines form Table B-22 Show ib-agent summary Command Field DescriptionsShow ib-agent summary Slot System chassis slot in which the device resides Type Agent Show ib-agent switchFollowing example displays a summary of all the SMA nodes Lid Decimal-base LID of this port Show ib-agent switch linear-frd-info Display the attributes of a single switchSwguid Swguid Guid of a specific InfiniBand switch Show ib-agent switch all mcast-info lidShow ib-agent switch all node-info Type, GUIDs, and capabilities Defaults This command has no defaults Show ib-agent switch all pkey-info Show ib-agent switch port-infoUsage Guidelines None Examples Out-ib-port Show ib-agent switch sl-vl-mapNode-guid Bit Guid of this node In-ib-port SL0 to VL mapping Show ib-agent switch switch-info Syntax is described in the table below Table B-30 Show interface ib Command Syntax DescriptionsShow interface ib Table B-31 Show interface ib Output Descriptions Link-trap Show interface ib sm Usage Guidelines None Show interface ib sm statistics Show interface mgmt-ethernet Show interface mgmt-ib Show interface mgmt-serial Show ip Table B-33 show ip Command Syntax Descriptions Show locationShow logging Syntax show ntp Table B-34 show logging Command Syntax DescriptionShow ntp End Show running-status Show power-supplyRelated Commands ntp Usage Guidelines Examples Show sensor Show system-services Show snmp Table B-36 show trace Command Syntax Descriptions Show terminalShow trace Username Specify the name of a specific user Table B-37 Show user Command Syntax DescriptionsShow user Shows all users in the user database Show version IP Commands Table B-38 IP CommandsArp ib Table B-39 Auto-negotiate Syntax Description Default ip ip-over-ib-mtuClear ib arp-cache Enter the IP address of the target host Ip domain-name Defaults The default ip-over-ib-mtu isDefaults The default domain name is an empty string Ip ip-over-ib-mtu Defaults The default is an empty string Defaults The default ip-over-ib-mtu unit isTable B-42 Command Syntax Description Ip name-server-one Ip name-server-two Table B-43 Command Syntax DescriptionTable B-44 Command Syntax Description Ip route Ib sm subnet-prefix InfiniBand CommandsTable B-45 InfiniBand Commands Arp ib Table B-46 ib sm subnet-prefix Syntax Description Following example creates a Partition, and adds Table B-47 Acceptable PKey ValuesFollowing example removes a specified subnet manager Sweep-interval is 10 seconds Ib-agent channel-adapter Ib-agent switchResponse-timeout is 2,000 microseconds Syntax is described in the following table Interface ib Config-if-ib submodeTable B-50 interface ib Command Syntax Description Link-trap Defaults The default interface name is card#/port# Commands syntax is described in the table belowDefaults The default is no link-trap Name Table B-53 Shutdown Command Syntax Description Administrative CommandsCommands stoats is described in the table below Table B-54 Administrative Commands Configures a radius-server. See radius-server Ftp-server enable Use to enable or disable ftp-servicesLogging Configures active log Reboots the chassis. See reload Action Boot-configTable B-56 boot-config Keywords Table B-55 action Keywords Table B-57 boot-config Arguments Table B-58 Broadcast Command ArgumentsBroadcast Card ConfigureTable B-59 clock Command Argument Descriptions Clock System upon boot-up Syntax Table B-60 copy Command Syntax DescriptionsTable B-61 Copy Command Argument Descriptions Copy Delete Arguments related to the delete command are described belowTable B-62 Delete Command Argument Descriptions Syntax delete fsfile To display installed system-images and image files Related Commands cardTable B-63 Dir Command Argument Descriptions Dir Syntax disable DisableEnable Syntax enable Ftp-server enable ExecExit Gateway Privileged-execute modes Help Defaults The default gateway IP address isRelated Commands None History Table B-69 hostname Command Argument Description InstallSynopsis The hostname command assigns a name to the system Hostname Table B-70 install Command Argument Descriptions Table B-71 interface Command KeywordsInterface Interface mgmt-ethernet Example Syntax ipip mask Related Commands ipInterface mgmt-ib No ipip mask Location Table B-72 Location Syntax Descriptions Logging LoginSynopsis The logout command ends the current CLI session Logout More Related Commands exitRelated Commands dir Ntp Syntax ping dest PingRadius-server Table B-73 Radius-server Syntax Descriptions Syntax reload Synopsis The reload command reboots the chassisReload Enabling/Disabling the Ethernet Management Port Snmp-server No snmp-server Command Modes Privilege Level Usage Guidelines Table B-74 Snmp-server Command Keyword DescriptionsTable B-75 snmp-server Command Argument Descriptions Defaults Telnet access is enabled by default Terminal length Defaults The default is 24 lines per screenDefaults The default time-out interval is 15 minutes Terminal time-out Syntax type type Table B-76 type Command Argument Type Username Table B-77 Default User Accounts WhoSyntax who Defaults This command has no defaultsWrite 216 HyperTerminal Prerequisite How to Use Windows HyperTerminalAttaching through Windows HyperTerminal Loading HyperTerminal Configuring the HyperTerminal Connection How to Use Windows HyperTerminal Figure C-3 Connect To Dialog Box Verifying the HyperTerminal Connection Figure C-4 COM Port Properties Dialog Box Figure C-5 HyperTerminal Login 222 Glossary 223 Ietf Glossary Snmp 225 226 Ib-rw, 104 setting levels, 104 unrestricted-rw Index227 CLI Index 229 See also HCAs hostname 231 Index See also ports Snmp Changing, 104 user access levels changing About, 104 adding, 101 creating, 100 user information235 Index