IBM L2/3 9.3.2 Data collection

Chapter 9. Nortel Networks Layer 2/3 GbE Switch Module troubleshooting 159

9.3.2 Data collection

In this particular step, it might be necessary to collect additional information to help isolate

problem cause as noted in the following sections.

Test connectivity

򐂰Use connectivity testing tools such as ping, traceroute, and Telnet

򐂰Note error messages that appear

Details from affected users

Ask more detailed questions:

򐂰Has anything changed recently?

򐂰How often is the problem occurring?

򐂰Is the problem readily reproduceable?

򐂰Is it occurring intermittently or at regular intervals?

Nortel GbESM (Alteon OS) switch dumps

Collect the following dump types:

򐂰TechSupport dump

– /maint/tsdmp

– Includes output from the following dumps:

• Configuration dump (/cfg/dump)

• Information dump (/info/dump)

• Statistics dump (/stats/dump)

򐂰Panic dump

– Manually force a panic if switch hangs

• /maint/panic

• <CTRL-SHIFT-”-”>

– /maint/uudmp

– Includes a core dump of memory registers

򐂰Trace buffers

– Display switch processor trace buffer contents

/maint/debug/sptb <port#>

– Display the management processor trace buffer contents

/maint/debug/tbuf <port#>

– Display the snap trace buffer contents

/maint/debug/snap <port#>

– Send traces to GbE Technical Support as requested

򐂰Port diagnostics

Perform a port burn-in test to check for switch port hardware problems. You may also

consider running port diagnostics from maintenance kernel.

Note: If file size != 65 KB, then the panic file is no good.

Contents

Main Page Page Page Contents Page Page Page Notices Trademarks Preface The team that wrote this Redpaper Become a published author Comments welcome Page Page Page overview 2.1 The IBM Eserver BladeCenter product family 2.1.1 IBM Eserver BladeCenter storage solutions 2.1.2 IBM Eserver BladeCenter system management Management foundation Advanced server management Deployment and update management 2.2 IBM Eserver BladeCenter architecture 2.2.1 The midplane Chapter 2. IBMEserver BladeCenter overview 7 ^ Midplane Lower Section IBM Figure 2-3 Internal picture of the upper and lower midplane of the BladeCenter chassis Figure 2-2 Midplane view ^ IBM BladeCenter - Management Module Ethernet Interface 2.2.3 Gigabit Ethernet path 10 2.3 IBM Eserver HS20 architecture ^ Midplane (Lower Section) IBM BladeCenter - Gigabit Ethernet path 8843 HS20 Block Diagram Servicing the IBM ^ HS20 (M/T 8843) and Blade Storage Expansion-II Option 2.4 Stand-alone configuration tools Page Page Page 3.1 Product description Page 3.2 Value proposition Product strength Leadership features and function Competitive advantage 3.3 Supported hardware Product shipment group Page Page 4.1 Nortel GbESM architecture overview Internal Layer 2 traffic flow in the Nortel Networks L2/3 GbESM 4.2 Nortel Networks L2/3 GbESM block diagram Chapter 4. Nortel Networks Layer 2/3 GbE Switch Module architecture 25 4.2.1 Nortel Networks L2/3 GbESM ports specific roles Figure 4-6 Connections on the Nortel GbESM IBM eServer BladeCenter Figure 4-5 Connections on the Nortel GbESM Nortel Networks L2/3 GbESM Module INT1 14 INT1 - 14 The management VLAN IP address information is not lost during factory reset Page Page 30 5.1 Nortel Networks L2/3 GbESM management connectivity 5.1.1 Out-of-band management Management Module (Path 1) Serial port (Path 2) 5.1.2 In-band management External Ethernet ports (Path 3A) Internal Ethernet ports (Path 3B) In-band management considerations 5.2 Nortel Networks L2/3 GbESM user interface 5.2.1 IBM Eserver BladeCenter Management Module and I2C 5.2.2 Command-line interface Main Menu commands Global commands Navigation commands Configuration control commands Additional commands Upgrading the firmware Capturing the current configuration Configuring user accounts 5.2.3 Browser Based Interface 5.2.4 SNMP management - IBM Director 5.3 Multiple Nortel Networks L2/3 GbESMs in a BladeCenter Page 6.1 IBM Eserver BladeCenter system 6.1.1 Management Module firmware 6.1.2 Management Module network interface Establishing a physical connection to the Management Module Accessing the Management Module Web interface Configuring the Management Module network interfaces 6.1.3 I/O module management tasks Nortel Networks L2/3 GbESM setup and configuration Enabling Nortel Networks L2/3 GbESM uplink ports through the Management Module Nortel Networks L2/3 GbESM firmware download Determining the level of Nortel switch software Obtaining the latest level of switch software Upgrading the switch software 6.2 Blade server initial configuration 6.2.1 Firmware update UpdateXpress Getting started Firmware update 6.2.2 Operating systems Microsoft Windows 2003 Broadcom driver installation Red Hat Linux Broadcom driver installation 6.2.3 Broadcom Advanced Control Suite installation Page 6.3 Firmware and device drivers used in this example Page Page 7.1 Standards and technologies 7.1.1 VLAN tagging - 802.1Q 7.1.2 Link Aggregation and LACP - 802.3ad and 802.3-2002 channel-group 7.1.3 Spanning Tree - 802.1D, 802.1w, 802.1s 7.1.4 Routing Information Protocol - RFC1058 and RFC2453 7.1.5 Open Shortest Path First (OSPF) - RFC1257, RFC2328, and others 7.1.6 Virtual Router Redundancy Protocol (VRRP) - RFC 3768 7.1.7 Where standards originate and how to get them 7.2 Summary of sample configurations 7.2.1 Basic Layer 2 configuration 7.2.2 Advanced Layer 2 configurations 7.2.3 Layer 3 configuration - static routing static routing 7.2.4 Layer 3 configurations - dynamic routing routing 7.3 Introduction to High Availability 7.3.1 Introduction to trunk failover disabled Configuration X 7.3.2 Introduction to NIC Teaming X 7.3.3 Introduction to VRRP VRRP Priority 7.3.4 Some important rules for ensuring High Availability 7.4 Guidelines for attaching the BladeCenter to a network 7.4.1 Guidelines and comments Cable type selection (cross-over or straight-through) Fiber connections Speed or duplex selection 7.5 Base configurations for examples in this document 7.5.1 Hardware and software used for lab environment IBM Eserver BladeCenter configuration Nortel Networks L2/3 GbESM Cisco 3560G switch 7.5.2 Preconfiguration preparation 7.5.3 Base configuration common to all examples Management Module settings for Nortel GbESMs IP address and VLAN scheme y Blade server configuration with BASP 7.6 Basic Layer 2 entry topology 70 Figure 7-3 Basic Layer 2 topology with 802.1Q tagging and trunk failover 7.6.1 Layer 2 configuration with 802.1Q tagging and trunk failover Figure 7-3 illustrates the first basic Layer 2 topology. Summary of disconnect procedure to be performed for each example Cisco 3560G switch configuration Page Nortel Networks L2/3 GbESM switch configuration Page Page 76 Re-enable the ports Enabling the ports on Core1 and Core2 Page 78 Page 7.6.2 Basic topology conclusions 7.7 Advanced Layer 2 topology sample configurations Page Summary of disconnect procedure, to be performed for each example 7.7.1 Dynamic link aggregation IEEE 802.3ad (LACP) Configure the Nortel GbESMs for IEEE 802.1Q tagged LACP trunk Configure the Cisco Core1 and Core2 for tagged LACP trunks Verify the LACP trunk status 7.7.2 Common Spanning Tree configuration - IEEE 802.1D and PVST Configure the Nortel GbESM_1 and GbESM_2 for Spanning Tree Groups Configure the Cisco Core1 and Core2 for Per VLAN Spanning Tree Reenable the ports Verify the Spanning Tree status Page 88 Page Design or topology remarks Full configuration snapshots Page 92 Page 94 7.7.3 Rapid Spanning Tree IEEE 802.1w Prepare the Nortel GbESM_1 and GbESM_2 for Rapid Spanning Tree Configure the Cisco Core1 and Core2 for Rapid-PVST Reenable the ports Verify the Spanning Tree status Design remarks Full configuration snapshots 98 Page 100 7.7.4 Multi-Spanning Tree IEEE 802.1s Configurations for use of MST Key configuration commands to verify MSTP Page Page Page 106 Page 7.8 Layer 3 topology sample configurations Implications of using Layer 3 Reasons for choosing a Layer 3 design 7.8.1 Layer 3 sample configuration with static routing and VRRP Merits of this configuration Summary of disconnect procedure to be performed for each example Summary of IP Addressing used in this sample Table7-1 summarizes the IP addresses used in this sample. Figure 7-6 also illustrates these. Configure VLANs and subnets for the uplink ports Page 114 VRRP configuration Configuration for GbESMs Configuration of Hot-standby for VRRP Configuration on Core switches to work with VRRP Configuration on server blades to work with VRRP Reconnect procedure when configuration is ready Verifying operation of the configuration 118 Page 120 Page 122 Page 124 7.8.2 Dynamic routing options OSPF/RIP Merits of this configuration 126 Summary of IP Addressing used in this sample Figure 7-7 Layer 3 Sample Configuration (Dynamic routing) Table7-2 summarizes the IP addresses in use. Figure 7-7 also shows these. Table 7-2 IP Addresses for Layer 3 Sample Configuration (Dynamic routing) Configure additional VLANs and subnets for the uplink ports 128 Page Configuring RIP Configuring OSPF Verifying operation of this configuration GbESM commands to verify RIP 132 Cisco commands to verify RIP GbESM commands to verify OSPF 134 Page 136 Cisco commands to verify OSPF Page 138 Page 140 Page 142 7.9 Configuration for Extreme switches 144 Page 146 Page 148 Page Page Page 8.1 SOL overview 8.2 General rules to establish an SOL connection 8.3 Configuring SOL for use with the Nortel GbESM 8.4 SOL use during Nortel Networks L2/3 GbESM experiments Page Page 9.1 Basic rules and unique symptoms 9.1.1 Basic rules both sides of the connection. This is an important best practice for any switch-to-switch 9.2 Nortel Networks L2/3 GbESM troubleshooting methodology 9.2.1 General comments on troubleshooting 9.3 Systematic approach 9.3.1 Problem definition 9.3.2 Data collection Test connectivity Details from affected users Nortel GbESM (Alteon OS) switch dumps Network logs 9.3.3 Data analysis 9.3.4 Action plan creation 9.3.5 Action plan implementation 9.3.6 Observation of results 9.3.7 Problem resolution 9.4 Troubleshooting tools Ping Traceroute Telnet/SSH Difference tool TFTP server Syslog server Web load generator Protocol analyzer Page Page 10.1 Placing a call to IBM 10.2 Online services 10.3 Ordering information Page Page Abbreviations and acronyms 170 Related publications IBM Redbooks Other publications Online resources Page How to get IBM Redbooks Help from IBM Page Page Redpaper Back cover INTERNATIONAL TECHNICAL SUPPORT ORGANIZATION Nortel Networks L2/3 Ethernet Switch Module for IBM Eserver BladeCenter