IBM L2/3 9.1 Basic rules and unique symptoms

156 Nortel Networks L2/3 Ethernet Switch Module for IBM Eserver BladeCenter

9.1 Basic rules and unique symptoms

Before going into detail about troubleshooting, it is important to first discuss certain common

rules and symptoms for this environment. Certain interactions within the IBMEserver

BladeCenter between the Nortel Networks L2/3 GbESM and the Management Module require

that certain important rules are followed. Failure to follow these rules can produce

unexpected results when deploying the IBMEserver BladeCenter containing an Nortel

Networks L2/3 GbESM. In this section, we summarize some of these rules and the

consequences of not adhering to them. We also discuss some symptoms that you might see

and possible solutions for these.

9.1.1 Basic rules

Review the following rules:

1. Do not attach cables to the Nortel Networks L2/3 GbESM until both sides of the

connection are configured.

– Symptoms: No upstream connectivity, upstream network failure from spanning tree

loop.

– Solution: Keep cables disconnected or ports shut down until properly configured on

both sides of the connection. This is an important best practice for any switch-to-switch

connections in a production network, not just between the Nortel Networks L2/3

GbESM and its upstream connections.

2. Do not put Blade servers on the subnet that the Nortel Networks L2/3 GbESM uses for its

management VLAN interface.

– Symptoms: Unable to reach the switch from server blades or from external devices

attached to the Management Module port.

– Solution: Use a separate subnets for the management module.

3. Make sure the Nortel Networks L2/3 GbESM firmware code is upgraded to the most

current version of the Alteon OS. You can find the latest code for the Nortel Networks L2/3

GbESM at:

http://www.ibm.com/support

When you go to this Web site, do the following:

a. Under Support topics, click Downloads and drivers.

b. Under Category, select BladeCenter (Blades).

c. Click BladeCenter chassis Hardware only.

d. Click Firmware.

e. Scroll to find the desired Nortel Networks L2/3 GbESM Firmware - IBMEserver

BladeCenter.

Note: It is not possible to put external ports on VLAN 4095, which is dedicated to the

Management Module. It is essential that the PVID of ports INT1-14 be a VLAN other

than 4095 and that ports INT1-14 are each members of at least one such

data-bearing VLAN

Contents

Main Page Page Page Contents Page Page Page Notices Trademarks Preface The team that wrote this Redpaper Become a published author Comments welcome Page Page Page overview 2.1 The IBM Eserver BladeCenter product family 2.1.1 IBM Eserver BladeCenter storage solutions 2.1.2 IBM Eserver BladeCenter system management Management foundation Advanced server management Deployment and update management 2.2 IBM Eserver BladeCenter architecture 2.2.1 The midplane Chapter 2. IBMEserver BladeCenter overview 7 ^ Midplane Lower Section IBM Figure 2-3 Internal picture of the upper and lower midplane of the BladeCenter chassis Figure 2-2 Midplane view ^ IBM BladeCenter - Management Module Ethernet Interface 2.2.3 Gigabit Ethernet path 10 2.3 IBM Eserver HS20 architecture ^ Midplane (Lower Section) IBM BladeCenter - Gigabit Ethernet path 8843 HS20 Block Diagram Servicing the IBM ^ HS20 (M/T 8843) and Blade Storage Expansion-II Option 2.4 Stand-alone configuration tools Page Page Page 3.1 Product description Page 3.2 Value proposition Product strength Leadership features and function Competitive advantage 3.3 Supported hardware Product shipment group Page Page 4.1 Nortel GbESM architecture overview Internal Layer 2 traffic flow in the Nortel Networks L2/3 GbESM 4.2 Nortel Networks L2/3 GbESM block diagram Chapter 4. Nortel Networks Layer 2/3 GbE Switch Module architecture 25 4.2.1 Nortel Networks L2/3 GbESM ports specific roles Figure 4-6 Connections on the Nortel GbESM IBM eServer BladeCenter Figure 4-5 Connections on the Nortel GbESM Nortel Networks L2/3 GbESM Module INT1 14 INT1 - 14 The management VLAN IP address information is not lost during factory reset Page Page 30 5.1 Nortel Networks L2/3 GbESM management connectivity 5.1.1 Out-of-band management Management Module (Path 1) Serial port (Path 2) 5.1.2 In-band management External Ethernet ports (Path 3A) Internal Ethernet ports (Path 3B) In-band management considerations 5.2 Nortel Networks L2/3 GbESM user interface 5.2.1 IBM Eserver BladeCenter Management Module and I2C 5.2.2 Command-line interface Main Menu commands Global commands Navigation commands Configuration control commands Additional commands Upgrading the firmware Capturing the current configuration Configuring user accounts 5.2.3 Browser Based Interface 5.2.4 SNMP management - IBM Director 5.3 Multiple Nortel Networks L2/3 GbESMs in a BladeCenter Page 6.1 IBM Eserver BladeCenter system 6.1.1 Management Module firmware 6.1.2 Management Module network interface Establishing a physical connection to the Management Module Accessing the Management Module Web interface Configuring the Management Module network interfaces 6.1.3 I/O module management tasks Nortel Networks L2/3 GbESM setup and configuration Enabling Nortel Networks L2/3 GbESM uplink ports through the Management Module Nortel Networks L2/3 GbESM firmware download Determining the level of Nortel switch software Obtaining the latest level of switch software Upgrading the switch software 6.2 Blade server initial configuration 6.2.1 Firmware update UpdateXpress Getting started Firmware update 6.2.2 Operating systems Microsoft Windows 2003 Broadcom driver installation Red Hat Linux Broadcom driver installation 6.2.3 Broadcom Advanced Control Suite installation Page 6.3 Firmware and device drivers used in this example Page Page 7.1 Standards and technologies 7.1.1 VLAN tagging - 802.1Q 7.1.2 Link Aggregation and LACP - 802.3ad and 802.3-2002 channel-group 7.1.3 Spanning Tree - 802.1D, 802.1w, 802.1s 7.1.4 Routing Information Protocol - RFC1058 and RFC2453 7.1.5 Open Shortest Path First (OSPF) - RFC1257, RFC2328, and others 7.1.6 Virtual Router Redundancy Protocol (VRRP) - RFC 3768 7.1.7 Where standards originate and how to get them 7.2 Summary of sample configurations 7.2.1 Basic Layer 2 configuration 7.2.2 Advanced Layer 2 configurations 7.2.3 Layer 3 configuration - static routing static routing 7.2.4 Layer 3 configurations - dynamic routing routing 7.3 Introduction to High Availability 7.3.1 Introduction to trunk failover disabled Configuration X 7.3.2 Introduction to NIC Teaming X 7.3.3 Introduction to VRRP VRRP Priority 7.3.4 Some important rules for ensuring High Availability 7.4 Guidelines for attaching the BladeCenter to a network 7.4.1 Guidelines and comments Cable type selection (cross-over or straight-through) Fiber connections Speed or duplex selection 7.5 Base configurations for examples in this document 7.5.1 Hardware and software used for lab environment IBM Eserver BladeCenter configuration Nortel Networks L2/3 GbESM Cisco 3560G switch 7.5.2 Preconfiguration preparation 7.5.3 Base configuration common to all examples Management Module settings for Nortel GbESMs IP address and VLAN scheme y Blade server configuration with BASP 7.6 Basic Layer 2 entry topology 70 Figure 7-3 Basic Layer 2 topology with 802.1Q tagging and trunk failover 7.6.1 Layer 2 configuration with 802.1Q tagging and trunk failover Figure 7-3 illustrates the first basic Layer 2 topology. Summary of disconnect procedure to be performed for each example Cisco 3560G switch configuration Page Nortel Networks L2/3 GbESM switch configuration Page Page 76 Re-enable the ports Enabling the ports on Core1 and Core2 Page 78 Page 7.6.2 Basic topology conclusions 7.7 Advanced Layer 2 topology sample configurations Page Summary of disconnect procedure, to be performed for each example 7.7.1 Dynamic link aggregation IEEE 802.3ad (LACP) Configure the Nortel GbESMs for IEEE 802.1Q tagged LACP trunk Configure the Cisco Core1 and Core2 for tagged LACP trunks Verify the LACP trunk status 7.7.2 Common Spanning Tree configuration - IEEE 802.1D and PVST Configure the Nortel GbESM_1 and GbESM_2 for Spanning Tree Groups Configure the Cisco Core1 and Core2 for Per VLAN Spanning Tree Reenable the ports Verify the Spanning Tree status Page 88 Page Design or topology remarks Full configuration snapshots Page 92 Page 94 7.7.3 Rapid Spanning Tree IEEE 802.1w Prepare the Nortel GbESM_1 and GbESM_2 for Rapid Spanning Tree Configure the Cisco Core1 and Core2 for Rapid-PVST Reenable the ports Verify the Spanning Tree status Design remarks Full configuration snapshots 98 Page 100 7.7.4 Multi-Spanning Tree IEEE 802.1s Configurations for use of MST Key configuration commands to verify MSTP Page Page Page 106 Page 7.8 Layer 3 topology sample configurations Implications of using Layer 3 Reasons for choosing a Layer 3 design 7.8.1 Layer 3 sample configuration with static routing and VRRP Merits of this configuration Summary of disconnect procedure to be performed for each example Summary of IP Addressing used in this sample Table7-1 summarizes the IP addresses used in this sample. Figure 7-6 also illustrates these. Configure VLANs and subnets for the uplink ports Page 114 VRRP configuration Configuration for GbESMs Configuration of Hot-standby for VRRP Configuration on Core switches to work with VRRP Configuration on server blades to work with VRRP Reconnect procedure when configuration is ready Verifying operation of the configuration 118 Page 120 Page 122 Page 124 7.8.2 Dynamic routing options OSPF/RIP Merits of this configuration 126 Summary of IP Addressing used in this sample Figure 7-7 Layer 3 Sample Configuration (Dynamic routing) Table7-2 summarizes the IP addresses in use. Figure 7-7 also shows these. Table 7-2 IP Addresses for Layer 3 Sample Configuration (Dynamic routing) Configure additional VLANs and subnets for the uplink ports 128 Page Configuring RIP Configuring OSPF Verifying operation of this configuration GbESM commands to verify RIP 132 Cisco commands to verify RIP GbESM commands to verify OSPF 134 Page 136 Cisco commands to verify OSPF Page 138 Page 140 Page 142 7.9 Configuration for Extreme switches 144 Page 146 Page 148 Page Page Page 8.1 SOL overview 8.2 General rules to establish an SOL connection 8.3 Configuring SOL for use with the Nortel GbESM 8.4 SOL use during Nortel Networks L2/3 GbESM experiments Page Page 9.1 Basic rules and unique symptoms 9.1.1 Basic rules both sides of the connection. This is an important best practice for any switch-to-switch 9.2 Nortel Networks L2/3 GbESM troubleshooting methodology 9.2.1 General comments on troubleshooting 9.3 Systematic approach 9.3.1 Problem definition 9.3.2 Data collection Test connectivity Details from affected users Nortel GbESM (Alteon OS) switch dumps Network logs 9.3.3 Data analysis 9.3.4 Action plan creation 9.3.5 Action plan implementation 9.3.6 Observation of results 9.3.7 Problem resolution 9.4 Troubleshooting tools Ping Traceroute Telnet/SSH Difference tool TFTP server Syslog server Web load generator Protocol analyzer Page Page 10.1 Placing a call to IBM 10.2 Online services 10.3 Ordering information Page Page Abbreviations and acronyms 170 Related publications IBM Redbooks Other publications Online resources Page How to get IBM Redbooks Help from IBM Page Page Redpaper Back cover INTERNATIONAL TECHNICAL SUPPORT ORGANIZATION Nortel Networks L2/3 Ethernet Switch Module for IBM Eserver BladeCenter