Troubleshooting and Recovery, Supported Interfaces | Dell 2 manual

17

Troubleshooting and Recovery

This section explains how to perform tasks related to recovering and troubleshooting problems on the remote system using the CMC web interface.

•Viewing chassis information.

•Viewing the event logs.

•Gathering configuration information, error status, and error logs.

•Using the diagnostic console.

•Managing power on a remote system.

•Managing Lifecycle Controller jobs on a remote system.

•Reset components.

•Troubleshooting Network Time Protocol (NTP) problems.

•Troubleshooting network problems.

•Troubleshooting alerting problems.

•Resetting forgotten administrator password.

•Saving and restoring Chassis configuration settings and certificates.

•Viewing error codes and logs.

Gathering Configuration Information, Chassis Status, and Logs Using RACDUMP

The racdump subcommand provides a single command to get comprehensive chassis status, configuration state information, and the historic event logs.

The racdump subcommand displays the following information:

•General system/RAC information

•CMC information

•Chassis information

•Session information

•Sensor information

•Firmware build information

Supported Interfaces

•CLI RACADM

•Remote RACADM

•Telnet RACADM

205

Image 205

Dell 2 manual Troubleshooting and Recovery, Supported Interfaces, CLI Racadm

Contents

Page Page Contents Logging in to CMC Racadm Updating Firmware Viewing and Modifying CMC Network LAN Settings Configuring CMC Configuring Servers Configuring User Accounts and Privileges 119 Configuring CMC To Send Alerts 114Configuring CMC For Single Sign-On Or Smart Card Login 144 Using FlexAddress and FlexAdress Plus Cards 160 Configuring CMC to Use Command Line Consoles 149 Managing Fabrics Managing and Monitoring Power Managing PCIe Slots 201 Managing Chassis Storage 191 Using LCD Panel Interface 217 Troubleshooting and Recovery 205Frequently Asked Questions 226 228 227229 231 Overview Key Features What Is New In This ReleaseManagement Features Security Features Chassis Overview PSU Drive enclosure Minimum CMC Version Supported Remote Access Connections Supported Web Browsers Supported PlatformsManaging Licenses Types of Licenses License Operations Acquiring Licenses Managing Licenses Using Racadm Managing Licenses Using CMC Web InterfaceLicensable Features in CMC Snmp WS-MANSSH Other Documents You May Need Viewing Localized Versions of the CMC Web Interface Accessing documents from Dell support site Page Installing CMC Hardware Installing and Setting Up CMCChecklist To Set up Chassis Before You Begin Installing Racadm on a Linux Management Station Installing Remote Access Software on a Management StationBasic CMC Network Connection Uninstalling Racadm From a Linux Management Station Configuring a Web BrowserProxy Server Internet Explorer Certificate Revocation List CRL Fetching Microsoft Phishing FilterDownloading Files From CMC With Internet Explorer Mozilla FireFox Configuring Initial CMC Network Setting Up Initial Access to CMCEnabling Animations In Internet Explorer Configure CMC? Configuring CMC Network Using LCD Panel InterfacePage Auto-Apply settings to newly-inserted servers? Apply settings to installed servers?Dhcp IPs will auto-increment by slot number Apply All Enclosure Settings? Interfaces and Protocols to Access CMC Telnet console using an encrypted transport layer for higher Setting Date and Time on CMC Setting Chassis Physical Location and Chassis NameLaunching CMC Using Other Systems Management Tools Downloading and Updating CMC Firmware Setting Date and Time on CMC Using CMC Web Interface Configuring LEDs to Identify Components on the ChassisSetting Date and Time on CMC Using Racadm Configuring LED Blinking Using CMC Web Interface Configuring iDRAC Launch Method Using CMC Web Interface Configuring CMC PropertiesCmc-active Understanding Redundant CMC Environment Configuring Login Lockout Policy Attributes Using About Standby CMC CMC Failsafe ModeActive CMC Election Process Configuring Power Button Configuring Front PanelConfiguring LCD Accessing a Server Using KVM Mapping the Server to KVM Using LCD Mapping a Server to KVM Using CMC Web InterfaceMapping a Server to a DVD Drive Logging in to CMC Accessing CMC Web Interface Logging in to CMC Using a Smart Card Logging In To CMC Using Serial, Telnet, Or SSH Console Logging in to CMC Using Single Sign-on Logging in to CMC Using Public Key Authentication Accessing CMC Using Racadm Changing Default Login Password Using Web Interface Changing Default Login PasswordMultiple CMC Sessions Changing Default Login Password Using Racadm Enabling or Disabling Default Password Warning Message Bios Viewing Currently Installed Firmware VersionsUpdating Firmware Downloading CMC Firmware Updating the CMC Firmware Viewing Currently Installed Firmware Versions Using Racadm Signed CMC Firmware Image Updating CMC Firmware Using Web Interface Updating Chassis Infrastructure Firmware Updating CMC Firmware Using Racadm Updating Server iDRAC Firmware Using Web Interface Updating Chassis Infrastructure Firmware Using RacadmUpdating Server iDRAC Firmware Updating Server Component Firmware Interface Lifecycle Controller may not be enabled Server Component Update SequenceBios NIC RAID Enabling Lifecycle Controller Filtering Components for Firmware Updates Viewing Firmware Inventory Filtering Components for Firmware Updates Using RacadmRacadm getversion -l -m module -f filter Viewing Firmware Inventory Using CMC Web Interface Configuring Network Share Using CMC Web Interface Saving Chassis Inventory Report Using CMC Web InterfaceViewing Firmware Inventory Using Racadm Lifecycle Controller Job Operations Re-installing Server Component Firmware Using Web Interface Reinstalling Server Component FirmwareRolling Back Server Component Firmware Upgrading Server Component Firmware Pre-requisites for Using Network Share Update Mode Server Component Single Click Update Using Network Share Deleting Scheduled Server Component Firmware Jobs Recovering iDRAC Firmware Using CMC Updating Storage Component Using CMC Web Interface Viewing Chassis and Component Summaries Chassis Graphics Cpld Selected Component Information For Power Off Viewing Chassis Summary Viewing Server Model Name and Service TagViewing Chassis Controller Information and Status Viewing Information and Health Status of All Servers Viewing Information and Health Status of the IOM Viewing Health Status and Information for Individual ServerViewing Information and Health Status of Fans Racadm fanoffset -s offlowmediumhigh Fan number is less than the lower critical threshold Configuring Fans Viewing Front Panel Properties Viewing LCD Information and Health Viewing KVM Information and Health StatusPage Viewing and Modifying CMC Network LAN Settings Configuring CMC Racadm config -g cfgLanNetworking -o cfgNicIPv4Enable Viewing and Modifying CMC Network LAN Settings Using RacadmRacadm config -g cfgIpv6LanNetworking -o cfgIPv6Enable Enabling the CMC Network Interface Enabling or Disabling Dhcp for DNS IP Addresses Racadm config -g cfgLanNetworking -o cfgDNSServersFromDHCP Configuring DNS Settings IPv4 and IPv6 Setting Static DNS IP addressesRacadm config -g cfgLanNetworking -o cfgDNSRegisterRac Racadm config -g cfgLanNetworking -o cfgDNSRacName name Setting the Maximum Transmission Unit MTU IPv4 and IPv6 Configuring CMC Network and Login Security SettingsConfiguring IP Range Attributes Using CMC Web Interface Racadm config -g cfgNetTuning -o cfgNetTuningNicSpeed speed Configuring IP Range Attributes Using Racadm Configuring Virtual LAN Tag Properties for CMCConfiguring Virtual LAN Tag Properties for CMC Using Racadm Racadm config -g cfgLanNetworking -o cfgNicVLanPriority Configuring ServicesRacadm setniccfg -v Vlan id Vlan priority Racadm setniccfg -v 1 Configuring Services Using Racadm Configuring Services Using CMC Web InterfaceCfgRacTuning CfgRacTuneRemoteRacadmEnable Setting Up Chassis Group Configuring CMC Extended Storage Card Removing a Member from the Leader Adding Members To Chassis Group Disbanding a Chassis Group Accessing the Web page of a Member Chassis or ServerDisabling an Individual Member at the Member Chassis Server Inventory for MCM group Propagating Leader Chassis Properties to Member ChassisSaving Server Inventory Report SLOT-01 Exported Data1PB8VF1 Chassis Group Inventory and Firmware Version Viewing Selected Chassis Inventory Using Web InterfaceViewing Chassis Group Inventory Data Format Configuring Multiple CMCs Using Racadm Creating a CMC Configuration File Parsing Rules CfgUserAdmin cfgUserAdminUserName= Username Racadm getconfig -g groupname -i indexCommand, racadm getconfig -f myfile.cfg Modifying the CMC IP Address Viewing and Ending CMC Sessions Using Racadm Viewing and Ending CMC Sessions Using Web InterfaceViewing and Ending CMC Sessions Configuring Slot Names Configuring ServersDrac Configuring iDRAC QuickDeploy Network Settings Configuring iDRAC Network SettingsPage Is 255.255.255.0 , the QuickDeploy IP QuickDeploy Subnet error message isAddress range is not fully within CfgLanNetworking CfgIPv6LanNetworking CfgRacTuning Modifying iDRAC Network Settings Using RacadmAssigning QuickDeploy IP Address to Servers Configuring iDRAC Virtual LAN Tag Settings Using Racadm Configuring iDRAC Virtual LAN Tag SettingsCfgRemoteHosts CfgSerial CfgSessionManagement Racadm setniccfg -m server-1 -v 1 PXE Setting First Boot DeviceCD/DVD Configuring Remote File Share Configuring Server FlexAddressSetting First Boot Device Using Racadm Page Adding or Saving Profile Accessing Server Profiles Importing Profile Applying ProfileExporting Profile Deleting Profile Editing Profile Viewing Stored Profile Settings Viewing Profile SettingsCompletion Status And Troubleshooting Viewing Profile Log Assigning Server Profiles to Slots Launching Remote Console Launching iDRAC using Single Sign-OnLaunching iDRAC from Server Status Launching iDRAC from Servers Status Launching Remote Console from Server Status Launching Remote Console from Chassis HealthLaunching Remote Console from Servers Status Enabling Or Disabling Alerts Using CMC Web Interface Configuring CMC To Send AlertsFiltering Alerts Using CMC Web Interface Enabling Or Disabling Alerts Configuring Snmp Trap Alert Destinations Configuring Alert DestinationsSetting Event Alerts Using Racadm Racadm config -g cfgAlerting -o cfgAlertingEnable Configuring Snmp Trap Alert Destinations Using RacadmRacadm config -g cfgTraps -o cfgTrapsEnable 1 -i index Configuring Email Alert Settings Using CMC Web Interface Configuring Email Alert SettingsConfiguring EMail Alert Settings Using Racadm Page Types of Users Configuring User Accounts and Privileges Update, and CMC reset Page CMC Group Privileges Configuring Local Users Modifying Root User Administrator Account SettingsConfiguring Local Users Using CMC Web Interface Adding CMC User Using Racadm Configure Local Users Using RacadmRacadm getconfig -g cfgUserAdmin -i index # cfgUserAdminIndex=XX CfgUserAdminUserName= Disabling CMC User Racadm getconfig -g cfgUserAdmin -iEnabling CMC User With Permissions Supported Active Directory Authentication Mechanisms Configuring Active Directory UsersStandard Schema Active Directory Overview Configuring Standard Schema Active Directory 128 Racadm sslcertupload -t 0x2 -f ADS root CA certificate Active Directory Schema Extensions Extended Schema Active Directory OverviewOverview of Schema Extensions Dell base OID Configuring Extended Schema Active Directory Using Dell Schema Extender Extending Active Directory Schema OID False Attribute dellPrivilegeMemberAttribute dellProductMembersFALSE Attribute dellIsLoginUser Attribute dellIsCardConfigAdminAttribute dellIsUserConfigAdmin Attribute delIsLogClearAdmin Attribute dellSchemaVersion Attribute dellIsDebugCommandAdminAttribute dellAssociationMembersFALSE Attribute dellPermissionsMask1 Creating RAC Device Object Adding CMC Users And Privileges To Active Directory Adding Objects To Association Object Page Racadm sslcertdownload -t 0x1 -f RAC SSL certificate Configuring Generic Ldap Users Authentication of Ldap Users Configuring the Generic Ldap Directory to Access CMCAuthorization Of Ldap Users Service Name.tcp.Search Domain Configuring Generic Ldap Directory Service Using RacadmLdap.tcp.dell.com Ldap.tcp.domainname.comPage System Requirements Configuring CMC For Single Sign-On Or Smart Card Login Generating Kerberos Keytab File Prerequisites For Single Sign-On Or Smart Card LoginClient Systems Configuring Browser For SSO Login Configuring CMC For Active Directory SchemaConfiguring Browser For Smart Card Login Internet Explorer Uploading Keytab File CfgSmartCardLogonEnable CfgSmartCardCRLEnable Racadm -g cfgActiveDirectory -o cfgADSSOEnable CMC Command Line Console Features Configuring CMC to Use Command Line ConsolesCMC Command Line Interface Commands Racadm Using SSH With CMC Using Telnet Console With CMCSupported SSH Cryptography Schemes Exit, logout, and quit Racadm getssninfo IP Address Login Type User Date/Time Configure Public Key Authentication Over SSHX.x 06/16/2009 090000 Generating Public Keys for Systems Running Windows Ssh-keygen -t rsa -b 1024 -C testingGenerating Public Keys for Systems Running Linux Racadm Syntax Notes for CMC Configuring Linux Minicom Configuring Terminal Emulation SoftwareViewing Public Keys Adding Public Keys Configuring Minicom Version Connecting to Servers or I/O Module Using Connect CommandRequired Minicom Settings Racadm config -g cfgSerial -o cfgSerialConsoleColumns Connect switch-n Serial --unit=1 --speed=57600 terminal --timeout=10 serial Configuring Windows for Serial Console Redirection Co2345respawn/sbin/agetty -h -L 57600 ttyS1 ansi Console=ttyS1,57600 Vc/1 Vc/2 Vc/3 TtyS1 159 About FlexAddress Using FlexAddress and FlexAdress Plus Cards About FlexAddress Plus Racadm featurecard -sViewing FlexAddress Activation Status Feature card inserted is valid No feature card insertedRacadm feature -s Racadm feature -s No features active on the chassis Racadm feature -d -c flexaddress Configuring FlexAddressRacadm racresetcfg -c flex Deactivating FlexAddress Configuring FlexAddress for Chassis-Level Fabric and Slots Viewing WWN/MAC Address Information Fabric ConfigurationPage Racadm getflexaddr Viewing WWN/MAC Address Information Using Racadm Command Messages $racadm feature -d -c $racadm setflexaddr Error Insufficient userPrivileges to perform $racadm setflexaddr Operation FlexAddress Dell Software License Agreement Limited Warranty Open Source Software General Configuring Network Settings for IOM Fresh Power-up ScenarioManaging Fabrics Monitoring IOM Health Configuring Network Settings for IOM Using Racadm Managing Power Control Operation for I/O ModulesEnabling or Disabling LED Blinking for I/O Modules Managing and Monitoring Power Redundancy Policies Power Supply Redundancy PolicyGrid Redundancy Policy Grid Redundancy Levels Dynamic Power Supply Engagement Power Budgeting For Hardware Modules Default Redundancy ConfigurationPower Supply Redundancy Grid RedundancyPage Assigning Priority Levels To Servers Server Slot Power Priority Settings Viewing Power Consumption Status Assigning Priority Levels To Servers Using RacadmViewing Power Budget Status Using CMC Web Interface Viewing Power Consumption Status Using CMC Web Interface Viewing Power Budget Status Using Racadm Redundancy Status and Overall Power HealthPower Management After PSU Failure Power Management After Removing PSU Power required for new server is not Configuring Power Budget and Redundancy Maximum Power Conservation Mode Power Conservation and Power BudgetServer Power Reduction to Maintain Power Budget 110V PSUs AC Operation Remote Logging External Power Management Racadm config -g cfgChassisPower -o cfgChassisPowerCap Configuring Power Budget and Redundancy Using Racadm Executing Power Control Operations on the Chassis Executing Power Control Operations Racadm chassisaction -m chassis action Executing Power Control Operations on a Server Executing Power Control Operations on the IOM Using Racadm Executing Power Control Operations on the IOMRacadm chassisaction -m switch action Viewing Status of the Storage Components Managing Chassis StorageViewing the Storage Topology Assigning Virtual Adapters To Slots Using CMC Web Interface Fault-Tolerance in Storage Controllers Viewing Controller Properties Using Racadm Viewing Controller Properties Using CMC Web InterfaceImporting or Clearing Foreign Configuration Configuring Storage Controller Settings Using Racadm Configuring Storage Controller Settings Shared Perc Controllers Racadm raid enablepercAdapterFQDD Enabling or Diasabling RAID Controller Using RacadmRacadm raid disablepercAdapterFQDD Assigning Global Hot Spares Using CMC Web Interface Viewing Physical Disk Drives Properties Using RacadmAssigning Global Hot Spares Using Racadm Recovering Physical Disks Viewing Virtual Disk Properties Using Racadm Viewing Virtual Disk Properties Using CMC Web InterfaceCreating Virtual Disk Using CMC Web Interface Applying Virtual Adapter Access Policy To Virtual Disks Viewing Enclosure Properties Using CMC Web Interface Modifying Virtual Disk Properties Using CMC Web Interface Assigning PCIe Slots To Servers Using CMC Web Interface Viewing PCIe Slot Properties Using CMC Web InterfaceManaging PCIe Slots PCIe Power Ride-Through Managing PCIe Slots Using Racadm Racadm getpciecfg -r Viewing PCIe Ridethrough Properties Status Using Racadm Configuring PCIe Ride-through Properties Status Using Racadm Supported Interfaces Troubleshooting and RecoveryCLI Racadm First Steps to Troubleshoot a Remote System Power TroubleshootingDownloading Snmp Management Information Base MIB File Page Viewing Hardware Logs Using CMC Web Interface Troubleshooting AlertsViewing Event Logs Viewing Hardware Log Viewing Hardware Logs Using Racadm Using Diagnostic ConsoleViewing Chassis Logs Using Racadm Viewing Chassis Logs Using the Web Interface Saving or Restoring Chassis Configuration Resetting Components Racadm config -g cfgRemoteHosts -o cfgRhostsNtpMaxDist Troubleshooting Network Time Protocol NTP ErrorsRacadm config -g cfgRemoteHosts -o cfgRhostsNtpEnable Jan 8 200240 cmc ntpd1423 synchronized to LOCAL0, stratum Interpreting LED Colors and Blinking Patterns AC OK Troubleshooting Non-responsive CMCDC OK Obtain Recovery Information from DB-9 Serial Port Observing LEDs to Isolate the ProblemRecovering Firmware Image Troubleshooting Controller Troubleshooting Network ProblemsPage LCD Navigation Using LCD Panel Interface KVM Mapping Menu Main Menu DVD Mapping SettingsEnclosure Menu IP Summary Menu Default Screen DiagnosticsFront Panel LCD Messages LCD Language LCD Module and Server Status Information IOM a RPM Current blower speed in RPM Related. For more information, see LCD Error Messages Web interface Racadm subcommand Transport Error RC=-1 Frequently Asked QuestionsRacadm command failed Racadm chassislog help clear Racadm getconfig -g cfgOobSnmp Managing and Recovering a Remote System Active Directory FlexAddress and FlexAddressPlus No feature card inserted Error cant open file No feature card inserted IOM