Cisco Systems 78-16845-01 appendix Appendix J Troubleshooting Equipment Management Problems

Page 46

Appendix J Troubleshooting

J.10.4 Equipment Management Problems

J.10.4.6 CTM Database Is Inconsistent with Switch Data After Successful Coldstart (CTM Server Stop/Start) or Periodic Resync

After a successful node resynchronization triggered by the periodic resync, the CTM database is found to be inconsistent with the switch data.

Step 1 Collect the ooemc log files and all config upload files for the node. The ooemc implements a node-based cache. You can dump and save the cache.

Step 2 Get the process ID of the ooemc process that manages the node.

Step 3 Save the config upload files and ooemc log files.

Defect information—Collect the ooemc and config upload files from the /opt/svplus/log directory.

Possible alternative workaround: Manually resync the node. If the problem persists, perform a coldstart. If the problem is still not resolved, collect the log files and report the problem.

J.10.4.7 CTM CM GUI Shows Incomplete Connections After Successful Node Resync and Dbroker Sync-Up

CTM shows that some of connections are incomplete after the node resync process and dbroker synchronization process.

Step 1 Determine which ooemc process manages the node. The ooemc will manage the connection segment in the CTM database, which terminates on the MGX. Find the child ID of the ooemc process that manages your node. The purpose of finding the child ID of the ooemc process is to find the correct ooemc log files for inspection. The debugging process for this issue mainly focuses on log file inspection.

Step 2 After you have identified the ooemc log files, grep NotifyDataBroker from the files to see some log message examples before you modify your grep format. This will help you more efficiently grep the pair of log messages that correspond to the local and remote ends of one endpoint (primary endpoint or secondary endpoint) of a connection segment. In other words, each end (either primary end or secondary end) of a connection segment managed by ooemc should be logged with one pair of messages, one for local end and one for remote end. This pair of messages corresponds to one atm_connection segment entry in the database. Verify the information that ooemc sends to dbroker. Dbroker will refer to these messages from ooemc and construct a user_connection database entry. CTM Connection Management (CM) GUI displays connection information based on the user_connection database entry. Whether or not the connection is displayed as complete or incomplete depends on the information in the user_connection database entry. It is important to verify that ooemc sent the correct information to dbroker. Also verify that the segment database entries are populated correctly.

Step 3 If the correct number of messages have been forwarded to dbroker and the data in the messages is correct, but the user_connection database entry still contains invalid data, see J.10.13.2 Data Inconsistency, page J-129for more debugging information, or contact the dbroker development engineer (DE) for further investigation. If the problem is due to wrong data populated in segment database entries, then the EM DE should look at the problem.

Defect information:

Collect ooemc and dbroker log files from the /opt/svplus/log directory.

Cisco Transport Manager Release 6.0 User Guide

 

J-46

78-16845-01

 

 

 

Image 46
Contents Troubleshooting Server Problems Conditions that Affect CTM Server PerformanceCTM Server Does Not Respond Cannot Connect to the CTM Server NE Connection State Is Listed as Unavailable Snmp Traps Are Not Forwarded from NEs Launching Tables Results in Database ErrorsTrap Port Is Unavailable Problems with ONS 15530 and ONS 15540 Configuration Performance Monitoring Data Is Not DisplayedCTC-Based NE Is Not Reachable CTC-Based NE Is Not DiscoveredONS 15501, ONS 15540, or ONS 15530 Is Not Reachable Circuit State for Monitor Circuits Reads Duplicate ID Circuits Are Not DisplayedNE Model Type Appears as Unknown Cannot Copy CTC Binary to the CTM Server Memory Backup, Memory Restore, or Software Download FailsGetinfo.sh Script Fails Using the get nestate Command Database Is Not AvailableAre the CTM Client and the CTM Server Connected? Database Timeout OccursCannot Log In as Provisioner or Operator Cannot Authenticate User Message Appears Model Index Is UnknownCTC Fails to Start Cannot Delete an NEAdded a New Software Version to the Wrong NE Cannot Move an NE Between Subnetworks Cannot Delete a SubnetworkCannot Schedule Jobs NE Displays Incorrect Configuration Management Data Bandwidth-Intensive Operations Are BlockedCannot Customize the Network Map How Do I Collect Thread Dumps? SnmptrapClick OK How Do I Replace the Alarm Interface Panel? Client Debug MessagesCTM GateWay/TL1 Problems Cisco CRS-1 and Catalyst 6509 Problems Cat6509 enable# set snmp trap enable all CTM Does Not Receive Traps from the CatalystCTM Does Not Receive syslog Events from the Catalyst SyslogCTM Does Not Collect Historical PM Data for the CRS-1 Memory Backup or Restore Fails for the CRS-1 Click NE Service How Do I Configure Telnet on the CRS-1? Cannot Delete an Out-of-Service NE from CTMHow Do I Configure SSH on the CRS-1? How Do I View the Snmp Configuration on the Catalyst 6509? How Do I Load a Software Image on the Catalyst 6509?Output is similar to the following example Reporting XR 12000 Issues CTM Cannot Discover an XR 12000 Node4 XR 12000 Alarm and Event Reporting Issues 5 XR 12000 PM Collection Issues6 XR 12000 BGP and CDP Neighbor Discovery Issues 7 XR 12000 NE Explorer Applications Are DisabledDiscovery Mechanism Discovery Issues at StartupNo Nodes Are Discovered Output of the switch CLI, selnd, and dbnds Node Name Is Incorrect in the DatabaseNode Alarm Is Shown Incorrectly in the Database Node IP Is Incorrect in the DatabaseReachable Node Is Shown as Unreachable CTM State Is IncorrectDiscovery Issues at Runtime Collect the output of the switch CLI, selnd, and dbnds Equipment Management Problems Node Mode Remains Ls -ltr *.9 SMCARD01SRM.CF Node Resync Mode Is SMCARD01RPM.CFEMCSnmpFunccProcFuncGenNodeBulkFile1 entering Appendix J Troubleshooting Equipment Management Problems Appendix J Troubleshooting Equipment Management Problems Appendix J Troubleshooting Equipment Management Problems Database Table Population Through Traps and Snmp Upload Appendix J Troubleshooting Equipment Management Problems Appendix J Troubleshooting Equipment Management Problems You should see something like the following Figure J-1shows the end-to-end architecture Figure J-1 CTM End-to-End Architecture Basic Issues Topology Discovery Issues Appendix J Troubleshooting CTM CLI Network Element Discovery Issues All the Cards Under a Node Are MissingCisco Transport Manager Release 6.0 User Guide 78-16845-01 Appendix J Troubleshooting Alarm Processing Basics XML Schema for Alarm Rules Figure J-2 Client/Server Architecture for Alarm ComponentsFigure J-3 Correlated Alarm Rule Diagram Alarm Severity and Object Severity Figure J-5 Transient Alarm Rule DiagramCisco Transport Manager Release 6.0 User Guide 78-16845-01 Alarm List Shows Alarm that Does Not Exist on Platform Transient Event Has Disappeared Unexpectedly Chassis View Problems Chassis View BasicsLines Not Displayed in the Chassis View Card Not Displayed in the Chassis ViewRPM Card Status Not Updated Ethernet Status Not Updated on PXM CardsHIST, CBRX, and Cbtx Status Not Updated in MGX Nodes 10.6.4 DCA/DCB Status Not Displayed on PXM CardsLines Not Selectable Configuration Center ManagementConfiguration Center Framework Wrong Tooltip Is DisplayedCannot Launch the Configuration Center Choose Tools Configuration Center from any applicationAppendix J Troubleshooting Configuration Center Management Element Management-Drag and Drop Within Configuration Center Element Tab-Internal Frame Does Not LaunchCross Application-Configuration Center as Drag Source Appendix J Troubleshooting Configuration Center Management Configuration Center-Element Management XML Parsing ErrorSnmp No Data Error Details, Create, Delete, and Refresh Buttons Are Not EnabledSnmp Timeout Error Snmp Set ErrorObject Not Found In Tree View Error Element Data Inconsistent with Switch Configuration Center GUI-Framework Issues Connection Management ProblemsConfig Server Reports Error Messages Appendix J Troubleshooting Connection Management Problems Appendix J Troubleshooting Connection Management Problems Cross Application-Connection List as Drag Source Configuration Center GUI-CM Server-Reported Errors Configuration Centers Connection Tab Does Not RespondDATABASEERRORDatabase error %s Appendix J Troubleshooting Connection Management Problems Cmsvr Errors Cannot Create One-Segment Hybrid Unsupported Service Type-General node-slot-port-version Cmsvr-Connection Diagnostic Issues TestConn or TestDelay Failed with Switch ErrorConnection Up/Down/Reroute Failed Connection Trace FailuresCmgrd High-Level Connection Management Subsystem Architecture Figure J-6 Connection Management SubsystemCmgrd-Sdbroker Errors Related key index entries cmgrd, sdbroker, vpi/vci Appendix J Troubleshooting Connection Management Problems Cmgrd Errors 100101 102 Cmgrd-Switch Errors 103104 105 106 Diagnostics Center Problems 107Diagnostics Center Framework 108Cannot Launch Diagnostics Center 109110 Drop Target Displays Incorrect Object or Object Data Diagnostics Center Does Not Respond GUI Is Grayed Out111 Diagnostics Center Server-Specific Issues 112Snmpcommtimeout Error While Polling MIB Objects XML Parser Error113 Polling of Real-Time Counters Fails Node Sync-Up Status Are Not Shown in the Diagnostics CenterNode Resync Fails 114Unable to Start, Stop, or Modify Bert Operations Some Real-Time Counters Are Not Shown115 Unable to Start/Stop IMA Link Test Patterns Cannot Perform Connection Diagnostics116 Miscellaneous Issues or Problems in Diagnostics Center Performance Management Collection and Parsing ProblemsPM Collection Issues Pmcollector-Generic Troubleshooting118 PM Collection Fails for the Node-Log File Shows Snmp failed 119PM Collection Fails for the Node-Log File Shows Ftp failed 120StatsParser-Generic Troubleshooting Statistics Report ProblemsPM Parsing Issues 121You Collected Data but You See No Data Available You See FDNs for Other Entities when You Generate a ReportStatistics Report You Generated a Report but Do Not See Data for a Long TimeRtmProxy Service Agent ProblemsUtilization Report Value Is Greater than 100% 123Not Receiving Any Traps Registration with RtmProxy Failed124 Nodal Community Strings Do Not Work Audit Trail Logging Mechanism ProblemsManager Gets Deregistered 125Audit Trail Log File Naming Convention Miscellaneous ProblemsAuditLogger.conf Usage 12610.13.1 NTS Nodes Stay in Mode 1 After ColdStartConfig Change or Provisioning Activity Not Reflected on CTM 127How To Interpret NTS Log 128Data Inconsistency Connection Inconsistency Between the Switch and GUI129 Each output line is similar to this 130Process ID of the sdbroker will be returned 131Inconsistent Connection Secondary Status Inconsistent Connection Status132 Incomplete Connections 133CTM FTP Daemon Generic TroubleshootingFTP Username and Password FTP Daemon OverviewFTP Sessions in Switch Cwmftpd-File Not Available On Switch135 10.13.3.8 499 Session Limit Reached 10.13.3.7 421 Session Limit Reached136 General Log Information Failed to Acquire a Session with a SwitchFailed to Acquire Session After All Retries 137138
Related manuals
Manual 68 pages 52.71 Kb