Page 47
Appendix J Troubleshooting
J.10.4 Equipment Management Problems
•Collect config upload files from the /opt/svplus/tmp directory.
Possible alternative workaround is to manually resync the node. If the problem persists, perform a coldstart. If the problem is still not resolved, collect the log files and report the problem.
J.10.4.8 CTM GUI Shows Mismatched Information Between GUI Views and Database Data
After an initial ooemc node resync, such as a coldstart process, GUI views such as Network Monitor tree view, Inspector View, and Chassis View display information that does not match newly provisioned or updated database data. The problem persists even after each subsequent manual or periodic resync.
Step 1 Determine which ooemc process manages the node and find the child ID of the ooemc process that manages your node. The purpose of finding the child ID of the ooemc process is to find the correct ooemc log files for inspection. The debugging process for this issue mainly focuses on log file inspection.
Step 2 Initiate an initial node resync, such as coldstart. Ooemc starts sending newly provisioned or updated database data to the network management (NM) server. The current supported database tables for NM server message forwarding include the following:
•node
•card
•line
•ausm_port
•cesm_port
•frp
•rpm_port
•svc_port
•virtual_port
•aps
•ima_group
•ima_link
•linedistribution
•au4tug3
•controller
•license_in_use
•mfr_bundle
•mfr_link
•peripheral
•redundantcard
•sensor
Step 3 To determine whether or not ooemc sent newly provisioned or updated database data to the NM server, grep ComposeNMSMsg from the ooemc log files. The key words in log messages that correspond to the supported database tables in step 2 are the following:
•EMC_DBProperty_Node_c::ComposeNMSMsg
Cisco Transport Manager Release 6.0 User Guide
Contents
Troubleshooting
Conditions that Affect CTM Server Performance
Server Problems
CTM Server Does Not Respond
Cannot Connect to the CTM Server
NE Connection State Is Listed as Unavailable
Trap Port Is Unavailable
Launching Tables Results in Database Errors
Snmp Traps Are Not Forwarded from NEs
Performance Monitoring Data Is Not Displayed
Problems with ONS 15530 and ONS 15540 Configuration
ONS 15501, ONS 15540, or ONS 15530 Is Not Reachable
CTC-Based NE Is Not Discovered
CTC-Based NE Is Not Reachable
NE Model Type Appears as Unknown
Circuits Are Not Displayed
Circuit State for Monitor Circuits Reads Duplicate ID
Getinfo.sh Script Fails
Memory Backup, Memory Restore, or Software Download Fails
Cannot Copy CTC Binary to the CTM Server
Database Is Not Available
Using the get nestate Command
Cannot Log In as Provisioner or Operator
Database Timeout Occurs
Are the CTM Client and the CTM Server Connected?
Model Index Is Unknown
Cannot Authenticate User Message Appears
Added a New Software Version to the Wrong NE
Cannot Delete an NE
CTC Fails to Start
Cannot Schedule Jobs
Cannot Delete a Subnetwork
Cannot Move an NE Between Subnetworks
Bandwidth-Intensive Operations Are Blocked
NE Displays Incorrect Configuration Management Data
Cannot Customize the Network Map
Snmptrap
How Do I Collect Thread Dumps?
Click OK
Client Debug Messages
How Do I Replace the Alarm Interface Panel?
CTM GateWay/TL1 Problems
Cisco CRS-1 and Catalyst 6509 Problems
CTM Does Not Receive Traps from the Catalyst
Cat6509 enable# set snmp trap enable all
Syslog
CTM Does Not Receive syslog Events from the Catalyst
CTM Does Not Collect Historical PM Data for the CRS-1
Memory Backup or Restore Fails for the CRS-1
Click NE Service
How Do I Configure SSH on the CRS-1?
Cannot Delete an Out-of-Service NE from CTM
How Do I Configure Telnet on the CRS-1?
How Do I Load a Software Image on the Catalyst 6509?
How Do I View the Snmp Configuration on the Catalyst 6509?
Output is similar to the following example
CTM Cannot Discover an XR 12000 Node
Reporting XR 12000 Issues
5 XR 12000 PM Collection Issues
4 XR 12000 Alarm and Event Reporting Issues
7 XR 12000 NE Explorer Applications Are Disabled
6 XR 12000 BGP and CDP Neighbor Discovery Issues
No Nodes Are Discovered
Discovery Issues at Startup
Discovery Mechanism
Node Name Is Incorrect in the Database
Output of the switch CLI, selnd, and dbnds
Node IP Is Incorrect in the Database
Node Alarm Is Shown Incorrectly in the Database
CTM State Is Incorrect
Reachable Node Is Shown as Unreachable
Discovery Issues at Runtime
Collect the output of the switch CLI, selnd, and dbnds
Equipment Management Problems
Node Mode Remains
Ls -ltr *.9
SMCARD01SRM.CF
SMCARD01RPM.CF
Node Resync Mode Is
EMCSnmpFunccProcFuncGenNodeBulkFile1 entering
Appendix J Troubleshooting Equipment Management Problems
Appendix J Troubleshooting Equipment Management Problems
Appendix J Troubleshooting Equipment Management Problems
Database Table Population Through Traps and Snmp Upload
Appendix J Troubleshooting Equipment Management Problems
Appendix J Troubleshooting Equipment Management Problems
You should see something like the following
Figure J-1shows the end-to-end architecture
Figure J-1 CTM End-to-End Architecture
Basic Issues
Topology Discovery Issues
Appendix J Troubleshooting
CTM CLI
All the Cards Under a Node Are Missing
Network Element Discovery Issues
Cisco Transport Manager Release 6.0 User Guide 78-16845-01
Appendix J Troubleshooting
Alarm Processing Basics
Figure J-2 Client/Server Architecture for Alarm Components
XML Schema for Alarm Rules
Figure J-3 Correlated Alarm Rule Diagram
Figure J-5 Transient Alarm Rule Diagram
Alarm Severity and Object Severity
Cisco Transport Manager Release 6.0 User Guide 78-16845-01
Alarm List Shows Alarm that Does Not Exist on Platform
Transient Event Has Disappeared Unexpectedly
Chassis View Basics
Chassis View Problems
Card Not Displayed in the Chassis View
Lines Not Displayed in the Chassis View
10.6.4 DCA/DCB Status Not Displayed on PXM Cards
Ethernet Status Not Updated on PXM Cards
HIST, CBRX, and Cbtx Status Not Updated in MGX Nodes
RPM Card Status Not Updated
Wrong Tooltip Is Displayed
Configuration Center Management
Configuration Center Framework
Lines Not Selectable
Choose Tools Configuration Center from any application
Cannot Launch the Configuration Center
Appendix J Troubleshooting Configuration Center Management
Element Tab-Internal Frame Does Not Launch
Element Management-Drag and Drop Within Configuration Center
Cross Application-Configuration Center as Drag Source
Appendix J Troubleshooting Configuration Center Management
XML Parsing Error
Configuration Center-Element Management
Details, Create, Delete, and Refresh Buttons Are Not Enabled
Snmp No Data Error
Snmp Set Error
Snmp Timeout Error
Object Not Found In Tree View Error
Element Data Inconsistent with Switch
Config Server Reports Error Messages
Connection Management Problems
Configuration Center GUI-Framework Issues
Appendix J Troubleshooting Connection Management Problems
Appendix J Troubleshooting Connection Management Problems
Cross Application-Connection List as Drag Source
Configuration Centers Connection Tab Does Not Respond
Configuration Center GUI-CM Server-Reported Errors
DATABASEERRORDatabase error %s
Appendix J Troubleshooting Connection Management Problems
Cmsvr Errors
Cannot Create One-Segment Hybrid
Unsupported Service Type-General node-slot-port-version
TestConn or TestDelay Failed with Switch Error
Cmsvr-Connection Diagnostic Issues
Connection Trace Failures
Connection Up/Down/Reroute Failed
Cmgrd
Figure J-6 Connection Management Subsystem
High-Level Connection Management Subsystem Architecture
Cmgrd-Sdbroker Errors
Related key index entries cmgrd, sdbroker, vpi/vci
Appendix J Troubleshooting Connection Management Problems
100
Cmgrd Errors
101
102
103
Cmgrd-Switch Errors
104
105
106
107
Diagnostics Center Problems
108
Diagnostics Center Framework
109
Cannot Launch Diagnostics Center
110
111
Diagnostics Center Does Not Respond GUI Is Grayed Out
Drop Target Displays Incorrect Object or Object Data
112
Diagnostics Center Server-Specific Issues
113
XML Parser Error
Snmpcommtimeout Error While Polling MIB Objects
114
Node Sync-Up Status Are Not Shown in the Diagnostics Center
Node Resync Fails
Polling of Real-Time Counters Fails
115
Some Real-Time Counters Are Not Shown
Unable to Start, Stop, or Modify Bert Operations
116
Cannot Perform Connection Diagnostics
Unable to Start/Stop IMA Link Test Patterns
Pmcollector-Generic Troubleshooting
Performance Management Collection and Parsing Problems
PM Collection Issues
Miscellaneous Issues or Problems in Diagnostics Center
118
119
PM Collection Fails for the Node-Log File Shows Snmp failed
120
PM Collection Fails for the Node-Log File Shows Ftp failed
121
Statistics Report Problems
PM Parsing Issues
StatsParser-Generic Troubleshooting
You Generated a Report but Do Not See Data for a Long Time
You See FDNs for Other Entities when You Generate a Report
Statistics Report
You Collected Data but You See No Data Available
123
Service Agent Problems
Utilization Report Value Is Greater than 100%
RtmProxy
124
Registration with RtmProxy Failed
Not Receiving Any Traps
125
Audit Trail Logging Mechanism Problems
Manager Gets Deregistered
Nodal Community Strings Do Not Work
126
Miscellaneous Problems
AuditLogger.conf Usage
Audit Trail Log File Naming Convention
127
Nodes Stay in Mode 1 After ColdStart
Config Change or Provisioning Activity Not Reflected on CTM
10.13.1 NTS
128
How To Interpret NTS Log
129
Connection Inconsistency Between the Switch and GUI
Data Inconsistency
130
Each output line is similar to this
131
Process ID of the sdbroker will be returned
132
Inconsistent Connection Status
Inconsistent Connection Secondary Status
133
Incomplete Connections
FTP Daemon Overview
Generic Troubleshooting
FTP Username and Password
CTM FTP Daemon
135
Cwmftpd-File Not Available On Switch
FTP Sessions in Switch
136
10.13.3.7 421 Session Limit Reached
10.13.3.8 499 Session Limit Reached
137
Failed to Acquire a Session with a Switch
Failed to Acquire Session After All Retries
General Log Information
138