Cisco Systems 78-16845-01 appendix Appendix J Troubleshooting Connection Management Problems

Page 99

Appendix J Troubleshooting

J.10.8 Connection Management Problems

Provisioning fails and the following error messages are displayed:

"No more vpi/vci available for local trunk end" and "Remote trunk"

"Local end of the connection already exists," "Remote end," and "Both end"

"Vpcon already exists for Local Vpi," "Remote Vpi," and "Both ends"

Each of these error messages identifies the reason for the failure of provisioning. They sometimes can be displayed in error. The reason is a mismatch between the switch and the DMD's cache. To find out if a mismatch has occurred, dump the DMD cache, using the command pkill -USR1 dmd and compare the values in the cache with those on the switch for endpoints associated with the error.

Defect Information—If the switch and DMD cache are not in sync, you will need the DMD logs, message logs, DMD cache dump, and the EM logs for the node in question. You will also need the switch CLI screen shot of the connection in question.

Possible alternative workaround—If there is a cache inconsistency, the only workaround is a complete cache resync (/opt/svplus/tools/CacheResync). If there is no DMD cache inconsistency, it is a normal operational scenario. Different connection add parameters should be chosen.

J.10.8.4.7 Cmgrd—Sdbroker Addition Errors: Adding a Connection Results in "EndPoint Exists Local/Remote/Both end of the connection already exists." Error

Related key index entries: cmgrd, sdbroker, endpoint exists

During an add connection request, the cmgrd will request intermediate endpoint vpi/vci from sdbroker. If the sdbroker incorrectly chooses endpoints that are already in use by the switch the cmgrd will try to provision these endpoints and return the error "endpoint exists" to the user.

Step 1 Determine which of the endpoints already exist on the switch. (If it is the intermediate endpoints then it is an sdbroker problem.)

a.When provisioning hybrid connections the CMGRD requests intermediate endpoints from the sdbroker. The sdbroker then requests them from DMD and then sends them back to CMGRD. First use the dbcmap command (/opt/svplus/tools/dbcmap) to identify the DMD that is used to process the node. Then dump that DMD's cache. Verify that the DMD does not contain the endpoints in question in its cache.

b.Determine why the DMD doesn't have the information on endpoints that exist on a switch. You first need to determine if the EM sent the add message to the DMD. See J.10.13.2.1 Connection Inconsistency Between the Switch and GUI, page J-129. If the add message is not found and the logs do not go back to coldstart, then you will need to check to see if the EM has done any processing of this endpoint. Check for the endpoint in the xxx_Connection table. If the endpoint is not there, check the EM to determine why it did not process the endpoint. If the endpoint is in the xxx_connection table, you know that the EM processed it, but you do not know if it forwarded it to the DMD.

Defect Information:

If the problem is an intermediate endpoint and it is on the switch, but the EM didn't process it or didn't' send the message to DMD, check the EM log and nts log from /opt/svplus/log.

If the problem is within the DMD you will need the DMD logs, DMD message logs, and the DMD cache dump.

Possible alternative workaround—None.

Cisco Transport Manager Release 6.0 User Guide

 

78-16845-01

J-99

 

Image 99
Contents Troubleshooting Conditions that Affect CTM Server Performance Server ProblemsCTM Server Does Not Respond Cannot Connect to the CTM Server NE Connection State Is Listed as Unavailable Launching Tables Results in Database Errors Snmp Traps Are Not Forwarded from NEsTrap Port Is Unavailable Performance Monitoring Data Is Not Displayed Problems with ONS 15530 and ONS 15540 ConfigurationCTC-Based NE Is Not Discovered CTC-Based NE Is Not ReachableONS 15501, ONS 15540, or ONS 15530 Is Not Reachable Circuits Are Not Displayed Circuit State for Monitor Circuits Reads Duplicate IDNE Model Type Appears as Unknown Memory Backup, Memory Restore, or Software Download Fails Cannot Copy CTC Binary to the CTM ServerGetinfo.sh Script Fails Database Is Not Available Using the get nestate CommandDatabase Timeout Occurs Are the CTM Client and the CTM Server Connected?Cannot Log In as Provisioner or Operator Model Index Is Unknown Cannot Authenticate User Message AppearsCannot Delete an NE CTC Fails to StartAdded a New Software Version to the Wrong NE Cannot Delete a Subnetwork Cannot Move an NE Between SubnetworksCannot Schedule Jobs Bandwidth-Intensive Operations Are Blocked NE Displays Incorrect Configuration Management DataCannot Customize the Network Map Snmptrap How Do I Collect Thread Dumps?Click OK Client Debug Messages How Do I Replace the Alarm Interface Panel?CTM GateWay/TL1 Problems Cisco CRS-1 and Catalyst 6509 Problems CTM Does Not Receive Traps from the Catalyst Cat6509 enable# set snmp trap enable allSyslog CTM Does Not Receive syslog Events from the CatalystCTM Does Not Collect Historical PM Data for the CRS-1 Memory Backup or Restore Fails for the CRS-1 Click NE Service Cannot Delete an Out-of-Service NE from CTM How Do I Configure Telnet on the CRS-1?How Do I Configure SSH on the CRS-1? How Do I Load a Software Image on the Catalyst 6509? How Do I View the Snmp Configuration on the Catalyst 6509?Output is similar to the following example CTM Cannot Discover an XR 12000 Node Reporting XR 12000 Issues5 XR 12000 PM Collection Issues 4 XR 12000 Alarm and Event Reporting Issues7 XR 12000 NE Explorer Applications Are Disabled 6 XR 12000 BGP and CDP Neighbor Discovery IssuesDiscovery Issues at Startup Discovery MechanismNo Nodes Are Discovered Node Name Is Incorrect in the Database Output of the switch CLI, selnd, and dbndsNode IP Is Incorrect in the Database Node Alarm Is Shown Incorrectly in the DatabaseCTM State Is Incorrect Reachable Node Is Shown as UnreachableDiscovery Issues at Runtime Collect the output of the switch CLI, selnd, and dbnds Equipment Management Problems Node Mode Remains Ls -ltr *.9 SMCARD01SRM.CF SMCARD01RPM.CF Node Resync Mode IsEMCSnmpFunccProcFuncGenNodeBulkFile1 entering Appendix J Troubleshooting Equipment Management Problems Appendix J Troubleshooting Equipment Management Problems Appendix J Troubleshooting Equipment Management Problems Database Table Population Through Traps and Snmp Upload Appendix J Troubleshooting Equipment Management Problems Appendix J Troubleshooting Equipment Management Problems You should see something like the following Figure J-1shows the end-to-end architecture Figure J-1 CTM End-to-End Architecture Basic Issues Topology Discovery Issues Appendix J Troubleshooting CTM CLI All the Cards Under a Node Are Missing Network Element Discovery IssuesCisco Transport Manager Release 6.0 User Guide 78-16845-01 Appendix J Troubleshooting Alarm Processing Basics Figure J-2 Client/Server Architecture for Alarm Components XML Schema for Alarm RulesFigure J-3 Correlated Alarm Rule Diagram Figure J-5 Transient Alarm Rule Diagram Alarm Severity and Object SeverityCisco Transport Manager Release 6.0 User Guide 78-16845-01 Alarm List Shows Alarm that Does Not Exist on Platform Transient Event Has Disappeared Unexpectedly Chassis View Basics Chassis View ProblemsCard Not Displayed in the Chassis View Lines Not Displayed in the Chassis View10.6.4 DCA/DCB Status Not Displayed on PXM Cards Ethernet Status Not Updated on PXM CardsHIST, CBRX, and Cbtx Status Not Updated in MGX Nodes RPM Card Status Not UpdatedWrong Tooltip Is Displayed Configuration Center ManagementConfiguration Center Framework Lines Not SelectableChoose Tools Configuration Center from any application Cannot Launch the Configuration CenterAppendix J Troubleshooting Configuration Center Management Element Tab-Internal Frame Does Not Launch Element Management-Drag and Drop Within Configuration CenterCross Application-Configuration Center as Drag Source Appendix J Troubleshooting Configuration Center Management XML Parsing Error Configuration Center-Element ManagementDetails, Create, Delete, and Refresh Buttons Are Not Enabled Snmp No Data ErrorSnmp Set Error Snmp Timeout ErrorObject Not Found In Tree View Error Element Data Inconsistent with Switch Connection Management Problems Configuration Center GUI-Framework IssuesConfig Server Reports Error Messages Appendix J Troubleshooting Connection Management Problems Appendix J Troubleshooting Connection Management Problems Cross Application-Connection List as Drag Source Configuration Centers Connection Tab Does Not Respond Configuration Center GUI-CM Server-Reported ErrorsDATABASEERRORDatabase error %s Appendix J Troubleshooting Connection Management Problems Cmsvr Errors Cannot Create One-Segment Hybrid Unsupported Service Type-General node-slot-port-version TestConn or TestDelay Failed with Switch Error Cmsvr-Connection Diagnostic IssuesConnection Trace Failures Connection Up/Down/Reroute FailedCmgrd Figure J-6 Connection Management Subsystem High-Level Connection Management Subsystem ArchitectureCmgrd-Sdbroker Errors Related key index entries cmgrd, sdbroker, vpi/vci Appendix J Troubleshooting Connection Management Problems 100 Cmgrd Errors101 102 103 Cmgrd-Switch Errors104 105 106 107 Diagnostics Center Problems108 Diagnostics Center Framework109 Cannot Launch Diagnostics Center110 Diagnostics Center Does Not Respond GUI Is Grayed Out Drop Target Displays Incorrect Object or Object Data111 112 Diagnostics Center Server-Specific IssuesXML Parser Error Snmpcommtimeout Error While Polling MIB Objects113 114 Node Sync-Up Status Are Not Shown in the Diagnostics CenterNode Resync Fails Polling of Real-Time Counters FailsSome Real-Time Counters Are Not Shown Unable to Start, Stop, or Modify Bert Operations115 Cannot Perform Connection Diagnostics Unable to Start/Stop IMA Link Test Patterns116 Pmcollector-Generic Troubleshooting Performance Management Collection and Parsing ProblemsPM Collection Issues Miscellaneous Issues or Problems in Diagnostics Center118 119 PM Collection Fails for the Node-Log File Shows Snmp failed120 PM Collection Fails for the Node-Log File Shows Ftp failed121 Statistics Report ProblemsPM Parsing Issues StatsParser-Generic TroubleshootingYou Generated a Report but Do Not See Data for a Long Time You See FDNs for Other Entities when You Generate a ReportStatistics Report You Collected Data but You See No Data Available123 Service Agent ProblemsUtilization Report Value Is Greater than 100% RtmProxyRegistration with RtmProxy Failed Not Receiving Any Traps124 125 Audit Trail Logging Mechanism ProblemsManager Gets Deregistered Nodal Community Strings Do Not Work126 Miscellaneous ProblemsAuditLogger.conf Usage Audit Trail Log File Naming Convention127 Nodes Stay in Mode 1 After ColdStartConfig Change or Provisioning Activity Not Reflected on CTM 10.13.1 NTS128 How To Interpret NTS LogConnection Inconsistency Between the Switch and GUI Data Inconsistency129 130 Each output line is similar to this131 Process ID of the sdbroker will be returnedInconsistent Connection Status Inconsistent Connection Secondary Status132 133 Incomplete ConnectionsFTP Daemon Overview Generic TroubleshootingFTP Username and Password CTM FTP DaemonCwmftpd-File Not Available On Switch FTP Sessions in Switch135 10.13.3.7 421 Session Limit Reached 10.13.3.8 499 Session Limit Reached136 137 Failed to Acquire a Session with a SwitchFailed to Acquire Session After All Retries General Log Information138
Related manuals
Manual 68 pages 52.71 Kb