Monitoring EMS Event Messages, ViewPoint on | HP NonStop NS manual

4

Monitoring EMS Event Messages

When to Use This Section on page 4-1

What Is the Event Management Service (EMS)? on page 4-1

Tools for Monitoring EMS Event Messages on page 4-1

OSM Event Viewer on page 4-2

OSM Event Viewer on page 4-2

ViewPoint on page 4-2

Web ViewPoint on page 4-2

Related Reading on page 4-2

When to Use This Section

Use this section for a brief description of the Event Management Service (EMS) and the tools used to monitor EMS event messages.

What Is the Event Management Service (EMS)?

The Event Management Service (EMS) is a collection of processes, tools, and interfaces that support the reporting and retrieval of event information. Information retrieved from EMS can help you to:

•

•

•

•

•

•

Monitor your system or network environment

Analyze circumstances that led up to a problem

Detect failure patterns

Adjust for changes in the run-time environment

Recognize and handle critical problems

Perform many other tasks required to maintain a productive computing operation

Tools for Monitoring EMS Event Messages

To view EMS event messages for an Integrity NonStop server, use one of these tools:

•

•

•

•

OSM Event Viewer

EMSDIST

ViewPoint

Web ViewPoint

HP Integrity NonStop NS-Series Operations Guide—529869-005

4-1

Image 75

HP NonStop NS manual Monitoring EMS Event Messages, What Is the Event Management Service EMS?, ViewPoint on

Contents

Product Version Supported Release Version Updates RVUs AbstractPart Number Published Document History Part Number Product Version Published Introduction to Integrity NonStop NS-Series Operations HP Integrity NonStop NS-Series Operations Guide Determining Your System Configuration Monitoring EMS Event Messages ServerNet Resources Monitoring and Recovery 10-5 10-210-3 10-4 11-8 11-111-2 11-5 Power Failures Preparation and Recovery 15-22 15-1915-20 15-21 Preventive Maintenance Related Reading Converting Numbers Figures Examples Tables Document History What’s New in This ManualManual Information New and Changed Information Xiv Who Should Use This Guide About This Guide Converting Numbers What Is in This Guide ServerNet Cluster Manual Where to Get More Information General Syntax Notation Support and Service LibraryNotation Conventions Hypertext Links Inspect OFF on Saveabend MaxattachInterrupts Allowsu on OFF Notation for Messages Enter RUN CodeCode Received Operator Backup UpEvent number = number Subject = first-subject-value Proc-nametrapped in SQL in SQL file system Change Bar Notation Introduction to Integrity NonStop NS-Series Operations What Are the Operator Tasks? When to Use This SectionMonitoring the System and Performing Recovery Operations Understanding the Operational Environment Performing Preventive Maintenance Preparing for and Recovering from Power FailuresStopping and Powering Off the System Powering On and Starting the System Updating Firmware Determining the Cause of a Problem a Systematic ApproachResponding to Spooler Problems Problem-Solving Worksheet Problem-Solving Worksheet Problem Facts Possible CausesSituation Facts Escalation Decision Category Questions to Ask Task 1a Determine the Facts About the ProblemTask 1 Get the Facts Task 1b Determine the Facts About the Situation Task 2a Identify the Most Likely Cause Task 2 Find and Eliminate the Cause of the Problem Task 3b Provide Documentation Task 2b Fix the Most Probable Cause of the ProblemTask 3 Escalate the Problem If Necessary Task 3a Determine Whether You Need to Escalate the Problem Task 4 Prevent Future Problems Logging On to an Integrity NonStop ServerSystem Consoles Select StartProgramsOutsideView32 Opening a Tacl WindowOpening a Tacl Window Directly From OutsideView Opening a Tacl Window From the Low-Level Link Launching OSM Applications Overview of OSM Applications Service Procedures Determining Your System Configuration Integrity NonStop NS16000 Systems Modular Hardware Components Integrity NonStop NS1000 Systems Integrity NonStop NS14000 Systems System Resource or Object Recording Your System ConfigurationTerms Used to Describe System Hardware Components Device Using SCF to Determine Your System Configuration SCF Configuration FilesSCF System Naming Conventions Using SCF to Display Subsystem Configuration Information Save ConfigurationSCF Assume WS $L1.#TERM1 Example 2-1. SCF Listdev Command Output SCF Listdev Listing the Devices on Your SystemSCF Listdev Backup processor number and PIN of the specified device Specified deviceSubsystem Name Logical Name Device Type Description Displaying Information for the TCP/IP Subsystem $ZTCO Displaying SCF Configuration Information for SubsystemsTCP/IP Subsystem SCF Assume Process $ZTCO Displaying Information for the Kernel Subsystem $ZZKRN Kernel SubsystemStorage Subsystem SCF Assume Process $ZZKRN Cbpoollen Example 2-2. SCF ADD Disk Command OutputInfo Disk $SYSTEM,OBEYFORM Mirrorlocation 11,1,12 SCF Assume Process $ZZLAN ServerNet LAN Systems Access Slsa SubsystemADD Adapter $ZZLAN.E0154 Location Type G4SA Accesslist Info Tape $*,OBEYFORM Subsystem Objects Controlled by SCF page 1 WAN SubsystemAdditional Subsystems Controlled by SCF Displaying Information for the WAN Subsystem $ZZWAN X25AM Subsystem Objects Controlled by SCF page 2 Info Proc $ZZKRN.# Displaying Configuration Information-SCF ExamplesExample 2-3. SCF Info Process Command Output Example 2-4. SCF Info SAC Command Output Info Line $line-name, Detail Example 2-5. SCF Info Process $ZZWAN Command OutputExample 2-6. SCF Info Line Command Output Info Process $ZZWAN Overview of Monitoring and Recovery Working With a Daily Checklist Functions of MonitoringMonitoring Tasks Task Operator’s name Date & time Tools for Checking the Status of System Hardware Monitoring System Components Monitored Using These Resource Tools See ServerNet Cluster 6770 Hardware Daily Tasks Checklist Additional Monitoring TasksGeneral Tasks Specific Tasks For More Information, See Top-Down Approach Monitoring and Resolving Problems-An ApproachUsing OSM to Monitor the System Using the OSM Service Connection OSM Management System Icons Indicate Problems Within Expanding the Tree Pane to Locate the Source of Problems Attributes Tab Using System Status Icons to Monitor Multiple Systems Alarm Summary Dialog Box Using Alarm and Problem Summaries Monitoring Problem Incident Reports Using SCF to Monitor the SystemSuppressing Problems and Alarms Recovery Operations for Problems Detected by OSM Determining Device States Example 3-1. SCF Status Tape Command SCF Object States SCF Object States page 1State Substate Explanation Servicing Special SCF Object States page 2 Automating Routine System Monitoring Example 3-2. System Monitoring Command File Example 3-3. System Monitoring Output File page 1 Example 3-3. System Monitoring Output File page 2 Example 3-3. System Monitoring Output File page 3 Using the Status LEDs to Monitor the System Status LEDs and Their Functions page 1Location LED Name Color Function FC-AL I/O Status LEDs and Their Functions page 2 Task Tool For information, see Related ReadingStatus LEDs and Their Functions page 3 Related Reading for Monitoring ViewPoint on What Is the Event Management Service EMS?Monitoring EMS Event Messages Tools for Monitoring EMS Event Messages Related Reading for Monitoring EMS Event Messages OSM Event ViewerViewPoint Web ViewPoint Processes Monitoring Recovery Types of ProcessesSystem Processes Generic Processes Processes IOPs Monitoring System Processes Monitoring Processes Monitoring the Status of All Generic Processes Monitoring IOPsMonitoring Generic Processes Monitoring the Status of $ZZKRN CLCI-TACL $CLCI Stopped PID Recovery Operations for Processes Supported for the subsystem manager processesInitiates the operation of a generic process Communications Subsystems Communications Subsystems Monitoring and Recovery Local Area Networks LANs and Wide Area Networks WANs Object Connectivity By LAN Service Provider Subsystems Supported To monitor the status of an adapter Monitoring Communications Subsystems and Their ObjectsMonitoring the Slsa Subsystem Monitoring the Status of an Adapter and Its Components SCF Status PIF pif-name SCF Status Adapter $ZZLANSCF Status SAC sac-name SCF Status SAC $ZZLAN.G11123 SCF Status LIF $ZZLAN.L11021A , Detail Monitoring the WAN SubsystemMonitoring Status for a Swan Concentrator SCF Status PIF $ZZLAN.G11123 SCF Status Device $ZZWAN.#device-name Monitoring Status for a Data Communications DeviceSystem displays a listing similar to SCF Status Adapter $ZZWAN SCF Status Process $ZZWAN To monitor a single WANBoot process, typeSCF Status Process $ZZWAN.#boot-process Monitoring WAN Processes SCF Listdev Tcpip Monitoring CLIPsMonitoring the NonStop TCP/IP Subsystem Monitoring the NonStop TCP/IP Process SCF Status Route $ZTCO Monitoring NonStop TCP/IP RoutesMonitoring NonStop TCP/IP Subnets Monitoring Line-Handler Process Status Info Process $NCP, Lineset SCF Status Line $LHCS6S, Detail Tracing a Communications Line Recovery Operations for Communications Subsystems Related Reading for Communications Lines and Devices page 1For Information About Refer to Related Reading for Communications Lines and Devices page 2 WAN Subsystem Configuration and Management Manual ServerNet Communications Network ServerNet Resources Monitoring and Recovery Integrity NonStop NS16000 System Integrity NonStop NS16000 ServerNet Connectivity Integrity NonStop NS14000 System with Ioam Enclosure Integrity NonStop NS14000 ServerNet Connectivity System I/O ServerNet Connections Monitoring the Status of the ServerNet FabricsIntegrity NonStop NS1000 ServerNet Connectivity Monitoring the ServerNet Fabrics Using OSM SCF Status Servernet $ZSNET Monitoring the ServerNet Fabrics Using SCF Normal ServerNet Fabric States Identifying ServerNet Fabric Problems Recovery Operations for a Down Processor Recovery Operations for the ServerNet FabricsRecovery Operations for a Down Disk Due to a Fabric Failure Recovery Operations for a Down Path Between Processors Adapters and Modules Monitoring and Recovery Adapters and Modules Fibre Channel ServerNet Adapter FcsaGigabit Ethernet 4-Port Adapter G4SA Port ServerNet Extender 4PSE Monitoring I/O Adapters and Modules Monitoring the FCSAs SCF Status Adapter $ZZSTO.#FCSA*, DetailState Description SCF Status Adapter $ZZLAN.G1123 Monitoring the G4SAs Service, Device, and Enabled States for the G4SA page 1 Monitoring the 4PSEs Recovery Operations for I/O Adapters and Modules ServerNet/DA Manual Related Reading for I/O Adapters and Modules Processors and Components Monitoring and Recovery Overview of the NonStop Blade Complex LSU CPU Term Description Monitoring Processors Automatically Using TfdsMonitoring and Maintaining Processors Summary, these terms describe the Nsaa processor Processor Status Display Monitoring Processor Status Using the OSM Low-Level Link OSM Representation of Processor Complex Viewsys Identifying Processor ProblemsMonitoring Processor Performance Using ViewSys Processor or System Hangs Freeze code = %nnnnnn OSM Alarms and Attribute ValuesProcessor Halts Halt code = %nnnnnn Recovery Operations for a Processor Halt Recovery Operations for Processors Halting One or More Processors Reloading a Single Processor on a Running ServerSelect Processor ActionsHalt Click Perform action Using Tacl Reload to Perform Reload Select FileStart Terminal EmulatorReload / run-option , run-option Fabric Noswitch Primenoprime fabric Omitblade ABCNoswitch Primenoprime Select Reload, click Perform action Using the OSM Service Connection to Perform Reload Recovery Operations for a System Hang Freezing the System and Freeze-Enabled Processors Enabling/Disabling Processor and System FreezeDumping a Processor to Disk See Using Rcvdump to Dump a Processor to Disk on Using Rcvdump to Dump a Processor to Disk Before You BeginFUP Purgedata dumpfile Blade Element Reintegration CPU n has been dumped to dumpfileFUP Info dumpfile Replacing Processor Memory Troubleshooting and Recovery Operations for Disk DumpsSubmitting Information to Your Service Provider Backing Up a Processor Dump to Tape Submitting Tapes of Processor Dumps Submitting Tapes of Configuration and Operations FilesOther Files to Submit to Your Service Provider Backup $tape, CPU0,$SYSTEM.SYS00.CONFTEXT Additional Information Required by Your Service Provider For Information About Tool See Disk Drives Monitoring and Recovery Overview of Disk Drives Internal Scsi Disk DrivesFor information about M8xxx Fibre Channel Disk Drives Enterprise Storage System ESS DisksFor information about See Monitoring Disk Drives Monitoring Disk Drives With OSMTask See Status Disk $*, SUB Magnetic Monitoring Disk Drives With SCF Status $DATA02-M Status Disk $DATA09, DetailTo display the status of the disk $DATA01 Status $DATA01 Status Disk $ To display the status of all disks Status Disk $DATA00 To display the detailed status of the disk $DATA01Status $DATA01, Detail To display status of all paths for $DATA00 Monitoring the Size of Database Files Monitoring the Use of Space on a Disk VolumePrimary and Backup Path States for Disk Drives Monitoring the State of Disk Drives FUP Info DATA1.MEMOS, Detail Monitoring Disk Configuration and PerformanceExample To check the size of the file DATA1.MEMOS Identifying Disk Drive Problems Possible Causes of Common Disk Drive ProblemsProblems Possible Symptoms Command Description Recovery Operations for Disk DrivesThese SCF commands control Disk objects Common Recovery Operations for Disk Drives page 1 Customer Support Center or your service provider Common Recovery Operations for Disk Drives page 2 Start Disk $volume Recovery Operations for a Down Disk or Down Disk PathReset Disk $volume Reset Disk $WD8 Recovery Operations for a Nearly Full Database File FUP Alter MEMOS, Maxextents Info MEMOS, DetailReport such as this one is sent to your home terminal 10-16 Overview of Tape Drives Tape Drives Monitoring and Recovery Monitoring Tape Drive Status With OSM Monitoring Tape Drives OSM Monitoring Tape Drives Connected to an Fcsa OSM Monitoring Tape Drives Connected to an IOMF2 Listing such as this one is sent to your home terminal Monitoring Tape Drive Status With SCFListing similar to this one is sent to your home terminal SCF Status Tape $TAPE0 Monitoring Tape Drive Status With Mediacom Mediacom Status TapedriveMediacom Status Tapedrive $TAPE0 Monitoring the Status of Labeled-Tape Operations Identifying Tape Drive ProblemsCommon Tape Drive Problems Symptom Problem Possible Causes Performing an OSM Action on a Multiple Tape Drives Recovery Operations for Tape DrivesRecovery Operations Using the OSM Service Connection Performing an OSM Action on a Tape Drive Recovery Operations Using SCF SCF Command DescriptionRelated Reading for Tapes and Tape Drives page 1 Related Reading for Tapes and Tape Drives page 2 Overview of Printers and Terminals Printers and Terminals Monitoring and Recovery Spoolcom DEV $LASER Monitoring Printer and Collector Process StatusMonitoring Printer Status Monitoring Collector Process Status Recovery Operations for a Full Collector Process Recovery Operations for Printers and Terminals 12-4 Applications Monitoring and Recovery Monitoring TMFTMF States on ~ Status TMF Monitoring Data VolumesMonitoring the Status of TMF Tmfcom TMF States page 1 TMF StatesTmfcom responds with output similar to TMF subsystem can be in any of the states listed in Table = Status Pathway Monitoring the Status of PathwayTMF States page 2 Status *, Prog $*.*.PATHMON Pathmon States Pathcom responds with output such as= Status Pathmon Request is waiting for an object that has been locked by Another requesterRequest is waiting for a RUN Program to finish ESS Cabinets on Power Failures Preparation and Recovery External Devices System Response to Power FailuresNonStop NS-Series Cabinets Modular Cabinets NonStop S-Series I/O Enclosures Air Conditioning Preparing for Power FailureConfigure OSM Power Fail Support ESS Cabinets Maintain Batteries Power Failure RecoveryMonitor Power Supplies Monitor Batteries Setting System Time Procedure to Recover From a Power Failure 14-6 Alerts on Starting and Stopping the System Powering On a System Select Hard Reset Click Perform Action Powering On the System From a Low Power StatePowering On the System From a No Power State Select Power On System 15-4 Normal System Load Starting a SystemLoading the System Alerts System Load Disks System Load to a Specific Processor Path Group Module Slot System Load Paths for a Normal System LoadSystem Load Paths in Order of Use Disk Drive Enclosure Data Travels Configuration File Performing a System Load Starting Other System Components Click Start system System Load Dialog Box Performing a System Load From a Specific Processor Reload 01 15, Prime Reloading Processors Using the Reload CommandReloading Processors Using OSM Reloading Processors Logical Processor Reload Parameters Minimizing the Frequency of Planned Outages Stopping Application, Devices, and ProcessesAnticipating and Planning for Change RUN Stopscm = SHUTDOWN2, Mode OrderlyVolume $DSMSCM.ZDSMSCM Stop DSM/SCM Tmfcom Stop TMF Halting All Processors Using OSMStopping the System Spoolcom supervisor-name, SPOOLER, Drain From the Processors Actions menu, select Halt Powering Off a SystemSystem Power-Off Using OSM System Power-Off Using SCF Troubleshooting and Recovery Operations Emergency Power-Off ProcedureFans Are Not Turning System Does Not Appear to Be Powered On Components Fail When Testing the PowerGreen LED Is Not Lit After POSTs Finish Info Subsys $ZZKRN Recovering From a System Load Failure Getting a Corrupt System Configuration File Analyzed Recovering From a Reload FailureBackup $TAPE, $SYSTEM.ZSYSCONF.CONFSAVE, Listall Opening Startup Event Stream and Startup Tacl Windows Exiting the OSM Low-Level Link 15-23 Related Reading for Starting and Stopping a System NonStop NS-Series Hardware Installation Guide Startup on Shutdown on Creating Startup and Shutdown Files Managed Configuration Services MCS Automating System Startup and ShutdownStartup Processes That Represent the System Console ShutdownFor More Information $ZHOME Alternative Example Command Files Ciin File Modifying a Ciin File Establishing a Ciin File If a Ciin File Is Not Specified or Enabled in OSM Conftext Ciin EntryCiin File Option Results Reload /TERM $ZHOME, OUT $ZHOME Example Ciin Files = Start Term TERM1, TERM2, TERM3, TERM4, TERM5, TERM6 Writing Efficient Startup and Shutdown Command FilesCommand File Syntax = Start Term Use Parallel Processing Avoid Manual Intervention How Process Persistence Affects Configuration and Startup Tips for Startup FilesInvestigate Product-Specific Techniques Startup File Examples System Startup FileObey $SYSTEM.STARTUP.STRTSYS 16-13 Obey $SYSTEM.STARTUP.SPLWARM TCP/IP Stack Configuration and Startup FileSpooler Warm-Start File TMF Warm-Start File 16-15 16-16 CP6100 Lines Startup File ATP6100 Lines Startup FileLines Startup File Printer Line Startup File Expand-Over-IP Line Startup FileExpand Direct-Connect Line Startup File Shutdown File Examples Tips for Shutdown Files Obey $SYSTEM.SHUTDOWN.STOPSYS System Shutdown File CP6100 Lines Shutdown File ATP6100 Lines Shutdown FileLines Shutdown File Printer Line Shutdown File Expand-Over-IP Line Shutdown FileDirect-Connect Line Shutdown File Tmfcom / in $SYSTEM.SHUTDOWN.TMFSTOP, OUT $ZHOME Spooler Shutdown FileTMF Shutdown File Obey $SYSTEM.SHUTDOWN.SPLDRAIN 16-24 Preventive Maintenance Checking Air Temperature and HumidityMonitoring Physical Facilities Cleaning System Components Handling and Storing Cartridge Tapes Cleaning Tape Drives 17-4 HP Integrity NonStop NS-Series Operations Guide-529869-005 Page Tools and Utilities for Operations When to Use This Appendix Event Management Service Analyzer Emsa Disk Compression Program DcomDisk Space Analysis Program Dsap Nskcom and the Kernel-Managed Swap Facility Kmsf File Utility Program FUPMeasure NonStop NET/MASTER Subsystem Control Facility SCF Pathcom HP Tandem Advanced Command Language Tacl Web ViewPointViewPoint ViewSys Related Reading Table C-1. Related Reading for Tools and Utilities page 1Tool Documentation Description NET/MASTER MS Table C-1. Related Reading for Tools and Utilities page 2 Management Manual Table C-1. Related Reading for Tools and Utilities page 3 Table C-1. Related Reading for Tools and Utilities page 4 Recovery Guide Output Table C-1. Related Reading for Tools and Utilities page 5Page Converting Numbers Overview of Numbering Systems Table D-1. Descriptions of Number SystemsNumber System Base Description Binary Value Decimal Value Binary to Decimal Octal Value Decimal Value 1375 765 Octal to Decimal Hexadecimal Decimal Hexadecimal to Decimal Hexadecimal Value Decimal Value HBA10 47632 Figure D-3. Hexadecimal to Decimal Conversion Decimal Value Binary Value 354 B101100010 Decimal to BinaryResult is Step Division Quotient Remainder Decimal to Octal Step Division QuotientDecimal Value Octal Value Decimal to Hexadecimal Decimal HexadecimalDecimal Value Hexadecimal Value Page Canadian Compliance Safety and ComplianceRegulatory Compliance Statements FCC Compliance Statements-2 Laser Compliance European Union Notice Safety Caution Waste Electrical and Electronic Equipment Weee Important Safety Information Statements-6 See Conftext file IndexNumbers Port ServerNet Extender 4PSE Dcom 10-15,B-2 FUP Nskcom B-3 Index-5 SACs SCF B-4 commands System shutdown file 16-20 TMF LinesSpooler 16-14 Startup files Tacl 9-22,16-5,B-5 Special Characters Index-8