Identifying Disk Drive Problems | HP NonStop NS manual

Disk Drives: Monitoring and Recovery

Identifying Disk Drive Problems

Identifying Disk Drive Problems

For recovery operations, refer to Recovery Operations for Disk Drives on page 10-12.

Table 10-2. Possible Causes of Common Disk Drive Problems

	Problems	Possible Symptoms
	Disk is full or does not have enough space.	Error 43 (unable to obtain disk space for file
		extent) occurs.
	Disk free space is fragmented.	extent) occurs.
	Disk free space is fragmented.	If the disk is full, an application might go down.
		If the disk is full, an application might go down.

	One disk in a mirrored pair is down.	The storage subsystem generates an event
		message but the application continues to run.

	An unmirrored disk is down, or both disks	Users report access problems, applications go
	in a mirrored pair are down.	down, and the storage subsystem generates
		event messages.

	Performance problems occur due to path	Users report poor application performance.
	switches or a cache size that is too small.
	Defective tracks or sectors exist.	Output from SCF INFO DISK, BAD command
		indicates unspared defective sectors.
	Disk errors exceed a certain limit.	Intm-errors-exceeded message
	Slow I/O operations exceed a certain limit.	Slow-IOs-threshold-exceeded message

Internal SCSI Disk Drives

The most common disk drive problems on a NonStop S-series server include:

•

•

•

•

Space problems such as full disks or free-space fragmentation

Stopped disks

Performance problems

Defective tracks or sectors

M8xxx Fibre Channel Disk Drives

The most common disk problems on an Integrity NonStop NS-series server are intm- errors-exceeded and slow-IOs-threshold-exceeded errors on the Fibre Channel loop.

Such errors are often normal. However, if they cause problems on a Fibre Channel loop, power the affected disk down and up again. This procedure can solve the problem temporarily.

Unless you are a qualified service provider, you cannot perform any physical actions on disk drives. However, operators can use OSM and SCF commands.

HP Integrity NonStop NS-Series Operations Guide—529869-005

10-11

Image 145

HP NonStop NS Identifying Disk Drive Problems, Possible Causes of Common Disk Drive Problems, Problems Possible Symptoms

Contents

Abstract Product Version Supported Release Version Updates RVUsPart Number Published Document History Part Number Product Version Published Introduction to Integrity NonStop NS-Series Operations HP Integrity NonStop NS-Series Operations Guide Determining Your System Configuration Monitoring EMS Event Messages ServerNet Resources Monitoring and Recovery 10-3 10-210-4 10-5 11-2 11-111-5 11-8 Power Failures Preparation and Recovery 15-20 15-1915-21 15-22 Preventive Maintenance Related Reading Converting Numbers Figures Examples Tables Manual Information What’s New in This ManualNew and Changed Information Document History Xiv Who Should Use This Guide About This Guide Converting Numbers What Is in This Guide ServerNet Cluster Manual Where to Get More Information Notation Conventions Support and Service LibraryHypertext Links General Syntax Notation Interrupts MaxattachAllowsu on OFF Inspect OFF on Saveabend Enter RUN Code Notation for MessagesCode Received Event number = number Subject = first-subject-value Backup UpProc-nametrapped in SQL in SQL file system Operator Change Bar Notation Introduction to Integrity NonStop NS-Series Operations Monitoring the System and Performing Recovery Operations When to Use This SectionUnderstanding the Operational Environment What Are the Operator Tasks? Stopping and Powering Off the System Preparing for and Recovering from Power FailuresPowering On and Starting the System Performing Preventive Maintenance Responding to Spooler Problems Determining the Cause of a Problem a Systematic ApproachProblem-Solving Worksheet Updating Firmware Problem Facts Possible Causes Problem-Solving WorksheetSituation Facts Escalation Decision Task 1 Get the Facts Task 1a Determine the Facts About the ProblemTask 1b Determine the Facts About the Situation Category Questions to Ask Task 2a Identify the Most Likely Cause Task 2 Find and Eliminate the Cause of the Problem Task 3 Escalate the Problem If Necessary Task 2b Fix the Most Probable Cause of the ProblemTask 3a Determine Whether You Need to Escalate the Problem Task 3b Provide Documentation Logging On to an Integrity NonStop Server Task 4 Prevent Future ProblemsSystem Consoles Opening a Tacl Window Directly From OutsideView Opening a Tacl WindowOpening a Tacl Window From the Low-Level Link Select StartProgramsOutsideView32 Launching OSM Applications Overview of OSM Applications Service Procedures Determining Your System Configuration Integrity NonStop NS16000 Systems Modular Hardware Components Integrity NonStop NS1000 Systems Integrity NonStop NS14000 Systems Terms Used to Describe System Hardware Components Recording Your System ConfigurationDevice System Resource or Object SCF Configuration Files Using SCF to Determine Your System ConfigurationSCF System Naming Conventions Save Configuration Using SCF to Display Subsystem Configuration InformationSCF Assume WS $L1.#TERM1 SCF Listdev Listing the Devices on Your System Example 2-1. SCF Listdev Command OutputSCF Listdev Specified device Backup processor number and PIN of the specified deviceSubsystem Name Logical Name Device Type Description TCP/IP Subsystem Displaying SCF Configuration Information for SubsystemsSCF Assume Process $ZTCO Displaying Information for the TCP/IP Subsystem $ZTCO Storage Subsystem Kernel SubsystemSCF Assume Process $ZZKRN Displaying Information for the Kernel Subsystem $ZZKRN Info Disk $SYSTEM,OBEYFORM Example 2-2. SCF ADD Disk Command OutputMirrorlocation 11,1,12 Cbpoollen ADD Adapter $ZZLAN.E0154 Location Type G4SA Accesslist ServerNet LAN Systems Access Slsa SubsystemInfo Tape $*,OBEYFORM SCF Assume Process $ZZLAN Additional Subsystems Controlled by SCF WAN SubsystemDisplaying Information for the WAN Subsystem $ZZWAN Subsystem Objects Controlled by SCF page 1 X25AM Subsystem Objects Controlled by SCF page 2 Example 2-3. SCF Info Process Command Output Displaying Configuration Information-SCF ExamplesExample 2-4. SCF Info SAC Command Output Info Proc $ZZKRN.# Example 2-6. SCF Info Line Command Output Example 2-5. SCF Info Process $ZZWAN Command OutputInfo Process $ZZWAN Info Line $line-name, Detail Overview of Monitoring and Recovery Functions of Monitoring Working With a Daily ChecklistMonitoring Tasks Task Operator’s name Date & time Tools for Checking the Status of System Hardware Monitoring System Components Monitored Using These Resource Tools See ServerNet Cluster 6770 Hardware Additional Monitoring Tasks Daily Tasks ChecklistGeneral Tasks Specific Tasks For More Information, See Using OSM to Monitor the System Monitoring and Resolving Problems-An ApproachUsing the OSM Service Connection Top-Down Approach OSM Management System Icons Indicate Problems Within Expanding the Tree Pane to Locate the Source of Problems Attributes Tab Using System Status Icons to Monitor Multiple Systems Alarm Summary Dialog Box Using Alarm and Problem Summaries Suppressing Problems and Alarms Using SCF to Monitor the SystemRecovery Operations for Problems Detected by OSM Monitoring Problem Incident Reports Determining Device States Example 3-1. SCF Status Tape Command SCF Object States page 1 SCF Object StatesState Substate Explanation Servicing Special SCF Object States page 2 Automating Routine System Monitoring Example 3-2. System Monitoring Command File Example 3-3. System Monitoring Output File page 1 Example 3-3. System Monitoring Output File page 2 Example 3-3. System Monitoring Output File page 3 Status LEDs and Their Functions page 1 Using the Status LEDs to Monitor the SystemLocation LED Name Color Function FC-AL I/O Status LEDs and Their Functions page 2 Status LEDs and Their Functions page 3 Related ReadingRelated Reading for Monitoring Task Tool For information, see Monitoring EMS Event Messages What Is the Event Management Service EMS?Tools for Monitoring EMS Event Messages ViewPoint on ViewPoint OSM Event ViewerWeb ViewPoint Related Reading for Monitoring EMS Event Messages Types of Processes Processes Monitoring RecoverySystem Processes Generic Processes Processes IOPs Monitoring System Processes Monitoring Processes Monitoring Generic Processes Monitoring IOPsMonitoring the Status of $ZZKRN Monitoring the Status of All Generic Processes CLCI-TACL $CLCI Stopped PID Supported for the subsystem manager processes Recovery Operations for ProcessesInitiates the operation of a generic process Communications Subsystems Communications Subsystems Monitoring and Recovery Local Area Networks LANs and Wide Area Networks WANs Object Connectivity By LAN Service Provider Subsystems Supported Monitoring the Slsa Subsystem Monitoring Communications Subsystems and Their ObjectsMonitoring the Status of an Adapter and Its Components To monitor the status of an adapter SCF Status SAC sac-name SCF Status Adapter $ZZLANSCF Status SAC $ZZLAN.G11123 SCF Status PIF pif-name Monitoring Status for a Swan Concentrator Monitoring the WAN SubsystemSCF Status PIF $ZZLAN.G11123 SCF Status LIF $ZZLAN.L11021A , Detail System displays a listing similar to Monitoring Status for a Data Communications DeviceSCF Status Adapter $ZZWAN SCF Status Device $ZZWAN.#device-name SCF Status Process $ZZWAN.#boot-process To monitor a single WANBoot process, typeMonitoring WAN Processes SCF Status Process $ZZWAN Monitoring the NonStop TCP/IP Subsystem Monitoring CLIPsMonitoring the NonStop TCP/IP Process SCF Listdev Tcpip Monitoring NonStop TCP/IP Subnets Monitoring NonStop TCP/IP RoutesMonitoring Line-Handler Process Status SCF Status Route $ZTCO Info Process $NCP, Lineset SCF Status Line $LHCS6S, Detail Tracing a Communications Line Related Reading for Communications Lines and Devices page 1 Recovery Operations for Communications SubsystemsFor Information About Refer to Related Reading for Communications Lines and Devices page 2 WAN Subsystem Configuration and Management Manual ServerNet Communications Network ServerNet Resources Monitoring and Recovery Integrity NonStop NS16000 System Integrity NonStop NS16000 ServerNet Connectivity Integrity NonStop NS14000 System with Ioam Enclosure Integrity NonStop NS14000 ServerNet Connectivity Monitoring the Status of the ServerNet Fabrics System I/O ServerNet ConnectionsIntegrity NonStop NS1000 ServerNet Connectivity Monitoring the ServerNet Fabrics Using OSM SCF Status Servernet $ZSNET Monitoring the ServerNet Fabrics Using SCF Normal ServerNet Fabric States Identifying ServerNet Fabric Problems Recovery Operations for a Down Disk Due to a Fabric Failure Recovery Operations for the ServerNet FabricsRecovery Operations for a Down Path Between Processors Recovery Operations for a Down Processor Adapters and Modules Monitoring and Recovery Fibre Channel ServerNet Adapter Fcsa Adapters and ModulesGigabit Ethernet 4-Port Adapter G4SA Port ServerNet Extender 4PSE Monitoring I/O Adapters and Modules SCF Status Adapter $ZZSTO.#FCSA*, Detail Monitoring the FCSAsState Description SCF Status Adapter $ZZLAN.G1123 Monitoring the G4SAs Service, Device, and Enabled States for the G4SA page 1 Monitoring the 4PSEs Recovery Operations for I/O Adapters and Modules ServerNet/DA Manual Related Reading for I/O Adapters and Modules Processors and Components Monitoring and Recovery Overview of the NonStop Blade Complex LSU CPU Monitoring and Maintaining Processors Monitoring Processors Automatically Using TfdsSummary, these terms describe the Nsaa processor Term Description Processor Status Display Monitoring Processor Status Using the OSM Low-Level Link OSM Representation of Processor Complex Monitoring Processor Performance Using ViewSys Identifying Processor ProblemsProcessor or System Hangs Viewsys Processor Halts OSM Alarms and Attribute ValuesHalt code = %nnnnnn Freeze code = %nnnnnn Recovery Operations for a Processor Halt Recovery Operations for Processors Reloading a Single Processor on a Running Server Halting One or More ProcessorsSelect Processor ActionsHalt Click Perform action Select FileStart Terminal Emulator Using Tacl Reload to Perform ReloadReload / run-option , run-option Noswitch Noswitch Primenoprime fabric Omitblade ABCPrimenoprime Fabric Select Reload, click Perform action Using the OSM Service Connection to Perform Reload Recovery Operations for a System Hang Enabling/Disabling Processor and System Freeze Freezing the System and Freeze-Enabled ProcessorsDumping a Processor to Disk See Using Rcvdump to Dump a Processor to Disk on Before You Begin Using Rcvdump to Dump a Processor to DiskFUP Purgedata dumpfile CPU n has been dumped to dumpfile Blade Element ReintegrationFUP Info dumpfile Submitting Information to Your Service Provider Troubleshooting and Recovery Operations for Disk DumpsBacking Up a Processor Dump to Tape Replacing Processor Memory Other Files to Submit to Your Service Provider Submitting Tapes of Configuration and Operations FilesBackup $tape, CPU0,$SYSTEM.SYS00.CONFTEXT Submitting Tapes of Processor Dumps Additional Information Required by Your Service Provider For Information About Tool See Disk Drives Monitoring and Recovery Internal Scsi Disk Drives Overview of Disk DrivesFor information about Enterprise Storage System ESS Disks M8xxx Fibre Channel Disk DrivesFor information about See Monitoring Disk Drives With OSM Monitoring Disk DrivesTask See Status Disk $*, SUB Magnetic Monitoring Disk Drives With SCF To display the status of the disk $DATA01 Status Disk $DATA09, DetailStatus $DATA01 Status $DATA02-M Status Disk $ To display the status of all disks Status $DATA01, Detail To display the detailed status of the disk $DATA01To display status of all paths for $DATA00 Status Disk $DATA00 Primary and Backup Path States for Disk Drives Monitoring the Use of Space on a Disk VolumeMonitoring the State of Disk Drives Monitoring the Size of Database Files Example Monitoring Disk Configuration and PerformanceTo check the size of the file DATA1.MEMOS FUP Info DATA1.MEMOS, Detail Possible Causes of Common Disk Drive Problems Identifying Disk Drive ProblemsProblems Possible Symptoms These SCF commands control Disk objects Recovery Operations for Disk DrivesCommon Recovery Operations for Disk Drives page 1 Command Description Customer Support Center or your service provider Common Recovery Operations for Disk Drives page 2 Reset Disk $volume Recovery Operations for a Down Disk or Down Disk PathReset Disk $WD8 Start Disk $volume FUP Alter MEMOS, Maxextents Info MEMOS, Detail Recovery Operations for a Nearly Full Database FileReport such as this one is sent to your home terminal 10-16 Overview of Tape Drives Tape Drives Monitoring and Recovery Monitoring Tape Drive Status With OSM Monitoring Tape Drives OSM Monitoring Tape Drives Connected to an Fcsa OSM Monitoring Tape Drives Connected to an IOMF2 Listing similar to this one is sent to your home terminal Monitoring Tape Drive Status With SCFSCF Status Tape $TAPE0 Listing such as this one is sent to your home terminal Mediacom Status Tapedrive Monitoring Tape Drive Status With MediacomMediacom Status Tapedrive $TAPE0 Common Tape Drive Problems Identifying Tape Drive ProblemsSymptom Problem Possible Causes Monitoring the Status of Labeled-Tape Operations Recovery Operations Using the OSM Service Connection Recovery Operations for Tape DrivesPerforming an OSM Action on a Tape Drive Performing an OSM Action on a Multiple Tape Drives SCF Command Description Recovery Operations Using SCFRelated Reading for Tapes and Tape Drives page 1 Related Reading for Tapes and Tape Drives page 2 Overview of Printers and Terminals Printers and Terminals Monitoring and Recovery Monitoring Printer Status Monitoring Printer and Collector Process StatusMonitoring Collector Process Status Spoolcom DEV $LASER Recovery Operations for a Full Collector Process Recovery Operations for Printers and Terminals 12-4 Monitoring TMF Applications Monitoring and RecoveryTMF States on Monitoring the Status of TMF Monitoring Data VolumesTmfcom ~ Status TMF Tmfcom responds with output similar to TMF StatesTMF subsystem can be in any of the states listed in Table TMF States page 1 TMF States page 2 Monitoring the Status of PathwayStatus *, Prog $*.*.PATHMON = Status Pathway Pathcom responds with output such as Pathmon States= Status Pathmon Another requester Request is waiting for an object that has been locked byRequest is waiting for a RUN Program to finish ESS Cabinets on Power Failures Preparation and Recovery NonStop NS-Series Cabinets Modular Cabinets System Response to Power FailuresNonStop S-Series I/O Enclosures External Devices Configure OSM Power Fail Support Preparing for Power FailureESS Cabinets Air Conditioning Monitor Power Supplies Power Failure RecoveryMonitor Batteries Maintain Batteries Setting System Time Procedure to Recover From a Power Failure 14-6 Alerts on Starting and Stopping the System Powering On a System Powering On the System From a No Power State Powering On the System From a Low Power StateSelect Power On System Select Hard Reset Click Perform Action 15-4 Loading the System Starting a SystemAlerts Normal System Load System Load Disks System Load to a Specific Processor System Load Paths in Order of Use System Load Paths for a Normal System LoadDisk Drive Enclosure Path Group Module Slot Data Travels Configuration File Performing a System Load Starting Other System Components Click Start system System Load Dialog Box Performing a System Load From a Specific Processor Reloading Processors Using OSM Reloading Processors Using the Reload CommandReloading Processors Reload 01 15, Prime Logical Processor Reload Parameters Stopping Application, Devices, and Processes Minimizing the Frequency of Planned OutagesAnticipating and Planning for Change Volume $DSMSCM.ZDSMSCM = SHUTDOWN2, Mode OrderlyStop DSM/SCM RUN Stopscm Stopping the System Halting All Processors Using OSMSpoolcom supervisor-name, SPOOLER, Drain Tmfcom Stop TMF System Power-Off Using OSM Powering Off a SystemSystem Power-Off Using SCF From the Processors Actions menu, select Halt Emergency Power-Off Procedure Troubleshooting and Recovery OperationsFans Are Not Turning Components Fail When Testing the Power System Does Not Appear to Be Powered OnGreen LED Is Not Lit After POSTs Finish Info Subsys $ZZKRN Recovering From a System Load Failure Recovering From a Reload Failure Getting a Corrupt System Configuration File AnalyzedBackup $TAPE, $SYSTEM.ZSYSCONF.CONFSAVE, Listall Opening Startup Event Stream and Startup Tacl Windows Exiting the OSM Low-Level Link 15-23 Related Reading for Starting and Stopping a System NonStop NS-Series Hardware Installation Guide Startup on Shutdown on Creating Startup and Shutdown Files Automating System Startup and Shutdown Managed Configuration Services MCSStartup Shutdown Processes That Represent the System ConsoleFor More Information $ZHOME Alternative Example Command Files Ciin File Modifying a Ciin File Establishing a Ciin File Conftext Ciin Entry If a Ciin File Is Not Specified or Enabled in OSMCiin File Option Results Reload /TERM $ZHOME, OUT $ZHOME Example Ciin Files Command File Syntax Writing Efficient Startup and Shutdown Command Files= Start Term = Start Term TERM1, TERM2, TERM3, TERM4, TERM5, TERM6 Use Parallel Processing Avoid Manual Intervention Tips for Startup Files How Process Persistence Affects Configuration and StartupInvestigate Product-Specific Techniques System Startup File Startup File ExamplesObey $SYSTEM.STARTUP.STRTSYS 16-13 Spooler Warm-Start File TCP/IP Stack Configuration and Startup FileTMF Warm-Start File Obey $SYSTEM.STARTUP.SPLWARM 16-15 16-16 ATP6100 Lines Startup File CP6100 Lines Startup FileLines Startup File Expand-Over-IP Line Startup File Printer Line Startup FileExpand Direct-Connect Line Startup File Shutdown File Examples Tips for Shutdown Files Obey $SYSTEM.SHUTDOWN.STOPSYS System Shutdown File ATP6100 Lines Shutdown File CP6100 Lines Shutdown FileLines Shutdown File Expand-Over-IP Line Shutdown File Printer Line Shutdown FileDirect-Connect Line Shutdown File TMF Shutdown File Spooler Shutdown FileObey $SYSTEM.SHUTDOWN.SPLDRAIN Tmfcom / in $SYSTEM.SHUTDOWN.TMFSTOP, OUT $ZHOME 16-24 Checking Air Temperature and Humidity Preventive MaintenanceMonitoring Physical Facilities Cleaning System Components Handling and Storing Cartridge Tapes Cleaning Tape Drives 17-4 HP Integrity NonStop NS-Series Operations Guide-529869-005 Page Tools and Utilities for Operations When to Use This Appendix Disk Compression Program Dcom Event Management Service Analyzer EmsaDisk Space Analysis Program Dsap Measure File Utility Program FUPNonStop NET/MASTER Nskcom and the Kernel-Managed Swap Facility Kmsf Subsystem Control Facility SCF Pathcom Web ViewPoint HP Tandem Advanced Command Language TaclViewPoint ViewSys Table C-1. Related Reading for Tools and Utilities page 1 Related ReadingTool Documentation Description NET/MASTER MS Table C-1. Related Reading for Tools and Utilities page 2 Management Manual Table C-1. Related Reading for Tools and Utilities page 3 Table C-1. Related Reading for Tools and Utilities page 4 Recovery Guide Output Table C-1. Related Reading for Tools and Utilities page 5Page Converting Numbers Table D-1. Descriptions of Number Systems Overview of Numbering SystemsNumber System Base Description Binary Value Decimal Value Binary to Decimal Octal Value Decimal Value 1375 765 Octal to Decimal Hexadecimal Decimal Hexadecimal to Decimal Hexadecimal Value Decimal Value HBA10 47632 Figure D-3. Hexadecimal to Decimal Conversion Result is Decimal to BinaryStep Division Quotient Remainder Decimal Value Binary Value 354 B101100010 Step Division Quotient Decimal to OctalDecimal Value Octal Value Decimal Hexadecimal Decimal to HexadecimalDecimal Value Hexadecimal Value Page Regulatory Compliance Statements Safety and ComplianceFCC Compliance Canadian Compliance Statements-2 Laser Compliance European Union Notice Safety Caution Waste Electrical and Electronic Equipment Weee Important Safety Information Statements-6 Numbers IndexPort ServerNet Extender 4PSE See Conftext file Dcom 10-15,B-2 FUP Nskcom B-3 Index-5 System shutdown file 16-20 TMF Lines SACs SCF B-4 commandsSpooler 16-14 Startup files Tacl 9-22,16-5,B-5 Special Characters Index-8