Part 3. Implementing recovery and restart | IBM SC34-7012-01

Part 3. Implementing recovery and restart

This part describes the way you implement recovery and restart for CICS regions.

© Copyright IBM Corp. 1982, 2010

99

Image 111

IBM SC34-7012-01 manual Part 3. Implementing recovery and restart

Contents

Recovery and Restart Guide Page Recovery and Restart Guide Copyright IBM Corporation 1982 Contents Restart Part 3. Implementing recovery Programming for recovery 141 Part 4. Appendixes 241 Disaster recovery 223 Who should read this book How to use this bookPreface What this book is about Viii Cics TS for z/OS 4.1 Recovery and Restart Guide Copyright IBM Corp Cics TS for z/OS 4.1 Recovery and Restart Guide Part 1. Cics recovery and restart concepts Cics TS for z/OS 4.1 Recovery and Restart Guide Recovery and restart facilities Maintaining the integrity of dataLogging changes Role of Cics Minimizing the effect of failures Cics backward recovery backout Recoverable resources Emergency restart backout Dynamic transaction backout Forward recovery of Cics data sets Cics forward recovery Failures that require Cics recovery processing Cics recovery processing following a communication failureForward recovery for non-VSAM resources XCF/MRO partner failures Cics recovery processing following a system failure Cics recovery processing following a transaction failure Recovery and restart facilities Cics TS for z/OS 4.1 Recovery and Restart Guide Resource recovery in Cics Units of workShunted units of work Active and retained states for locks Locks Synchronization points Units of work and syncpoints Examples of synchronization points Backout of units of work Cics recovery manager DB2 Coordinating updates to local resources Managing indoubt units of work Coordinating updates in distributed units of work Resynchronization after system or connection failure Cics system logInformation recorded on the system log Forward recovery logs System activity keypointsUser journals and automatic journaling Resource recovery in Cics Cics TS for z/OS 4.1 Recovery and Restart Guide Shutdown and restart recovery Normal shutdown processingFirst quiesce stage Third quiesce stage Second quiesce stage Shunted units of work at shutdown Warm keypoints Flushing journal buffers Immediate shutdown processing Perform Shutdown ImmediatePerform Immediate not recommended Dfhcesd Shutdown requested by the operating system Shutdown assist transaction Uncontrolled termination Global catalog Cataloging Cics resources Local catalog Shutdown initiated by Cics log manager Effect of problems with the system log DFHRM0403 and DFHRM0404 How the state of the Cics region is reconstructed About this task Overriding the type of start indicatorWarm restart Emergency restart Recovery of data during an emergency restart Cold startAn initial start of Cics Dynamic RLS restart MNPS, multinode persistent sessions Recovery with Vtam persistent sessionsRunning with persistent sessions support SNPS, single-node persistent sessions Situations in which sessions are not reestablished Situations in which Vtam does not retain sessions Running without persistent sessions support Regions that do have persistent sessions support Cics TS for z/OS 4.1 Recovery and Restart Guide Part 2. Recovery and restart processes Cics TS for z/OS 4.1 Recovery and Restart Guide Starting Cics with the START=COLD parameter Cics cold start Vsam Files Data tables Temporary storageTransient data Temporary storage data sharing server Library resources Resource definitions dynamically installedTransactions Journal names and journal models Committing and cataloging resources installed from the CSD Monitoring and statisticsTerminal control resources Starting Cics with the START=INITIAL parameter Installable set installDistributed transaction resources Dump table Cics cold start Cics TS for z/OS 4.1 Recovery and Restart Guide Rebuilding the Cics state after a normal shutdown Cics warm restart Reconnecting to Smsvsam for RLS access Data set name blocksRecreating non-RLS retained locks Trigger levels for Terminal and System only TDINTRA=NOEMPTY the default No autoinstall for programs Start requests Autoinstall for programs Tcam and sequential Bsam devices CSD-defined resource definitions Urimap definitions and virtual hosts Cics TS for z/OS 4.1 Recovery and Restart Guide Driving backout processing for in-flight units of work Recovering after a Cics failureRecovering information from the system log Cics emergency restart Effect of delayed recovery on Pltpi processing Other backout processingRebuilding the Cics state after an abnormal termination Reconnecting to Smsvsam for RLS access RLS restart processing and orphan locksRecreating non-RLS retained locks Start requests Distributed transaction resources Cics TS for z/OS 4.1 Recovery and Restart Guide Automatic restart management Cics ARM processingRestrictions Before you begin Registering with ARMWaiting for predecessor subsystems De-registering from ARM Failing to register ARM couple data setsCics restart JCL and parameters Connecting to Vtam Workload policiesCics Start options Server ARM processing Covr transactionMessages associated with automatic restart Automatic restart of Cics data-sharing servers ARMELEMENTNAME=elementname Server commands for ARM supportWaiting on events during initialization Server initialization parameters for ARM support Commit-failed Unit of work recovery and abend processingUnit of work recovery In-flight-failed Transaction backout Backout-failedIndoubt-failed Files Bdam files and Vsam Esds filesCics data tables Intrapartition transient data Auxiliary temporary storageStart requests Restart of started transactions Start with recoverable data ProtectStart with nonrecoverable data no Protect Start with nonrecoverable data Protect Basic mapping support BMS messages Exec Cics Cancel requests Backout-failed recovery Disposition of data sets after backout failures ErrorTransient data Retrying backout-failed units of work Open error Logical delete not performedSmsvsam server failure Coupling facility cache structure failure DFSMSdss non-BWO backup in progressDuplicate key error Smsvsam server recycle during backout Commit-failed recovery Lock structure full errorNone of the above Indoubt failure recovery Investigating an indoubt failure Cemt Inquire UOW details of UOW AC0CD65E5D990800 Cemt Inquire UOWENQ-details of a lock associated with a UOW Cache failure support Recovery from failures associated with the coupling facility Lost locks recovery Rebuilding the lock structureNotifying Cics of Smsvsam restart Performing lost locks recovery for failed units of work MVS system recovery and sysplex recovery Connection failure to a coupling facility cache structureConnection failure to a coupling facility lock structure Exit code Transaction abend processing Transaction restart Abnormal termination of a task Processing operating system abends and program checks Actions taken at transaction failure Unit of work recovery and abend processing Cics TS for z/OS 4.1 Recovery and Restart Guide Terminal error program Dfhtep Communication error processingTerminal error processing Node error program Dfhznep Intersystem communication failures Part 3. Implementing recovery and restart Cics TS for z/OS 4.1 Recovery and Restart Guide Planning aspects of recovery Questions relating to recovery requirementsApplication design considerations Validate the recovery requirements statement Designing the end user’s restart procedure End user’s standby proceduresCommunications between application and user Security System definitions for recovery-related functionsSystem recovery table SRT Resource definitions for recovery Program list table PLT Documentation and test plansTransient data queues Temporary storage table Cics TS for z/OS 4.1 Recovery and Restart Guide Defining system and general log streams General log streams Defining log streams to MVSDefining system log streams System log streams Specifying a Journalmodel resource definition Without a Journalmodel definitionWith a Journalmodel definition Model log streams for Cics system logs Recovery considerations Sharing system logger structures between 2 MVS images Activity keypointing Varying the model log stream nameAbout this task System log scan during restart Keeping system log data to a minimum Moving units of work to the secondary log Log-tail deletion Writing user-recovery data Avoiding retention periods on the system logRetrieving user records from the system log Defining forward recovery log streams About this task ProcedureLong-running transactions What to do next Model log streams for Cics general logs Defining the log of logs Merging data on shared general log streams Reading log streams offline Log of logs failure Adjusting local time Effect of daylight saving time changesTime stamping log and journal records Offline utility program, Dfhjup Cics TS for z/OS 4.1 Recovery and Restart Guide DTIMOUTNO1-6800 Defining recoverability for CICS-managed resourcesRecovery for transactions Defining transaction recovery attributes WAITTIME00,00,00dd,hh,mm Indoubt options for distributed transactions Sharing data sets with batch jobs Recovery for filesFile-owning regions and RLS access Vsam files Backward recovery Basic direct access method BdamDefining files as recoverable resources Forward recovery Recoveryall Vsam files accessed in non-RLS mode Inquiring on recovery attributes Vsam files accessed in RLS mode Overriding open failures at the Xfcnrec global user exit File recovery attribute consistency checking non-RLSBdam files CSD data set Cics responses to file open requests Backward recovery Implementing forward recovery with user-written utilitiesImplementing forward recovery with Cics Vsam Recovery Recovery for intrapartition transient data Physical recovery Logical recovery No recovery Forward recovery Input extrapartition data sets Recovery for extrapartition transient data Using post-initialization Pltpi programs Recovery for temporary storageOutput extrapartition data sets Configuring Cics to support persistent messages Recovery for Web services Results What to do next Defining local queues in a service provider Persistent message processing Error processing Defining recoverability for CICS-managed resources Cics TS for z/OS 4.1 Recovery and Restart Guide Programming for recovery Designing applications for recoverySplitting the application into transactions Cics TS for z/OS 4.1 Recovery and Restart Guide SAA-compatible applications Program designDividing transactions into units of work Processing dialogs with users Conversational processingPseudoconversational processing Cics recoverable resources Mechanisms for passing data between transactionsMain storage areas Designing to avoid transaction deadlocks Implications of interval control Start requests Using transient data queues Implications of automatic task initiation TD trigger levelTerminal paging through BMS Transaction failures Managing transaction and system failures Handle Abend commands Exec Cics Syncpoint Rollback commandDynamic transaction backout Transaction restart after DTB Use of the program error program DfhpepSystem failures Handling abends and program level abend exits Processing the Ioerr condition Command Information provided Start Transid commands PL/I programs and error handlingLocking enqueuing on resources in application programs Implicit locking for files Nonrecoverable files Recoverable files Cics TS for z/OS 4.1 Recovery and Restart Guide Implicit enqueuing on logically recoverable TD destinations Implicit enqueuing on recoverable temporary storage queuesWriteq TD Readq TD Deleteq TD Sequential methods HSAM, HISAM, Shisam Implicit enqueuing on DL/I databases with DbctlExplicit enqueuing by the application programmer Direct methods HDAM, Hidam Deadlock Possibility of transaction deadlock User exits for transaction backout Where you can add your own code Xrcinit exit Xrcinpt exitXfcbfail global user exit Coding transaction backout exits Xfcldel global user exitXfcbover global user exit Xfcbout global user exit CICS-supplied PEP Using a program error program PEP Your own PEP Omitting the PEP Cics TS for z/OS 4.1 Recovery and Restart Guide Quiescing RLS data sets Resolving retained locks on recoverable resources Illustration of the quiesce flow across two Cics regions RLS quiesce and unquiesce functions MVS1 MVS2 Cics AOR1 Cics AOR2 Other quiesce interface functions Non-BWO data set backup start Forward recovery complete Non-BWO data set backup endBWO backup start BWO backup end Quiesce coupling facility cache available Switching from RLS to non-RLS access modeLost locks recovery complete Exception for read-only operations What can prevent a switch to non-RLS access mode? Investigating which retained locks are held and why Inquire Dsname Resolving retained locks and preserving data integrity Shcds List subcommands Choosing data availability over data integrity Batch-enabling sample programs Cemt command examples Uowdsnfail Overriding retained locks Permitnonrlsupdate subcommandDenynonrlsupdate subcommand Special case lost locks Post-batch processing Coupling facility data table retained locks Moving recoverable data sets that have retained locks Procedure for moving a data set with retained locksUsing the Repro method Shcds Frsetrr Using the Export and Import functions Rebuilding alternate indexes Forward recovery of data sets accessed in RLS mode Forward recovery procedures Recovery of data set with volume still available Alter the new data set name Recovery of data set with loss of volumeIssue the Frbind subcommand Issue the Frresetrr subcommand Volume recovery procedure using Cfvol Quiesce Forward recovery procedures Example of recovery using data set backup Shcds LISTSUBSYSDS’ADSWA01D’ Vary SMS,SMSVSAM,FORCEDELETELOCKSTRUCTURE Inquire Uowdsnfail Status Results Example of recovery using volume backup Above command produced the following message Catalog recovery Forward recovery of data sets accessed in non-RLS mode Procedure for failed RLS mode forward recovery operationClose all files Forward recovery procedures Uowactioncommit Backout Force Procedure for failed non-RLS mode forward recovery operation Cics TS for z/OS 4.1 Recovery and Restart Guide Backup-while-open BWO BWO and backupsBWO and concurrent copy Component name Full DFSMS/MVS name Previous product Name BWO requirements Hardware requirements Which data sets are eligible for BWOVsam control interval or control area split Specifying BWO using access method services How you request BWOResults Specifying BWO on Cics file resource definitions Systems administration Removing BWO attributesBatch jobs BWO processing First file opened in non-RLS mode against a cluster File openingBack-level data sets Subsequent files opened when use count is zero Subsequent files opened when use count is not zero Restriction for Vsam upgrade set File closing non-RLS mode Immediate or uncontrolled shutdown Data set backup and restoreShutdown and restart Controlled shutdown Invalid state changes for BWO attributes Vsam access method services Non-SMS managed storage Forward recovery loggingData set restore Data sets Recovery point non-RLS mode Recovering Vsam spheres with AIXs ASM XOPTSCICS,NOEPILOG,SP An assembler program that calls Dfsms callable services DS 8C MVC MVC SUCMSG18,SUCTXT1 SUCMSG21,SUCTXT2 Cics TS for z/OS 4.1 Recovery and Restart Guide Why have a disaster recovery plan? Disaster recovery Disaster recovery testing Six tiers of solutions for off-site recovery Tier 0 no off-site dataTier 1 physical removal Disaster recovery tier 1 physical removal Tier 2 physical removal with hot site Tier 3 electronic vaultingTier Summarizes the tier 3 solution Tier 0-3 solutions Tier 0 Tier 4 active secondary site Disaster recovery tier 4 active secondary site Tier 6 minimal to zero data loss Tier 5 two-site, two-phase commit Summarizes the tier 6 solution Tier 4 Tier 4-6 solutions Peer-to-peer remote copy Pprc and extended remote copy XRC Disaster recovery and high availability Use Pprc for high value transactions Pprc or XRC? Other benefits of Pprc and XRC Remote Recovery Data Facility Disaster recovery personnel considerations Choosing between Rrdf and 3990-6 solutions Disaster recovery facilities MVS system logger recovery supportReturning to your primary site Cics VR shadowing Cics Vsam Recovery Qsam copyRemote Recovery Data Facility support Remote site recovery for RLS-mode data sets Final summary 241 Part 4. Appendixes Cics TS for z/OS 4.1 Recovery and Restart Guide 243 Trademarks Cics books for Cics Transaction Server for z/OS Bibliography Administration and Management CICSPlex SM books for Cics Transaction Server for z/OSOther Cics publications Accessibility Cics TS for z/OS 4.1 Recovery and Restart Guide Index Airdelay DL/I 251 Tpend Readers’ Comments Wed Like to Hear from You Readers’ Comments Wed Like to Hear from You Page SC34-7012-01