HP Serviceguard Metrocluster instruction

Building Disaster Recovery Serviceguard Solutions Using Continentalclusters A.08.00

HP Part Number: 698669-001

Published: February 2013

Image 1

HP Serviceguard Metrocluster manual HP Part Number Published February

Contents

HP Part Number Published February Legal Notices Contents Administering Continentalclusters Disaster recovery rehearsal in Continentalclusters Troubleshooting Continentalclusters Package attributes Sample Continentalclusters Ascii configuration file 115Legacy packages 100 114 Glossary 145 Index 150 Introduction Sample Continentalclusters Configuration Creating the Serviceguard clusters at both the sites Building the Continentalclusters configurationEasy deployment method Traditional deployment method # cmapplyconf -v -C /etc/cmcluster/cluster.config Setting up securityCsshsetup -r -k rsa -f host-list-file # passwd conclusr Creating data replication between the clusters Userhost Anyserviceguardnode Userrole MonitorUsername conclusr Using software based logical replication Using any other array based physical replication technology # mkdir /dev/vgxx # mknod /dev/vgxx/group c 64 0xnn0000 Creating and Exporting LVM Volume Groups# vgchange -a n vgname # vgexport -s -p -m mapfilename vgname Creating VxVM Disk Groups Dts/xpca/dtspkgdir # cmmakepkg -m dts/ccxpca temp.configDts/xpca/dtspkgdir pkgdirname Devicegroup # cmapplyconf -P temp.config # cmcheckconf -P temp.config# cmmakepkg -m dts/cccaeva temp.config Fence Drgroupname DtapplicationstartuppolicyDC1STORAGEWORLDWIDENAME DC1SMISLIST Dts/dts/dtspkgdir # cmmakepkg -m dts/ccsrdf temp.configRdfmode # cmhaltpkg packagename # cmrunpkg -n nodename packagename # cmmakepkg -m dts/cc3parrc pkgName.config# cmmodpkg -e packagename # cmcheckconf -P pkgName.config Configuring the monitor package# cmapplyconf -P pkgName.config # cd /etc/cmcluster # cmqueryconcl -C cmconcl.config Creating a Continentalclusters configurationCluster information # cmcheckconf -P ccmonpkg.conf Recovery groups, the following parameters are available Recovery groups Monitoring definitions has the following parameters Monitoring definitions Verify the Continentalclusters configuration Checking and applying the Continentalclusters configuration# cmcheckconcl -v -C cmconcl.config # cmapplyconcl -v -C cmconcl.config Testing Individual Packages Testing the ContinentalclustersTesting Continentalclusters Operations Starting the Continentalclusters monitor package # cmviewconcl Testing the Continentalclusters Building the Continentalclusters configuration Verifying that recovery is required Performing recovery in case of disasterPreparing the storage manually in the recovery cluster Receiving notification # cmrecovercl Recovering the entire cluster after a cluster alarm# vxdg deport disk group name Recovering a single cluster in an N-1 configuration Recovering the entire cluster after a cluster alert# cmrecovercl -f Viewing the Continentalclusters status Switching the Primary and Recovery Cluster Roles Restoring disaster recovery cluster after a disaster Run this command Check and apply the Continentalclusters configurationRestart the monitor packages on every cluster Creating a new Primary Cluster # cmapplyconf -p ccmonpkg.config Creating a new Recovery ClusterRestart the monitor package on the recovery cluster # cmrunpkg ccmonpkg Overview of Disaster Recovery rehearsal Disaster recovery rehearsal in ContinentalclustersConfiguring Continentalclusters Disaster Recovery rehearsal Configuring maintenance mode in Continentalclusters # vgcreate /dev/vgcc -f /dev/sda1 # cmgetconf -p ccmonpkg ccnew.config# lvcreate -L size vgname Mke2fs -j Lvol # mkdir directorypath Modifying Continentalclusters configuration Configuring Continentalclusters rehearsal packages# cmhaltpkg ccmonpkg # cmapplyconf -P ccnew.config # cmcheckconf -P packagename Client access IP address at recovery cluster Precautions to be taken while performing DR RehearsalCluster role switch during rehearsal Verify data replication environment Move the recovery group into maintenance modePrepare the replication environment for DR rehearsal Start rehearsal Recovering the primary cluster disaster during DR Rehearsal Cleanup of secondary mirror copyRestore replication environment for recovery Move the recovery group out of maintenance mode Run cmrrcovercl command Limitations of DR rehearsal feature Sadta Configuration in Continentalclusters Configuring the primary cluster with a single site Setting up replication Setting up the complex workload in the primary cluster Configuring the recovery cluster with a single site# vxdg -s init cvmdgname replicateddisk1 replicateddisk2 # vxdg -g cvmdgname set activation=sw # cmmakepkg -m sg/cfsall /etc/cmcluster/cfspkg1.ascii # newfs -F vxfs /dev/vx/rdsk/cvmdgname/cvmdgvolname# cmcheckconf -P cfspkg1.ascii # cmapplyconf -P cfspkg1.ascii # mkdir /dev/vgname # mknod /dev/vgname/group c 64 0xnn0000 # cmapplyconf -v -P cvmdgpkgname.conf# lvcreate -L Xxxx /dev/vgname # vgimport -s -m mapfilename vgname Configuring the complex workload at the primary cluster # cmmakepkg -m dts/sc -m dts/cccaeva cwsc.config # cmmakepkg -m dts/sc -m dts/ccxpca cwsc.config# cmmakepkg -m dts/sc -m dts/ccsrdf cwsc.config # cmmakepkg -m dts/sc -m dts/cc3parrc cwsc.config # cmapplyconf -v -P /etc/cmcluster/cwsc/cwsc.config Suspending the replication to the recovery cluster# cmviewcl -v -p pkgname # vxdg -stfC import cvmdgname Where node3 and node4 are the nodes at the recovery cluster# cmrunpkg siteBcfspkgname # vgchange -c n vgname # vgchange -c y vgname Configuring Site Safety Latch dependencies# vgchange -c n -S n vgname # vgchange -c y -S y vgname Configuring Continentalclusters Resuming the replication to the recovery cluster Checking the status of clusters, nodes, and packages Administering Continentalclusters Configured Event Status Duration Last Notification Sent Package Status State Pkgswitch Node Startup and Switching Characteristics Enabling and disabling maintenance mode# cmrecovercl -d recoverygroup1 Network Attributes Starting a recovery package forcefully Recovering a cluster when the storage array or disks fail# cmrecovercl -e recoverygroup1 # cmrecovercl -g RecoveryGroupName # cmapplyconf -C cluster.config Adding a Recovery Group to ContinentalclustersAdding or Removing a Node from a Cluster # cmforceconcl cmrunpkg -n node3 Pkg1 Modifying Continentalclusters configuration Modifying a package in a recovery groupRemoving a recovery group from the Continentalclusters Modifying a recovery group with a new rehearsal package Removing a rehearsal package from a recovery groupBehavior of Serviceguard commands in Continentalclusters Changing monitoring definitions Renaming the Continentalclusters Verifying the status of Continentalclusters daemons# ps -ef grep cmclsentryd # ps -ef grep cmclapplyd Maintaining the data replication environment Deleting the Continentalclusters configuration# what /usr/sbin/cmviewconcl Using the pairresync command Continuous Access EVA Link Suspend and Resume Modes Maintaining 3PAR Remote Copy data replication environment Maintaining EMC Srdf data replication environment Maintaining a Node Administering Continentalclusters using Sadta configurationMaintaining the Site Maintaining Site Controller Package Starting a Complex Workload Deleting the Site Controller Package# cfsdgadm deleteems pkg1dg /dts/mcsc/cwsc Start the Site Controller package by enabling it Shutting Down a Complex Workload Moving a Complex Workload to the Recovery ClusterRestarting a Failed Site Controller Package # cmviewcl -l package Troubleshooting Continentalclusters Error Messages Troubleshooting ContinentalclustersReviewing Messages and Log Files Reviewing Messages and Log Files of Monitoring Daemon Troubleshooting Continentalclusters Error Messages PackageName on cluster ClusterName Migrating to Continentalclusters A.08.00 Verify the configuration and the status of the cluster Continentalclusters Worksheets Recovery Group WorksheetData Center Worksheet Cluster Event Worksheet Recovery Checklist Replication configuration Continentalclusters Site configuration Configuring a CRS sub-cluster using CFS CRS Sub-cluster configuration using CFS RAC database configuration RAC database configuration RAC MNP Site Controller package configuration Site Controller package configuration Configuration file parameters for Continentalclusters Datasenderpackage clustername/packagename Primarypackage Clustername/Packagename Continentalclusters Command and Daemon Reference Cmcluster/cmoncl.config Cmrunpkg or cmmodpkg commandCmrecovercl -f Cmdeleteconcl -f Recovery group name Cmrecovercl -r-gCmviewconcl Autofencedatasplit Package attributes Autopsuepsus Autopsusssws Autosvolpsue Autosvolpfus Clustertype Autosvolpsus Horctakeover command, -t timeout. HorctimeoutPkg Startup Timeout Horctimeout Continuous Access link timeout value Waittime DTS PKG DIRMonpollinterval Monnotificationfrequency Monnotificationconsole MonnotificationsyslogAutoresync Start with an error Before startup of the package on destination volumeIf Waittime is 0 default value, and if the state of DR Fails to start with an error Package Attributes for Continentalcluster with EMC Srdf Querytimeout DefaultEtc/cmcluster/ packagename/FORCEFLAG Consistencygroups is set to Group in an M by N configuration ConfigurationsWhere the Metrocluster Srdf environment file is located Supported mode are sync for synchronous and async for Migrating legacy to modular packages Legacy packagesMigrating legacy monitor package # cmcheckconf -P modularccmonpkg.conf # cmmigratepkg -p packagename -o modularccmonpkg.conf# cmapplyconf -P modularccmonpkg.conf # cmmigratepkg -p packagename -s -o modularsgconf Pathtoenvfile modularsgecmcc.conf Validate the new modular package configuration file# cmcheckconf -P modularsgecmcc.conf # cmapplyconf -P modularsgecmcc.conf # cmmigratepkg -p packagename -s -o modularsg.conf Configuring the monitor package in legacy style Configuring legacy packagesValidate the new package configuration file # mkdir /etc/cmcluster/ccmonpkg # cmcheckconf -P ccmonpkg.config Use the cmcheckconf command to validate the package# mkdir /etc/cmcluster/pkgname # cd /etc/cmcluster/pkgname # cmmakepkg -p pkgname.ascii # cmmakepkg -s pkgname.cntl Create a package control script EVA Legacy packages # mkdir /etc/cmcluster/pkgname Set Pkgdir to /etc/cmcluster/pkgname Example 1 Example Etc/cmcluster/pkgname Configuring storage devices for complex workload# vxdg -s init cvmdgname replicateddisk1\ replicateddisk2 # cfsdgadm add cvmdgnamecvmdgpkgname all=sw\ node1node2 # cfsdgadm activate cvmdgname Where node1 and node2 are the nodes in the Source Disk Site# newfs -F vxfs \ Dev/vx/rdsk/cvmdgname/cvmdgvolname # mkdir /cfs # chmod 775 /cfs # mkdir /cfs/cvmdgname # cfsmount /cfs/cvmdgname EVA Continentalclusters Ascii configuration file Sample Continentalclusters Ascii configuration fileSample Continentalclusters Ascii configuration file Clustername #### #### Recoverygroupname Datasenderpackage Notification Console #### #### Clusterevent # Cluster Information Clusteralarm Seconds Notification Recoverygroupname RG2 Sample input and output files for cmswitchconcl command Recoverygroupname RG2 Primarypackage Oracle RAC instances in a Continentalclusters environment Supported Continentalclusters and RAC configuration # cfsdgadm add disk group name all=sw # vxdctl -c mode# cfsmntadm add racdgl vol4 /cfs/mntl all=rw # cfsdgadm activate diskgroupPage # cd your own directory # cp /opt/cmconcl/scripts/ccrac.config ccrac.config.mycopyCopy the file Copy the edited file to the final directory # cp ccrac.config.mycopy /etc/cmconcl/ccrac/ccrac.configPage Failover of Oracle RAC instances to the recovery site Oracle RAC instance package can be started in sequence Failback of Oracle RAC instances after a failover # cfsumount Mount Point Name Rehearsing Oracle RAC databases in Continentalclusters# cfsdgadm deactivate Disk Group Name # /opt/cmconcl/bin/ccracmgmt.ksh -i indexNumber \ start Sample Oracle RAC database with ASM in Sadta Configure a primary cluster with a single site Installing and configuring Oracle Clusterware Configure a recovery cluster with a single siteInstalling Oracle Real Application Clusters RAC software Configuring the ASM disk group in the primary cluster Halting the RAC database in the primary cluster Creating the Oracle RAC database in the primary cluster # chown -R oracleoinstall /opt/app/oracle/admin Page # chown -R oracleoinstall /opt/app/oracle/admin/hrdb Run the following command at the remote site# srvctl status # cmviewcl -v -p ASMDGpkgname Site cc2site1 # cmapplyconf -v -P sitecontrollerconfigurationfile # cmrunpkg siteController1 See also quorum server Glossary Span 100s or 1000s of kilometers Or system availability Format /dev/devtype/filename Svol Index Symbols 151