HP Serviceguard Metrocluster guide

Figure 1 Sample Continentalclusters Configuration

WAN

ccmonpkg

Site A

Node 1

Monitor

Package

	PRI_SCM_DB_PKG	Recovery	REC_SCM_DB_PKG
	PRI_SCM_DB_PKG	Group Packages	REC_SCM_DB_PKG
		Group Packages
	REC_CRM_DB_PKG	Recovery	PRI_CRM_DB_PKG
	REC_CRM_DB_PKG	Group	PRI_CRM_DB_PKG
		Packages
	cconfpkg	Continentalclusters	cconfpkg
	cconfpkg	Conﬁguration	cconfpkg
		Package
	Site A		Site B
	Node 2		Node 1
	FC Switch		FC Switch

WAN

		Data Replication Links
	WAN	Data Replication Links	WAN
	WAN		WAN
	Converters		Converters

ccmonpkg

Site B

Node 2

Monitor

Package

Site A Disk Array	Site B Disk Array
Site A Cluster (Primary)	Site B Cluster (Recovery)

For more information about Continentalclusters concepts, see Understanding and Designing Serviceguard Disaster Recovery Architectures manual available at http://www.hp.com/go/ hpux-serviceguard-docs.

9

Image 9

HP Serviceguard Metrocluster manual Sample Continentalclusters Configuration

Contents

HP Part Number Published February Legal Notices Contents Administering Continentalclusters Disaster recovery rehearsal in Continentalclusters Troubleshooting Continentalclusters Package attributes Sample Continentalclusters Ascii configuration file 115Legacy packages 100 114 Glossary 145 Index 150 Introduction Sample Continentalclusters Configuration Creating the Serviceguard clusters at both the sites Building the Continentalclusters configurationEasy deployment method Traditional deployment method # cmapplyconf -v -C /etc/cmcluster/cluster.config Setting up securityCsshsetup -r -k rsa -f host-list-file # passwd conclusr Userhost Anyserviceguardnode Userrole Monitor Creating data replication between the clustersUsername conclusr Using software based logical replication Using any other array based physical replication technology # mkdir /dev/vgxx # mknod /dev/vgxx/group c 64 0xnn0000 Creating and Exporting LVM Volume Groups# vgchange -a n vgname # vgexport -s -p -m mapfilename vgname Creating VxVM Disk Groups Dts/xpca/dtspkgdir # cmmakepkg -m dts/ccxpca temp.configDts/xpca/dtspkgdir pkgdirname Devicegroup # cmapplyconf -P temp.config # cmcheckconf -P temp.config# cmmakepkg -m dts/cccaeva temp.config Fence Drgroupname DtapplicationstartuppolicyDC1STORAGEWORLDWIDENAME DC1SMISLIST Dts/dts/dtspkgdir # cmmakepkg -m dts/ccsrdf temp.configRdfmode # cmhaltpkg packagename # cmmakepkg -m dts/cc3parrc pkgName.config # cmrunpkg -n nodename packagename# cmmodpkg -e packagename Configuring the monitor package # cmcheckconf -P pkgName.config# cmapplyconf -P pkgName.config # cd /etc/cmcluster # cmqueryconcl -C cmconcl.config Creating a Continentalclusters configurationCluster information # cmcheckconf -P ccmonpkg.conf Recovery groups, the following parameters are available Recovery groups Monitoring definitions has the following parameters Monitoring definitions Verify the Continentalclusters configuration Checking and applying the Continentalclusters configuration# cmcheckconcl -v -C cmconcl.config # cmapplyconcl -v -C cmconcl.config Testing Individual Packages Testing the ContinentalclustersTesting Continentalclusters Operations Starting the Continentalclusters monitor package # cmviewconcl Testing the Continentalclusters Building the Continentalclusters configuration Verifying that recovery is required Performing recovery in case of disasterPreparing the storage manually in the recovery cluster Receiving notification Recovering the entire cluster after a cluster alarm # cmrecovercl# vxdg deport disk group name Recovering a single cluster in an N-1 configuration Recovering the entire cluster after a cluster alert# cmrecovercl -f Viewing the Continentalclusters status Switching the Primary and Recovery Cluster Roles Restoring disaster recovery cluster after a disaster Check and apply the Continentalclusters configuration Run this commandRestart the monitor packages on every cluster Creating a new Primary Cluster # cmapplyconf -p ccmonpkg.config Creating a new Recovery ClusterRestart the monitor package on the recovery cluster # cmrunpkg ccmonpkg Overview of Disaster Recovery rehearsal Disaster recovery rehearsal in ContinentalclustersConfiguring Continentalclusters Disaster Recovery rehearsal Configuring maintenance mode in Continentalclusters # vgcreate /dev/vgcc -f /dev/sda1 # cmgetconf -p ccmonpkg ccnew.config# lvcreate -L size vgname Mke2fs -j Lvol # mkdir directorypath Modifying Continentalclusters configuration Configuring Continentalclusters rehearsal packages# cmhaltpkg ccmonpkg # cmapplyconf -P ccnew.config # cmcheckconf -P packagename Precautions to be taken while performing DR Rehearsal Client access IP address at recovery clusterCluster role switch during rehearsal Verify data replication environment Move the recovery group into maintenance modePrepare the replication environment for DR rehearsal Start rehearsal Recovering the primary cluster disaster during DR Rehearsal Cleanup of secondary mirror copyRestore replication environment for recovery Move the recovery group out of maintenance mode Run cmrrcovercl command Limitations of DR rehearsal feature Sadta Configuration in Continentalclusters Configuring the primary cluster with a single site Setting up replication Setting up the complex workload in the primary cluster Configuring the recovery cluster with a single site# vxdg -s init cvmdgname replicateddisk1 replicateddisk2 # vxdg -g cvmdgname set activation=sw # cmmakepkg -m sg/cfsall /etc/cmcluster/cfspkg1.ascii # newfs -F vxfs /dev/vx/rdsk/cvmdgname/cvmdgvolname# cmcheckconf -P cfspkg1.ascii # cmapplyconf -P cfspkg1.ascii # mkdir /dev/vgname # mknod /dev/vgname/group c 64 0xnn0000 # cmapplyconf -v -P cvmdgpkgname.conf# lvcreate -L Xxxx /dev/vgname # vgimport -s -m mapfilename vgname Configuring the complex workload at the primary cluster # cmmakepkg -m dts/sc -m dts/cccaeva cwsc.config # cmmakepkg -m dts/sc -m dts/ccxpca cwsc.config# cmmakepkg -m dts/sc -m dts/ccsrdf cwsc.config # cmmakepkg -m dts/sc -m dts/cc3parrc cwsc.config Suspending the replication to the recovery cluster # cmapplyconf -v -P /etc/cmcluster/cwsc/cwsc.config# cmviewcl -v -p pkgname Where node3 and node4 are the nodes at the recovery cluster # vxdg -stfC import cvmdgname# cmrunpkg siteBcfspkgname Configuring Site Safety Latch dependencies # vgchange -c n vgname # vgchange -c y vgname# vgchange -c n -S n vgname # vgchange -c y -S y vgname Configuring Continentalclusters Resuming the replication to the recovery cluster Checking the status of clusters, nodes, and packages Administering Continentalclusters Configured Event Status Duration Last Notification Sent Package Status State Pkgswitch Node Startup and Switching Characteristics Enabling and disabling maintenance mode# cmrecovercl -d recoverygroup1 Network Attributes Starting a recovery package forcefully Recovering a cluster when the storage array or disks fail# cmrecovercl -e recoverygroup1 # cmrecovercl -g RecoveryGroupName # cmapplyconf -C cluster.config Adding a Recovery Group to ContinentalclustersAdding or Removing a Node from a Cluster # cmforceconcl cmrunpkg -n node3 Pkg1 Modifying a package in a recovery group Modifying Continentalclusters configurationRemoving a recovery group from the Continentalclusters Modifying a recovery group with a new rehearsal package Removing a rehearsal package from a recovery groupBehavior of Serviceguard commands in Continentalclusters Changing monitoring definitions Renaming the Continentalclusters Verifying the status of Continentalclusters daemons# ps -ef grep cmclsentryd # ps -ef grep cmclapplyd Deleting the Continentalclusters configuration Maintaining the data replication environment# what /usr/sbin/cmviewconcl Using the pairresync command Continuous Access EVA Link Suspend and Resume Modes Maintaining 3PAR Remote Copy data replication environment Maintaining EMC Srdf data replication environment Administering Continentalclusters using Sadta configuration Maintaining a NodeMaintaining the Site Maintaining Site Controller Package Starting a Complex Workload Deleting the Site Controller Package# cfsdgadm deleteems pkg1dg /dts/mcsc/cwsc Start the Site Controller package by enabling it Shutting Down a Complex Workload Moving a Complex Workload to the Recovery ClusterRestarting a Failed Site Controller Package # cmviewcl -l package Troubleshooting Continentalclusters Error Messages Troubleshooting ContinentalclustersReviewing Messages and Log Files Reviewing Messages and Log Files of Monitoring Daemon Troubleshooting Continentalclusters Error Messages PackageName on cluster ClusterName Migrating to Continentalclusters A.08.00 Verify the configuration and the status of the cluster Recovery Group Worksheet Continentalclusters WorksheetsData Center Worksheet Cluster Event Worksheet Recovery Checklist Replication configuration Continentalclusters Site configuration Configuring a CRS sub-cluster using CFS CRS Sub-cluster configuration using CFS RAC database configuration RAC database configuration RAC MNP Site Controller package configuration Site Controller package configuration Configuration file parameters for Continentalclusters Datasenderpackage clustername/packagename Primarypackage Clustername/Packagename Continentalclusters Command and Daemon Reference Cmcluster/cmoncl.config Cmrunpkg or cmmodpkg commandCmrecovercl -f Cmdeleteconcl -f Cmrecovercl -r-g Recovery group nameCmviewconcl Autofencedatasplit Package attributes Autopsuepsus Autopsusssws Autosvolpsue Autosvolpfus Clustertype Autosvolpsus Horctakeover command, -t timeout. HorctimeoutPkg Startup Timeout Horctimeout Continuous Access link timeout value Waittime DTS PKG DIRMonpollinterval Monnotificationfrequency Monnotificationsyslog MonnotificationconsoleAutoresync Start with an error Before startup of the package on destination volumeIf Waittime is 0 default value, and if the state of DR Fails to start with an error Querytimeout Default Package Attributes for Continentalcluster with EMC SrdfEtc/cmcluster/ packagename/FORCEFLAG Consistencygroups is set to Group in an M by N configuration ConfigurationsWhere the Metrocluster Srdf environment file is located Supported mode are sync for synchronous and async for Legacy packages Migrating legacy to modular packagesMigrating legacy monitor package # cmcheckconf -P modularccmonpkg.conf # cmmigratepkg -p packagename -o modularccmonpkg.conf# cmapplyconf -P modularccmonpkg.conf # cmmigratepkg -p packagename -s -o modularsgconf Pathtoenvfile modularsgecmcc.conf Validate the new modular package configuration file# cmcheckconf -P modularsgecmcc.conf # cmapplyconf -P modularsgecmcc.conf # cmmigratepkg -p packagename -s -o modularsg.conf Configuring the monitor package in legacy style Configuring legacy packagesValidate the new package configuration file # mkdir /etc/cmcluster/ccmonpkg # cmcheckconf -P ccmonpkg.config Use the cmcheckconf command to validate the package# mkdir /etc/cmcluster/pkgname # cd /etc/cmcluster/pkgname # cmmakepkg -p pkgname.ascii # cmmakepkg -s pkgname.cntl Create a package control script EVA Legacy packages # mkdir /etc/cmcluster/pkgname Set Pkgdir to /etc/cmcluster/pkgname Example 1 Example Etc/cmcluster/pkgname Configuring storage devices for complex workload# vxdg -s init cvmdgname replicateddisk1\ replicateddisk2 # cfsdgadm add cvmdgnamecvmdgpkgname all=sw\ node1node2 # cfsdgadm activate cvmdgname Where node1 and node2 are the nodes in the Source Disk Site# newfs -F vxfs \ Dev/vx/rdsk/cvmdgname/cvmdgvolname # mkdir /cfs # chmod 775 /cfs # mkdir /cfs/cvmdgname # cfsmount /cfs/cvmdgname EVA Sample Continentalclusters Ascii configuration file Continentalclusters Ascii configuration fileSample Continentalclusters Ascii configuration file Clustername #### #### Recoverygroupname Datasenderpackage Notification Console #### #### Clusterevent # Cluster Information Clusteralarm Seconds Notification Recoverygroupname RG2 Sample input and output files for cmswitchconcl command Recoverygroupname RG2 Primarypackage Oracle RAC instances in a Continentalclusters environment Supported Continentalclusters and RAC configuration # cfsdgadm add disk group name all=sw # vxdctl -c mode# cfsmntadm add racdgl vol4 /cfs/mntl all=rw # cfsdgadm activate diskgroupPage # cp /opt/cmconcl/scripts/ccrac.config ccrac.config.mycopy # cd your own directoryCopy the file Copy the edited file to the final directory # cp ccrac.config.mycopy /etc/cmconcl/ccrac/ccrac.configPage Failover of Oracle RAC instances to the recovery site Oracle RAC instance package can be started in sequence Failback of Oracle RAC instances after a failover # cfsumount Mount Point Name Rehearsing Oracle RAC databases in Continentalclusters# cfsdgadm deactivate Disk Group Name # /opt/cmconcl/bin/ccracmgmt.ksh -i indexNumber \ start Sample Oracle RAC database with ASM in Sadta Configure a primary cluster with a single site Installing and configuring Oracle Clusterware Configure a recovery cluster with a single siteInstalling Oracle Real Application Clusters RAC software Configuring the ASM disk group in the primary cluster Halting the RAC database in the primary cluster Creating the Oracle RAC database in the primary cluster # chown -R oracleoinstall /opt/app/oracle/admin Page # chown -R oracleoinstall /opt/app/oracle/admin/hrdb Run the following command at the remote site# srvctl status # cmviewcl -v -p ASMDGpkgname Site cc2site1 # cmapplyconf -v -P sitecontrollerconfigurationfile # cmrunpkg siteController1 See also quorum server Glossary Span 100s or 1000s of kilometers Or system availability Format /dev/devtype/filename Svol Index Symbols 151