Example Adding a Heartbeat LAN | HP Serviceguard guide

Example: Adding a Heartbeat LAN

Suppose that a subnet 15.13.170.0 is shared by nodes ftsys9 and ftsys10 in a two-node cluster cluster1, and you want to add it to the cluster configuration as a heartbeat subnet. Proceed as follows.

1.Run cmquerycl to get a cluster configuration template file that includes networking information for interfaces that are available to be added to the cluster configuration:

cmquerycl -c cluster1 -C clconfig.ascii

NOTE: As of Serviceguard A.11.18, cmquerycl -cproduces output that includes commented-out entries for interfaces that are not currently part of the cluster configuration, but are available.

The networking portion of the resulting clconfig.ascii file looks something like this:

NODE_NAMEftsys9

NETWORK_INTERFACE lan1

HEARTBEAT_IP 192.3.17.18

#NETWORK_INTERFACE lan0

#STATIONARY_IP 15.13.170.18

NETWORK_INTERFACE lan3

#Possible standby Network Interfaces for lan1, lan0: lan2.

NODE_NAMEftsys10

NETWORK_INTERFACE lan1

HEARTBEAT_IP 192.3.17.19

#NETWORK_INTERFACE	lan0
# STATIONARY_IP	15.13.170.19
NETWORK_INTERFACE	lan3

# Possible standby Network Interfaces for lan0, lan1: lan2

2.Edit the file to uncomment the entries for the subnet that is being added lan0 in this example), and change STATIONARY_IP to HEARTBEAT_IP:

NODE_NAMEftsys9

NETWORK_INTERFACE lan1

HEARTBEAT_IP 192.3.17.18

NETWORK_INTERFACE lan0

HEARTBEAT_IP 15.13.170.18

NETWORK_INTERFACE lan3

#Possible standby Network Interfaces for lan1, lan0: lan2.

NODE_NAMEftsys10

NETWORK_INTERFACE lan1

HEARTBEAT_IP 192.3.17.19

NETWORK_INTERFACE lan0

HEARTBEAT_IP 15.13.170.19

NETWORK_INTERFACE lan3

# Possible standby Network Interfaces for lan0, lan1: lan2

3.Verify the new configuration: cmcheckconf -C clconfig.ascii

4.Apply the changes to the configuration and distribute the new binary configuration file to all cluster nodes:

cmapplyconf -C clconfig.ascii

If you were configuring the subnet for data instead, and wanted to add it to a package configuration, you would now need to:

1.Halt the package

2.Add the new networking information to the package configuration file

Reconfiguring a Cluster 299

Image 299

HP Serviceguard manual Example Adding a Heartbeat LAN, Cmquerycl -c cluster1 -C clconfig.ascii

Contents

Managing Serviceguard Twentieth Edition Legal Notices Contents Contents Planning and Documenting an HA Cluster Contents Building an HA Cluster Configuration 158 Contents Configuring Packages and Their Services 227 Contents Cluster and Package Maintenance 261 279 Troubleshooting Your Cluster 321 Contents 349 Integrating HA Applications with Serviceguard 354Software Upgrades 357 IPv6 Network Support 378 Blank Planning Worksheets 369Migrating from LVM to VxVM Data Storage 374 377 Index 398 Using Serviceguard Manager 385396 Publishing History Publishing History Preface Related Publications Securing Serviceguard and other Serviceguard white papers What is Serviceguard? Serviceguard at a Glance Shows what happens in a failover situation Failover Typical Cluster After Failover About Veritas CFS and CVM from Symantec Using SAM Using Serviceguard Manager Usr/sbin/sam -w What are the Distributed Systems Administration Utilities?Roadmap for Configuring Clusters and Packages Tasks in Configuring a Serviceguard Cluster Redundancy of Cluster Components Understanding Serviceguard Hardware Configurations Rules and Restrictions Redundant Network Components Redundant LANs Redundant Ethernet Configuration Restrictions Cross-Subnet ConfigurationsConfiguration Tasks For legacy packages, see Configuring Cross-Subnet Failover For More Information Supported Disk Interfaces Redundant Disk StorageReplacing Failed Network Cards About Multipathing Disk Arrays using RAID Levels and Multiple Data PathsData Protection Disk Mirroring Replacing Failed I/O Cards Monitoring LVM Disks Through Event Monitoring ServiceMonitoring VxVM and CVM Disks Replacing Failed Disk Mechanisms Mirrored Disks Connected for High Availability Sample Scsi Disk Configurations Cluster with High Availability Disk Array Sample Fibre Channel Disk Configuration Larger Clusters Redundant Power Supplies Point to Point Connections to Storage Devices Active/Standby Model Eight-Node Cluster with XP or EMC Disk Array Serviceguard Daemons Understanding Serviceguard Software ComponentsServiceguard Architecture Cluster Daemon cmcld Configuration Daemon cmclconfd Cluster Object Manager Daemon cmomd Cluster Logical Volume Manager Daemon cmlvmdFile Management Daemon cmfileassistd Syslog Log Daemon cmlogd Lock LUN Daemon cmdisklockd Service Assistant Daemon cmservicedQuorum Server Daemon qs Network Manager Daemon cmnetd CFS Components Configuring the ClusterHow the Cluster Manager Works Proxy Daemon cmproxyd Manual Startup of Entire Cluster Heartbeat Messages Cluster Lock Automatic Cluster StartupDynamic Cluster Re-formation Cluster Quorum to Prevent Split-Brain Syndrome Lock Requirements Use of a Lock LUN or LVM Lock Disk as the Cluster Lock Dual Lock Disk Use of the Quorum Server as the Cluster LockSingle Lock Disk or LUN Quorum Server Operation No Cluster Lock Failover Packages How the Package Manager WorksPackage Types Non-failover Packages Failover Packages’ Switching Behavior Deciding When and Where to Run and Halt Failover Packages Before Package Switching Automatic Rotating Standby Package Configuration Data Rotating Standby Configuration before Failover Failback Policy Configurednode Policy Packages after Failover Automatic Failback Configuration After Failover Using the Generic Resources Monitoring Service Using Older Package Configuration Files Understanding Serviceguard Software Components See also Using EMS to Monitor Volume Groups Using the Event Monitoring ServiceUsing the EMS HA Monitors See also Using Generic Resources to Monitor Volume Groups What Makes a Package Run? How Packages Run Legacy Package Time Line Showing Important Events Before the Control Script Starts Package Time Line Legacy Package During Run Script Execution Normal and Abnormal Exits from the Run Script Service Startup with cmrunservWhile Services are Running During Halt Script Execution When a Package is Halted with a Command Legacy Package Time Line for Halt Script Execution Normal and Abnormal Exits from the Halt Script Package Control Script Error and Exit ConditionsError Conditions and Package Movement for Failover Packages Stationary and Relocatable IP Addresses How the Network Manager Works Types of IP Addresses Adding and Deleting Relocatable IP Addresses Load Sharing Monitoring LAN Interfaces and Detecting Failure Link Level Local Switching Cluster Before Local Network Switching Cmmodnet -e interface Where interface is the primary interface Monitoring LAN Interfaces and Detecting Failure IP Level Remote Switching How the IP Monitor Works Reasons To Use IP Monitoring Failure and Recovery Detection Times See also Reporting Link-Level and IP-Level Failures Example 1 If Local Switching is ConfiguredReporting Link-Level and IP-Level Failures Constraints and Limitations Cmmodnet -e lan2 Example 2 If There Is No Local SwitchingAutomatic Port Aggregation Support for HP-UX Vlan Vlan ConfigurationsWhat is VLAN? Additional Heartbeat Requirements Volume Managers for Data StorageConfiguration Restrictions Types of Redundant Storage White papers About Device File Names Device Special FilesExamples of Mirrored Storage Physical Disks Within Shared Storage Units Examples of Storage on Disk Arrays Multiple Devices Configured in Volume Groups Multiple Paths to LUNs Propagation of Disk Groups in VxVM Types of Volume ManagerHP-UX Logical Volume Manager LVM Veritas Volume Manager VxVM For heartbeat requirements, see Redundant Heartbeat Subnets Veritas Cluster Volume Manager CVMCluster Startup Time with CVM Propagation of Disk Groups with CVM Redundant Heartbeat Subnets Comparison of Volume Managers Pros and Cons of Volume Managers with Serviceguard Example System Reset When a Node FailsWhat Happens when a Node Times Out Responses to Failures Responses to Hardware Failures Responses to Package and Generic Resources Failures Responses to Package and Service Failures Network Communication Failure Service Restarts Planning for Expansion Serviceguard Memory RequirementsPlanning and Documenting an HA Cluster General Planning Hardware Planning Sample Cluster Configuration Nnn.nnn.nnn.nnn SPU InformationNetwork Information LAN Information Scsi Addressing in Cluster Configuration Under Cluster Configuration Parameters Diskinfo Disk I/O Information Hardware Configuration Worksheet Power Supply Planning Cluster Lock Planning Power Supply Configuration Worksheet Quorum Server Worksheet Using a Quorum ServerCluster Lock Disk and Re-formation Time LVM Planning Using Generic Resources to Monitor Volume Groups Using EMS to Monitor Volume Groups LVM Worksheet For more information, see Using the EMS HA MonitorsCVM and VxVM Planning CVM and VxVM Worksheet Cluster Configuration Planning Points To Note Where cDSFs ResideAbout Cluster-wide Device Special Files cDSFs Limitations of cDSFs LVM Commands and cDSFsAbout Easy Deployment Limitations of Easy Deployment Heartbeat Subnet and Cluster Re-formation TimeAdvantages of Easy Deployment Localhost ipv6-localhost ipv6-loopback What Is IPv4-only Mode?What Is IPv6-Only Mode? Rules and Restrictions for IPv6-Only Mode IPV6 or ANY What Is Mixed Mode?Recommendations for IPv6-Only Mode Name of the cluster as it will appear in the output Cluster Configuration ParametersRules and Restrictions for Mixed Mode Cluster configuration file Planning and Documenting an HA Cluster 99 and Specifying a Quorum Server Happens when You Change the Quorum ConfigurationGo/hpux-serviceguard-docs under HP Serviceguard IPv4-Only,IPv6-Only, and Mixed Mode page 106 for See also About Hostname Address Families IPv4-Only Hpux-serviceguard-docs under HP ServiceguardIPv6-Only, and Mixed Mode page 106 for important Sitepreferred or Configuration file see Configuring PackagesTheir Services page 227 and these in turn must Cluster Is Running Configuration Planning page 125 must be specified Cluster Configuration Planning To that LAN, to risk timeout without being serviced You cannot change the heartbeat configuration whileCVM/CFS on HP Serviceguard A.11.20 April Protocols and services. RPC assumes that each network Lock LUN page 189 for more information Also What Happens when You Change the QuorumConfiguration Online page 49 for important information See IPv6 Address Types Failbackpolicy When You Change the Quorum Configuration OnlineCluster is running, see Updating the Cluster Lock Disk See About Package Weights page 144 for more Planning and Documenting an HA Cluster IP-Level Failures See also What Happens when a Node Times Out88, Cluster Daemon cmcld page 41, 69, Monitoring LAN Interfaces and Detecting How Serviceguard will handle the recovery of the primary Default isConfiguration file specifies one of two ways to decide When a network interface card has failed See Monitoring LAN Interfaces and Detecting Failure IP Planning and Documenting an HA Cluster Access Control Policies also known as Role Based Access Package Configuration PlanningCluster Configuration Next Step Logical Volume and File System Planning CVM 4.1 and later without CFS CVM 4.1 and later with CFS Using the Volume Monitor About the Volume Monitor Or --log-level Or --helpOr --version Or --log-file Usr/sbin/cmvolmond -t 10 /dev/vg00/lvol1 VolumepathPlanning for NFS-mounted File Systems Usr/sbin/cmvolmond /dev/vg01/lvol1 /dev/vg01/lvol2 Package Configuration Planning Package Failover Behavior Choosing Switching and Failover Behavior Cmmakepkg -i $SGCONF/pkg1/pkg1.conf -m sg/genericresource Parameters for Configuring Generic ResourcesConfiguring a Generic Resource Extended generic resource Cmapplyconf -P $SGCONF/pkg1/pkg1.conf Cmcheckconf -v -P $SGCONF/pkg1/pkg1.conf Cmsetresource -r sfmdisk -s up Cmviewcl -v -f line -p pkg1 grep genericresourceCmrunpkg pkg1 Cmgetresource -r sfmdisk Online Reconfiguration of Generic Resources Parameters for Configuring EMS Resources Assume that we want to make pkg1 depend on pkg2 About Package DependenciesSimple Dependencies Rules for Simple Dependencies Planning and Documenting an HA Cluster Dragging Rules for Simple Dependencies Planning and Documenting an HA Cluster Extended Dependencies Rules for Exclusionary Dependencies See Rules for differentnode and anynode Dependencies Rules for differentnode and anynode Dependencies What Happens when a Package Fails Cmmakepkg 1m manpage About Package WeightsPackage Weights and Node Capacities Configuring Weights and Capacities For pkg2 Weightname packagelimit weightvalueSimple Method Nodename node1 Capacityname packagelimit Defining Capacities Comprehensive MethodPoints to Keep in Mind Nodename node2 Clustername cluster23 Nodename node1 Defining Default Weights Defining Weights Weightname B Weightvalue Weightname a WeightvalueWeightname B Weightvalue Weightname a Cmquerycl 1m manpage Rules and Guidelines About External Scripts Pevmonitoringinterval Using Serviceguard Commands in an External Script Cmviewcl -v -f line displays a lasthaltfailed flag Determining Why a Package Has Shut DownAbout Cross-Subnet Failover Lasthaltfailed Implications for Application Deployment Configuring a Package to Fail Over across Subnets Example Configuring ipsubnetnode Configuring a Package Next StepsConfiguring nodename Configuring monitoredsubnetaccess Planning for Changes in Cluster Size Preparing Your Systems Configuring the Cluster Building an HA Cluster ConfigurationInstalling and Updating Serviceguard Where Serviceguard Files Are Kept Etc/cmcluster.conf Creating Cluster-wide Device Special Files cDSFsBefore You Start Creating cDSFs for a Group of Nodes Cmpreparecl -n node1 -n node2 -n node3 -n node4 Csshsetup -r node2Csshsetup -r -f /etc/cmcluster/sshhosts Cmpreparecl -n nodename -n nodename Removing a Node from a cDSF Group Using Easy DeploymentDisplaying the cDSF Configuration Adding a Node to a cDSF Group Cmquerycl -N $SGCONF/mynetwork Using Easy Deployment Commands to Configure the ClusterFor example Preparing Your Systems Building an HA Cluster Configuration PVG bus1 /dev/cdisk/disk14 /dev/cdisk/disk15 Format for entries in cmclnodelist is as follows Configuring Root-Level AccessAllowing Root Access to an Unconfigured Node About identd Ensuring that the Root User on Another Node Is Recognized Any of the aliases. Examples Configuring Name ResolutionOfficial hostname, as defined by hosts 4, for example Safeguarding against Loss of Name Resolution Services For NIS, enter two lines Ensuring Consistency of Kernel Configuration Enabling the Network Time Protocol Tuning Network and Kernel Parameters Make the new disk a boot disk Creating Mirrors of Root Logical Volumes Backing Up Cluster Lock Disk Information Choosing Cluster Lock Disks Setting Up a Lock LUN This will create three device files, for example Creating a Disk Partition on an HP Integrity SystemUsr/sbin/idisk -w -p -f partition.txt /dev/rdsk/c1t4d0 Usr/sbin/idisk -w -p -f partition.txt /dev/rdisk/disk12 Defining the Lock LUN Excluding Devices from Probing Creating a Storage Infrastructure with LVM Setting Up and Running the Quorum Server Using the EMS Disk Monitor Using the Generic Resources Disk Monitor Creating Volume Groups Using Mirrored Individual Data Disks Lvchange -t 60 /dev/vg01/lvol1 Creating Logical VolumesSetting Logical Volume Timeouts Creating File Systems Distributing the Volume Group Distributing Volume Groups to Other NodesVerify the configuration Deactivating the Volume Group Create a directory to mount the disk Deactivate the volume group on ftsys10Still on ftsys9, copy the map file to ftsys10 Converting Disks from LVM to VxVM Making Physical Volume Group Files ConsistentCreating Additional Volume Groups Creating a Storage Infrastructure with VxVM Creating Disk Groups Initializing Disks for VxVMInitializing Disks Previously Used by LVM Re-Importing Disk Groups Deporting Disk Groups Cmquerycl -v -C $SGCONF/clust1.conf -n ftsys9 -n ftsys10 Configuring the ClusterClearimport at System Reboot Time Here is an example of the command enter it all one line Specifying the Address Family for the Heartbeat Cmquerycl OptionsSpeeding up the Process Specifying the Address Family for the Cluster Hostnames Full Network Probing Specifying a Lock DiskSpecifying the Cluster Lock Generating a Network Template File See also Choosing Cluster Lock Disks Specifying a Lock LUNCmquerycl -v -n ftsys9 -n ftsys10 Vgchange -c y /dev/vglock Will produce the output such as the following Specifying a Quorum ServerObtaining Cross-Subnet Information Cmquerycl -q QSHost QSAddr -n ftsys9 -n ftsys10 -C Configuring the Cluster Identifying Heartbeat Subnets Specifying Maximum Number of Configured PackagesModifying the Membertimeout Parameter Controlling Access to the Cluster Access Roles How Access Roles Work Setting up Access-Control Policies Levels of Access Userrole must be one of these three values Monitor Fulladmin Packageadmin Username root Username john Userhost bitRole Conflicts Package versus Cluster Roles Adding Volume GroupsVerifying the Cluster Configuration Distributing the Binary Configuration File Differences between Legacy CFS and Modular CFS Storing Volume Group and Cluster Lock Configuration DataModular CFS packages v/s Legacy CFS packages Operational commands for Legacy CFS and Modular CFS Delete a mount point, check point, or snapshot in a package Cfscluster status Cfscluster config -t 900 -sPreparing the Cluster and the System Multi-node Package Cfsdgadm display Creating the Disk GroupsCreating the Disk Group Cluster Packages Cfsdgadm add logdata all=sw Vxprint logfiles Creating VolumesUse the vxprint command to verify Cfsdgadm showpackage logdata Cmmakepkg -m sg/cfsall /etc/cmcluster/cfspkg1.ascii Create a package configuration fileFor instructions on creating modular CFS packages, see Cmapplyconf -P /etc/cmcluster/cfspkg1.ascii Apply the package configuration fileCmcheckconf -P /etc/cmcluster/cfspkg1.ascii Cfsconcurrentmountunmountoperations CmviewclBdf Cvmconcurrentdgoperations Current primary, a primary migration is triggered to Cmmakepkg -m sg/cfsall /etc/cmcluster/ckpt1.asciiPackage. For more information, see the manpage See the mountvxfs 1m manpage Cmmakepkg -m sg/cfsall snap1.ascii Create a package configuration file for the snapshot imageVxassist -g cvmdg3 make vol1 100m vxvol -g cvmdg3 startall Mountvxfs 1m manpage Mount pointsSnapshotmountoptions Information about the mount options, see Online reconfiguration of modular CFS package parameters Cmcheckconf -P cfspkg1.ascii Cmviewcl -v -f line -p cfspkg1 Verify the output Apply the configurationCmapplyconf -P cfspkg1.ascii Legacy Style of Packaging Modular Style of Packaging Managing Disk Groups and Mount Points Using Legacy Packages Cfsmount /tmp/checklogfiles Creating Checkpoint and Snapshot Packages for CFSFsckptadm -n create check2 /tmp/logdata/logfiles Associate it with the cluster and mount it Cfsmount /local/snap1 cmviewcl It is persistentVxassist -g dg1 make vol1 100m vxvol -g dg1 startall Associate it with the cluster You need to do the tasks described in the following sections Initializing Disks for CVM Initializing the Veritas Volume ManagerPreparing the Cluster for Use with CVM Identifying the Master Node Mirror Detachment Policies with CVM Adding Disk Groups to the Package ConfigurationUsr/lib/vxvm/bin/vxdisksetup -i c4t3d4 Vxdg -s init logdata c0t3d2 Managing the Running Cluster Using Dsau during ConfigurationChecking Cluster Operation with Serviceguard Manager Checking Cluster Operation with Serviceguard Commands Setting up Autostart Features Preventing Automatic Activation of LVM Volume Groups Managing a Single-Node Cluster Here is an example of the /etc/rc.config.d/cmcluster fileChanging the System Message Change the cmclconfd entry in /etc/inetd.conf to Deleting the Cluster ConfigurationDisabling identd Single-Node Operation Building an HA Cluster Configuration Configuring Packages and Their Services Types of Package Failover, Multi-Node, System Multi-Node Choosing Package Modules Failoverpolicy Failbackpolicy Ipsubnet Ipaddress Cmmakepkg -m sg/all $SGCONF/sg-all Differences between Failover and Multi-Node PackagesPackage Modules and Parameters Base Package Modules Optional Modules Optional Package ModulesBase Modules Locallanfailoverallowed Externalscript Package Parameter Explanations Cmmakepkg $SGCONF/sg-all Autorun NodefailfastenabledNodename Haltscripttimeout Runscripttimeout Loglevel SuccessorhalttimeoutScriptlogfile Operationsequence Dependencyname Priority For more information, see About Package Dependencies Dependencycondition Dependencylocation Specifies where the dependencycondition must be metWeightname, weightvalue Clusterinterconnectsubnet MonitoredsubnetaccessLocallanfailoverallowed Monitoredsubnet Ipsubnet Ipaddress New for A.11.18 for both modular and legacy packagesIpsubnet Ipaddress ServicenameSee the package configuration file for more examples Ipsubnetnode Servicehalttimeout ServicecmdServicerestart Servicefailfastenabled Genericresourceupcriteria Defines when the status of a generic resource is evaluatedGenericresourceevaluationtype Resourcestart ResourcenameName of a resource to be monitored Resourcepollinginterval Cannot lock /etc/lvmconf//lvmlock still trying EnablethreadedvgchangeResourceupvalue Concurrentvgchangeoperations Vxvolcmd VgchangecmdCvmactivationcmd Vxvmdgretry KillprocessesaccessingrawdevicesCvmdg Vxvmdg Fsmountretrycount Fsfsckopt -s Fstype vxfsConcurrentfsckoperations Concurrentmountandumountoperations Fstype FsnameFsserver Fsdirectory Pev FsmountoptFsumountopt Fsfsckopt Additional Parameters Used Only by Legacy Packages UsernameUserhost Userrole Mkdir $SGCONF/pkg1 Generating the Package Configuration FileBefore You Start Cmmakepkg Examples Next Step Editing the Configuration File See About Package Dependencies page 137 for more information Packagetype. Enter failover, multinode, or systemmultinode Editing the Configuration File Vg vg01 Vg vg02 Verifying and Applying the Package Configuration # vxdg -tfC import dg01 Adding the Package to the ClusterHow Control Scripts Manage VxVM Disk Groups Cmviewcl -r A.11.16 Cluster and Package MaintenanceReviewing Cluster and Package Status Viewing Dependencies Node Status and State Viewing CFS Multi-Node InformationTypes of Cluster and Package States Cluster Status Reviewing Cluster and Package Status Down Unknown Normal Running Status Examples of Cluster and Package StatesFailover and Failback Policies CFS Package Status Quorum Server Status Status After Halting a Package Then run cmviewcl -v, we’ll see If we use the following commandStatus After Moving the Package to Another Node Status After Halting a Node Status After Auto Run is EnabledOutput of the cmviewcl command is now as follows After we halt ftsys10 with the following command Viewing Information about System Multi-Node Packages This output can be seen on both ftsys9 and ftsys10Viewing Information about Unowned Packages Cmviewcl -v -p SG-CFS-pkg Checking Status of the Cluster File System CFSStatus of the Packages in a Cluster File System Ftsys9 Sw sw Status of CFS Modular Disk Group and Mount Point PackagesCmviewcl -v -p mpdg1 Status of Legacy CVM Disk Group Packages Cfsmntadm display -v /tmp/logdata/logfiles Checking the Cluster Configuration and ComponentsFtsys10 Status of Legacy CFS Mount Point Packages User-created files if you specify them Etc/nsswitch.conf Etc/servicesChecking Cluster Components Cmapplyconf 1m Verifying Cluster Components Run cmcheckconf -C See the cron 1m manpage for more information Setting up Periodic Cluster VerificationManaging the Cluster and Nodes Limitations Cmruncl -v -n ftsys9 -n ftsys10 Starting the Cluster When all Nodes are DownAdding Previously Configured Nodes to a Running Cluster Using Serviceguard Commands to Start the Cluster Cmhaltnode -f -v ftsys9 Removing Nodes from Participation in a Running ClusterHalting the Entire Cluster Cmrunnode -v ftsys8 Rules and Restrictions Automatically Restarting the ClusterWhat You Can Do Managing the Cluster and Nodes Additional Points To Note Halting the Cluster and Detaching its Packages Halting a Node and Detaching its PackagesHalting a Detached Package Cmrunnode node1 Managing Packages and ServicesStarting a Package Halting a Package that Has Dependencies Using Serviceguard Commands to Start a PackageHalting a Package Starting a Package that Has Dependencies Moving a Failover Package Changing Package Switching BehaviorUsing Serviceguard Commands to Halt a Package Changing Package Switching with Serviceguard Commands Cmmodpkg -d -n lptest3 pkg1 Maintaining a Package Maintenance ModeSee Performing Maintenance Using Maintenance Mode Cluster and Package Maintenance Procedure Performing Maintenance Using Maintenance Mode Cmrunpkg -m sg/packageip pkg1 Excluding Modules in Partial-Startup Maintenance Mode Types of Changes to the Cluster Configuration Reconfiguring a ClusterCmrunpkg -e sg/service pkg1 Cmrunpkg -m sg/services -e sg/packageip pkg1 Previewing the Effect of Cluster Changes Change to the Cluster Configuration Cmmodpkg -e -t pkg1 Using Preview mode for Commands and in Serviceguard ManagerMode see Maintaining a Package Maintenance Mode What You Can Preview You would see output something like this Using cmevalCmeval -v newstate.in Updating the Cluster Lock LUN Configuration Online Updating the Cluster Lock ConfigurationReconfiguring a Halted Cluster Updating the Cluster Lock Disk Configuration Online Cmgetconf -c cluster1 temp.ascii Reconfiguring a Running ClusterCmapplyconf -C clconfig.ascii Adding Nodes to the Cluster While the Cluster is Running What You Can Do Cmquerycl -C clconfig.ascii -c cluster1 -n ftsys8 -n ftsys9 What You Must Keep in Mind Example Adding a Heartbeat LAN Cmquerycl -c cluster1 -C clconfig.ascii Removing a LAN or Vlan Interface from a Node Cmgetconf clconfig.ascii See also Replacing LAN or Fibre Channel Cards Changing the LVM Configuration while the Cluster is RunningChanging the VxVM or CVM Storage Configuration Cmgetconf -c clustername clconfig.ascii Configuring a Legacy PackageCreating the Legacy Package Configuration Mkdir /etc/cmcluster/pkg1 Configuring a Package in StagesEditing the Package Configuration File Cluster and Package Maintenance Cmmakepkg -s /etc/cmcluster/pkg1/pkg1.sh Creating the Package Control Script Customizing the Package Control Script Support for Additional Products Adding Serviceguard Commands in Customer Defined Functions Cmcheckconf -v -P /etc/cmcluster/pkg1/pkg1.conf Verifying the Package ConfigurationDistributing the Configuration Copying Package Control Scripts with HP-UX commands Configuring nodename Configuring Cross-Subnet Failover IP0 = SUBNET0 IP1 = SUBNET1 Reconfiguring a PackageConfiguring monitoredsubnetaccess Creating Subnet-Specific Package Control Scripts Cmgetconf -p pkg1 pkg1.conf Reconfiguring a Package on a Running ClusterMigrating a Legacy Package to a Modular Package Adding a Package to a Running Cluster Reconfiguring a Package on a Halted Cluster Unmount the shared file system cfsumount mount point Deleting a Package from a Running ClusterCmhaltpkg mypkg Cmdeleteconf -p mypkg Cmapplyconf -v -P app1.conf Cmmodpkg -R -s myservice pkg1 Resetting the Service Restart CounterAllowable Package States During Reconfiguration Types of Changes to Packages Change servicerestart modular package Locallanfailoverallowed Change vxvolcmd Cfsmountoptions Changes that Will Trigger Warnings Responding to Cluster Events Single-Node Operation Disabling ServiceguardRemoving Serviceguard from a System Testing the Package Manager Troubleshooting Your ClusterTesting Cluster Operation Start the Cluster using Serviceguard Manager Monitoring Hardware Testing the Cluster ManagerTesting the Network Manager Hardware Monitors and Persistence Requests Using System Fault Management ServiceUsing Event Monitoring Service Using EMS Event Monitoring Service Hardware Monitors Replacing Disks Using HP Isee HP Instant Support Enterprise EditionReplacing a Faulty Array Mechanism Replacing a Faulty Mechanism in an HA Enclosure Replacing a Lock LUN Replacing a Lock Disk Replacing Scsi Host Bus Adapters Online Hardware Maintenance with In-line Scsi TerminatorCmdisklock reset /dev/dsk/c0t1d1 Replacing I/O Cards Replacing LAN or Fibre Channel Cards Offline ReplacementOnline Replacement After Replacing the Card Replacing a Failed Quorum Server System Reviewing the System Log File Troubleshooting ApproachesUsing cmquerycl and cmcheckconf Using cmviewcl Reviewing Package IP Addresses Cmreadlog /var/opt/cmom/cmomd.log Reviewing Object Manager Log FilesSample System Log Entries Following is an example of a successful package starting Reviewing the System Multi-node Package Files Reviewing Serviceguard Manager Log FilesReviewing Configuration Files Using the cmcheckconf Command Serviceguard Command Hangs Solving ProblemsUsing the cmviewconf Command Reviewing the LAN Configuration Nslookup ftsys9 Networking and Security Configuration ErrorsCluster Re-formations Caused by Temporary Conditions Package Control Script Hangs or Failures System Administration Errors Fuser -kulogical-volume umount logical-volume Llt, gab Vxfen W cvm Cfs Problems with Cluster File System CFS Force Import and Deport After Node Failure Problems with VxVM Disk GroupsPackage Movement Errors Node and Network Failures Access denied to quorum server Troubleshooting the Quorum ServerAuthorization File Problems Timeout Problems Messages Enterprise Cluster Master Toolkit Automating Application Operation Designing Highly Available Cluster Applications Define Application Startup and Shutdown Controlling the Speed of Application FailoverInsulate Users from Outages Minimize Data Loss Use Raw VolumesReplicate Non-Data File Systems Evaluate the Use of JFS Balance Checkpoint Frequency with Performance Use Restartable TransactionsUse Checkpoints Design for Multiple Servers Avoid Node-Specific Information Designing Applications to Run on Multiple SystemsDesign for Replicated Data Sites Allow Multiple Instances on Same System Avoid Using SPU IDs or MAC AddressesAssign Unique Names to Applications Obtain Enough IP Addresses Bind to Relocatable IP Addresses Use uname2 With CareBind to a Fixed Port Call bind before connect Give Each Application its Own Volume GroupUse Multiple Destinations for SNA Applications Avoid File Locking Help menu for ndd -h ipstrongesmodel Etc/rc.config.d/nddconf as followsUsr/sbin/route add net default 128.17.17.1 1 source Restoring Client Connections Usr/sbin/route delete net default 128.17.17.1 1 source Be Able to Monitor Applications Handling Application FailuresCreate Applications to be Failure Tolerant Do Not Change the Data Layout Between Releases Reducing Time Needed for Application Upgrades and PatchesProvide for Rolling Upgrades Minimizing Planned Downtime Documenting Maintenance Operations Providing Online Application Reconfiguration Defining Baseline Application Behavior on a Single System Integrating HA Applications with ServiceguardChecklist for Integrating HA Applications Integrating HA Applications in Multiple Systems Move it back Testing the Cluster Special Considerations for Upgrade to Serviceguard A.11.19 Software UpgradesSpecial Considerations for Upgrade to Serviceguard A.11.20 Rolling Upgrade Using DRD Types of UpgradeHow To Tell when the Cluster Re-formation Is Complete Rolling Upgrade Non-Rolling Upgrade Using DRD Guidelines for Rolling UpgradeRestrictions for DRD Upgrades Non-Rolling Upgrade Limitations of Rolling Upgrades Performing a Rolling Upgrade Migrating cmclnodelist entries from A.11.15 or earlier Running the Rolling UpgradeKeeping Kernels Consistent Running the Rolling Upgrade Using DRD Performing a Rolling Upgrade Using DRD Halt the first node, as follows Example of a Rolling UpgradeStep Running Cluster with Packages Moved to Node Repeat the process on node 2. Halt the node, as follows Node 1 Rejoining the Cluster Performing a Non-Rolling Upgrade Guidelines for Non-Rolling Upgrade Steps for a Non-Rolling Upgrade Using DRD Performing a Non-Rolling Upgrade Using DRDLimitations of Non-Rolling Upgrades using DRD Checklist for Migration Guidelines for Migrating a Cluster with Cold Install Worksheet for Hardware Planning Power Supply WorksheetBlank Planning Worksheets Quorum Server Worksheet LVM Volume Group and Physical Volume Worksheet VxVM Disk Group and Disk Worksheet Cluster Configuration Worksheet Package Configuration Worksheet Package Configuration Worksheet Loading VxVM Migrating Volume GroupsMigrating from LVM to VxVM Data Storage Mntdg0202, respectively Customizing Packages for VxVM Restart the package Removing LVM Volume GroupsCustomizing Packages for CVM Migrating from Legacy CFS Packages to Modular CFS Packages Textual Representation of IPv6 Addresses IPv6 Network SupportIPv6 Address Types IPv4 Compatible IPv6 Addresses IPv6 Address PrefixUnicast Addresses IPv4 and IPv6 Compatibility Multicast Addresses Aggregatable Global Unicast AddressesLink-Local Addresses Site-Local Addresses Network Configuration Restrictions Ndd -set /dev/ip6 ip6nddadsolicitcountn Example ConfigurationsLocal Primary/Standby LAN Patterns Ndd -get /dev/ip6 ip6nddadsolicitcount Example Configurations 384 IPv6 Network Support About the Online Help System Using Serviceguard ManagerBefore Using HP Serviceguard Manager Setting Up Accessing Serviceguard Manager Opt/hpsmh/bin/hpsmh autostart Launching Serviceguard ManagerAccessing Serviceguard Manager Scenario 1 Single cluster management System Management Homepage with Serviceguard Manager From the left-hand panel, expand Cluster by Type Expand HP Serviceguard, and click on a Serviceguard clusterSign Maximum and Minimum Values for Parameters Membertimeout Launching Monitoring Scripts Monitoring Script for Generic ResourcesSample scripts Launching Monitoring Scripts Template of a Monitoring Script I L I T Y N C T I O N S Monitoring Script for Generic Resources Template of a Monitoring Script Migrating EMS Resources to Generic Resources Identify the equivalent SFM style resource monitor Start the package APA Index 399 Cvmactivationcmd Firstclusterlockpv LAN INONLYORINOUT, 69 Inout Pollingtarget defined Qsaddr Servicename Vxvmdg