Using Serviceguard Manager 385, 396, Index 398

I Using Serviceguard Manager	385
About the Online Help System	385
Before Using HP Serviceguard Manager: Setting Up	385
Accessing Serviceguard Manager	385
Launching Serviceguard Manager	386
Scenario 1 - Single cluster management	386
Scenario 2- Multi-Cluster Management for Serviceguard A.11.17 Cluster or Earlier	387
Scenario 3- Multi-Cluster Management for Serviceguard A.11.17.01 Cluster or Later	388
J Maximum and Minimum Values for Parameters	389
K Monitoring Script for Generic Resources	390
Launching Monitoring Scripts	390
Template of a Monitoring Script	392
L Migrating EMS Resources to Generic Resources	396
Example of Migration of an EMS Resource to a Generic Resource	397
Index	398

Contents 17

Page 17

Image 17

HP manual Using Serviceguard Manager 385, 396, Index 398

Contents

Managing Serviceguard Twentieth Edition Legal Notices Contents Contents Planning and Documenting an HA Cluster Contents Building an HA Cluster Configuration 158 Contents Configuring Packages and Their Services 227 Contents Cluster and Package Maintenance 261 279 Troubleshooting Your Cluster 321 Contents 349 Integrating HA Applications with Serviceguard 354Software Upgrades 357 Migrating from LVM to VxVM Data Storage 374 Blank Planning Worksheets 369377 IPv6 Network Support 378 Index 398 Using Serviceguard Manager 385396 Publishing History Publishing History Preface Related Publications Securing Serviceguard and other Serviceguard white papers What is Serviceguard? Serviceguard at a Glance Shows what happens in a failover situation Failover Typical Cluster After Failover About Veritas CFS and CVM from Symantec Using SAM Using Serviceguard Manager Usr/sbin/sam -w What are the Distributed Systems Administration Utilities?Roadmap for Configuring Clusters and Packages Tasks in Configuring a Serviceguard Cluster Redundancy of Cluster Components Understanding Serviceguard Hardware Configurations Rules and Restrictions Redundant Network Components Redundant LANs Redundant Ethernet Configuration Configuration Tasks Cross-Subnet ConfigurationsFor legacy packages, see Configuring Cross-Subnet Failover Restrictions For More Information Supported Disk Interfaces Redundant Disk StorageReplacing Failed Network Cards Data Protection Disk Arrays using RAID Levels and Multiple Data PathsDisk Mirroring About Multipathing Monitoring VxVM and CVM Disks Monitoring LVM Disks Through Event Monitoring ServiceReplacing Failed Disk Mechanisms Replacing Failed I/O Cards Mirrored Disks Connected for High Availability Sample Scsi Disk Configurations Cluster with High Availability Disk Array Sample Fibre Channel Disk Configuration Larger Clusters Redundant Power Supplies Point to Point Connections to Storage Devices Active/Standby Model Eight-Node Cluster with XP or EMC Disk Array Serviceguard Daemons Understanding Serviceguard Software ComponentsServiceguard Architecture Cluster Daemon cmcld Configuration Daemon cmclconfd File Management Daemon cmfileassistd Cluster Logical Volume Manager Daemon cmlvmdSyslog Log Daemon cmlogd Cluster Object Manager Daemon cmomd Quorum Server Daemon qs Service Assistant Daemon cmservicedNetwork Manager Daemon cmnetd Lock LUN Daemon cmdisklockd How the Cluster Manager Works Configuring the ClusterProxy Daemon cmproxyd CFS Components Manual Startup of Entire Cluster Heartbeat Messages Dynamic Cluster Re-formation Automatic Cluster StartupCluster Quorum to Prevent Split-Brain Syndrome Cluster Lock Lock Requirements Use of a Lock LUN or LVM Lock Disk as the Cluster Lock Dual Lock Disk Use of the Quorum Server as the Cluster LockSingle Lock Disk or LUN Quorum Server Operation No Cluster Lock Package Types How the Package Manager WorksNon-failover Packages Failover Packages Failover Packages’ Switching Behavior Deciding When and Where to Run and Halt Failover Packages Before Package Switching Automatic Rotating Standby Package Configuration Data Rotating Standby Configuration before Failover Failback Policy Configurednode Policy Packages after Failover Automatic Failback Configuration After Failover Using the Generic Resources Monitoring Service Using Older Package Configuration Files Understanding Serviceguard Software Components Using the EMS HA Monitors Using the Event Monitoring ServiceSee also Using Generic Resources to Monitor Volume Groups See also Using EMS to Monitor Volume Groups What Makes a Package Run? How Packages Run Legacy Package Time Line Showing Important Events Before the Control Script Starts Package Time Line Legacy Package During Run Script Execution Normal and Abnormal Exits from the Run Script Service Startup with cmrunservWhile Services are Running During Halt Script Execution When a Package is Halted with a Command Legacy Package Time Line for Halt Script Execution Normal and Abnormal Exits from the Halt Script Package Control Script Error and Exit ConditionsError Conditions and Package Movement for Failover Packages Stationary and Relocatable IP Addresses How the Network Manager Works Types of IP Addresses Adding and Deleting Relocatable IP Addresses Load Sharing Monitoring LAN Interfaces and Detecting Failure Link Level Local Switching Cluster Before Local Network Switching Cmmodnet -e interface Where interface is the primary interface Monitoring LAN Interfaces and Detecting Failure IP Level Remote Switching How the IP Monitor Works Reasons To Use IP Monitoring Failure and Recovery Detection Times Reporting Link-Level and IP-Level Failures Example 1 If Local Switching is ConfiguredConstraints and Limitations See also Reporting Link-Level and IP-Level Failures Cmmodnet -e lan2 Example 2 If There Is No Local SwitchingAutomatic Port Aggregation Support for HP-UX Vlan Vlan ConfigurationsWhat is VLAN? Configuration Restrictions Volume Managers for Data StorageTypes of Redundant Storage Additional Heartbeat Requirements White papers About Device File Names Device Special FilesExamples of Mirrored Storage Physical Disks Within Shared Storage Units Examples of Storage on Disk Arrays Multiple Devices Configured in Volume Groups Multiple Paths to LUNs HP-UX Logical Volume Manager LVM Types of Volume ManagerVeritas Volume Manager VxVM Propagation of Disk Groups in VxVM Cluster Startup Time with CVM Veritas Cluster Volume Manager CVMPropagation of Disk Groups with CVM For heartbeat requirements, see Redundant Heartbeat Subnets Redundant Heartbeat Subnets Comparison of Volume Managers Pros and Cons of Volume Managers with Serviceguard What Happens when a Node Times Out System Reset When a Node FailsResponses to Failures Example Responses to Hardware Failures Responses to Package and Generic Resources Failures Responses to Package and Service Failures Network Communication Failure Service Restarts Planning and Documenting an HA Cluster Serviceguard Memory RequirementsGeneral Planning Planning for Expansion Hardware Planning Sample Cluster Configuration Network Information SPU InformationLAN Information Nnn.nnn.nnn.nnn Scsi Addressing in Cluster Configuration Under Cluster Configuration Parameters Diskinfo Disk I/O Information Hardware Configuration Worksheet Power Supply Planning Cluster Lock Planning Power Supply Configuration Worksheet Quorum Server Worksheet Using a Quorum ServerCluster Lock Disk and Re-formation Time LVM Planning Using Generic Resources to Monitor Volume Groups Using EMS to Monitor Volume Groups LVM Worksheet For more information, see Using the EMS HA MonitorsCVM and VxVM Planning CVM and VxVM Worksheet Cluster Configuration Planning Points To Note Where cDSFs ResideAbout Cluster-wide Device Special Files cDSFs Limitations of cDSFs LVM Commands and cDSFsAbout Easy Deployment Limitations of Easy Deployment Heartbeat Subnet and Cluster Re-formation TimeAdvantages of Easy Deployment What Is IPv6-Only Mode? What Is IPv4-only Mode?Rules and Restrictions for IPv6-Only Mode Localhost ipv6-localhost ipv6-loopback IPV6 or ANY What Is Mixed Mode?Recommendations for IPv6-Only Mode Rules and Restrictions for Mixed Mode Cluster Configuration ParametersCluster configuration file Name of the cluster as it will appear in the output Planning and Documenting an HA Cluster Go/hpux-serviceguard-docs under HP Serviceguard Happens when You Change the Quorum ConfigurationIPv4-Only,IPv6-Only, and Mixed Mode page 106 for 99 and Specifying a Quorum Server See also About Hostname Address Families IPv4-Only Hpux-serviceguard-docs under HP ServiceguardIPv6-Only, and Mixed Mode page 106 for important Sitepreferred or Configuration file see Configuring PackagesTheir Services page 227 and these in turn must Cluster Is Running Configuration Planning page 125 must be specified Cluster Configuration Planning CVM/CFS on HP Serviceguard A.11.20 April You cannot change the heartbeat configuration whileProtocols and services. RPC assumes that each network To that LAN, to risk timeout without being serviced Configuration Online page 49 for important information Also What Happens when You Change the QuorumSee IPv6 Address Types Lock LUN page 189 for more information Failbackpolicy When You Change the Quorum Configuration OnlineCluster is running, see Updating the Cluster Lock Disk See About Package Weights page 144 for more Planning and Documenting an HA Cluster 88, Cluster Daemon cmcld page 41, See also What Happens when a Node Times Out69, Monitoring LAN Interfaces and Detecting IP-Level Failures Configuration file specifies one of two ways to decide Default isWhen a network interface card has failed How Serviceguard will handle the recovery of the primary See Monitoring LAN Interfaces and Detecting Failure IP Planning and Documenting an HA Cluster Cluster Configuration Next Step Package Configuration PlanningLogical Volume and File System Planning Access Control Policies also known as Role Based Access CVM 4.1 and later without CFS CVM 4.1 and later with CFS Using the Volume Monitor About the Volume Monitor Or --version Or --helpOr --log-file Or --log-level Planning for NFS-mounted File Systems VolumepathUsr/sbin/cmvolmond /dev/vg01/lvol1 /dev/vg01/lvol2 Usr/sbin/cmvolmond -t 10 /dev/vg00/lvol1 Package Configuration Planning Package Failover Behavior Choosing Switching and Failover Behavior Configuring a Generic Resource Parameters for Configuring Generic ResourcesExtended generic resource Cmmakepkg -i $SGCONF/pkg1/pkg1.conf -m sg/genericresource Cmapplyconf -P $SGCONF/pkg1/pkg1.conf Cmcheckconf -v -P $SGCONF/pkg1/pkg1.conf Cmrunpkg pkg1 Cmviewcl -v -f line -p pkg1 grep genericresourceCmgetresource -r sfmdisk Cmsetresource -r sfmdisk -s up Online Reconfiguration of Generic Resources Parameters for Configuring EMS Resources Simple Dependencies About Package DependenciesRules for Simple Dependencies Assume that we want to make pkg1 depend on pkg2 Planning and Documenting an HA Cluster Dragging Rules for Simple Dependencies Planning and Documenting an HA Cluster Extended Dependencies Rules for Exclusionary Dependencies See Rules for differentnode and anynode Dependencies Rules for differentnode and anynode Dependencies What Happens when a Package Fails Package Weights and Node Capacities About Package WeightsConfiguring Weights and Capacities Cmmakepkg 1m manpage Simple Method Weightname packagelimit weightvalueNodename node1 Capacityname packagelimit For pkg2 Defining Capacities Comprehensive MethodPoints to Keep in Mind Nodename node2 Clustername cluster23 Nodename node1 Defining Default Weights Defining Weights Weightname B Weightvalue Weightname a WeightvalueWeightname B Weightvalue Weightname a Cmquerycl 1m manpage Rules and Guidelines About External Scripts Pevmonitoringinterval Using Serviceguard Commands in an External Script About Cross-Subnet Failover Determining Why a Package Has Shut DownLasthaltfailed Cmviewcl -v -f line displays a lasthaltfailed flag Implications for Application Deployment Configuring a Package to Fail Over across Subnets Example Configuring nodename Configuring a Package Next StepsConfiguring monitoredsubnetaccess Configuring ipsubnetnode Planning for Changes in Cluster Size Installing and Updating Serviceguard Building an HA Cluster ConfigurationWhere Serviceguard Files Are Kept Preparing Your Systems Configuring the Cluster Before You Start Creating Cluster-wide Device Special Files cDSFsCreating cDSFs for a Group of Nodes Etc/cmcluster.conf Csshsetup -r -f /etc/cmcluster/sshhosts Csshsetup -r node2Cmpreparecl -n nodename -n nodename Cmpreparecl -n node1 -n node2 -n node3 -n node4 Displaying the cDSF Configuration Using Easy DeploymentAdding a Node to a cDSF Group Removing a Node from a cDSF Group Cmquerycl -N $SGCONF/mynetwork Using Easy Deployment Commands to Configure the ClusterFor example Preparing Your Systems Building an HA Cluster Configuration PVG bus1 /dev/cdisk/disk14 /dev/cdisk/disk15 Format for entries in cmclnodelist is as follows Configuring Root-Level AccessAllowing Root Access to an Unconfigured Node About identd Ensuring that the Root User on Another Node Is Recognized Any of the aliases. Examples Configuring Name ResolutionOfficial hostname, as defined by hosts 4, for example Safeguarding against Loss of Name Resolution Services For NIS, enter two lines Ensuring Consistency of Kernel Configuration Enabling the Network Time Protocol Tuning Network and Kernel Parameters Make the new disk a boot disk Creating Mirrors of Root Logical Volumes Backing Up Cluster Lock Disk Information Choosing Cluster Lock Disks Setting Up a Lock LUN Usr/sbin/idisk -w -p -f partition.txt /dev/rdsk/c1t4d0 Creating a Disk Partition on an HP Integrity SystemUsr/sbin/idisk -w -p -f partition.txt /dev/rdisk/disk12 This will create three device files, for example Defining the Lock LUN Excluding Devices from Probing Creating a Storage Infrastructure with LVM Setting Up and Running the Quorum Server Using the EMS Disk Monitor Using the Generic Resources Disk Monitor Creating Volume Groups Using Mirrored Individual Data Disks Setting Logical Volume Timeouts Creating Logical VolumesCreating File Systems Lvchange -t 60 /dev/vg01/lvol1 Verify the configuration Distributing Volume Groups to Other NodesDeactivating the Volume Group Distributing the Volume Group Create a directory to mount the disk Deactivate the volume group on ftsys10Still on ftsys9, copy the map file to ftsys10 Creating Additional Volume Groups Making Physical Volume Group Files ConsistentCreating a Storage Infrastructure with VxVM Converting Disks from LVM to VxVM Creating Disk Groups Initializing Disks for VxVMInitializing Disks Previously Used by LVM Re-Importing Disk Groups Deporting Disk Groups Clearimport at System Reboot Time Configuring the ClusterHere is an example of the command enter it all one line Cmquerycl -v -C $SGCONF/clust1.conf -n ftsys9 -n ftsys10 Speeding up the Process Cmquerycl OptionsSpecifying the Address Family for the Cluster Hostnames Specifying the Address Family for the Heartbeat Specifying the Cluster Lock Specifying a Lock DiskGenerating a Network Template File Full Network Probing Cmquerycl -v -n ftsys9 -n ftsys10 Specifying a Lock LUNVgchange -c y /dev/vglock See also Choosing Cluster Lock Disks Obtaining Cross-Subnet Information Specifying a Quorum ServerCmquerycl -q QSHost QSAddr -n ftsys9 -n ftsys10 -C Will produce the output such as the following Configuring the Cluster Modifying the Membertimeout Parameter Specifying Maximum Number of Configured PackagesControlling Access to the Cluster Identifying Heartbeat Subnets Access Roles How Access Roles Work Setting up Access-Control Policies Levels of Access Userrole must be one of these three values Monitor Fulladmin Packageadmin Username root Username john Userhost bitRole Conflicts Package versus Cluster Roles Adding Volume GroupsVerifying the Cluster Configuration Distributing the Binary Configuration File Differences between Legacy CFS and Modular CFS Storing Volume Group and Cluster Lock Configuration DataModular CFS packages v/s Legacy CFS packages Operational commands for Legacy CFS and Modular CFS Delete a mount point, check point, or snapshot in a package Cfscluster status Cfscluster config -t 900 -sPreparing the Cluster and the System Multi-node Package Creating the Disk Group Cluster Packages Creating the Disk GroupsCfsdgadm add logdata all=sw Cfsdgadm display Use the vxprint command to verify Creating VolumesCfsdgadm showpackage logdata Vxprint logfiles Cmmakepkg -m sg/cfsall /etc/cmcluster/cfspkg1.ascii Create a package configuration fileFor instructions on creating modular CFS packages, see Cmapplyconf -P /etc/cmcluster/cfspkg1.ascii Apply the package configuration fileCmcheckconf -P /etc/cmcluster/cfspkg1.ascii Bdf CmviewclCvmconcurrentdgoperations Cfsconcurrentmountunmountoperations Package. For more information, see the manpage Cmmakepkg -m sg/cfsall /etc/cmcluster/ckpt1.asciiSee the mountvxfs 1m manpage Current primary, a primary migration is triggered to Cmmakepkg -m sg/cfsall snap1.ascii Create a package configuration file for the snapshot imageVxassist -g cvmdg3 make vol1 100m vxvol -g cvmdg3 startall Snapshotmountoptions Mount pointsInformation about the mount options, see Mountvxfs 1m manpage Online reconfiguration of modular CFS package parameters Cmcheckconf -P cfspkg1.ascii Cmviewcl -v -f line -p cfspkg1 Verify the output Apply the configurationCmapplyconf -P cfspkg1.ascii Legacy Style of Packaging Modular Style of Packaging Managing Disk Groups and Mount Points Using Legacy Packages Fsckptadm -n create check2 /tmp/logdata/logfiles Creating Checkpoint and Snapshot Packages for CFSAssociate it with the cluster and mount it Cfsmount /tmp/checklogfiles Vxassist -g dg1 make vol1 100m vxvol -g dg1 startall It is persistentAssociate it with the cluster Cfsmount /local/snap1 cmviewcl You need to do the tasks described in the following sections Preparing the Cluster for Use with CVM Initializing the Veritas Volume ManagerIdentifying the Master Node Initializing Disks for CVM Usr/lib/vxvm/bin/vxdisksetup -i c4t3d4 Adding Disk Groups to the Package ConfigurationVxdg -s init logdata c0t3d2 Mirror Detachment Policies with CVM Checking Cluster Operation with Serviceguard Manager Using Dsau during ConfigurationChecking Cluster Operation with Serviceguard Commands Managing the Running Cluster Setting up Autostart Features Preventing Automatic Activation of LVM Volume Groups Managing a Single-Node Cluster Here is an example of the /etc/rc.config.d/cmcluster fileChanging the System Message Disabling identd Deleting the Cluster ConfigurationSingle-Node Operation Change the cmclconfd entry in /etc/inetd.conf to Building an HA Cluster Configuration Configuring Packages and Their Services Types of Package Failover, Multi-Node, System Multi-Node Choosing Package Modules Failoverpolicy Failbackpolicy Ipsubnet Ipaddress Package Modules and Parameters Differences between Failover and Multi-Node PackagesBase Package Modules Cmmakepkg -m sg/all $SGCONF/sg-all Optional Modules Optional Package ModulesBase Modules Locallanfailoverallowed Externalscript Package Parameter Explanations Cmmakepkg $SGCONF/sg-all Autorun NodefailfastenabledNodename Haltscripttimeout Runscripttimeout Scriptlogfile SuccessorhalttimeoutOperationsequence Loglevel Dependencyname Priority For more information, see About Package Dependencies Dependencycondition Dependencylocation Specifies where the dependencycondition must be metWeightname, weightvalue Locallanfailoverallowed MonitoredsubnetaccessMonitoredsubnet Clusterinterconnectsubnet Ipsubnet Ipaddress New for A.11.18 for both modular and legacy packagesIpsubnet See the package configuration file for more examples ServicenameIpsubnetnode Ipaddress Servicerestart ServicecmdServicefailfastenabled Servicehalttimeout Genericresourceupcriteria Defines when the status of a generic resource is evaluatedGenericresourceevaluationtype Name of a resource to be monitored ResourcenameResourcepollinginterval Resourcestart Resourceupvalue EnablethreadedvgchangeConcurrentvgchangeoperations Cannot lock /etc/lvmconf//lvmlock still trying Vxvolcmd VgchangecmdCvmactivationcmd Cvmdg KillprocessesaccessingrawdevicesVxvmdg Vxvmdgretry Concurrentfsckoperations Fsfsckopt -s Fstype vxfsConcurrentmountandumountoperations Fsmountretrycount Fsserver FsnameFsdirectory Fstype Fsumountopt FsmountoptFsfsckopt Pev Userhost UsernameUserrole Additional Parameters Used Only by Legacy Packages Before You Start Generating the Package Configuration FileCmmakepkg Examples Mkdir $SGCONF/pkg1 Next Step Editing the Configuration File See About Package Dependencies page 137 for more information Packagetype. Enter failover, multinode, or systemmultinode Editing the Configuration File Vg vg01 Vg vg02 Verifying and Applying the Package Configuration # vxdg -tfC import dg01 Adding the Package to the ClusterHow Control Scripts Manage VxVM Disk Groups Reviewing Cluster and Package Status Cluster and Package MaintenanceViewing Dependencies Cmviewcl -r A.11.16 Types of Cluster and Package States Viewing CFS Multi-Node InformationCluster Status Node Status and State Reviewing Cluster and Package Status Down Unknown Normal Running Status Examples of Cluster and Package StatesFailover and Failback Policies CFS Package Status Quorum Server Status Status After Halting a Package Then run cmviewcl -v, we’ll see If we use the following commandStatus After Moving the Package to Another Node Output of the cmviewcl command is now as follows Status After Auto Run is EnabledAfter we halt ftsys10 with the following command Status After Halting a Node Viewing Information about System Multi-Node Packages This output can be seen on both ftsys9 and ftsys10Viewing Information about Unowned Packages Cmviewcl -v -p SG-CFS-pkg Checking Status of the Cluster File System CFSStatus of the Packages in a Cluster File System Cmviewcl -v -p mpdg1 Status of CFS Modular Disk Group and Mount Point PackagesStatus of Legacy CVM Disk Group Packages Ftsys9 Sw sw Ftsys10 Checking the Cluster Configuration and ComponentsStatus of Legacy CFS Mount Point Packages Cfsmntadm display -v /tmp/logdata/logfiles User-created files if you specify them Etc/nsswitch.conf Etc/servicesChecking Cluster Components Cmapplyconf 1m Verifying Cluster Components Run cmcheckconf -C Managing the Cluster and Nodes Setting up Periodic Cluster VerificationLimitations See the cron 1m manpage for more information Adding Previously Configured Nodes to a Running Cluster Starting the Cluster When all Nodes are DownUsing Serviceguard Commands to Start the Cluster Cmruncl -v -n ftsys9 -n ftsys10 Halting the Entire Cluster Removing Nodes from Participation in a Running ClusterCmrunnode -v ftsys8 Cmhaltnode -f -v ftsys9 Rules and Restrictions Automatically Restarting the ClusterWhat You Can Do Managing the Cluster and Nodes Additional Points To Note Halting the Cluster and Detaching its Packages Halting a Node and Detaching its PackagesHalting a Detached Package Cmrunnode node1 Managing Packages and ServicesStarting a Package Halting a Package Using Serviceguard Commands to Start a PackageStarting a Package that Has Dependencies Halting a Package that Has Dependencies Using Serviceguard Commands to Halt a Package Changing Package Switching BehaviorChanging Package Switching with Serviceguard Commands Moving a Failover Package Cmmodpkg -d -n lptest3 pkg1 Maintaining a Package Maintenance ModeSee Performing Maintenance Using Maintenance Mode Cluster and Package Maintenance Procedure Performing Maintenance Using Maintenance Mode Cmrunpkg -m sg/packageip pkg1 Excluding Modules in Partial-Startup Maintenance Mode Cmrunpkg -e sg/service pkg1 Reconfiguring a ClusterCmrunpkg -m sg/services -e sg/packageip pkg1 Types of Changes to the Cluster Configuration Previewing the Effect of Cluster Changes Change to the Cluster Configuration Mode see Maintaining a Package Maintenance Mode Using Preview mode for Commands and in Serviceguard ManagerWhat You Can Preview Cmmodpkg -e -t pkg1 You would see output something like this Using cmevalCmeval -v newstate.in Reconfiguring a Halted Cluster Updating the Cluster Lock ConfigurationUpdating the Cluster Lock Disk Configuration Online Updating the Cluster Lock LUN Configuration Online Cmapplyconf -C clconfig.ascii Reconfiguring a Running ClusterAdding Nodes to the Cluster While the Cluster is Running Cmgetconf -c cluster1 temp.ascii What You Can Do Cmquerycl -C clconfig.ascii -c cluster1 -n ftsys8 -n ftsys9 What You Must Keep in Mind Example Adding a Heartbeat LAN Cmquerycl -c cluster1 -C clconfig.ascii Removing a LAN or Vlan Interface from a Node Cmgetconf clconfig.ascii See also Replacing LAN or Fibre Channel Cards Changing the LVM Configuration while the Cluster is RunningChanging the VxVM or CVM Storage Configuration Cmgetconf -c clustername clconfig.ascii Configuring a Legacy PackageCreating the Legacy Package Configuration Mkdir /etc/cmcluster/pkg1 Configuring a Package in StagesEditing the Package Configuration File Cluster and Package Maintenance Cmmakepkg -s /etc/cmcluster/pkg1/pkg1.sh Creating the Package Control Script Customizing the Package Control Script Support for Additional Products Adding Serviceguard Commands in Customer Defined Functions Distributing the Configuration Verifying the Package ConfigurationCopying Package Control Scripts with HP-UX commands Cmcheckconf -v -P /etc/cmcluster/pkg1/pkg1.conf Configuring nodename Configuring Cross-Subnet Failover Configuring monitoredsubnetaccess Reconfiguring a PackageCreating Subnet-Specific Package Control Scripts IP0 = SUBNET0 IP1 = SUBNET1 Cmgetconf -p pkg1 pkg1.conf Reconfiguring a Package on a Running ClusterMigrating a Legacy Package to a Modular Package Adding a Package to a Running Cluster Reconfiguring a Package on a Halted Cluster Cmhaltpkg mypkg Cmdeleteconf -p mypkg Deleting a Package from a Running ClusterCmapplyconf -v -P app1.conf Unmount the shared file system cfsumount mount point Cmmodpkg -R -s myservice pkg1 Resetting the Service Restart CounterAllowable Package States During Reconfiguration Types of Changes to Packages Change servicerestart modular package Locallanfailoverallowed Change vxvolcmd Cfsmountoptions Changes that Will Trigger Warnings Responding to Cluster Events Single-Node Operation Disabling ServiceguardRemoving Serviceguard from a System Testing Cluster Operation Troubleshooting Your ClusterStart the Cluster using Serviceguard Manager Testing the Package Manager Monitoring Hardware Testing the Cluster ManagerTesting the Network Manager Using Event Monitoring Service Using System Fault Management ServiceUsing EMS Event Monitoring Service Hardware Monitors Hardware Monitors and Persistence Requests Replacing a Faulty Array Mechanism Using HP Isee HP Instant Support Enterprise EditionReplacing a Faulty Mechanism in an HA Enclosure Replacing Disks Replacing a Lock LUN Replacing a Lock Disk Cmdisklock reset /dev/dsk/c0t1d1 Online Hardware Maintenance with In-line Scsi TerminatorReplacing I/O Cards Replacing Scsi Host Bus Adapters Replacing LAN or Fibre Channel Cards Offline ReplacementOnline Replacement After Replacing the Card Replacing a Failed Quorum Server System Using cmquerycl and cmcheckconf Using cmviewcl Troubleshooting ApproachesReviewing Package IP Addresses Reviewing the System Log File Sample System Log Entries Reviewing Object Manager Log FilesFollowing is an example of a successful package starting Cmreadlog /var/opt/cmom/cmomd.log Reviewing Configuration Files Reviewing Serviceguard Manager Log FilesUsing the cmcheckconf Command Reviewing the System Multi-node Package Files Using the cmviewconf Command Solving ProblemsReviewing the LAN Configuration Serviceguard Command Hangs Nslookup ftsys9 Networking and Security Configuration ErrorsCluster Re-formations Caused by Temporary Conditions Package Control Script Hangs or Failures System Administration Errors Fuser -kulogical-volume umount logical-volume Llt, gab Vxfen W cvm Cfs Problems with Cluster File System CFS Package Movement Errors Problems with VxVM Disk GroupsNode and Network Failures Force Import and Deport After Node Failure Authorization File Problems Troubleshooting the Quorum ServerTimeout Problems Access denied to quorum server Messages Enterprise Cluster Master Toolkit Automating Application Operation Designing Highly Available Cluster Applications Define Application Startup and Shutdown Controlling the Speed of Application FailoverInsulate Users from Outages Replicate Non-Data File Systems Use Raw VolumesEvaluate the Use of JFS Minimize Data Loss Use Checkpoints Use Restartable TransactionsDesign for Multiple Servers Balance Checkpoint Frequency with Performance Avoid Node-Specific Information Designing Applications to Run on Multiple SystemsDesign for Replicated Data Sites Assign Unique Names to Applications Avoid Using SPU IDs or MAC AddressesObtain Enough IP Addresses Allow Multiple Instances on Same System Bind to Relocatable IP Addresses Use uname2 With CareBind to a Fixed Port Use Multiple Destinations for SNA Applications Give Each Application its Own Volume GroupAvoid File Locking Call bind before connect Help menu for ndd -h ipstrongesmodel Etc/rc.config.d/nddconf as followsUsr/sbin/route add net default 128.17.17.1 1 source Restoring Client Connections Usr/sbin/route delete net default 128.17.17.1 1 source Be Able to Monitor Applications Handling Application FailuresCreate Applications to be Failure Tolerant Provide for Rolling Upgrades Reducing Time Needed for Application Upgrades and PatchesMinimizing Planned Downtime Do Not Change the Data Layout Between Releases Documenting Maintenance Operations Providing Online Application Reconfiguration Defining Baseline Application Behavior on a Single System Integrating HA Applications with ServiceguardChecklist for Integrating HA Applications Integrating HA Applications in Multiple Systems Move it back Testing the Cluster Special Considerations for Upgrade to Serviceguard A.11.19 Software UpgradesSpecial Considerations for Upgrade to Serviceguard A.11.20 How To Tell when the Cluster Re-formation Is Complete Types of UpgradeRolling Upgrade Rolling Upgrade Using DRD Restrictions for DRD Upgrades Guidelines for Rolling UpgradeNon-Rolling Upgrade Non-Rolling Upgrade Using DRD Limitations of Rolling Upgrades Performing a Rolling Upgrade Migrating cmclnodelist entries from A.11.15 or earlier Running the Rolling UpgradeKeeping Kernels Consistent Running the Rolling Upgrade Using DRD Performing a Rolling Upgrade Using DRD Halt the first node, as follows Example of a Rolling UpgradeStep Running Cluster with Packages Moved to Node Repeat the process on node 2. Halt the node, as follows Node 1 Rejoining the Cluster Performing a Non-Rolling Upgrade Guidelines for Non-Rolling Upgrade Steps for a Non-Rolling Upgrade Using DRD Performing a Non-Rolling Upgrade Using DRDLimitations of Non-Rolling Upgrades using DRD Checklist for Migration Guidelines for Migrating a Cluster with Cold Install Worksheet for Hardware Planning Power Supply WorksheetBlank Planning Worksheets Quorum Server Worksheet LVM Volume Group and Physical Volume Worksheet VxVM Disk Group and Disk Worksheet Cluster Configuration Worksheet Package Configuration Worksheet Package Configuration Worksheet Loading VxVM Migrating Volume GroupsMigrating from LVM to VxVM Data Storage Mntdg0202, respectively Customizing Packages for VxVM Restart the package Removing LVM Volume GroupsCustomizing Packages for CVM Migrating from Legacy CFS Packages to Modular CFS Packages Textual Representation of IPv6 Addresses IPv6 Network SupportIPv6 Address Types Unicast Addresses IPv6 Address PrefixIPv4 and IPv6 Compatibility IPv4 Compatible IPv6 Addresses Link-Local Addresses Aggregatable Global Unicast AddressesSite-Local Addresses Multicast Addresses Network Configuration Restrictions Local Primary/Standby LAN Patterns Example ConfigurationsNdd -get /dev/ip6 ip6nddadsolicitcount Ndd -set /dev/ip6 ip6nddadsolicitcountn Example Configurations 384 IPv6 Network Support Before Using HP Serviceguard Manager Setting Up Using Serviceguard ManagerAccessing Serviceguard Manager About the Online Help System Accessing Serviceguard Manager Launching Serviceguard ManagerScenario 1 Single cluster management Opt/hpsmh/bin/hpsmh autostart System Management Homepage with Serviceguard Manager From the left-hand panel, expand Cluster by Type Expand HP Serviceguard, and click on a Serviceguard clusterSign Maximum and Minimum Values for Parameters Membertimeout Launching Monitoring Scripts Monitoring Script for Generic ResourcesSample scripts Launching Monitoring Scripts Template of a Monitoring Script I L I T Y N C T I O N S Monitoring Script for Generic Resources Template of a Monitoring Script Migrating EMS Resources to Generic Resources Identify the equivalent SFM style resource monitor Start the package APA Index 399 Cvmactivationcmd Firstclusterlockpv LAN INONLYORINOUT, 69 Inout Pollingtarget defined Qsaddr Servicename Vxvmdg

I Using Serviceguard Manager

385

396

Index

398

Contents 17