Understanding Serviceguard Software Components | HP manual

3 Understanding Serviceguard Software Components

This chapter gives a broad overview of how the Serviceguard software components work. The following topics are presented:

•Serviceguard Architecture

•How the Cluster Manager Works (page 44)

•How the Package Manager Works (page 50)

•How Packages Run

•How the Network Manager Works (page 67)

•Volume Managers for Data Storage

•Responses to Failures (page 88)

If you are ready to start setting up Serviceguard clusters, skip ahead to Chapter 5: “Building an HA Cluster Configuration” (page 158).

Serviceguard Architecture

The following figure shows the main software components used by Serviceguard. This chapter discusses these components in some detail.

NOTE: Veritas CFS may not yet be supported on the version of HP-UX you are running; see “About Veritas CFS and CVM from Symantec” (page 23).

Figure 10 Serviceguard Software Components

Serviceguard Daemons

Serviceguard uses the following daemons:

•/usr/lbin/cmclconfd—Serviceguard Configuration Daemon

•/usr/lbin/cmcld—Serviceguard Cluster Daemon

40 Understanding Serviceguard Software Components

Image 40

HP manual Understanding Serviceguard Software Components, Serviceguard Architecture, Serviceguard Daemons

Contents

Managing Serviceguard Twentieth Edition Legal Notices Contents Contents Planning and Documenting an HA Cluster Contents Building an HA Cluster Configuration 158 Contents Configuring Packages and Their Services 227 Contents Cluster and Package Maintenance 261 279 Troubleshooting Your Cluster 321 Contents Software Upgrades 357 Integrating HA Applications with Serviceguard 354349 Blank Planning Worksheets 369 Migrating from LVM to VxVM Data Storage 374377 IPv6 Network Support 378 396 Using Serviceguard Manager 385Index 398 Publishing History Publishing History Preface Securing Serviceguard and other Serviceguard white papers Related Publications Serviceguard at a Glance What is Serviceguard? Failover Shows what happens in a failover situation About Veritas CFS and CVM from Symantec Typical Cluster After Failover Using Serviceguard Manager Using SAM Roadmap for Configuring Clusters and Packages What are the Distributed Systems Administration Utilities?Usr/sbin/sam -w Tasks in Configuring a Serviceguard Cluster Understanding Serviceguard Hardware Configurations Redundancy of Cluster Components Redundant Network Components Rules and Restrictions Redundant Ethernet Configuration Redundant LANs Cross-Subnet Configurations Configuration TasksFor legacy packages, see Configuring Cross-Subnet Failover Restrictions For More Information Replacing Failed Network Cards Redundant Disk StorageSupported Disk Interfaces Disk Arrays using RAID Levels and Multiple Data Paths Data ProtectionDisk Mirroring About Multipathing Monitoring LVM Disks Through Event Monitoring Service Monitoring VxVM and CVM DisksReplacing Failed Disk Mechanisms Replacing Failed I/O Cards Sample Scsi Disk Configurations Mirrored Disks Connected for High Availability Sample Fibre Channel Disk Configuration Cluster with High Availability Disk Array Redundant Power Supplies Larger Clusters Active/Standby Model Point to Point Connections to Storage Devices Eight-Node Cluster with XP or EMC Disk Array Serviceguard Architecture Understanding Serviceguard Software ComponentsServiceguard Daemons Configuration Daemon cmclconfd Cluster Daemon cmcld Cluster Logical Volume Manager Daemon cmlvmd File Management Daemon cmfileassistdSyslog Log Daemon cmlogd Cluster Object Manager Daemon cmomd Service Assistant Daemon cmserviced Quorum Server Daemon qsNetwork Manager Daemon cmnetd Lock LUN Daemon cmdisklockd Configuring the Cluster How the Cluster Manager WorksProxy Daemon cmproxyd CFS Components Heartbeat Messages Manual Startup of Entire Cluster Automatic Cluster Startup Dynamic Cluster Re-formationCluster Quorum to Prevent Split-Brain Syndrome Cluster Lock Use of a Lock LUN or LVM Lock Disk as the Cluster Lock Lock Requirements Single Lock Disk or LUN Use of the Quorum Server as the Cluster LockDual Lock Disk No Cluster Lock Quorum Server Operation How the Package Manager Works Package TypesNon-failover Packages Failover Packages Deciding When and Where to Run and Halt Failover Packages Failover Packages’ Switching Behavior Before Package Switching Package Configuration Data Automatic Rotating Standby Rotating Standby Configuration before Failover Configurednode Policy Packages after Failover Failback Policy Automatic Failback Configuration After Failover Using Older Package Configuration Files Using the Generic Resources Monitoring Service Understanding Serviceguard Software Components Using the Event Monitoring Service Using the EMS HA MonitorsSee also Using Generic Resources to Monitor Volume Groups See also Using EMS to Monitor Volume Groups How Packages Run What Makes a Package Run? Before the Control Script Starts Legacy Package Time Line Showing Important Events During Run Script Execution Package Time Line Legacy Package While Services are Running Service Startup with cmrunservNormal and Abnormal Exits from the Run Script When a Package is Halted with a Command During Halt Script Execution Legacy Package Time Line for Halt Script Execution Error Conditions and Package Movement for Failover Packages Package Control Script Error and Exit ConditionsNormal and Abnormal Exits from the Halt Script How the Network Manager Works Stationary and Relocatable IP Addresses Adding and Deleting Relocatable IP Addresses Types of IP Addresses Monitoring LAN Interfaces and Detecting Failure Link Level Load Sharing Local Switching Cluster Before Local Network Switching Where interface is the primary interface Cmmodnet -e interface Remote Switching Monitoring LAN Interfaces and Detecting Failure IP Level Reasons To Use IP Monitoring How the IP Monitor Works Failure and Recovery Detection Times Example 1 If Local Switching is Configured Reporting Link-Level and IP-Level FailuresConstraints and Limitations See also Reporting Link-Level and IP-Level Failures Automatic Port Aggregation Example 2 If There Is No Local SwitchingCmmodnet -e lan2 What is VLAN? Vlan ConfigurationsSupport for HP-UX Vlan Volume Managers for Data Storage Configuration RestrictionsTypes of Redundant Storage Additional Heartbeat Requirements Examples of Mirrored Storage About Device File Names Device Special FilesWhite papers Physical Disks Within Shared Storage Units Multiple Devices Configured in Volume Groups Examples of Storage on Disk Arrays Multiple Paths to LUNs Types of Volume Manager HP-UX Logical Volume Manager LVMVeritas Volume Manager VxVM Propagation of Disk Groups in VxVM Veritas Cluster Volume Manager CVM Cluster Startup Time with CVMPropagation of Disk Groups with CVM For heartbeat requirements, see Redundant Heartbeat Subnets Comparison of Volume Managers Redundant Heartbeat Subnets Pros and Cons of Volume Managers with Serviceguard System Reset When a Node Fails What Happens when a Node Times OutResponses to Failures Example Responses to Hardware Failures Responses to Package and Service Failures Responses to Package and Generic Resources Failures Service Restarts Network Communication Failure Serviceguard Memory Requirements Planning and Documenting an HA ClusterGeneral Planning Planning for Expansion Sample Cluster Configuration Hardware Planning SPU Information Network InformationLAN Information Nnn.nnn.nnn.nnn Under Cluster Configuration Parameters Scsi Addressing in Cluster Configuration Disk I/O Information Diskinfo Power Supply Planning Hardware Configuration Worksheet Power Supply Configuration Worksheet Cluster Lock Planning Cluster Lock Disk and Re-formation Time Using a Quorum ServerQuorum Server Worksheet Using Generic Resources to Monitor Volume Groups LVM Planning Using EMS to Monitor Volume Groups CVM and VxVM Planning For more information, see Using the EMS HA MonitorsLVM Worksheet Cluster Configuration Planning CVM and VxVM Worksheet About Cluster-wide Device Special Files cDSFs Where cDSFs ResidePoints To Note About Easy Deployment LVM Commands and cDSFsLimitations of cDSFs Advantages of Easy Deployment Heartbeat Subnet and Cluster Re-formation TimeLimitations of Easy Deployment What Is IPv4-only Mode? What Is IPv6-Only Mode?Rules and Restrictions for IPv6-Only Mode Localhost ipv6-localhost ipv6-loopback Recommendations for IPv6-Only Mode What Is Mixed Mode?IPV6 or ANY Cluster Configuration Parameters Rules and Restrictions for Mixed ModeCluster configuration file Name of the cluster as it will appear in the output Planning and Documenting an HA Cluster Happens when You Change the Quorum Configuration Go/hpux-serviceguard-docs under HP ServiceguardIPv4-Only,IPv6-Only, and Mixed Mode page 106 for 99 and Specifying a Quorum Server IPv6-Only, and Mixed Mode page 106 for important Hpux-serviceguard-docs under HP ServiceguardSee also About Hostname Address Families IPv4-Only Their Services page 227 and these in turn must Configuration file see Configuring PackagesSitepreferred or Configuration Planning page 125 must be specified Cluster Is Running Cluster Configuration Planning You cannot change the heartbeat configuration while CVM/CFS on HP Serviceguard A.11.20 AprilProtocols and services. RPC assumes that each network To that LAN, to risk timeout without being serviced Also What Happens when You Change the Quorum Configuration Online page 49 for important informationSee IPv6 Address Types Lock LUN page 189 for more information Cluster is running, see Updating the Cluster Lock Disk When You Change the Quorum Configuration OnlineFailbackpolicy See About Package Weights page 144 for more Planning and Documenting an HA Cluster See also What Happens when a Node Times Out 88, Cluster Daemon cmcld page 41,69, Monitoring LAN Interfaces and Detecting IP-Level Failures Default is Configuration file specifies one of two ways to decideWhen a network interface card has failed How Serviceguard will handle the recovery of the primary See Monitoring LAN Interfaces and Detecting Failure IP Planning and Documenting an HA Cluster Package Configuration Planning Cluster Configuration Next StepLogical Volume and File System Planning Access Control Policies also known as Role Based Access CVM 4.1 and later without CFS CVM 4.1 and later with CFS About the Volume Monitor Using the Volume Monitor Or --help Or --versionOr --log-file Or --log-level Volumepath Planning for NFS-mounted File SystemsUsr/sbin/cmvolmond /dev/vg01/lvol1 /dev/vg01/lvol2 Usr/sbin/cmvolmond -t 10 /dev/vg00/lvol1 Package Configuration Planning Choosing Switching and Failover Behavior Package Failover Behavior Parameters for Configuring Generic Resources Configuring a Generic ResourceExtended generic resource Cmmakepkg -i $SGCONF/pkg1/pkg1.conf -m sg/genericresource Cmcheckconf -v -P $SGCONF/pkg1/pkg1.conf Cmapplyconf -P $SGCONF/pkg1/pkg1.conf Cmviewcl -v -f line -p pkg1 grep genericresource Cmrunpkg pkg1Cmgetresource -r sfmdisk Cmsetresource -r sfmdisk -s up Parameters for Configuring EMS Resources Online Reconfiguration of Generic Resources About Package Dependencies Simple DependenciesRules for Simple Dependencies Assume that we want to make pkg1 depend on pkg2 Planning and Documenting an HA Cluster Dragging Rules for Simple Dependencies Planning and Documenting an HA Cluster Extended Dependencies See Rules for differentnode and anynode Dependencies Rules for Exclusionary Dependencies What Happens when a Package Fails Rules for differentnode and anynode Dependencies About Package Weights Package Weights and Node CapacitiesConfiguring Weights and Capacities Cmmakepkg 1m manpage Weightname packagelimit weightvalue Simple MethodNodename node1 Capacityname packagelimit For pkg2 Points to Keep in Mind Comprehensive MethodDefining Capacities Clustername cluster23 Nodename node1 Nodename node2 Defining Weights Defining Default Weights Weightname B Weightvalue Weightname a Weightname a WeightvalueWeightname B Weightvalue Rules and Guidelines Cmquerycl 1m manpage About External Scripts Pevmonitoringinterval Using Serviceguard Commands in an External Script Determining Why a Package Has Shut Down About Cross-Subnet FailoverLasthaltfailed Cmviewcl -v -f line displays a lasthaltfailed flag Configuring a Package to Fail Over across Subnets Example Implications for Application Deployment Configuring a Package Next Steps Configuring nodenameConfiguring monitoredsubnetaccess Configuring ipsubnetnode Planning for Changes in Cluster Size Building an HA Cluster Configuration Installing and Updating ServiceguardWhere Serviceguard Files Are Kept Preparing Your Systems Configuring the Cluster Creating Cluster-wide Device Special Files cDSFs Before You StartCreating cDSFs for a Group of Nodes Etc/cmcluster.conf Csshsetup -r node2 Csshsetup -r -f /etc/cmcluster/sshhostsCmpreparecl -n nodename -n nodename Cmpreparecl -n node1 -n node2 -n node3 -n node4 Using Easy Deployment Displaying the cDSF ConfigurationAdding a Node to a cDSF Group Removing a Node from a cDSF Group For example Using Easy Deployment Commands to Configure the ClusterCmquerycl -N $SGCONF/mynetwork Preparing Your Systems Building an HA Cluster Configuration PVG bus1 /dev/cdisk/disk14 /dev/cdisk/disk15 Allowing Root Access to an Unconfigured Node Configuring Root-Level AccessFormat for entries in cmclnodelist is as follows Ensuring that the Root User on Another Node Is Recognized About identd Official hostname, as defined by hosts 4, for example Configuring Name ResolutionAny of the aliases. Examples Safeguarding against Loss of Name Resolution Services Ensuring Consistency of Kernel Configuration For NIS, enter two lines Tuning Network and Kernel Parameters Enabling the Network Time Protocol Creating Mirrors of Root Logical Volumes Make the new disk a boot disk Choosing Cluster Lock Disks Backing Up Cluster Lock Disk Information Setting Up a Lock LUN Creating a Disk Partition on an HP Integrity System Usr/sbin/idisk -w -p -f partition.txt /dev/rdsk/c1t4d0Usr/sbin/idisk -w -p -f partition.txt /dev/rdisk/disk12 This will create three device files, for example Excluding Devices from Probing Defining the Lock LUN Setting Up and Running the Quorum Server Creating a Storage Infrastructure with LVM Using the Generic Resources Disk Monitor Using the EMS Disk Monitor Using Mirrored Individual Data Disks Creating Volume Groups Creating Logical Volumes Setting Logical Volume TimeoutsCreating File Systems Lvchange -t 60 /dev/vg01/lvol1 Distributing Volume Groups to Other Nodes Verify the configurationDeactivating the Volume Group Distributing the Volume Group Still on ftsys9, copy the map file to ftsys10 Deactivate the volume group on ftsys10Create a directory to mount the disk Making Physical Volume Group Files Consistent Creating Additional Volume GroupsCreating a Storage Infrastructure with VxVM Converting Disks from LVM to VxVM Initializing Disks Previously Used by LVM Initializing Disks for VxVMCreating Disk Groups Deporting Disk Groups Re-Importing Disk Groups Configuring the Cluster Clearimport at System Reboot TimeHere is an example of the command enter it all one line Cmquerycl -v -C $SGCONF/clust1.conf -n ftsys9 -n ftsys10 Cmquerycl Options Speeding up the ProcessSpecifying the Address Family for the Cluster Hostnames Specifying the Address Family for the Heartbeat Specifying a Lock Disk Specifying the Cluster LockGenerating a Network Template File Full Network Probing Specifying a Lock LUN Cmquerycl -v -n ftsys9 -n ftsys10Vgchange -c y /dev/vglock See also Choosing Cluster Lock Disks Specifying a Quorum Server Obtaining Cross-Subnet InformationCmquerycl -q QSHost QSAddr -n ftsys9 -n ftsys10 -C Will produce the output such as the following Configuring the Cluster Specifying Maximum Number of Configured Packages Modifying the Membertimeout ParameterControlling Access to the Cluster Identifying Heartbeat Subnets How Access Roles Work Access Roles Levels of Access Setting up Access-Control Policies Monitor Fulladmin Packageadmin Userrole must be one of these three values Role Conflicts Username john Userhost bitUsername root Verifying the Cluster Configuration Adding Volume GroupsPackage versus Cluster Roles Distributing the Binary Configuration File Modular CFS packages v/s Legacy CFS packages Storing Volume Group and Cluster Lock Configuration DataDifferences between Legacy CFS and Modular CFS Operational commands for Legacy CFS and Modular CFS Delete a mount point, check point, or snapshot in a package Preparing the Cluster and the System Multi-node Package Cfscluster config -t 900 -sCfscluster status Creating the Disk Groups Creating the Disk Group Cluster PackagesCfsdgadm add logdata all=sw Cfsdgadm display Creating Volumes Use the vxprint command to verifyCfsdgadm showpackage logdata Vxprint logfiles For instructions on creating modular CFS packages, see Create a package configuration fileCmmakepkg -m sg/cfsall /etc/cmcluster/cfspkg1.ascii Cmcheckconf -P /etc/cmcluster/cfspkg1.ascii Apply the package configuration fileCmapplyconf -P /etc/cmcluster/cfspkg1.ascii Cmviewcl BdfCvmconcurrentdgoperations Cfsconcurrentmountunmountoperations Cmmakepkg -m sg/cfsall /etc/cmcluster/ckpt1.ascii Package. For more information, see the manpageSee the mountvxfs 1m manpage Current primary, a primary migration is triggered to Vxassist -g cvmdg3 make vol1 100m vxvol -g cvmdg3 startall Create a package configuration file for the snapshot imageCmmakepkg -m sg/cfsall snap1.ascii Mount points SnapshotmountoptionsInformation about the mount options, see Mountvxfs 1m manpage Online reconfiguration of modular CFS package parameters Cmviewcl -v -f line -p cfspkg1 Cmcheckconf -P cfspkg1.ascii Cmapplyconf -P cfspkg1.ascii Apply the configurationVerify the output Legacy Style of Packaging Modular Style of Packaging Managing Disk Groups and Mount Points Using Legacy Packages Creating Checkpoint and Snapshot Packages for CFS Fsckptadm -n create check2 /tmp/logdata/logfilesAssociate it with the cluster and mount it Cfsmount /tmp/checklogfiles It is persistent Vxassist -g dg1 make vol1 100m vxvol -g dg1 startallAssociate it with the cluster Cfsmount /local/snap1 cmviewcl You need to do the tasks described in the following sections Initializing the Veritas Volume Manager Preparing the Cluster for Use with CVMIdentifying the Master Node Initializing Disks for CVM Adding Disk Groups to the Package Configuration Usr/lib/vxvm/bin/vxdisksetup -i c4t3d4Vxdg -s init logdata c0t3d2 Mirror Detachment Policies with CVM Using Dsau during Configuration Checking Cluster Operation with Serviceguard ManagerChecking Cluster Operation with Serviceguard Commands Managing the Running Cluster Preventing Automatic Activation of LVM Volume Groups Setting up Autostart Features Changing the System Message Here is an example of the /etc/rc.config.d/cmcluster fileManaging a Single-Node Cluster Deleting the Cluster Configuration Disabling identdSingle-Node Operation Change the cmclconfd entry in /etc/inetd.conf to Building an HA Cluster Configuration Configuring Packages and Their Services Choosing Package Modules Types of Package Failover, Multi-Node, System Multi-Node Failoverpolicy Failbackpolicy Ipsubnet Ipaddress Differences between Failover and Multi-Node Packages Package Modules and ParametersBase Package Modules Cmmakepkg -m sg/all $SGCONF/sg-all Base Modules Optional Package ModulesOptional Modules Locallanfailoverallowed Package Parameter Explanations Externalscript Cmmakepkg $SGCONF/sg-all Nodename NodefailfastenabledAutorun Runscripttimeout Haltscripttimeout Successorhalttimeout ScriptlogfileOperationsequence Loglevel Priority Dependencyname Dependencycondition For more information, see About Package Dependencies Weightname, weightvalue Specifies where the dependencycondition must be metDependencylocation Monitoredsubnetaccess LocallanfailoverallowedMonitoredsubnet Clusterinterconnectsubnet Ipsubnet New for A.11.18 for both modular and legacy packagesIpsubnet Ipaddress Servicename See the package configuration file for more examplesIpsubnetnode Ipaddress Servicecmd ServicerestartServicefailfastenabled Servicehalttimeout Genericresourceevaluationtype Defines when the status of a generic resource is evaluatedGenericresourceupcriteria Resourcename Name of a resource to be monitoredResourcepollinginterval Resourcestart Enablethreadedvgchange ResourceupvalueConcurrentvgchangeoperations Cannot lock /etc/lvmconf//lvmlock still trying Cvmactivationcmd VgchangecmdVxvolcmd Killprocessesaccessingrawdevices CvmdgVxvmdg Vxvmdgretry Fsfsckopt -s Fstype vxfs ConcurrentfsckoperationsConcurrentmountandumountoperations Fsmountretrycount Fsname FsserverFsdirectory Fstype Fsmountopt FsumountoptFsfsckopt Pev Username UserhostUserrole Additional Parameters Used Only by Legacy Packages Generating the Package Configuration File Before You StartCmmakepkg Examples Mkdir $SGCONF/pkg1 Editing the Configuration File Next Step Packagetype. Enter failover, multinode, or systemmultinode See About Package Dependencies page 137 for more information Editing the Configuration File Vg vg01 Vg vg02 Verifying and Applying the Package Configuration How Control Scripts Manage VxVM Disk Groups Adding the Package to the Cluster# vxdg -tfC import dg01 Cluster and Package Maintenance Reviewing Cluster and Package StatusViewing Dependencies Cmviewcl -r A.11.16 Viewing CFS Multi-Node Information Types of Cluster and Package StatesCluster Status Node Status and State Reviewing Cluster and Package Status Unknown Down Failover and Failback Policies Examples of Cluster and Package StatesNormal Running Status Quorum Server Status CFS Package Status Status After Halting a Package Status After Moving the Package to Another Node If we use the following commandThen run cmviewcl -v, we’ll see Status After Auto Run is Enabled Output of the cmviewcl command is now as followsAfter we halt ftsys10 with the following command Status After Halting a Node Viewing Information about Unowned Packages This output can be seen on both ftsys9 and ftsys10Viewing Information about System Multi-Node Packages Status of the Packages in a Cluster File System Checking Status of the Cluster File System CFSCmviewcl -v -p SG-CFS-pkg Status of CFS Modular Disk Group and Mount Point Packages Cmviewcl -v -p mpdg1Status of Legacy CVM Disk Group Packages Ftsys9 Sw sw Checking the Cluster Configuration and Components Ftsys10Status of Legacy CFS Mount Point Packages Cfsmntadm display -v /tmp/logdata/logfiles Checking Cluster Components Etc/nsswitch.conf Etc/servicesUser-created files if you specify them Verifying Cluster Components Cmapplyconf 1m Run cmcheckconf -C Setting up Periodic Cluster Verification Managing the Cluster and NodesLimitations See the cron 1m manpage for more information Starting the Cluster When all Nodes are Down Adding Previously Configured Nodes to a Running ClusterUsing Serviceguard Commands to Start the Cluster Cmruncl -v -n ftsys9 -n ftsys10 Removing Nodes from Participation in a Running Cluster Halting the Entire ClusterCmrunnode -v ftsys8 Cmhaltnode -f -v ftsys9 What You Can Do Automatically Restarting the ClusterRules and Restrictions Managing the Cluster and Nodes Additional Points To Note Halting a Detached Package Halting a Node and Detaching its PackagesHalting the Cluster and Detaching its Packages Starting a Package Managing Packages and ServicesCmrunnode node1 Using Serviceguard Commands to Start a Package Halting a PackageStarting a Package that Has Dependencies Halting a Package that Has Dependencies Changing Package Switching Behavior Using Serviceguard Commands to Halt a PackageChanging Package Switching with Serviceguard Commands Moving a Failover Package See Performing Maintenance Using Maintenance Mode Maintaining a Package Maintenance ModeCmmodpkg -d -n lptest3 pkg1 Cluster and Package Maintenance Performing Maintenance Using Maintenance Mode Procedure Excluding Modules in Partial-Startup Maintenance Mode Cmrunpkg -m sg/packageip pkg1 Reconfiguring a Cluster Cmrunpkg -e sg/service pkg1Cmrunpkg -m sg/services -e sg/packageip pkg1 Types of Changes to the Cluster Configuration Change to the Cluster Configuration Previewing the Effect of Cluster Changes Using Preview mode for Commands and in Serviceguard Manager Mode see Maintaining a Package Maintenance ModeWhat You Can Preview Cmmodpkg -e -t pkg1 Cmeval -v newstate.in Using cmevalYou would see output something like this Updating the Cluster Lock Configuration Reconfiguring a Halted ClusterUpdating the Cluster Lock Disk Configuration Online Updating the Cluster Lock LUN Configuration Online Reconfiguring a Running Cluster Cmapplyconf -C clconfig.asciiAdding Nodes to the Cluster While the Cluster is Running Cmgetconf -c cluster1 temp.ascii Cmquerycl -C clconfig.ascii -c cluster1 -n ftsys8 -n ftsys9 What You Can Do What You Must Keep in Mind Cmquerycl -c cluster1 -C clconfig.ascii Example Adding a Heartbeat LAN Cmgetconf clconfig.ascii Removing a LAN or Vlan Interface from a Node Changing the VxVM or CVM Storage Configuration Changing the LVM Configuration while the Cluster is RunningSee also Replacing LAN or Fibre Channel Cards Creating the Legacy Package Configuration Configuring a Legacy PackageCmgetconf -c clustername clconfig.ascii Editing the Package Configuration File Configuring a Package in StagesMkdir /etc/cmcluster/pkg1 Cluster and Package Maintenance Creating the Package Control Script Cmmakepkg -s /etc/cmcluster/pkg1/pkg1.sh Customizing the Package Control Script Adding Serviceguard Commands in Customer Defined Functions Support for Additional Products Verifying the Package Configuration Distributing the ConfigurationCopying Package Control Scripts with HP-UX commands Cmcheckconf -v -P /etc/cmcluster/pkg1/pkg1.conf Configuring Cross-Subnet Failover Configuring nodename Reconfiguring a Package Configuring monitoredsubnetaccessCreating Subnet-Specific Package Control Scripts IP0 = SUBNET0 IP1 = SUBNET1 Migrating a Legacy Package to a Modular Package Reconfiguring a Package on a Running ClusterCmgetconf -p pkg1 pkg1.conf Reconfiguring a Package on a Halted Cluster Adding a Package to a Running Cluster Deleting a Package from a Running Cluster Cmhaltpkg mypkg Cmdeleteconf -p mypkgCmapplyconf -v -P app1.conf Unmount the shared file system cfsumount mount point Allowable Package States During Reconfiguration Resetting the Service Restart CounterCmmodpkg -R -s myservice pkg1 Change servicerestart modular package Types of Changes to Packages Locallanfailoverallowed Change vxvolcmd Cfsmountoptions Responding to Cluster Events Changes that Will Trigger Warnings Removing Serviceguard from a System Disabling ServiceguardSingle-Node Operation Troubleshooting Your Cluster Testing Cluster OperationStart the Cluster using Serviceguard Manager Testing the Package Manager Testing the Network Manager Testing the Cluster ManagerMonitoring Hardware Using System Fault Management Service Using Event Monitoring ServiceUsing EMS Event Monitoring Service Hardware Monitors Hardware Monitors and Persistence Requests Using HP Isee HP Instant Support Enterprise Edition Replacing a Faulty Array MechanismReplacing a Faulty Mechanism in an HA Enclosure Replacing Disks Replacing a Lock Disk Replacing a Lock LUN Online Hardware Maintenance with In-line Scsi Terminator Cmdisklock reset /dev/dsk/c0t1d1Replacing I/O Cards Replacing Scsi Host Bus Adapters Online Replacement Offline ReplacementReplacing LAN or Fibre Channel Cards Replacing a Failed Quorum Server System After Replacing the Card Troubleshooting Approaches Using cmquerycl and cmcheckconf Using cmviewclReviewing Package IP Addresses Reviewing the System Log File Reviewing Object Manager Log Files Sample System Log EntriesFollowing is an example of a successful package starting Cmreadlog /var/opt/cmom/cmomd.log Reviewing Serviceguard Manager Log Files Reviewing Configuration FilesUsing the cmcheckconf Command Reviewing the System Multi-node Package Files Solving Problems Using the cmviewconf CommandReviewing the LAN Configuration Serviceguard Command Hangs Cluster Re-formations Caused by Temporary Conditions Networking and Security Configuration ErrorsNslookup ftsys9 System Administration Errors Package Control Script Hangs or Failures Fuser -kulogical-volume umount logical-volume Problems with Cluster File System CFS Llt, gab Vxfen W cvm Cfs Problems with VxVM Disk Groups Package Movement ErrorsNode and Network Failures Force Import and Deport After Node Failure Troubleshooting the Quorum Server Authorization File ProblemsTimeout Problems Access denied to quorum server Messages Enterprise Cluster Master Toolkit Designing Highly Available Cluster Applications Automating Application Operation Insulate Users from Outages Controlling the Speed of Application FailoverDefine Application Startup and Shutdown Use Raw Volumes Replicate Non-Data File SystemsEvaluate the Use of JFS Minimize Data Loss Use Restartable Transactions Use CheckpointsDesign for Multiple Servers Balance Checkpoint Frequency with Performance Design for Replicated Data Sites Designing Applications to Run on Multiple SystemsAvoid Node-Specific Information Avoid Using SPU IDs or MAC Addresses Assign Unique Names to ApplicationsObtain Enough IP Addresses Allow Multiple Instances on Same System Bind to a Fixed Port Use uname2 With CareBind to Relocatable IP Addresses Give Each Application its Own Volume Group Use Multiple Destinations for SNA ApplicationsAvoid File Locking Call bind before connect Usr/sbin/route add net default 128.17.17.1 1 source Etc/rc.config.d/nddconf as followsHelp menu for ndd -h ipstrongesmodel Usr/sbin/route delete net default 128.17.17.1 1 source Restoring Client Connections Create Applications to be Failure Tolerant Handling Application FailuresBe Able to Monitor Applications Reducing Time Needed for Application Upgrades and Patches Provide for Rolling UpgradesMinimizing Planned Downtime Do Not Change the Data Layout Between Releases Providing Online Application Reconfiguration Documenting Maintenance Operations Checklist for Integrating HA Applications Integrating HA Applications with ServiceguardDefining Baseline Application Behavior on a Single System Integrating HA Applications in Multiple Systems Testing the Cluster Move it back Special Considerations for Upgrade to Serviceguard A.11.20 Software UpgradesSpecial Considerations for Upgrade to Serviceguard A.11.19 Types of Upgrade How To Tell when the Cluster Re-formation Is CompleteRolling Upgrade Rolling Upgrade Using DRD Guidelines for Rolling Upgrade Restrictions for DRD UpgradesNon-Rolling Upgrade Non-Rolling Upgrade Using DRD Performing a Rolling Upgrade Limitations of Rolling Upgrades Keeping Kernels Consistent Running the Rolling UpgradeMigrating cmclnodelist entries from A.11.15 or earlier Performing a Rolling Upgrade Using DRD Running the Rolling Upgrade Using DRD Step Example of a Rolling UpgradeHalt the first node, as follows Running Cluster with Packages Moved to Node Node 1 Rejoining the Cluster Repeat the process on node 2. Halt the node, as follows Guidelines for Non-Rolling Upgrade Performing a Non-Rolling Upgrade Limitations of Non-Rolling Upgrades using DRD Performing a Non-Rolling Upgrade Using DRDSteps for a Non-Rolling Upgrade Using DRD Guidelines for Migrating a Cluster with Cold Install Checklist for Migration Blank Planning Worksheets Power Supply WorksheetWorksheet for Hardware Planning LVM Volume Group and Physical Volume Worksheet Quorum Server Worksheet Cluster Configuration Worksheet VxVM Disk Group and Disk Worksheet Package Configuration Worksheet Package Configuration Worksheet Migrating from LVM to VxVM Data Storage Migrating Volume GroupsLoading VxVM Customizing Packages for VxVM Mntdg0202, respectively Customizing Packages for CVM Removing LVM Volume GroupsRestart the package Migrating from Legacy CFS Packages to Modular CFS Packages IPv6 Address Types IPv6 Network SupportTextual Representation of IPv6 Addresses IPv6 Address Prefix Unicast AddressesIPv4 and IPv6 Compatibility IPv4 Compatible IPv6 Addresses Aggregatable Global Unicast Addresses Link-Local AddressesSite-Local Addresses Multicast Addresses Network Configuration Restrictions Example Configurations Local Primary/Standby LAN PatternsNdd -get /dev/ip6 ip6nddadsolicitcount Ndd -set /dev/ip6 ip6nddadsolicitcountn Example Configurations 384 IPv6 Network Support Using Serviceguard Manager Before Using HP Serviceguard Manager Setting UpAccessing Serviceguard Manager About the Online Help System Launching Serviceguard Manager Accessing Serviceguard ManagerScenario 1 Single cluster management Opt/hpsmh/bin/hpsmh autostart System Management Homepage with Serviceguard Manager Sign Expand HP Serviceguard, and click on a Serviceguard clusterFrom the left-hand panel, expand Cluster by Type Membertimeout Maximum and Minimum Values for Parameters Sample scripts Monitoring Script for Generic ResourcesLaunching Monitoring Scripts Launching Monitoring Scripts Template of a Monitoring Script I L I T Y N C T I O N S Monitoring Script for Generic Resources Template of a Monitoring Script Migrating EMS Resources to Generic Resources Start the package Identify the equivalent SFM style resource monitor Index APA 399 Cvmactivationcmd Firstclusterlockpv LAN INONLYORINOUT, 69 Inout Pollingtarget defined Qsaddr Servicename Vxvmdg