155 | HP Serviceguard Extension for RAC (SGeRAC) manual

T

temp.dbf

Oracle demo database files, 45, 58 troubleshooting

monitoring hardware, 131 replacing disks, 132

U

upgrade DRD, 149

upgrade restrictions DRD, 149

V

volume group

creating for a cluster, 42

creating physical volumes for clusters, 41 volume groups

adding shared volume groups, 130 displaying for RAC, 46 exporting to other nodes, 46

making changes to shared volume groups, 128 making shareable, 127

making unshareable, 128

W

worksheet

logical volume planning, 31 worksheets

physical volume planning, 151 worksheets for planning

blanks, 151

155

Image 155

HP Serviceguard Extension for RAC (SGeRAC) manual 155

Contents

Using Serviceguard Extension for RAC Version A.11.20 Legal Notices Contents Contents SGeRAC Toolkit for Oracle RAC 10g or later Support of Oracle RAC ASM with SGeRAC Maintenance 113 Index 153 Troubleshooting 138 Software Upgrades 139Blank Planning Worksheets 151 Advantages of using SGeRAC User Guide Overview Typographical Conventions Problem Reporting Where to find Documentation on the Web What is a Serviceguard Extension for RAC Cluster? Introduction to Serviceguard Extension for RAC Group Membership Using Packages in a Cluster Group Membership Daemon Serviceguard Extension for RAC Architecture About Veritas CFS and CVM from Symantec Storage Configuration OptionsServiceguard Extension for RAC Package Dependencies SMS bundle Overview of SGeRAC and Oracle 10g, 11gR1, and 11gR2 RAC How Cluster Interconnect Subnet Works Overview of SGeRAC Cluster Interconnect Subnet Monitoring Configuring Packages for Oracle Listeners Configuring Packages for Oracle RAC Instances Before Node Failure Node Failure Up to Four Nodes with Scsi Storage Larger Clusters Four-Node RAC Cluster Point-to-Point Connections to Storage Devices Eight-Node Cluster with EVA, XP or EMC Disk Array GMS Authorization Administering Clusters with Serviceguard Manager Overview of Serviceguard ManagerStarting Serviceguard Manager Monitoring Clusters with Serviceguard Manager Configuring Clusters with Serviceguard Manager Group Membership API NMAPI2 Cluster TimeoutsServiceguard Cluster Timeout Interface Areas Shared Storage Oracle Cluster Software Listener Automated Startup and Shutdown RAC InstancesNetwork Monitoring Manual Startup and Shutdown Network Planning for Cluster Communication Planning Storage for Oracle 10g/11gR1/11gR2 RAC Planning Storage for Oracle Cluster Software Storage Planning with CFS Volume Planning with SlvmVolume Planning with CVM RAW Logical Volume Name Size MB About Multipathing Installing Serviceguard Extension for RACAbout Device Special Files About Cluster-wide Device Special Files cDSFs Where cDSFs Reside Cluster Communication Network Monitoring Configuration File Parameters Single Network for Cluster Communications Single Network for Cluster Communications SG-HB/RAC-IC Traffic Separation Alternate Configuration-Multiple RAC Databases Guidelines for Changing Cluster Parameters When Cluster Interconnect Subnet Monitoring is usedWhen Cluster Interconnect Subnet Monitoring is not Used Cluster Interconnect Monitoring Restrictions Creating a Storage Infrastructure with LVMLimitations of Cluster Communication Network Monitor HP Serviceguard Extension for RAC Building Volume Groups for RAC on Mirrored DisksCreating Volume Groups and Logical Volumes # ls -l /dev/*/group Building Mirrored Logical Volumes for RAC with LVM Commands# mkdir /dev/vgrac # mknod /dev/vgrac/group c 64 0xhh0000 # lvcreate -m 1 -M y -s g -n system.dbf -L 408 dev/vgrac Creating Mirrored Logical Volumes for RAC Data Files# lvcreate -m 1 -M y -s g -n redo1.log -L 408 /dev/vgrac Creating RAC Volume Groups on Disk Arrays Oracle Demo Database Files Creating Logical Volumes for RAC on Disk Arrays Exporting with LVM Commands Displaying the Logical Volume InfrastructureExporting the Logical Volume Infrastructure Creating a Storage Infrastructure with CFS Installing Oracle Real Application Clusters # cmruncl # cmviewcl Initializing the Veritas Volume Manager# cmapplyconf -C clm.asc # vxdg -s init cfsdg1 c4t4d0 # cfscluster config -s# vxdctl -c mode # /etc/vx/bin/vxdisksetup -i c4t4d0 #cfsmntadm add cfsdg1 vol2 /cfs/mnt2 all=rw # newfs -F vxfs /dev/vx/rdsk/cfsdg1/vol2# newfs -F vxfs /dev/vx/rdsk/cfsdg1/vol3 # cfsmntadm add cfsdg1 vol1 /cfs/mnt1 all=rw # cfsmntadm delete /cfs/mnt2 Deleting CFS from the Cluster# bdf grep cfs # cfsmntadm delete /cfs/mnt1 # cfsdgadm delete cfsdg1 # cfscluster unconfig# vxinstall Creating a Storage Infrastructure with CVM Using CVM 5.x or later # vxdg -s init opsdg c4t4d0 # cmapplyconf -P /etc/cmcluster/cvm/VxVM-CVM-pkg.conf Using CVMPreparing the Cluster for Use with CVM # vxedit set diskdetpolicy=global DiskGroupName # vxdg list # /usr/lib/vxvm/bin/vxdisksetup -i /dev/dsk/c0t3d2# cmruncl # vxdg -s init opsdg c0t3d2 # vxassist -g opsdg make logfiles 1024m Creating VolumesOracle Demo Database Files Mirror Detachment Policies with CVM Adding Disk Groups to the Cluster Configuration # mkdir -p /mnt/app/crs/oracle/product/10.2.0/crs # groupadd oinstall # groupadd dba # groupadd oper# useradd -u 203 -g oinstall -G dba,oper oracle # passwd oracle # chmod 755 /dev/vgrac # usermod -d /cfs/mnt1/oracle oracle Mnt/app/oracle/oradata/ver10/ver10raw.conf Installing RAC Binaries on a Local File System Installing Oracle 10g, 11gR1, or 11gR2 Cluster SoftwareInstalling Oracle 10g/11gR1/11gR2 RAC Binaries Installing on Local File System $ netca Installing RAC Binaries on Cluster File SystemCreating a RAC Demo Database Creating a RAC Demo Database on Slvm or CVM #swlist VRTSodm Verifying Oracle Disk Manager is ConfiguredCreating a RAC Demo Database on CFS # cat /dev/odm/stats Configuring Oracle to Use Oracle Disk Manager LibraryVerify that Oracle Disk Manager is Running #ll -L /opt/VRTSodm/lib/libodm.sl $ cd $ORACLEHOME/lib # kcmodule -P state odm $ORACLEHOME/bin/srvctl modify database -d dbname -y manual Preparing Oracle Cluster Software for Serviceguard PackagesConfigure Serviceguard Packages $ORACRSHOME/bin/crsctl disable crs Sbin/init.d/init.crs start Overview Why ASM over SLVM?Introduction SG/SGeRAC Support for ASM on HP-UX 11i Configuring Slvm Volume Groups for ASM Disk Groups Sample Command Sequence for Configuring Slvm Volume Groups Support of Oracle RAC ASM with SGeRAC ASM support with SG/SGeRAC A.11.17.01 or later ASM over Slvm Configuring Slvm Volume Groups for ASM Disk Groups Sample Command Sequence for Configuring Slvm Volume Groups ASM over Raw disk Additional Hints on ASM Integration with SGeRAC Installation, Configuration, Support, and Troubleshooting ASM may require Modified Backup/Restore Procedures Additional Documentation on the Web and Scripts Background SGeRAC Toolkit for Oracle RAC 10g or later Background SGeRAC Toolkit for Oracle RAC 10g or later Page Serviceguard Extension for RAC Toolkit operation Startup and shutdown of the combined Oracle RAC-SGeRAC stack SGeRAC Toolkit for Oracle RAC 10g or later Asmdg MNP Use case 3 Database storage in ASM over Slvm Use Case 2 Setup Use Case 2 performing maintenance with ASM disk groups Use Case 1 Performing maintenance with Oracle Clusterware SGeRAC Toolkit for Oracle RAC 10g or later Internal structure of SGeRAC for Oracle Clusterware Support for the SGeRAC Toolkit Contents Support for the SGeRAC Toolkit CFS-DG1-MNP CFS-DG2-MNP Readme SGeRAC Toolkit for Oracle RAC 10g or later DEPENDENCYCONDITIONDG-MNP-PKG=UP Dependencylocationsamenode ORACLEHOME, CHECKINTERVAL...MAINTENANCEFLAG Dependencyname DEPENDENCYCONDITIONOC-MNP-PKG=UP Dependencylocationsamenode Set to any name desired for the RAC MNP For a package using CFS Oracrshome Asmdisk Group Asmtkitdir SGeRAC Toolkit for Oracle RAC 10g or later Asmdg MNP SGeRAC Toolkit for Oracle RAC 10g or later Support for the SGeRAC Toolkit SGeRAC Toolkit for Oracle RAC 10g or later DEPENDENCYCONDITIONOC-DGMP-PKG=UP Dependencylocationsamenode Additional Documentation on the Web Conclusion # cmviewcl -r A.11.16 Types of Cluster and Package StatesExamples of Cluster and Package States Name Status State Types of Cluster and Package States Package Status and State Cluster StatusNode Status and State Status of Group Membership Package Switching AttributesService Status Normal Running Status Examples of Cluster and Package StatesNetwork Status Failover and Failback Policies CVM Package Status Quorum Server Status Status After Moving the Package to Another Node Output of the cmviewcl -vcommand is as follows Viewing Data on Unowned Packages Status After Package Switching is EnabledStatus After Halting a Node Status Nodename Name Checking the Cluster Configuration and Components Verifying Cluster Components Checking Cluster Components Cmcheckconf -P Example Setting up Periodic Cluster Verification Online Node Addition and Deletion Online ReconfigurationLimitations Managing the Shared Storage Making LVM Volume Groups Shareable Deactivating a Shared Volume Group Activating an LVM Volume Group in Shared ModeMaking Offline Changes to Shared Volume Groups Making a Volume Group Unshareable On node 2, issue the following command Changing the CVM Storage Configuration Removing Serviceguard Extension for RAC from a SystemAdding Additional Shared LVM Volume Groups Adding Disk Hardware Using Event Monitoring ServiceUsing EMS Hardware Monitors Monitoring Hardware # lvreduce -m 0 /dev/vgsg01/lvolname /dev/dsk/c2t3d0 Replacing a Mechanism in a Disk Array Configured with LVM# vgcfgrestore /dev/vgsg01 /dev/dsk/c2t3d0 Replacing Disks # pvchange -a n pv path # vgcfgrestore -n vg name pv raw path# lvextend -m 1 /dev/vgsg01 /dev/dsk/c2t3d0 # lvsync /dev/vgsg01/lvolname Replacing a Lock Disk Online Hardware Maintenance with Inline Scsi Terminator W Scsi Buses with Inline Terminators Online Replacement Replacement of I/O CardsReplacement of LAN Cards Offline Replacement Monitoring RAC Instances Troubleshooting Rolling Software Upgrades Software Upgrades Upgrading Serviceguard to SGeRAC cluster Cmhaltnode nodename Autostartcmcld = Steps for Rolling Upgrades Keeping Kernels Consistent Example of Rolling Upgrade Step Running Cluster Before Rolling Upgrade Node 1 Upgraded to SG/SGeRAC # cmhaltnode -f node2 Running Cluster After Upgrades Limitations of Rolling Upgrades Migrating an SGeRAC Cluster with Cold Install Non-Rolling Software UpgradesLimitations of Non-Rolling Upgrades Restrictions for DRD Upgrades Upgrade Using DRDRolling Upgrade Using DRD Non-Rolling Upgrade Using DRD Software Upgrades Oracle Logical Volume Worksheet LVM Volume Group and Physical Volume Worksheet Blank Planning Worksheets Index RAC 155