HP Serviceguard Extension for SAP (SGeSAP) manual Option 1 SGeSAP NFS Cluster

Page 22

Whether it needs to be kept as a local copy on internal disks of each node of the cluster.

Whether it needs to be shared on a SAN storage device to allow failover and exclusive activation.

Whether it needs to provide shared access to more than one node of the cluster at the same time.

NOTE: SGeSAP packages and service monitors require SAP tools. Patching the SAP kernel sometimes also patches SAP tools. Depending on what SAP changed, this might introduce additional dependencies on shared libraries that weren't required before the patch. Depending on the SHLIB_PATH settings of the root user it might no longer be possible for SGeSAP to execute the SAP tools after applying the patch. The introduced additional libraries are not found. Creating local copies of the complete central executable directory prevents this issue.

The following sections detail the three different storage options.

Option 1: SGeSAP NFS Cluster

With this storage setup SGeSAP makes extensive use of exclusive volume group activation. Concurrent shared access is provided via NFS services. Automounter and cross-mounting concepts are used in order to allow each node of the cluster to switch roles between serving and using NFS shares. It is possible to access the NFS file systems from servers outside of the cluster, which is an intrinsic part of many SAP configurations.

Common Directories that are Kept Local

The following common directories and their files are kept local on each node of the cluster:

/home/<SID>adm — the home directory of the SAP system administrator with node specific startup log files

/usr/sap/<SID>/SYS/exe/run — the directory that holds a local copy of all SAP instance executables, libraries and tools (optional for kernel 7.x and higher)

/usr/sap/tmp — the directory in which the SAP operating system collector keeps monitoring data of the local operating system

/usr/sap/hostctrl — the directory in which SAP control services for the local host are kept (kernel 7.x and higher)

/etc/cmcluster — the directory in which Serviceguard keeps its legacy configuration files and the node specific package runtime directories

Depending on database vendor and version, it might in addition be required to keep local database client software. Details can be found in the database sections below.

Part of the content of the local group of directories must be synchronized manually between all nodes of the cluster. Serviceguard provides a tool cmcp(1m) that allows easy replication of a file to all cluster nodes.

SAP instance (startup) profile names contain either local hostnames or virtual hostnames. SGeSAP will always prefer profiles that use local hostnames to allow individual startup profiles for each host, which might be useful if the failover hardware differs in size.

In clustered SAP environments prior to 7.x releases it is required to install local executables. Local executables help to prevent several causes for package startup or package shutdown hangs due to the unavailability of the centralized executable directory. Availability of executables delivered with packaged SAP components is mandatory for proper package operation. Experience has shown that it is a good practice to create local copies for all files in the central executable directory. This includes shared libraries delivered by SAP.

NOTE: SGeSAP packages and service monitors require SAP tools. Patching the SAP kernel sometimes also patches SAP tools. Depending on what SAP changed, this might introduce additional dependencies on shared libraries that weren't required before the patch. Depending on the SHLIB_PATH settings of the root user it might no longer be possible for SGeSAP to execute the SAP tools after applying the patch. The introduced additional libraries are not found. Creating local copies of the complete central executable directory prevents this issue.

To automatically synchronize local copies of the executables, SAP components deliver the sapcpe mechanism. With every startup of the instance, sapcpe matches new executables stored centrally with those stored locally.

22 Planning the Storage Layout

Image 22
Contents Managing Serviceguard Extension for SAP Version B.05.00 Page Table of Contents SAP Supply Chain Management SGeSAP Cluster Administration 137SAP Master Data Management MDM 111 List of Figures Page List of Tables Page Printing History About this ManualRelated Documentation INSTNR, INRDesigning SGeSAP Cluster Scenarios General Concepts of SGeSAPMutual Failover Scenarios Using the Two Package Concept Mutual Failover Scenarios Using the Two Package Concept Robust Failover Using the One Package Concept Follow-and-Push Clusters with Replicated Enqueue Replicated Enqueue Clustering for Abap and Java Instances Applications/sap/enqor/SID ersINSTNRDedicated NFS Packages Handling of Redundant Dialog Instances Dedicated Failover Host Replicated Enqueue Clustering for Abap and Java Instances Planning the Storage Layout SAP Instance Storage ConsiderationsOption 1 SGeSAP NFS Cluster Common Directories that are Kept LocalDirectories that Reside on Shared Disks System and Environment Specific Volume Groups Option 2 SGeSAP NFS Idle Standby ClusterOption 3 SGeSAP CFS Cluster Database Instance Storage Considerations Directories that Reside on CFS$ORACLEHOME/common/nls/admin/data Oracle Single Instance RdbmsOracle Real Application Clusters File System Layout for NFS-based Oracle ClustersMaxdb Storage Considerations Globals IndepData=/sapdb/data IndepPrograms=/sapdb/programs Sapnfs Planning the Storage Layout Step-by-Step Cluster Conversion Pgname = pkgtypeINSTNRSIDStep-by-Step Cluster Conversion SAP Preparation SAP Pre-Installation ConsiderationsSAP Netweaver High Availability Cmmakepkg -m sgesap/db Cmmakepkg -m sgesap/ci Cmmakepkg -m sgesap/sapinstance -m ... /sap.configCmmakepkg -m sgesap/db -m sgesap/ci Cmmakepkg -m sgesap/scs Cmmakepkg -m sgesap/ersPage SAPINSTMASTERDVD/IMXOS/SAPINST/UNIX/OS Replicated Enqueue ConversionCreate a new mountpoint Volume group needs to be created for the Ascs instanceSplitting an Abap Central Instance Su sidadm Mkdir /usr/sap/SID/ASCSINSTNRIf the used SAP kernel has a release older than Sapsystemname =SIDSAPSYSTEM=INSTNR2 Instancename =ASCSINSTNRSAPLOCALHOST=DRELOC Creation of Replication Instance Volume group needs to be created for the ERS instanceSu sidadm Mkdir /usr/sap/SID/ERSINSTNR Mkdir -p exe/servicehttp/sapmc Sapstartsrv Sapcontrol ServicehttpScsid = INR PF = $DIRPROFILE/SIDERSINSTNRAREPRELOCHP-UX Configuration Cluster Filesystem Configuration Directory Structure ConfigurationLs -d /usr/sap/???/*INSTNR Installation Step IS049 Non-CFS Directory Structure Conversion Mkdir /usr/sap/SID.newMaxdb Database Step SD040 Installation Step IS050 Filename localcopy checkexistCluster Node Synchronization Open the password file, /etc/passwd, on the primary sideOpen the groupfile, /etc/group, on the primary side Look at the service file, /etc/services, on the primary side Copy the sidadm home directory to the backup nodesSu orasid Mkdir -p /oracle/SID Exit STARTPROFILE=STARTDVEBMGSINRprimarySTARTPROFILE=STARTDVEBMGSINRsecondary Su sqddbsid Mkdir -p /sapdb/DBSIDMkdir /dev/vgdbSIDmknod /dev/vgdbSID/group c 64 Cluster Node ConfigurationMkdir -p /usr/sap/tmp Iddsa Iddsa.pub Swlist grep sshSsh-keygen -t dsa Ssh hostN date Ssh -l sidadm hostN dateHosts filesNOTFOUND=continue UNAVAIL=continue TRYAGAIN=ns External Application Server Host ConfigurationConfigure /etc/nsswitch.conf to avoid problems Set -uDB=ORACLE Modular Package ConfigurationDBRELOC=0.0.0.0 Sapsystem C11 Modular Package Configuration Step-by-Step Cluster Conversion Example entries for the package configuration file Sapextinstance DVEBMGS10 Sapextsystem QAS Sbin/init.d/sapinit referenced by /sbin/rc3.d/S###sapinit Cmapplyconf -P ./sap.config Created configuration files need to be edited Legacy Package ConfigurationServiceguard Configuration Mkdir -p /etc/cmcluster/SIDServicename ciC11disp Servicename ciC11msServicefailfastenabled YES Servicehalttimeout Touch /etc/cmcluster/SID/debugResourcename /applications/sap/enqor/C11ascs Applications/sap/enqor/SIDersinstnrENQORSCSPKGNAMEC11=foobar ENQORREPPKGNAMEC11=foorep Specification of the Packaged SAP Components SGeSAP ConfigurationDistribute the package setup to all failover nodes NFSRELOC=0.0.0.0 DNAME0=D CINAME=DVEBMGS CINR=00AREPNAME=ERS AREPNR=01 AREPRELOC=0.0.0.0 DNAME1=DJCIRELOC=0.0.0.0 Configuration of Application Server HandlingJCINAME=SCS JCINR=01 REPNAME=ERS REPNR=00Page $STARTWITHPKG, $STOPWITHPKG, $RESTARTDURINGFAILOVER Restart Stop StartASPSTART=1 ASTREAT0=$RESTARTDURINGFAILOVER ASPLATFORM0=SG-PACKAGEWAITOWNAS=1 WAITOWNAS=2 Optional Parameters and Customizable FunctionsWAITOWNAS=0 RFCADAPTERSTART=1 SAPROUTERSTART0=1 SAPROUTERPORT0=-S3299RFCADAPTERCMD=runadapter.sh SAPCCMSRSTART=1 Global DefaultsSAPSTARTSRVSTART=1 SAPSTARTSRVSTOP=1 Legacy Package Configuration JMSSERVBASE=3600 HA NFS Toolkit ConfigurationEXEDIR= /usr/sap/SID/SYS/exe/runU Packagename sapnfsExport/sapmnt/SID Export/usr/sap/trans Auto FS ConfigurationHANFSSCRIPTEXTENSION=pkgtype Sapmnt/SID Usr/sap/transAUTOMASTER=/etc/automaster AUTOOPTIONS=-f $AUTOMASTER AUTOMASTER=/etc/automaster AUTOMOUNTOPTIONS=-f $AUTOMASTERAUTOMOUNTDOPTIONS= -L AUTOFS=1 NFSCLIENT=1 NFSSERVER=1 NUMNFSD=4 NUMNFSIOD=4Database Configuration Perform the following step as sidadm Additional Steps for OraclePerform the following steps as orasid If you use more than one SAP system inside of your cluster Lsnrctl start LISTENERSID1/2Additional steps for Oracle 9i Rdbms Additional Steps for MaxdbAdditional steps for Oracle 10g Rdbms SAP Application Server Configuration Rdisp/enqname = relocciSIDinstnr SAP Abap Engine specific configuration stepsCdpro Sapmnt/SID/profile/SIDINSTNAMEINRSID/dbhost = relocdb SaplocalhostsidinstnrBatch jobs can be scheduled to run on a particular instance Rdisp/starticman=TRUE SAP J2EE Engine specific installation stepsIcm/hostnamefull=relocatibleip Admin/host/SID relocdb Jdbc/pool/SID/Url jdbcoraclethin@relocdb1527SIDPage SAP Supply Chain Management More About Hot Standby Planning the Volume Manager Setup Option 1 Simple Clusters with Separated PackagesOption 2 Non-MAXDB Environments Option 3 Full FlexibilityMaxdb Storage Considerations Option 4 Hot Standby liveCacheLiveCache Installation Step LC010 HP-UX Setup for Options 1, 2 Synchronize the /etc/group and /etc/passwd filesMkdir /sapdb Do the following to continueCopy file /etc/opt/sdb to the second cluster node Mkdir -p /sapdb/data Mkdir /sapdb/LCSIDHosts filesNOTFOUND=continue UNAVAIL=continue \ TRYAGAIN=ns HP-UX Setup for OptionIf you use DNS HORCCMRCF=1 HORCMINST=0Cmmakepkg -m sgesap/livecache lcLCSID.config SGeSAP Modular Package ConfigurationPackagename lcLCSID SGeSAP Modular Package Configuration Cp /opt/cmcluster/sap/*.functions /etc/cmcluster SGeSAP Legacy Package ConfigurationCreate standard package control and configuration files Packagetype FailoverCold Admin Warm Online LCPRIMARYROLE=primarynode LCSECONDARYROLE=secondarynodeAVOIDWWIDCHECK=1 Livecache Service MonitoringLCCOPYMECHANISM=BUSINESSCOPY LCSTANDBYRESTART=1APO Setup Changes Mv .XUSER.62 .XUSER.62.ORG Default key Su lcsidadm Dbmcli -ux SAPLCSID,password -ul# dbmcli on hostname Lcsid quit For option General Serviceguard Setup ChangesSapdb/programsrelocsapnfss/export/sapdb/programs Last step is to reconfigure the cluster with cmapplyconf1m Master Data Management Overview Master Data Management User Interface ComponentsMDM Server Components SAP Netweaver XI componentsMDM SGeSAP File System Layout Installation and Configuration ConsiderationsPrerequisites Oracle/MDMMultiple MDM Serviceguard packages FOUR+ONE Single or Multiple MDM Serviceguard Package ConfigurationsSingle MDM Serviceguard Package ONE Opt/MDMMkdir -p /oracle/MDM Run ioscan and insf to probe/install new disk devicesMkdir -p /export/home/mdmuser Mkdir -p /home/mdmuser Mkdir -p /opt/MDM Installation and Configuration Considerations Nodename Runscripttimeout NotimeoutHaltscripttimeout Notimeout Runscript /etc/cmcluster/MDMNFS/mdmNFS.control.scriptScp -p /etc/auto.direct clunode2/etc/auto.direct Scp -rp /etc/cmcluster/MDMNFS clunode2/etc/cmcluster/MDMNFSSbin/init.d/nfs.client stop Sbin/init.d/nfs.client start Haltscript /etc/cmcluster/MDM/mgroupMDM.control.script Runscript /etc/cmcluster/MDM/mgroupMDM.control.scriptRunscript /etc/cmcluster/MDM/mdbMDM.control.script Runscript /etc/cmcluster/MDM/mdsMDM.control.script Haltscript /etc/cmcluster/MDM/mdbMDM.control.scriptHaltscript /etc/cmcluster/MDM/mdsMDM.control.script Runscript /etc/cmcluster/MDM/mdssMDM.control.script Runscript /etc/cmcluster/MDM/mdisMDM.control.scriptHaltscript /etc/cmcluster/MDM/mdisMDM.control.script Haltscript /etc/cmcluster/MDM/mdssMDM.control.scriptHaltscript /etc/cmcluster/MDM/masterMDM.control.script Runscript /etc/cmcluster/MDM/masterMDM.control.scriptSsh -X clunode1 -l oramdm Sidmdm KITS/ora9208/Disk1/runInstller Specify File LocationsSetup Step MDM218 SAP Master Data Management MDM MDM = Description = Addresslist = Listener = Descriptionlist = Description = Addresslist = Connectdata = Server = Dedicated Servicename = MDMSidlistlistener = Sidlist = Siddesc = Globaldbname = MDM Installation and Configuration Considerations Create a /home/mdmuser/mdss directory Global SAP Master Data Management MDM MDMLISTENERNAME=LISTENER Single Serviceguard package configure sap.configMDMUSER=mdmuser MDMMONITORINTERVAL=60 MDMPASSWORD= MDMREPOSITORYSPEC=MDMMDSRELOC=172.16.11.96 MDMMGROUPDEPEND=mdb mds mdis mdssCmapplyconf -P /etc/cmcluster/MDM/mdsMDM.config Servicefailfastenabled no ServicehalttimeoutVi /etc/cmcluster/MDM/mdsMDM.config Servicename mdsMDMmon Cmrunpkg mgroupMDM Cmhaltpkg mgroupMDMCmrunpkg masterMDM Cmhaltpkg masterMDM Change Management System Level ChangesSGeSAP Cluster Administration SAP Software Changes Swlist -l bundle T2357BA T2803BA Upgrading SAP SoftwareMixed Clusters Swlist -l bundle B7885BA T2803BACd /sapmnt/SID Ln -s /sapmnt/SIDexelocal exe Cd /sapmnt/SID Mv exe exepaCd /sapmnt/SID Mkdir exeipf Cd /sapmnt Ln -s /sapmnt/SID/exepa /sapmnt/SIDexelocal
Related manuals
Manual 35 pages 22.06 Kb