HP Veritas Volume Manager 5.0 -UX 11i v3 399

400Administering cluster functionality

Overview of cluster volume management

Figure 13-1

Example of a 4-node cluster

Redundant private network

Node 0

Node 1

Node 2

Node 3

(master)

(slave)

Redundant SCSI or Fibre Channel connectivity

Cluster-shareable

disks

Cluster-shareable disk

groups

To the cluster monitor, all nodes are the same. VxVM objects configured within shared disk groups can potentially be accessed by all nodes that join the cluster. However, the cluster functionality of VxVM requires that one node act as the master node; all other nodes in the cluster are slave nodes. Any node is capable of being the master node, and it is responsible for coordinating certain VxVM activities.

Note: You must run commands that configure or reconfigure VxVM objects on the master node. Tasks that must be initiated from the master node include setting up shared disk groups, creating and reconfiguring volumes, and performing snapshot operations.

VxVM determines that the first node to join a cluster performs the function of master node. If the master node leaves a cluster, one of the slave nodes is chosen to be the new master. In “Example of a 4-node cluster ,” node 0 is the master node and nodes 1, 2 and 3 are slave nodes.

Contents

Page Legal Notices Page Chapter Understanding Veritas Volume Manager Administering disks Administering dynamic multipathing (DMP) Creating and administering disk groups Chapter 5 Creating and administering subdisks Creating and administering plexes Creating volumes Administering volumes Administering volume snapshots Creating and administering volume sets Configuring off-hostprocessing Administering hot-relocation Administering cluster functionality Administering sites and remote mirrors Using Storage Expert Performance monitoring and tuning Appendix A Commands summary Appendix B Configuring Veritas Volume Manager Glossary Index Understanding Veritas Volume Manager Page VxVM and the operating system How VxVM handles storage management VxVM accesses all disks as entire physical disks without partitions Disk arrays balance Multipathed disk arrays Device discovery Enclosure-basednaming channel hub or switch enc0_0 enc0_1 Page for details of the standard and the enclosure-based naming schemes, and how to switch between them Virtual objects Virtual objects in VxVM include the following: ■Disk groups ■VM disks ■Subdisks ■Volumes are composed of one or more plexes vol01 vol02 Disk groups VM disks Subdisks Plexes Volumes Page Page Volume layouts in VxVM Layout methods Concatenation and spanning Page Page Striping (RAID-0) Figure 1-14Striping across three columns Stripe su1 su2 su3 Page Page Mirroring (RAID-1) Mirroring different Disk duplexing Striping plus mirroring (mirrored-stripeor RAID-0+1) Mirroring plus striping (striped-mirror, RAID-1+0or RAID-10) Page RAID-5(striping with parity) Data Data Parity Data Parity Data Parity Data Data Traditional RAID-5arrays Veritas Volume Manager RAID-5arrays Left-symmetriclayout Page RAID-5logging Layered volumes Page Page Online relayout Page Page Figure 1-30Example of increasing the stripe width for the columns in a volume Limitations of online relayout Note the following limitations of online relayout: ■Log plexes cannot be transformed Online relayout involving ■The number of mirrors in a mirrored volume cannot be changed using relayout Transformation characteristics You can determine the transformation direction by using the Transformations and volume length Resizing a volume Volume resynchronization Dirty region logging Log subdisks and plexes log plex Sequential DRL voldrl_max_seq_dirty Note: DRL adds a small I/O overhead for most write access patterns SmartSync recovery accelerator Data volume configuration Volume snapshots Page Comparison of snapshot features FastResync FastResync enhancements FastResync provides two fundamental enhancements to VxVM: detached returning mirror FastResync allows you to refresh and How non-persistentFastResync works with snapshots Persistent FastResync DCO volume versioning Version 0 DCO volume layout Version 20 DCO volume layout where the size of each map in bytes is: How persistent FastResync works with snapshots Figure 1-32Mirrored volume with persistent FastResync enabled Mirrored volume DCO DCO plex plex Figure 1-33Mirrored volume after completion of a snapstart operation snap objects For traditional snapshots, the For Effect of growing a volume on the FastResync map FastResync limitations The following limitations apply to FastResync: Persistent FastResync is supported for Neither Hot-relocation Volume sets Page Administering disks Disk devices Disk devices /dev/rdisk /dev/dsk Disk device naming in VxVM There are two different methods of naming disk devices: ■Operating system-basednaming ■Enclosure-basednaming Operating system-basednaming Page Private and public disk regions Most VM disks have two regions: ” on page 205) smallest the public area following the private area) When the vxconfigd daemon is started, VxVM obtains a list of known disk device addresses from the operating system and configures disk access records for them automatically cdsdisk disk that is suitable for moving between different operating Discovering and configuring newly added disk devices except #vxdisk scandisks !device=c1t1d0,c2t2d0 #vxdisk scandisks !ctlr=c1,c2 #vxdisk scandisks pctlr=8/12.8.0.255.0 Discovering disks and dynamically adding disk arrays third-party vendor. The support comes in the form of vendor-supplied libraries, and is added to an Adding support for a new disk array Enabling discovery of new devices Removing support for a disk array Third-partydriver coexistence Administering the Device Discovery Layer Listing details of supported disk arrays Excluding support for a disk array library Re-includingsupport for an excluded disk array library Listing excluded disk arrays Listing supported disks in the DISKS category Adding unsupported disk arrays to the DISKS category [length=serialno_length] [policy=ap] vendorid productid FUJITSU IBM Removing disks from the DISKS category Adding foreign devices Placing disks under VxVM control Changing the disk-namingscheme The following examples illustrate the use of command: Regenerating persistent device names Changing device naming for TPD-controlledenclosures #vxdmpadm setattr enclosure enclosure tpdmode=native|pseudo native Discovering the association between enclosure and OS based disk names #vxdisk list enclosure-based_name Persistent simple or nopriv disks in the boot disk group Installing and formatting disks Displaying and changing default disk layout attributes Adding a disk to VxVM Adding a disk to VxVM <pattern-list c3t0d0 c3t1d0 c3t2d0 c3t3d0 specifies fours disks at separate target IDs on controller To continue with the operation, enter Here are the disks selected. Output format: [Device] list of device names Continue operation? [y,n,q,?] (default: y) y Which disk group [<group>,none,list,q,?] step site_name If one or more disks already contains a file system At the following prompt vxdiskadm then proceeds to add the disks Reinitializing a disk Using vxdiskadd to place a disk under control of VxVM #vxdiskadd c0t1d0 Rootability VxVM root disk volume restrictions Root disk mirrors Booting root volumes Setting up a VxVM root disk and mirror primary #/etc/vx/bin/vxcp_lvmroot -R30 -v -bc0t4d0 #/etc/vx/bin/vxcp_lvmroot -mc1t1d0 -R30 -v -bc0t4d0 alternate #/etc/vx/bin/vxrootmir -v -bc1t1d0 Creating an LVM root disk from a VxVM root disk vxres_lvmroot #/etc/vx/bin/vxdestroy_lvmroot -vc0t1d0 #/etc/vx/bin/vxres_lvmroot -v -bc0t1d0 The -b option to vxres_lvmroot sets c0t1d0 as the primary boot device Adding swap volumes to a VxVM rootable system Adding persistent dump volumes to a VxVM rootable system Dynamic LUN expansion Page Removing disks Removing a disk with subdisks Removing a disk from VxVM control Removing and replacing disks To replace a disk At the following prompt, enter the name of the disk to be replaced (or enter Remove a disk for replacement Menu: VolumeManager/Disk/RemoveForReplace home src 114Administering disks The following devices are available as replacements: c0t1d0 You can choose one of these disks now, to replace mydg02 Select “none” if you do not wish to select a replacement disk VxVM NOTICE V-5-2-265Removal of disk mydg02 completed successfully Replacing a failed or removed disk To specify a disk that has replaced a failed or removed disk 116Administering disks c0t1d0 c1t1d0 You can choose one of these disks to replace mydg02 Choose "none" to initialize another disk to replace mydg02 Replace another disk? [y,n,q,?] (default: n) Enabling a disk Taking a disk offline Renaming a disk Reserving disks Displaying disk information Displaying disk information with vxdiskadm To display disk information Start the list (List disk information) all Controlling Powerfail Timeout Enabling or disabling PFTO Page Administering dynamic multipathing (DMP) How DMP works Page naming c2t99d0 enc0_0 How DMP monitors I/O on paths I/O throttling Path failover mechanism I/O throttling Path failover mechanism I/O throttling Load balancing DMP coexistence with HP-UXnative multipathing See “Changing the disk-namingscheme” on page See “Adding foreign devices” on page Migrating between DMP and HP-UXnative multipathing You can use the 3Restart all the volumes in each disk group: #vxvol -g diskgroup startall To migrate from HP-UXnative multipathing to DMP #vxddladm rmforeign blockdir=/dev/disk chardir=/dev/rdisk DMP in a clustered environment Disabling and enabling multipathing for specific devices ◆Select option 2 to exclude specified paths from the view of VxVM Enabling multipathing and making devices visible to VxVM Select menu item 18 main menu to re-enable ◆Select option 2 to make specified paths visible to VxVM ◆Select option 6 to enable multipathing for specified paths Enabling and disabling I/O for controllers and storage processors Displaying DMP database information Displaying the paths to a disk Page Administering DMP using vxdmpadm The above command displays output such as the following: #vxdmpadm getdmpnode enclosure=enc0 Displaying the members of a LUN group #vxdmpadm getlungroup dmpnodename=c11t0d10 Displaying paths controlled by a DMP node, controller or array port command combined with the directory: Displaying information about controllers operations being disabled on that controller by using the #vxdmpadm listctlr enclosure=enc0 type=ACME Displaying information about enclosures #vxdmpadm listenclosure enc0 The following command lists attributes for all enclosures in a system: The following is example output from this command: Displaying information about array ports Displaying information about TPD-controlleddevices Gathering and displaying I/O statistics To enable the gathering of statistics, enter this command: To reset the I/O counters to zero, use this command: memory 32k Page Setting the attributes of the paths to an enclosure Displaying the I/O policy Specifying the I/O policy Page 2 up to 2^31 as illustrated in the table below: Partition size in blocks Equivalent size in bytes dmp_pathswitch_blks_shift # vxdmpadm setattr enclosure enc0 iopolicy=balanced Page #vxdmpadm setattr arrayname DISK iopolicy=singleactive Scheduling I/O on the paths of an Asymmetric Active/Active array You can specify the attribute in conjunction with the (A/A-A) A/A-A Page Disabling I/O for paths, controllers or array ports Enabling I/O for paths, controllers or array ports Upgrading disk controller firmware Renaming an enclosure 156Administering dynamic multipathing (DMP) Configuring the response to I/O failures #vxdmpadm getattr {enclosure enc-name|arrayname name|arraytype type} \ recoveryoption See “Displaying recoveryoption values” on page 159 for more information #vxdmpadm getattr enclosure enc0 recoveryoption Configuring the I/O throttling mechanism {enclosure enc-name|arrayname name|arraytype type}\ recoveryoption {enclosure enc-name|arrayname name|arraytype type}\ recoveryoption=nothrottle 158Administering dynamic multipathing (DMP) #vxdmpadm setattr enclosure enc0 recoveryoption=nothrottle iotimeout queuedepth Note: The iotimeout and queuedepth attributes are mutually exclusive Displaying recoveryoption values Recovery option Possible settings Description 160Administering dynamic multipathing (DMP) Configuring DMP path restoration policies vxdmpadm disable Use the start restore command to configure one of the following policies: start restore ■check_all Stopping the DMP path restoration thread Displaying the status of the DMP path restoration thread Displaying information about the DMP error-handlingthread Configuring array policy modules Page Page Creating and administering disk groups Page Specifying a disk group to commands Rules for determining the default disk group Displaying the system-wideboot disk group Displaying and specifying the system-widedefault disk group Displaying disk group information Disk: c0t12d0 Creating a disk group Adding a disk to a disk group Removing a disk from a disk group Deporting a disk group Importing a disk group Handling disks with duplicated identifiers udid_mismatch c2t68d0 Writing a new UDID to a disk #vxdisk [-f] [-g diskgroup] updateudid disk #vxdisk updateudid c2t66d0 c2t67d0 Importing a disk group containing cloned disks option to the vxdg import command, as shown in this nconfig=all nlog=all Sample cases of operations on cloned disks Enabling configuration database copies on tagged disks These tags can be viewed by using the vxdisk listtag command: # vxdisk listtag t1: Importing cloned disks without tags #vxdg -nnewdg -ouseclonedev=on -oupdateid import mydg online udid_mismatch To import only the cloned disks into the mydg disk group: #vxdg -ouseclonedev=on -oupdateid import mydg EMC0_27 Importing cloned disks with tags to online clone_disk #vxdisk set EMC0_8 clone=off #vxdg -ouseclonedev=on -otag=t1 -oupdateid import mydg EMC0_15 Renaming a disk group Moving disks between disk groups Moving disk groups between systems Handling errors when importing disks The following error message indicates a recoverable error #vxdg -fimport diskgroup See “Handling conflicting configuration copies” on page 8 (Remove access to (deport) a disk group) Reserving minor numbers for disk groups device minor number base_minor #vxprint -lmydg | egrep minors minors: >=45000 #vxprint -gmydg -m| egrep base_minor base_minor=45000 #vxdg init diskgroup minor=base_minor disk_access_name minor=base_minor #xvdg init newdg minor=30000 c1d0t0 c1t1d0 Compatibility of disk groups between platforms Handling conflicting configuration copies Page 192Creating and administering disk groups in the output from the command). This case is illustrated below Example of a serial split brain condition that can be resolved automatically Example of a true serial split brain condition that cannot be resolved imported on host Y Shared disk group fails to import Expected A = 1 Expected B Expected A = 0 Expected B Correcting conflicting configuration information vxsplitlines #vxsplitlines -gnewdg The following splits were found in disk group newdg They are listed in da(dm) name pairs Reorganizing the contents of disk groups To perform online maintenance and upgrading of off-host processing Disk group move operation Page Disk group join operation self-contained Limitations of disk group split and join The disk group split and join feature has the following limitations: ■The reconfiguration must involve an integral number of physical disks ■Objects to be moved must not contain open volumes ■Disks cannot be moved between CDS and non-CDScompatible disk groups Listing objects potentially affected by a move #vxdg [-oexpand] listmove sourcedg targetdg object #vxdg listmove mydg newdg vol1 #vxdg listmove mydg newdg mydg01 #vxdg -oexpand listmove mydg newdg mydg01 Page 202Creating and administering disk groups Examples of disk groups that can and cannot be split Volume data plexes Split Page #vxprint #vxdg -oexpand move mydg rootdg mydg01 #vxrecover -g targetdg -m[volume ...] #vxvol -g targetdg startall Page The output from vxprint after the split shows the new disk group, mydg: #vxdg [-ooverride|verify] join sourcedg targetdg Note: You cannot specify rootdg as the source disk group for a join operation # vxprint Disabling a disk group Destroying a disk group Upgrading a disk group Page Features supported by disk group versions New features supported Previous version features supported version To list the version of a disk group, use this command: #vxdg list dgname #vxdg upgrade dgname Managing the configuration daemon in VxVM Backing up and restoring disk group configuration data Using vxnotify to monitor configuration changes Using vxnotify to monitor configuration changes Creating and administering subdisks Creating subdisks Displaying subdisk information Moving subdisks Splitting subdisks Joining subdisks Associating subdisks with plexes Associating subdisks with plexes #vxsd [-g diskgroup] assoc plex subdisk1 [subdisk2 subdisk3 ...] ] assoc #vxsd -gmydg assoc home-1 mydg02-01 mydg02-00 mydg02-01 #vxsd [-g diskgroup] -l offset assoc sparse_plex exact_size_subdisk Associating log subdisks Dissociating subdisks from plexes Removing subdisks Changing subdisk attributes Page Creating and administering plexes Creating plexes Creating a striped plex Displaying plex information ACTIVE plex state CLEAN plex state DCOSNP plex state EMPTY plex state IOFAIL plex state LOG plex state OFFLINE plex state SNAPATT plex state SNAPTMP plex state STALE plex state TEMP plex state TEMPRM plex state TEMPRMSD plex state Plex condition flags IOFAIL plex condition NODAREC plex condition NODEVICE plex condition Attaching and associating plexes Taking plexes offline Detaching plexes Reattaching plexes Moving plexes Copying volumes to plexes Dissociating and removing plexes Changing plex attributes Creating volumes Types of volume layouts Supported volume logs and maps Version 0 DCO volumes only support Persistent FastResync for the traditional Version 0 DCO volume layout Version 20 DCO volume layout Creating a volume with a version 20 DCO volume Creating a volume Using vxassist operation The vxassist utility helps you perform the following tasks: ■Creating volumes ■Creating mirrors for existing volumes ■Growing or shrinking existing volumes Setting default values for vxassist /etc/default/vxassist /etc/default #vxassist help showattrs The following is a sample vxassist defaults file: Discovering the maximum size of a volume Disk group alignment constraints on volumes Creating a volume on any disk Creating a volume on specific disks Specifying ordered allocation of storage to volumes ordered 1Concatenate disks 2Form columns 3Form mirrors 246Creating volumes Example of using ordered allocation to create a striped-mirror mydg01-01 mydg03-01 mydg04-01 Example of using concatenated disk space to create a mirrored mydg04-01 mydg06-01 mydg08-01 Mirrored-stripevolume Page Creating a mirrored volume Creating a volume with a version 0 DCO volume To upgrade a disk group to version 90, use the following command: #vxdg -T90 upgrade diskgroup For more information, see “Upgrading a disk group” on page layout \ logtype=dco [ndcomirror Creating a volume with a version 20 DCO volume Creating a volume with dirty region logging enabled Creating a striped volume 254Creating volumes #vxassist -b -gmydg make stripevol 30g layout=stripe \ mydg03 mydg04 mydg05 #vxassist -b -gmydg make stripevol 30g layout=stripe \ stripeunit=32k ncol=5 Creating a mirrored-stripevolume A mirrored-stripevolume mirrors several striped data plexes To create a striped-mirrorvolume, use the following command: Creating a striped-mirrorvolume Mirroring across targets, controllers or enclosures Creating a RAID-5volume Creating tagged volumes Creating a volume using vxmake mydg00-00 mydg03-00 mydg01-00 mydg04-00 Creating a volume using a vxmake description file #vxmake [-g diskgroup] < description_file Alternatively, you can specify the file to vxmake using the -d option: # vxmake [-g diskgroup] -d description_file Initializing and starting a volume Initializing and starting a volume created using vxmake Accessing a volume Administering volumes Displaying volume information Volume states ACTIVE volume state CLEAN volume state EMPTY volume state INVALID volume state NEEDSYNC volume state REPLAY volume state SYNC volume state Volume kernel states Monitoring and controlling tasks Managing tasks with vxtask vxtask operations EXITED pause Puts a running task in the paused state, causing it to suspend operation resume Causes a paused task to continue operation Using the vxtask command To list all tasks currently running on the system, use the following command: #vxtask list #vxtask -hlist foodg Stopping a volume Starting a volume Adding a mirror to a volume Adding a mirror to a volume Mirroring all volumes #/etc/vx/bin/vxmirror -g diskgroup -a #/etc/vx/bin/vxmirror -dyes nmirror=1 nomirror Removing a mirror Adding logs and maps to volumes Preparing a volume for DRL and instant snapshots Using a DCO and DCO volume with a Specifying storage for version 20 DCO plexes disk05 disk06 #vxsnap -gmydg prepare myvol ndcomirs=2 alloc=disk05,disk06 vol1_dco Using a DCO and DCO volume with a RAID-5volume Determining the DCO version number Determining if DRL is enabled on a volume #vxprint [-g diskgroup] -F%drl$DCONAME DRL is enabled if this command displays on #vxprint [-g diskgroup] -F%sequentialdrl$DCONAME Sequential DRL is enabled if this command displays on Upgrading existing volumes to use version 20 DCOs To upgrade a disk group to the latest version, use the following command: #vxprint [-g diskgroup] -F“%name” -e“v_hasdcolog” #vxassist [-g diskgroup] remove log volume [nlog=n] nlog #vxassist [-g diskgroup] snapback snapvol Adding traditional DRL logging to a mirrored volume vol03 #vxassist -gmydg addlog vol03 logtype=drl logtype=drlseq #vxassist -gmydg addlog volume logtype=drlseq [nlog=n] Removing a traditional DRL log To remove a DRL log, use the vxassist command as follows: #vxassist [-g diskgroup] remove log volume logtype=drl [nlog=n] mydg10 # vxassist -gmydg remove log vol01 !mydg10 logtype=drl Adding a RAID-5log Resizing a volume Resizing volumes using vxresize vxresize Resizing volumes using vxassist The following modifiers are used with the vxassist command to resize a volume: growto Increase volume to a specified length Extending to a given length To extend a volume to a specific length, use the following command: Shrinking to a given length Shrinking by a given length Resizing volumes using vxvol Setting tags on volumes Changing the read policy for mirrored volumes Removing a volume Moving volumes from a VM disk Moving volumes from a VM disk To move volumes from a disk 1Select menu item 6 (Move volumes from a disk) from the vxdiskadm main menu Enabling FastResync on a volume Checking whether FastResync is enabled on a volume Disabling FastResync Performing online relayout Permitted relayout transformations Relayout to From concat concat raid5 Supported relayout transformations for RAID-5volumes From raid5 From mirror-concat Supported relayout transformations for mirrored-stripevolumes From mirror-stripe Supported relayout transformations for unmirrored stripe and layered striped-mirrorvolumes From stripe or stripe-mirror Specifying a non-defaultlayout ncol ncol=-number stripeunit=size Specifying a plex for relayout Viewing the status of a relayout #vxrelayout -gmydg status vol04 might display output similar to this: See the vxrelayout(1M) manual page for more information about this command #vxtask monitor myconv Controlling the progress of a relayout Converting between layered and non-layeredvolumes Page Page Administering volume snapshots Page Traditional third-mirror break-offsnapshots Page Full-sizedinstant snapshots for details of the procedures for creating and using this type of snapshot Creating a volume for use as a full-sized instant or linked Space-optimizedinstant snapshots Emulation of third-mirror break-offsnapshots Linked break-offsnapshot volumes Cascaded snapshots Creating a snapshot of a snapshot Page Page Page Creating multiple snapshots Restoring the original volume from a snapshot Page Creating instant snapshots voliomem_maxpool_sz warning such as the following (for a system whereis set to 12MB): To make the volumes usable for instant snapshot operations, use on the volume, and then use re-prepare voliomem_maxpool_sz (in this example, 1MB): Preparing to create instant and ■“Creating and managing space-optimizedinstant snapshots” on page ■“Creating and managing full-sizedinstant snapshots” on page Preparing to create instant and break-offsnapshots To prepare a volume for the creation of instant and break-offsnapshots #vxprint -g volumedg -F%instant volume #vxprint -g volumedg -F%fastresync volume 2To prepare a volume for instant snapshots, use the following command: Creating a shared cache object To create a shared cache object Having decided on its characteristics, use the cachevol mydg16 Creating a volume for use as a full-sizedinstant or linked break-offsnapshot #LEN=‘vxprint [-g diskgroup] -F%len volume‘ ksh bash csh Creating and managing space-optimizedinstant snapshots Creating a shared cache object To create and manage a space-optimizedinstant snapshot To create a snap3myvol myvol snap4myvol mydg15 Use # fsck -Fvxfs /dev/vx/rdsk/diskgroup/snapshot You now have the following choices of what to do with a Creating and managing full-sizedinstant snapshots To create and manage a full-sizedinstant snapshot instant snapshot, use the following form of the snap1myvol #vxsnap -gmydg make source=myvol/snapvol=snap1myvol Page Creating and managing third-mirror break-offsnapshots To create and manage a third-mirror break-offsnapshot To create the snapshot, you can either take some of the existing SNAPATT # vxsnap -gmydg addmir vol1 nmirror=2 alloc=mydg10,mydg11 330Administering volume snapshots #vxsnap -gmydg snapwait vol1 nmirror=2 snapshot, use the following form of the state. (Such plexes could have been added to the volume by using the command.) Creating and managing linked break-offsnapshot volumes To create and manage a linked break-offsnapshot Use the following command to link the prepared snapshot volume # vxsnap [-g diskgroup] [-b]addmir volume mirvol=snapvol [mirdg=snapdg] snapdg mysnapdg #vxsnap -gmydg -baddmir vol1 mirvol=prepsnap \ mirdg=mysnapdg #vxsnap -gmydg snapwait vol1 mirvol=prepsnap \ mirdg=mysnapvoldg Creating multiple instant snapshots snapvol1 snapvol2 Creating a volume for use as a full-sized Creating instant snapshots of volume sets , and vxvset list #vxvset -gmydg list vset1 INDEX nmirrors vset1 snapvset1 #vxsnap -gmydg prepare vset1 #vxsnap -gmydg prepare snapvset1 Adding snapshot mirrors to a volume #vxsnap -gmydg addmir vol1 nmirror=2 alloc=mydg10,mydg11 plex Removing a snapshot mirror To remove a single snapshot mirror from a volume, use this command: Removing a linked break-offsnapshot volume To remove a linked break-offsnapshot volume from a volume, use this command: mirvol \ [mirdg snapvol Reattaching an instant snapshot Note: This operation is not supported for space-optimizedinstant snapshots snapmyvol #vxsnap -gmydg reattach snapmyvol source=myvol nmirror=1 SNAPTMP Reattaching a linked break-offsnapshot volume To reattach a linked snapshot volume, use the following form of the sourcedg snapdiskgroup Restoring a volume from an instant snapshot It is not possible to restore a volume from an unrelated volume snap3myvol #vxsnap -gmydg restore myvol source=snap3myvol Dissociating an instant snapshot Removing an instant snapshot Splitting an instant snapshot hierarchy Displaying instant snapshot information Page (1M) manual page for more information about using the commands Controlling instant snapshot synchronization Command Improving the performance of snapshot synchronization syncstart iosize size slow Tuning the autogrow attributes of a cache attributes determine how the VxVM cache daemon ) maintains the cache if the feature has been enabled and is running: Growing and shrinking a cache Removing a cache Creating traditional third-mirror break-offsnapshots Page It is also possible to make a snapshot plex from an existing plex in a volume See “Converting a plex into a snapshot plex” on page 351 for details 3Create a snapshot volume using the following command: #vxassist [-g diskgroup] snapshot [nmirror=N] volume snapshot For example, to create a snapshot of voldef, use the following command: Converting a plex into a snapshot plex To convert an existing plex into a snapshot plex for a volume on which Persistent FastResync is enabled, use the following command: #vxplex [-g diskgroup] -odcoplex=dcologplex convert \ state=SNAPDONE plex dcologplex Creating multiple snapshots #vxassist [-g diskgroup] snapshot volume1 volume2 ] snapshot reverses the order of the number and volume components in allvols Adding plexes to a snapshot volume Dissociating a snapshot volume Displaying snapshot information #vxassist snapprint [volume] Output from this command is shown in the following examples: #vxassist -gmydg snapprint SNAP-v2 Adding a version 0 DCO and DCO volume 3Use the following command to add a DCO and DCO volume to the existing volume: Specifying storage for version 0 DCO plexes #vxassist -gmydg addlog myvol logtype=dco dcolen=264 \ mydg05 mydg06 vol1 For more information, see the vxassist(1M) and vxdco(1M) manual pages Reattaching a version 0 DCO and DCO volume #vxdco [-g diskgroup] att volume dco_obj myvol_dco #vxdco -gmydg att myvol myvol_dco For more information, see the vxdco(1M) manual page Page Creating and administering volume sets Creating a volume set Adding a volume to a volume set Listing details of volume sets Stopping and starting volume sets Removing a volume from a volume set Raw device node access to component volumes Raw device node access to component volumes Note the following limitations of this feature: ■The disk group version must be 120 or greater Enabling raw device access when creating a volume set #vxvset [-g diskgroup] -omakedev=on [-o compvol_access={read-only|read-write}] [-o index] [-c"ch_addopt"] make vset vol [index] Displaying the raw device access settings for a volume set Controlling raw device access for an existing volume set The syntax for setting the compvol_access attribute on a volume set is: #vxvset [-g diskgroup] [-f] set compvol_access={read-only|read-write} vset The following example sets the attributes on a volume set Page Configuring off-hostprocessing Implementing off-hostprocessing solutions Implementing off-hostonline backup Page On the primary host, if you temporarily suspended updates to a volume in #vxdg deport snapvoldg #vxdg import snapvoldg #vxrecover -g snapvoldg -m snapvol #vxvol -g snapvoldg start snapvol Implementing decision support # vxprint -g volumedg -F%fastresync volume Page Page You can then resume the procedure from step 6 on page Page Administering hot-relocation How hot-relocationworks Finally Relocation of failing subdisks is not possible in the following cases: The failing subdisks are on ■There are insufficient spare disks or free disk space in the disk group The only available space is on a disk that already contains the Figure 12-1Example of hot-relocationfor a subdisk in a RAID-5volume mydg02-02 mydg03-02 mydg01 mydg02 mydg03 mydg04 mydg05 Partial disk failure mail messages for information on how to send the mail to users other than #vxstat -gmydg -s -ff home-02 src-02 mydg02-03 mydg02-04 Complete disk failure mail messages How space is chosen for relocation Configuring a system for hot-relocation Displaying spare disk information Marking a disk as a hot-relocationspare Removing a disk from use as a hot-relocationspare Excluding a disk from hot-relocationuse Making a disk available for hot-relocationuse Configuring hot-relocationto use only spare disks Moving and unrelocating subdisks Administering hot-relocation391 Moving and unrelocating subdisks Volume home Subdisk mydg02-03relocated to mydg05-01,but not yet recovered Moving and unrelocating subdisks using vxdiskadm 2This option prompts for the original disk media name first Enter the original disk name [<disk>,list,q,?] Currently there are no hot-relocated Moving and unrelocating subdisks using vxassist #vxassist -gmydg move home !mydg05 mydg02 !mydg05 Moving and unrelocating subdisks using vxunreloc The examples in the following sections demonstrate the use of vxunreloc Moving hot-relocatedsubdisks back to their original disk mydg01 #vxunreloc -gmydg mydg01 Moving hot-relocatedsubdisks back to a different disk #vxunreloc -gmydg -nmydg05 mydg01 Examining which subdisks were hot-relocatedfrom a disk #vxprint -gmydg -se'sd_orig_dmname="mydg01 Restarting vxunreloc after errors vxunreloc moves subdisks in three phases: UNRELOC Modifying the behavior of hot-relocation Page Administering cluster functionality Overview of cluster volume management Page Page Private and shared disk groups Activation modes of shared disk groups exclusivewrite Activation modes for shared disk groups Note: The default activation mode for shared disk groups is off (inactive) Activation mode Page Connectivity policy of shared disk groups ■All nodes in the cluster see exactly the same configuration ■Only the master node can change the configuration Global detach policy Local detach policy for information on how to use the command to set the disk detach policy on a shared disk group The table disk detach policies Disk group failure policy Behavior of master node for different failure policies Leave Disable (dgfailpolicy=leave) Guidelines for choosing detach and failure policies #vxdg -g diskgroup set diskdetpolicy=local dgfailpolicy=leave Effect of disk connectivity on cluster reconfiguration Limitations of shared disk groups Cluster initialization and configuration vxclustadm utility Node abort messages Reason Volume reconfiguration vxconfigd daemon ■master node ID ■role of the node ■network address of the vxconfigd daemon on each node (if applicable) vxconfigd daemon recovery If the #hagrp -freeze group Use the following command to # hagrp -unfreeze group Node shutdown clean node shutdown draining Multiple host failover configurations Import lock Failover Moving disk groups between systems Corruption of disk group configuration vxdg import where the reason can describe errors such as: for more information on Veritas Volume Manager error messages Administering VxVM in cluster environments Determining if a disk is shareable #vxdisk list accessname accessname Listing shared disk groups Example output from this command is displayed here: Creating a shared disk group Note: Shared disk groups can only be created on the master node #vxdg -sinit diskgroup [diskname=]devicename init Importing disk groups as shared Note: Shared disk groups can only be imported on the master node #vxdg -simport diskgroup Forcibly importing a disk group You can use the -f option to the vxdg command to import a disk group forcibly Converting a disk group from shared to private Note: Shared disk groups can only be deported on the master node Then reimport the disk group on any cluster node using this command: Changing the activation mode on a shared disk group Setting the disk detach policy on a shared disk group Setting the disk group failure policy on a shared disk group Creating volumes with exclusive open access by a node Setting exclusive open access to a volume by a node Displaying the cluster protocol version Displaying the supported cluster protocol version range Upgrading the cluster protocol version Recovering volumes in shared disk groups Obtaining cluster performance statistics Page Page Administering sites and remote mirrors Page Page Configuring sites for hosts and disks Configuring site-basedallocation on a disk group Configuring site consistency on a disk group Configuring site consistency on a volume Setting the siteread policy on a volume Site-basedallocation of storage to volumes Site-basedallocation of storage to volumes #vxassist -g diskgroup make volume size mirror=site a non-site-consistentmirrored volume with plexes at all of the sites: mirror=site \ allsites=on siteconsistent off Examples of storage allocation using sites nmirror=2 ccdg ordered make vol 2g Making an existing disk group site consistent Fire drill — testing the configuration Automatic site reattachment vxnotify vxdg detachsite Failure scenarios and recovery procedures Recovery from a loss of site connectivity #vxdg -g diskgroup -ooverridessb reattachsite sitename Recovery from host failure Recovery from storage failure Recovery from site failure Page Using Storage Expert About Storage Expert How Storage Expert works Before using Storage Expert Running Storage Expert Running Storage Expert info Discovering what a rule does #vxse_stripes2 info Displaying rule attributes and their default values mirror_threshold #vxse_drl1 list Rule result types Setting rule attributes Identifying configuration problems using Storage Expert Checking for large mirror volumes without a dirty region log (vxse_drl1) See “Preparing a volume for DRL and instant snapshots” on page To check whether a large mirrored volume has a mirrored DRL log, run rule Checking for RAID-5volumes without a RAID-5log (vxse_raid5log1) To check whether a RAID-5volume has an associated RAID-5log, run rule Disk groups Checking whether a configuration database is too full (vxse_dg1) See “Creating a disk group” on page Checking disk group configuration copies and logs (vxse_dg2) Checking “on disk config” size (vxse_dg3) Checking the number of configuration copies in a disk group (vxse_dg5) See “Creating and administering disk groups” on page Checking for non-importeddisk groups (vxse_dg6) To check for disk groups that are visible to VxVM but not imported, run rule See “Importing a disk group” on page Disk striping Checking the configuration of large mirrored-stripevolumes (vxse_mirstripe) See “Converting between layered and non-layeredvolumes” on page Checking the number of columns in RAID-5volumes (vxse_raid5) To check whether Checking the number of columns in striped volumes (vxse_stripes2) Disk sparing and relocation management Checking the number of spare disks in a disk group (vxse_spares) Hardware failures Checking for disk failures and disabled controllers (vxse_dc_failures) Checking the system name (vxse_host) Rule definitions and attributes See “Running a rule” on page Table 15-2 lists the available rule attributes and their default values Attribute Default Page Page Page Page Performance monitoring and tuning Performance guidelines Striping Mirroring Combining mirroring and striping RAID-5 Volume read policies Performance monitoring Tracing volume operations Printing volume statistics VxVM records the following I/O statistics: ■count of operations ■number of blocks transferred (one operation can involve more than one block) Using performance data Using I/O statistics #vxprint -gmydg -tvharchive The following is an extract from typical output: #vxassist -gmydg move archive !mydg03 dest_disk dest_disk #vxassist -gmydg move archive !mydg03 mydg04 Page Tuning VxVM Tuning guidelines for large systems Number of configuration copies for a disk group Changing the values of tunables Tunable parameters dmp_cache_open dmp_daemon_count dmp_delayq_interval page 160dmp_failed_io_threshold dmp_health_time dmp_health_time dmp_path_age dmp_log_level dmp_path_age dmp_probe_idle_lun dmp_queue_depth dmp_restore_cycles The value of this tunable is only changeable by using the dmp_restore_interval dmp_restore_policy The DMP restore policy, which can be set to 0 (CHECK_ALL) (CHECK_DISABLED), 2 (CHECK_PERIODIC), or 3 (CHECK_ALTERNATE) dmp_retry_count /kernel/drv/ vxdmp.conf dmp_stat_interval vol_checkpt_default vol_default_iodelay vol_fmr_logsz vol_max_vol vol_maxio vol_maxioctl vol_maxparallelio vol_maxspecialio vol_subdisk_num volcvm_smartsync voldrl_max_drtregs voldrl_max_seq_dirty voldrl_min_regionsz Note: If DRL sequential logging is configured, the value of voliomem_maxpool_sz voliot_errbuf_dflt voliot_iobuf_default voliot_iobuf_limit voliot_iobuf_max voliot_max_open volpagemod_max_memsz #vxtune volpagemod_max_memsz value volraid_minpool_sz volraid_rsrtransmax Page Commands summary ■ “Creating volumes” on page ■ “Administering volumes” on page ■ “Monitoring and controlling tasks” on page Table A-1 Obtaining information about objects in VxVM # vxinfo -gmydg myvol1 myvol2 # vxprint -gmydg myvol1 # vxprint -st -gmydg # vxprint -pt -gmydg # vxdiskadd c0t1d0 # vxedit -gmydg rename mydg03 mydg02 # vxedit -gmydg set reserve=on mydg02 spare=on mydg04 spare=off mydg04 # vxdisk offline c0t1d0 # vxdg -gmydg rmdisk c0t2d0 # vxdiskunsetup c0t3d0 Table A-3 # vxdg init mydg mydg01=c0t1d0 # vxsplitlines -gmydg # vxdg -nnewdg deport mydg # vxdg -oexpand listmove mydg newdg myvol1 # vxdg -oexpand move mydg newdg myvol1 # vxdg -oexpand split mydg # vxrecover -gmydg -sb # vxdg destroy mydg Table A-4 # vxmake -gmydg sd mydg02-01mydg02,0,8000 # vxsd -gmydg assoc vol01-01 mydg10-01:0 mydg11-01:1 mydg12-01:2 # vxsd -gmydg mv mydg01-01 # vxsd -gmydg -s1000m # vxunreloc -gmydg mydg01 # vxsd -gmydg -orm dis Table A-5 # vxmake -gmydg plex vol01-02 vol01-02 # vxplex -gmydg mv vol02-02 vol02-03 # vxplex -gmydg cp vol02 vol03-01 vol02-02 Table A-6 # vxassist -gmydg maxsize \ layout=raid5 nlog=2 mysvol 20g layout=stripe stripeunit=32 ncol=4 # vxmake -gmydg -Uraid5 vol r5vol plex=raidplex,raidlog1 raidlog2 # vxvol -gmydg start r5vol Table A-7 #vxassist -gmydg mirror \ myvol mydg10 # vxassist -gmydg remove mirror myvol !mydg11 # vxassist -gmydg growby # vxsnap -gmydg prepare \ myvol drl=on #vxsnap -gmydg make \ source=myvol/\ newvol=mysnpvol/\ nmiror=2 # vxassist -gmydg make cvol 1g layout=mirror init=active mydg16 mydg17 # vxmake -gmydg cache cobj cachevolname=cvol # vxsnap -gmydg unprepare \ myvol #vxassist -gmydg relayout \ vol2 layout=stripe vol3 layout=raid5 stripeunit=16 ncol=4 reverse vol3 #vxassist -gmydg remove \ myvol Table A-8 # vxrecover -gmydg -tmytask -bmydg05 # vxtask -h -gmydg list # vxtask pause mytask # vxtask -p -gmydg list # vxtask resume mytask # vxtask abort mytask Online manual pages Page Page Section 4 — file formats Manual pages in section 4 describe the format of files that are used by Veritas Volume Manager Table A-10 Section 4 manual pages Configuring Veritas Page Page Page Page Page Page Page Page Page Glossary 522 Glossary cluster A set of hosts (each termed a node) that share a set of disks cluster manager cluster-shareabledisk group Glossary log subdisk disabled path An alternative term for a device name disk access records Page Page Page Page Page Page Page Index