RAID-5 | HP Veritas Volume Manager 5.0 -UX 11i v3 guide

Performance monitoring and tuning

Performance guidelines

465

Combining mirroring and striping

Note: You need a full license to use this feature.

Mirroring and striping can be used together to achieve a significant improvement in performance when there are multiple I/O streams.

Striping provides better throughput because parallel I/O streams can operate concurrently on separate devices. Serial access is optimized when I/O exactly fits across all stripe units in one stripe.

Because mirroring is generally used to protect against loss of data due to disk failures, it is often applied to write-intensive workloads which degrades throughput. In such cases, combining mirroring with striping delivers both high availability and increased throughput.

A mirrored-stripe volume may be created by striping half of the available disks to form one striped data plex, and striping the remaining disks to form the other striped data plex in the mirror. This is often the best way to configure a set of disks for optimal performance with reasonable reliability. However, the failure of a single disk in one of the plexes makes the entire plex unavailable.

Alternatively, you can arrange equal numbers of disks into separate mirror volumes, and then create a striped plex across these mirror volumes to form a striped-mirror volume (see “Mirroring plus striping (striped-mirror,RAID-1+0 or RAID-10)” on page 43). The failure of a single disk in a mirror does not take the disks in the other mirrors out of use. A striped-mirror layout is preferred over a mirrored-stripe layout for large volumes or large numbers of disks.

RAID-5

Note: You need a full license to use this feature.

RAID-5 offers many of the advantages of combined mirroring and striping, but requires less disk space. RAID-5 read performance is similar to that of striping and RAID-5 parity offers redundancy similar to mirroring. Disadvantages of RAID-5 include relatively slow write performance.

RAID-5 is not usually seen as a way of improving throughput performance except in cases where the access patterns of applications show a high ratio of reads to writes.

Image 464

HP Veritas Volume Manager 5.0 -UX 11i v3 manual Combining mirroring and striping, RAID-5

Contents

Veritas Volume Manager Administrator’s Guide Legal Notices Page Contents Using vxdiskadd to place a disk under control of VxVM VxVM root disk volume restrictionsBooting root volumes Setting up a VxVM root disk and mirror Taking a disk offline 118 119133 136 207 167185 Setting default values for vxassist 241 215Displaying subdisk information 216 217 275 252274 Moving volumes from a VM disk 290 288289 290 Chapter Creating and administering volume sets 356 386 387388 390 434 441 Setup tasks after installation 511 463507 512 521 531 Understanding Veritas Volume Manager Veritas Enterprise Administrator User’s Guide VxVM and the operating system How data is stored How VxVM handles storage management Physical objects-physical disks Disk arrays Operating system Multipathed disk arraysDevice discovery Enclosure-based naming Fibre Channel hub or switch Provide redundant loop access C2t99d0 Virtual objects in VxVM include the following Virtual objectsCombining virtual objects in VxVM Connection between objects in VxVM Disk groups VM disks Subdisks Plexes Volumes How VxVM handles storage management Disk01-01 vol06-01 Implementation of layered volumes Volume layouts in VxVMImplementation of non-layered volumes Following sections describe each layout method Layout methodsConcatenation and spanning 12 Example of concatenation Data Disk01-01 Example of spanning Data +1 n+2 Striping RAID-0 Su1 Su2 Su3 Devname3 Devname2 Mirroring RAID-1 Striping plus mirroring mirrored-stripe or RAID-0+1 17 Mirrored-stripe volume laid out on six disks Mirroring plus striping striped-mirror, RAID-1+0 or RAID-10 Column Mirror Striped plex RAID-5 striping with parity Mirror volumesYou need a full license to use this feature Data Parity Traditional RAID-5 arrays Veritas Volume Manager RAID-5 arrays 22 Veritas Volume Manager RAID-5 array Stripe Left-symmetric layout Understanding Veritas Volume Manager RAID-5 logging Layered volumes Example of a striped-mirror layered volume Understanding Veritas Volume Manager Online relayout How online relayout works Three columns of length 5L/3 Striped volume Limitations of online relayout Transformation characteristics Transformations and volume length Resynchronization process Volume resynchronizationDirty flags Dirty region logging Dirty region logs Log subdisks and plexes Sequential DRL SmartSync recovery accelerator Data volume configuration Volume snapshots Redo log volume configuration Original Use on creation Requires less storage Comparison of snapshot featuresSpace than original Volume FastResync FastResync enhancements Non-persistent FastResync How non-persistent FastResync works with snapshots DCO volume versioningPersistent FastResync Version 0 DCO volume layout Version 20 DCO volume layout Where the size of each map in bytes is How persistent FastResync works with snapshots DCO FastResync Effect of growing a volume on the FastResync map FastResync limitations Volume sets Hot-relocation Volume sets Administering disks Disk devices There are two different methods of naming disk devices Disk device naming in VxVMOperating system-based naming # vxdisk path egrep diskname Private and public disk regions Configures disk access records for them automatically To boot the system.Typically, most disks on a system areConfigured as this disk type. However, it is not a suitable Format for boot, root or swap disks, for mirrors or Discovering and configuring newly added disk devices Following command scans for the devices c1t1d0 and c2t2d0Partial device discovery Next example discovers fabric devices For more information, see the vxdisk1M manual Discovering disks and dynamically adding disk arraysDisk categories Third-party driver coexistence Adding support for a new disk arrayEnabling discovery of new devices Removing support for a disk array Administering the Device Discovery Layer Listing details of supported disk arrays# vxddladm listsupport all # vxddladm listsupport libname=libraryname.sl Listing excluded disk arrays Excluding support for a disk array libraryRe-including support for an excluded disk array library Listing supported disks in the Disks category Adding unsupported disk arrays to the Disks category Length=serialnolength policy=ap # vxddladm addjbod vid=SEAGATE pid=ST318404LSUN18G# vxdmpadm listenclosure all # vxdisk list # vxddladm rmjbod vid=SEAGATE Removing disks from the Disks categoryAdding foreign devices Placing disks under VxVM control See Migrating between DMP and HP-UX native multipathing on Mode Format of output from VxVM command DefaultChanging the disk-naming scheme Legacy VxVM vxdmpadm Error V-5-1-10910 Invalid da-name # vxdmpadm getlungroup dmpnodename=disk25 This regenerates the persistent name database Restart the VxVM configuration demonRegenerating persistent device names # vxdmpadm getsubpaths dmpnodename=enclosure-basedname Changing device naming for TPD-controlled enclosures# vxdmpadm setattr enclosure enclosure tpdmode=nativepseudo Persistent simple or nopriv disks in the boot disk group # vxconfigd -kr reset Re-import the disk group using the following command Installing and formatting disksPersistent simple or nopriv disks in non-boot disk groups Adding a disk to VxVM Displaying and changing default disk layout attributesAdd or initialize disks Menu VolumeManager/Disk/AddDisks C3t0d0 c3t1d0 c3t2d0 c3t3d0 Continue operation? y,n,q,? default y y Use default disk names for the disks? y,n,q,? default yExclude disks from hot-relocation use? y,n,q,? default n n Add site tag to disks? y,n,q,? default n Continue with operation? y,n,q,? default y y Enter the desired format cdsdisk,hpdisk,q,? default cdsdiskEnter desired private region length privlen,q,? default Following disks Using vxdiskadd to place a disk under control of VxVM Add or initialize other disks? y,n,q,? default nReinitializing a disk Vxdiskadm then proceeds to add the disks Rootability VxVM root disk volume restrictions Root disk mirrors # /etc/vx/bin/vxcplvmroot -b c0t4d0 Booting root volumesSetting up a VxVM root disk and mirror # /etc/vx/bin/vxcplvmroot -R 30 -v -b c0t4d0 # /etc/vx/bin/vxcplvmroot -m c1t1d0 -R 30 -v -b c0t4d0 Creating an LVM root disk from a VxVM root disk Adding swap volumes to a VxVM rootable system Adding persistent dump volumes to a VxVM rootable systemView the changed swap configuration Display the initial crash dump configuration Removing a persistent dump volume You can now remove the volume if required# crashconf -ds /dev/vx/dsk/bootdg/dumpvol Dynamic LUN expansion 109 # vxvol -g diskgroup stop volume1 volume2 Removing disks Continue with operation? y,n,q,? default y Remove another disk? y,n,q,? default nRemoving a disk with subdisks VxVM Info V-5-2-268 Removal of disk mydg01 is complete # /usr/lib/vxvm/bin/vxdiskunsetup c#t#d# Removing a disk from VxVM controlRemoving and replacing disks Removing a disk with no subdisks # vxassist move mkting !mydg02 Are you sure you want do this? y,n,q,? default nTo replace a disk Following devices are available as replacements c0t1d0 Removing and replacing disks Following devices are available as replacements Replacing a failed or removed disk Replace another disk? y,n,q,? default n Then run the following command on the master node Select a disk device to enable address,list,q,? c0t2d0Enabling a disk # vxreattach -r accesname Taking a disk offline Enable another device? y,n,q,? default nDisable another device? y,n,q,? default n Reserving disks You would use the following command to rename the diskRenaming a disk VxVM returns a display similar to the following Displaying disk informationSee the vxedit1M manual page for more information Displaying disk information with vxdiskadm List disk information Menu VolumeManager/Disk/ListDiskEnter disk device or all address,all,q,? default all Controlling Powerfail Timeout Setting the Pfto valuesTo set the Pfto value on a disk, use the following command Displaying the Pfto values For example, to disable Pfto on the disk c5t0d6 Enabling or disabling Pfto Controlling Powerfail Timeout Administering dynamic multipathing DMP How DMP works How DMP works Enc00 How DMP monitors I/O on paths Path failover mechanism Load balancing Throttling Use the following commands to initiate the migration DMP coexistence with HP-UX native multipathingMigrating between DMP and HP-UX native multipathing # vxvol -g diskgroup stopall Restart all the volumes in each disk group # vxvol -g diskgroup startall DMP in a clustered environment Under the new naming scheme as Enabling or disabling controllers with shared disk groups Disabling and enabling multipathing for specific devicesDisabling multipathing and making devices invisible to VxVM Enabling multipathing and making devices visible to VxVM 135 Administering dynamic multipathing DMP Displaying the paths to a disk Typical output from the vxdisk list command is as followsDisplaying DMP database information Disabled Config Disabled LogDisabled Lockrgn C1t0d3 state=enabled Type=secondary # vxdmpadm getdmpnode nodename=c3t2d1 Administering DMP using vxdmpadmRetrieving information about a DMP node Displaying the members of a LUN group # vxdmpadm getdmpnode enclosure=enc0# vxdmpadm getlungroup dmpnodename=c11t0d10 # vxdmpadm getsubpaths dmpnodename=c2t66d0 Displaying information about controllers # vxdmpadm listctlr all Displaying information about array ports Following is example output from this commandDisplaying information about enclosures # vxdmpadm gettpdnode nodename=c7t0d10 # vxdmpadm getsubpaths tpdnodename=emcpower10Displaying information about TPD-controlled devices Gathering and displaying I/O statistics Examples of using the vxdmpadm iostat commandTo reset the I/O counters to zero, use this command # vxdmpadm iostat start memory=4096 # vxdmpadm iostat show pathname=c3t115d0# vxdmpadm iostat show dmpnodename=c0t0d0 # vxdmpadm iostat show enclosure=Disk Setting the attributes of the paths to an enclosure # vxdmpadm setattr path c2t10d0 pathtype=active# vxdmpadm setattr path c3t10d0 pathtype=nomanual # vxdmpadm setattr path c1t20d0 pathtype=nopreferred Displaying the I/O policy Specifying the I/O policy Following policies may be set Adaptive# vxdmpadm setattr enclosure enc1 iopolicy=adaptive Adaptiveminq 048 This is the default I/O policy for Active/Active A/A arrays # vxdmpadm setattr enclosure Disk iopolicy=minimumq# vxdmpadm setattr arrayname Sena iopolicy=priority # vxdmpadm setattr arraytype A/A iopolicy=round-robin Default setting for this attribute is useallpaths=no Example of applying load balancing in a SAN# vxdmpadm setattr arrayname Disk iopolicy=singleactive # vxdisk list c3t2d15 # vxdmpadm getattr enclosure ENC0 iopolicy DMP statistics are now reset# dd if=/dev/vx/rdsk/mydg/myvol1 of=/dev/null # vxdmpadm -c-fdisable path=pathname # vxdmpadm -c-fdisable ctlr=ctlrnameDisabling I/O for paths, controllers or array ports # vxdmpadm setattr enclosure ENC0 iopolicy=singleactive # vxdmpadm enable path=pathname # vxdmpadm enable ctlr=ctlrnameEnabling I/O for paths, controllers or array ports Upgrading disk controller firmware Re-enable the plex associated with the device Renaming an enclosureStop I/O to all disks through one controller of the HBA For the other controller on the HBA, enter # vxdmpadm setattr \ Configuring the response to I/O failures# vxdmpadm getattr enclosure enc0 recoveryoption Configuring the I/O throttling mechanism # vxdmpadm setattr arraytype A/A recoveryoption=default # vxdmpadm setattr enclosure enc0 recoveryoption=nothrottle HDS9500-ALUA0 Error-Retry Displaying recoveryoption values# vxdmpadm getattr enclosure HDS9500-ALUA0 recoveryoption Configuring DMP path restoration policies # vxdmpadm start restore interval=seconds policy=checkall This produces output such as the following Stopping the DMP path restoration threadDisplaying the status of the DMP path restoration thread Displaying information about the DMP error-handling thread Configuring array policy modulesTo add and configure an APM, use the following command One daemon should be shown as running # vxdmpadm -r cfgapm modulename Administering DMP using vxdmpadm Creating and administering disk groups Creating and administering disk groups Specifying a disk group to commands Block special device corresponding to this volume isSystem-wide reserved disk groups To nodg Rules for determining the default disk group Displaying the system-wide boot disk group# vxdg bootdg See the vxdg1M manual page for more information Displaying disk group information # vxdg list# vxdg list diskgroup # vxdisk -s list devicename Following is example output Creating a disk groupDisplaying free space in a disk group Adding a disk to a disk group # vxdiskadd c1t0d0# vxdg init mktdg mktdg01=c1t0d0 # vxdg -g diskgroup set cds=onoff # vxdiskunsetup devicename # vxdiskunsetup c1t0d0Removing a disk from a disk group # vxdg -g diskgroup rmdisk diskname Deporting a disk group # vxdisk -s list Importing a disk groupNewdg Select disk group to import group,list,q,? default list Select another disk group? y,n,q,? default nHandling disks with duplicated identifiers VxVM Info V-5-2-374 The import of newdg was successful Option to the vxdg import command, as shown in this # vxdisk -f-g diskgroup updateudid disk# vxdisk updateudid c2t66d0 c2t67d0 Writing a new Udid to a disk # vxdg -o useclonedev=on -o updateid import mydg # vxdisk -g diskgroup settag tagname disk# vxdisk settag mytaggeddisks c2t66d0 c2t67d0 # vxdisk listtag Enabling configuration database copies on tagged disks Sample cases of operations on cloned disks# vxdg -q listmeta diskgroup # vxdg -o useclonedev=on -o tag=mytaggeddisks import mydg # vxdisk -o alldgs list # vxdg -g mydg set tagmeta=on tag=t1 nconfig=all nlog=allImporting cloned disks without tags # /usr/symcli/bin/symmir -g mydg split DEV001 Symmir command is used to split off the BCV deviceTo import only the cloned disks into the mydg disk group # vxdg -n newdg -o useclonedev=on -o updateid import mydg Importing cloned disks with tagsState of the cloned disk is now shown as online clonedisk Disks are tagged as follows # vxdisk set EMC08 clone=off # vxdisk -o alldgs list This command results in output such as the following Renaming a disk group # vxdg -tC -n newdg import diskgroup Moving disks between disk groupsDgid 774226267.1025.tweety # vxrecover -g diskgroup -sb Moving disk groups between systems Handling errors when importing disks To clear the locks during import, use the following command# vxdisk clearimport devicename # vxdg -C import diskgroup # vxdg -f import diskgroup Following error message indicates a recoverable errorReserving minor numbers for disk groups # xvdg init newdg minor=30000 c1d0t0 c1t1d0 # vxprint -g mydg reminor Compatibility of disk groups between platforms # vxdg -g diskgroup set maxdev=4079 Handling conflicting configuration copies Example of a serial split brain condition in a cluster 191 Typical arrangement of a 2-node campus cluster Automatically Expected a = Expected B = Imported on host Y Correcting conflicting configuration information # vxsplitlines -g newdg Reorganizing the contents of disk groups Reorganizing the contents of disk groups 197 Disk group join operation Limitations of disk group split and join Moving DCO volumes between disk groups Listing objects potentially affected by a move 201 Split Snapshot Moving objects between disk groups # vxprint # vxdg -o expand move mydg rootdg mydg01 # vxdg -o expand split rootdg mydg rootdg07 rootdg08 Following commands would also achieve the same resultSplitting disk groups Joining disk groups # vxdg -o overrideverify join sourcedg targetdg Following command joins disk group mydg to rootdg Disabling a disk groupMydg Mydg05 C1t96d0 Mydg06 C1t98d0 Recovering a destroyed disk group Destroying a disk groupUpgrading a disk group Use the disk group ID to import the disk group Upgrading a disk group Features supported by disk group versions # vxdg list dgname To list the version of a disk group, use this command# vxdg upgrade dgname Managing the configuration daemon in VxVM # vxdg -T 120 init newdg newdg01=c0t3d0 Backing up and restoring disk group configuration data Using vxnotify to monitor configuration changes# vxnotify -f # vxnotify -s Using vxnotify to monitor configuration changes Creating subdisks # vxmake -g mydg sd mydg02-01 mydg02,0,8000 This command provides the following output Displaying subdisk information# vxprint -st # vxprint -g diskgroup -l subdisk # vxsd -g mydg mv mydg03-01 mydg12-01 mydg12-02 Moving subdisksSplitting subdisks # vxmake -g diskgroup plex plex sd=subdisk Joining subdisksAssociating subdisks with plexes # vxsd -g mydg assoc home-1 mydg02-01 mydg02-00 mydg02-01 # vxsd -g mydg -l 4096b assoc vol10-01 mydg15-01# vxsd -g mydg -l 1 assoc vol02-01 mydg11-01 # vxsd -g diskgroup assoc plex subdisk10 ... subdiskMN-1 # vxassist -g diskgroup addlog volume disk Associating log subdisks# vxsd -g diskgroup aslog plex subdisk # vxsd -g mydg aslog vol01-02 mydg02-01 To remove a subdisk, use the following command Dissociating subdisks from plexesRemoving subdisks Changing subdisk attributes # vxedit -g mydg set comment=subdisk comment mydg02-01 # vxedit -g mydg set putil0=DO-NOT-USE mydg02-01 Creating plexes # vxmake -g mydg plex vol01-02 sd=mydg02-01,mydg02-02 Plex states Creating a striped plexDisplaying plex information Dcosnp plex state Clean plex stateActive plex state Empty plex state Iofail plex stateLOG plex state Offline plex state Snaptmp plex state Stale plex stateTemp plex state Temprm plex state Plex condition flags Disabled plex kernel state Enabled plex kernel stateAttaching and associating plexes Plex kernel states # vxmend -g mydg off vol01-02 vol02-02 Taking plexes offline# vxmend -g diskgroup off plex Detaching plexes Reattaching plexes Start the volume using the following command # vxmend -g diskgroup fix clean plex# vxvol -g diskgroup start volume Moving plexes Copying volumes to plexes # vxplex -g diskgroup cp volume newplexDissociating and removing plexes # vxplex -g diskgroup -o rm dis plex Changing plex attributes # vxedit -g diskgroup set attribute=value ... plex# vxedit -g mydg set comment=plex comment tutil2=u vol01-02 # vxedit -g mydg set putil0=DO-NOT-USE vol01-02 Creating volumes Types of volume layouts RAID-5 Supported volume logs and maps Mirror and concatenated-mirror volumes Creating a volume Advanced approach Using vxassist Assisted approach # vxassist options make volume length attributes Vxassist Setting default values for vxassist Following is a sample vxassist defaults file Discovering the maximum size of a volume Disk group alignment constraints on volumes# vxassist -g diskgroup maxsize layout=layout attributes # vxassist -g dgrp maxsize layout=raid5 nlog=2 Creating a volume on any disk # vxassist -b -g diskgroup make volume length# vxassist -b make voldefault 10g # vxprint -g diskgroup -G -F %align # vxassist -b -g mydg make volspec 5g ctlrc1 !targetc1t5 Creating a volume on specific disks# vxassist -b -g mydg make volspec 5g mydg03 mydg04 Specifying ordered allocation of storage to volumes Mydg01 mydg02 mydg03 mydg04 mydg05 mydg06 mydg07 mydg08 Stripe volume Volume across controllers Creating a mirrored volume Creating a mirrored-concatenated volumeCreating a concatenated-mirror volume # vxassist -b -g mydg make volmir 5g layout=mirror Creating a volume with a version 0 DCO volume For more information, see Upgrading a disk group on # vxdg -T 90 upgrade diskgroup# vxvol -g diskgroup set logtype=drldrlseq volume # vxdg upgrade diskgroup Creating a volume with a version 20 DCO volumeCreating a volume with dirty region logging enabled # vxassist -b -g mydg make volzebra 10g layout=stripe Creating a striped volume# vxassist -b -g diskgroup make volume length layout=stripe Creating a mirrored-stripe volume Creating a striped-mirror volume Mirroring across targets, controllers or enclosures Creating a RAID-5 volume # vxassist -b -g mydg make volraid 10g layout=raid5 nlog=2 # vxassist -g diskgroup list tag=tagname volume Creating tagged volumes# vxassist -g diskgroup listtag volume Creating a volume using vxmake Creating a volume using a vxmake description file # vxmake -g diskgroup -d descriptionfile Mydg04-021/8000,mydg04-031/16000 Initializing and starting a volume# vxassist -b -g diskgroup make volume length layout=mirror Initializing and starting a volume created using vxmake # vxvol -g diskgroup init enable volume# vxvol -g diskgroup init active volume # vxvol -g diskgroup init zero volume Accessing a volume Administering volumes This is example output from this command Displaying volume informationVxprint command can also be applied to a single disk group Volume states Active volume stateClean volume state Empty volume state Volume kernel states Invalid volume stateNeedsync volume state Replay volume state Detached volume kernel state Disabled volume kernel stateEnabled volume kernel state Monitoring and controlling tasks Managing tasks with vxtask Vxtask operations Using the vxtask command Stopping a volume Putting a volume in maintenance mode To start all Disabled volumes, enter Starting a volumeAdding a mirror to a volume Mirroring all volumes Mirroring volumes on a VM disk# /etc/vx/bin/vxmirror -g diskgroup -a # /etc/vx/bin/vxmirror -d yes Mirror volumes on another disk? y,n,q,? default n # vxassist -gdiskgroup remove mirror volumeRemoving a mirror At the following prompt, press Return to make the mirror Adding logs and maps to volumes Preparing a volume for DRL and instant snapshots Specifying storage for version 20 DCO plexes Using a DCO and DCO volume with a RAID-5 volume # DCONAME=‘vxprint -g diskgroup -F%dconame volume‘Determining the DCO version number # vxprint -g diskgroup -F%version $DCONAME Determining if DRL is enabled on a volume Determining if DRL logging is active on a volumeThis command returns on if DRL logging is enabled Disabling and re-enabling DRL To re-enable sequential DRL on a volume, enter Upgrading existing volumes to use version 20 DCOsTo re-enable DRL on a volume, enter this command Use the following command on the volume to upgrade it Adding traditional DRL logging to a mirrored volume To remove a DRL log, use the vxassist command as follows # vxassist -g mydg addlog volume logtype=drlseq nlog=nRemoving a traditional DRL log # vxassist -g mydg addlog vol03 logtype=drl Adding a RAID-5 log using vxplex # vxassist -b -g diskgroup addlog volume loglen=lengthAdding a RAID-5 log # vxassist -g mydg addlog volraid Resizing a volume # vxprint -g diskgroup -ht volume# vxassist -g diskgroup maxgrow volume Removing a RAID-5 log Resizing volumes using vxresize Online JFS Full Base JFS LiteVxFS Mounted File System Unmounted File System Extending by a given length Resizing volumes using vxassistExtending to a given length Shrinking by a given length Resizing volumes using vxvolShrinking to a given length Setting tags on volumes # vxassist -g diskgroup removetag volume tagname Changing the read policy for mirrored volumes To set the read policy to select, use the following command Removing a volumeMoving volumes from a VM disk To move volumes from a disk Move volumes from another disk? y,n,q,? default nVxVM vxevac Info VxVM Info V-5-2-188 Evacuation of disk mydg02 is complete Enabling FastResync on a volume # vxvol -g diskgroup set fastresync=on volume Checking whether FastResync is enabled on a volume # vxprint -g diskgroup -F%fastresync volume# vxprint -g diskgroup -F%hasdcolog volume Disabling FastResync Performing online relayout # vxassist -g mydg relayout vol02 layout=stripe Permitted relayout transformations Supported relayout transformations for RAID-5 volumes Relayout to From raid5 ConcatRelayout to From mirror-concat Concat Mirror-concat Mirror-stripe Layered striped-mirror volumes Relayout to From mirror-stripe ConcatSupported relayout transformations for unmirrored stripe Relayout to From stripe or stripe-mirror Concat Specifying a non-default layout Specifying a plex for relayoutTagging a relayout operation # vxassist -g fsgrp relayout vol04 layout=raid5 ncol=4 Controlling the progress of a relayout To resume the operation, use the vxtask commandViewing the status of a relayout Converting between layered and non-layered volumes # vxrelayout -g mydg -o bg,slow=1000,iosize=10m start vol04# vxrelayout -g mydg -o bg reverse vol04 # vxassist -g mydg relayout vol1 ncol=5 # vxassist -g mydg convert vol1 layout=mirror-stripe Converting between layered and non-layered volumes Administering volume snapshots Administering volume snapshots Independent Volume Vxassist snapclear Traditional third-mirror break-off snapshots Traditional third-mirror break-off snapshots Full-sized instant snapshots 308 Administering volume snapshots Space-optimized instant snapshots Cycle Start Vxsnap make Vxsnap refresh Vxsnap prepare Emulation of third-mirror break-off snapshots Linked break-off snapshot volumes Cascaded snapshots Cascaded snapshots Creating a snapshot of a snapshot Creating a snapshot of a snapshot Create instant snapshot S2 of S1 Vxsnap make source=S1 Vxsnap dis S2 Restoring the original volume from a snapshot Creating multiple snapshots Restoring the original volume from a snapshot Creating instant snapshots Creating instant snapshots Preparing to create instant and break-off snapshots Creating a shared cache object # LEN=‘vxprint -g diskgroup -F%len volume‘ For example to start the cache object, cobjmydg# vxcache -g mydg start cobjmydg # RSZ=‘vxprint -g diskgroup -F%regionsz $DCONAME‘ Creating and managing space-optimized instant snapshots Creating instant snapshots # fsck -F vxfs /dev/vx/rdsk/diskgroup/snapshot Creating and managing full-sized instant snapshots # vxsnap -g mydg make source=myvol/snapvol=snap1myvol# vxsnap -g diskgroup syncwait snapvol # vxsnap -g mydg syncwait snap2myvol # vxprint -gdiskgroup -F%incompletesnapvol Creating and managing third-mirror break-off snapshots # vxsnap -g mydg addmir vol1 nmirror=2 alloc=mydg10,mydg11 # vxsnap -g mydg snapwait vol1 nmirror=2 Creating and managing linked break-off snapshot volumes # vxsnap -g diskgroup -b addmir volume mirvol=snapvol \ Mirdg=snapdg Reattach the snapshot volume with the original volume. See Creating multiple instant snapshots# vxsnap -g diskgroup make \ # vxsnap -g diskgroup make source=vol1/snapvol=snapvol1 \ # vxvset -g mydg list snapvset1 Creating instant snapshots of volume sets# vxvset -g mydg list vset1 Svol2 614400 Svol0 204800Svol1 409600 # vxsnap -g mydg rmmir vol1 Adding snapshot mirrors to a volumeRemoving a snapshot mirror Removing a linked break-off snapshot volume Adding a snapshot to a cascaded snapshot hierarchyRefreshing an instant snapshot # vxsnap -g mydg rmmir vol1 mirvol=prepsnap mirdg=mysnapdg Reattaching an instant snapshot # vxsnap -g mydg reattach snapmyvol source=myvol nmirror=1 Reattaching a linked break-off snapshot volume # vxsnap -g mydg snapwait myvol nmirror=1 Restoring a volume from an instant snapshot # vxsnap -g mydg restore myvol source=snap3myvolDissociating an instant snapshot # vxsnap -g snapdg snapwait myvol mirvol=prepsnap Removing an instant snapshot Splitting an instant snapshot hierarchy# vxsnap -g mydg dis snap2myvol # vxedit -g mydg -r rm snap2myvol Displaying instant snapshot information # vxsnap -g mydg split snap2myvol# vxsnap -g diskgroup print vol # vxsnap -g mydg print # vxsnap -g diskgroup -l -v -x list vol # vxsnap -g dg -vx list Controlling instant snapshot synchronization Vxsnap -g diskgroup syncresume \Vxsnap -b -g diskgroup syncstart \ Vxsnap -g diskgroup syncstop vol volset # vxcache -g diskgroup listvol cacheobject Listing the snapshots created on a cacheImproving the performance of snapshot synchronization Tuning the autogrow attributes of a cache # vxcache -g mydg set highwatermark=60 cobjmydg Removing a cache Finally, remove the cache object and its cache volumeGrowing and shrinking a cache Creating traditional third-mirror break-off snapshots # vxassist -g diskgroup snapstart voldef # vxassist -b -g diskgroup snapstart nmirror=N volume# vxassist -g diskgroup snapwait volume Create a snapshot volume using the following command # vxassist -g diskgroup snapshot nmirror=N volume snapshot# vxassist -g diskgroup snapshot voldef snapvol # vxedit -g diskgroup -rf rm snapshot Converting a plex into a snapshot plex Reattaching a snapshot volume Creating multiple snapshots# vxplex -g diskgroup convert state=SNAPDONE plex # vxassist -g diskgroup -o allvols snapshot # vxassist -g diskgroup snapback nmirror=number snapshot Adding plexes to a snapshot volume# vxassist -g diskgroup -o allplexes snapback snapshot # vxassist snapclear snapshot Dissociating a snapshot volume# vxprint -g diskgroup -F%rid $DCOVOL Output from this command is shown in the following examples # vxassist snapprint volumeDisplaying snapshot information # vxassist -g mydg snapprint Adding a version 0 DCO and DCO volume Specifying storage for version 0 DCO plexes # vxdco -g mydg dis myvoldco Removing a version 0 DCO and DCO volume# vxdco -g diskgroup -o rm dis dcoobj # vxdco -g mydg att myvol myvoldco Reattaching a version 0 DCO and DCO volumeFor more information, see the vxdco1M manual Adding a version 0 DCO and DCO volume Creating and administering volume sets Creating a volume set Adding a volume to a volume set Listing details of volume sets Stopping and starting volume sets Removing a volume from a volume set Raw device node access to component volumes# vxvset -g diskgroup -f rmvol volset volume # vxvset -g mydg start set1 # vxvset -g mydg list set1 Enabling raw device access when creating a volume set # vxvset -g diskgroup -fset makedev=onoff vset Displaying the raw device access settings for a volume setControlling raw device access for an existing volume set # vxvset -g mydg set makedev=off myvset2 # vxvset -g mydg set compvolaccess=read-write myvset2# vxvset -g mydg set makedev=on myvset2 Raw device node access to component volumes Configuring off-host processing Implementing off-host processing solutions Example implementation of off-host processing Implementing off-host online backup # vxprint -g volumedg -F%instant volume # vxvol -g volumedg set fastresync=on volume # fsck -F vxfs /dev/vx/rdsk/snapvoldg/snapvol # vxdg deport snapvoldg# vxdg import snapvoldg Implementing decision support # vxsnap -g volumedg snapwait volume mirvol=snapvol# vxprint -g volumedg -F%instantvolume 375 # mount -F vxfs /dev/vx/dsk/snapvoldg/snapvol \ mountpoint You can then resume the procedure from on Implementing off-host processing solutions Administering hot-relocation How hot-relocation works 381 Mydg05 # vxrecover -b -g mydg home src Partial disk failure mail messages# vxstat -g mydg -s -ff home-02 src-02 Sd mydg01-04 Sd mydg01-06 Sd mydg02-03 Sd mydg02-04 Failing disks mydg02 Complete disk failure mail messagesHow space is chosen for relocation Configuring a system for hot-relocation Mydg mydg02 C0t2d0 658007 Displaying spare disk information# vxdg -g diskgroup spare # vxedit -g mydg set spare=on mydg01 Mark another disk as a spare? y,n,q,? default nMarking a disk as a hot-relocation spare Where diskname is the disk media name Removing a disk from use as a hot-relocation spareExcluding a disk from hot-relocation use Following confirmation is displayed # vxedit -g diskgroup set nohotuse=off diskname Making a disk available for hot-relocation useTo use vxdiskadm to exclude a disk from hot-relocation use Configuring hot-relocation to use only spare disks Moving and unrelocating subdisksSpare=only To root Subject Attempting VxVM relocation on host teal Moving and unrelocating subdisks using vxdiskadm Unrelocate to a new disk y,n,q,? default nStatus message is displayed at the end of the operation Enter the original disk name disk,list,q,? # vxassist -g mydg move home !mydg05 mydg02 Moving and unrelocating subdisks using vxassistMoving and unrelocating subdisks using vxunreloc Forcing hot-relocated subdisks to accept different offsets Moving hot-relocated subdisks back to their original diskMoving hot-relocated subdisks back to a different disk # vxprint -g mydg -se sdorigdmname=mydg01 Restarting vxunreloc after errorsExamining which subdisks were hot-relocated from a disk Nohup vxrelocd -o slow=IOdelay root Modifying the behavior of hot-relocation# nohup vxrelocd root Nohup vxrelocd root user1 user2 # nohup /etc/vx/bin/vxrelocd root user1 user2 Alternatively, you can use the following commandSee the vxrelocd1M manual page for more information Administering cluster functionality Overview of cluster volume management 399 Example of a 4-node cluster Private and shared disk groups Two types of disk groups are defined Activation modes of shared disk groups Exclusivewrite ewReadonly ro Sharedread sr Activation mode Description Enableactivation=true Defaultactivationmode=activation-modeActivation modes for shared disk groups Sharedwrite sw Connectivity policy of shared disk groups Global detach policy Local detach policy Disk group failure policy Guidelines for choosing detach and failure policies Effect of disk connectivity on cluster reconfiguration Limitations of shared disk groups Cluster initialization and configuration Cluster reconfiguration Vxclustadm utility Various reasons that may be given are shown in Table# /etc/vx/bin/vxclustadm nodestate state out of cluster Reason user initiated stop Node abort messages Reason Description Volume reconfiguration Vxconfigd daemon Vxconfigd daemon recovery # hagrp -freeze group # hagrp -unfreeze group Node shutdown Cluster shutdown Multiple host failover configurationsNode abort Import lock Failover Corruption of disk group configuration Where the reason can describe errors such as Administering VxVM in cluster environments # vxdctl -c modeRequesting node status and discovering the master node See the vxdctl1M manual page for more information Example output from this command is displayed here Example output from this command is as followsDetermining if a disk is shareable Listing shared disk groups Creating a shared disk group # vxdg -s init diskgroup diskname=devicename Importing disk groups as shared Forcibly importing a disk group# vxdg -s import diskgroup # vxdg -s -f import diskgroup Converting a disk group from shared to private Changing the activation mode on a shared disk group Setting the disk detach policy on a shared disk groupDefault disk detach policy is global # vxdg -g diskgroup set activation=mode Creating volumes with exclusive open access by a node Setting exclusive open access to a volume by a node# vxdg -g diskgroup set dgfailpolicy=dgdisableleave # vxvol -g dskgrp set exclusive=on volmir This command produces output similar to the following This command produces out put similar to the followingDisplaying the cluster protocol version Displaying the supported cluster protocol version range Obtaining cluster performance statistics Recovering volumes in shared disk groupsUpgrading the cluster protocol version Vol Vol1 2421 600000 99.0 # vxstat -b Administering VxVM in cluster environments Administering Sites and remote mirrors Site-consistent volume with two plexes at each of two sites Example of a two-site configuration with remote storage only To remove the site name from a host, use this command Configuring sites for hosts and disksConfiguring site-based allocation on a disk group Configuring site consistency on a disk group Configuring site consistency on a volume# vxdg -g diskgroup set siteconsistent=on # vxdg -g diskgroup set siteconsistent=off RAID-5 volumes in a site-consistent disk group Setting the siteread policy on a volumeSite-based allocation of storage to volumes # vxassist -g diskgroup make volume size mirror=site Examples of storage allocation using sites Command Description Turn on site consistency for each volume in the disk group Making an existing disk group site consistentRegister a site record for each site with the disk group Turn on site consistency for the disk group Fire drill testing the configuration Recovery from simulated site failureAutomatic site reattachment Simulating site failure # kill -9 PID Failure scenarios and recovery procedures# ps -afe Recovery from a loss of site connectivity Recovery from host failureRecovery from storage failure Failure scenario Recovery technique Recovery from site failure Failure scenarios and recovery procedures About Storage Expert See the vxse1M manual Before using Storage Expert How Storage Expert worksRunning Storage Expert One of the following keywords must be specified Displaying rule attributes and their default values Discovering what a rule doesRunning a rule # vxsestripes2 info Setting rule attributes Rule result types Identifying configuration problems using Storage Expert Recovery time Vxseraid5log1 Checking minimum and maximum RAID-5 log sizes vxseraid5log2Checking for non-mirrored RAID-5 logs vxseraid5log3 Checking disk group configuration copies and logs vxsedg2 Checking on disk config size vxsedg3Disk groups Checking the version number of disk groups vxsedg4 Checking for non-imported disk groups vxsedg6 Checking volume redundancy vxseredundancyChecking states of plexes and volumes vxsevolplex Disk striping Checking the number of columns in RAID-5 volumes vxseraid5Volumes needing recovery Disk sparing and relocation management Hardware failuresRootability System name Checking the system name vxsehost Rule definitions and attributes Rule Description See Running a rule on Rule Attribute Default Description Value Nsdthreshold Vxseraid5 Toonarrowraid5Toowideraid5 R5maxsize Vxseredundancy Volumeredundancy Vxsestripes1 Default stripeunitVxserootmir Vxsespares Vxsevolplex Rule definitions and attributes Performance guidelines Data assignment Striping Mirroring Combining mirroring and striping RAID-5 Volume read policies Obtaining performance data Setting performance prioritiesPerformance monitoring By vxtrace Tracing volume operationsPrinting volume statistics Using performance data Using I/O statistics Following is an extract from typical output # vxprint -g mydg -tvh archiveMydg03-03 Archive-01 40960 C1t2d0 # vxassist -g mydg move archive !mydg03 destdisk 471 Using I/O tracing Tuning VxVMGeneral tuning guidelines Tuning guidelines for large systems Number of configuration copies for a disk group # vxedit set nconfig=5 bigdg Changing the values of tunables# vxdmpadm settune dmptunable=value # vxdmpadm gettune dmptunable Tunable parameters Dmppathswitchblksshift DmphealthtimeDmploglevel Dmppathage Dmprestorecycles DmprestoreintervalDmpprobeidlelun Dmpqueuedepth Dmprestorepolicy DmpretrytimeoutDmpscsitimeout Dmpretrycount Volcheckptdefault VoldefaultiodelayDmpstatinterval Volfmrlogsz Volmaxioctl VolmaxvolVolmaxio Volsubdisknum VolmaxparallelioVolmaxspecialio Volcvmsmartsync VoldrlmaxdrtregsVoldrlmaxseqdirty Voldrlminregionsz Voliotiobufdefault VoliomemmaxpoolszVolioterrbufdflt Voliotiobuflimit Volpagemodmaxmemsz VoliotiobufmaxVoliotmaxopen Volraidminpoolsz Volraidrsrtransmax Tuning VxVM Commands summary If you are using a C shell csh or tcsh, use the commands Vxdisk -g diskgroup list diskname # vxdisk -g mydg listVxdg list diskgroup # vxdg list mydg Table A-2 Administering disks Command Description # vxedit -g mydg rename \ Mydg03 mydg02# vxedit -g mydg set \ Reserve=on mydg02 # vxdiskunsetup c0t3d0 Spare=on mydg04Spare=off mydg04 # vxdisk offline c0t1d0 # vxdg init mydg \ Mydg01=c0t1d0# vxsplitlines -g mydg # vxdg -n newdg deport mydg # vxdg -o expand listmove \ Mydg newdg myvol1Newdg myvol1 Newdg myvol2 myvol3 # vxrecover -g mydg -sb # vxdg destroy mydg# vxmake -g mydg sd \ Mydg02-01 mydg02,0,8000 # vxsd -g mydg assoc \ Vol01-01 mydg10-010 \Mydg11-011 mydg12-012 Mydg02-01 # vxsd -g mydg -o rm dis \ # vxmake -g mydg plex \Vol01-02 \ Sd=mydg02-01,mydg02-02 Vol02-02 vol02-03 Vol01-02# vxplex -g mydg mv \ Table A-6 Creating volumes Command Description # vxplex -g mydg cp vol02 \Vol03-01 Vol02-02 Volume length layout=striperaid5 \ Vxassist -b -g diskgroup make \Stripeunit=W ncol=N attributes Mysvol 20g layout=stripe \ # vxmake -g mydg -Uraid5\ Vol r5vol \Plex=raidplex,raidlog1,\ Raidlog2 Table A-7 Administering volumes Command Description Drl=onsequentialoff # vxsnap -gmydg prepare \ myvol drl=on # vxassist -g mydg make \ Cvol 1g layout=mirror \Init=active mydg16 mydg17 Cachevolname=cvol # vxsnap -g mydg unprepare \ myvol # vxassist -gmydg relayout \ vol2 layout=stripeVol3 layout=raid5 \ Stripeunit=16 ncol=4 # vxrecover -g mydg \ # vxassist -g mydg remove \ myvolMytask -b mydg05 # vxtask -h -g mydg list # vxtask pause mytask # vxtask -p -g mydg list# vxtask resume mytask # vxtask abort mytask Administrative commands Online manual pages Table A-9 Manual pages Name Description Vxrecover VxmendVxmirror Vxnotify File formats Device driver interfaces Configuring Veritas Volume Manager Setup tasks after installation Adding disks to disk groups Adding unsupported disk arrays as JBODsAdding foreign devices Guidelines for configuring storage Mirroring guidelines Dirty region logging guidelines Striping guidelines RAID-5 guidelines Hot-relocation guidelines 517 Controlling VxVM’s view of multipathed devices Configuring cluster supportAccessing volume devices Configuring shared disk groups Converting existing VxVM disk groups to shared disk groups Reconfiguration tasks Glossary Single copy of a configuration database Page See disk enclosure Jbod Page RAID Page Page Page Index CDS Page CVM Page Page DMP Empty Page Page Page Page Iofail 228 Nodarec 228 Nodevice 228 Recover 228 Removed Page Page Page Page Page TPD Page Detached Disabled Enabled Page Page Page Page Page