Veritas Volume Manager Administrator’s Guide
Legal Notices
Page
Contents
Setting up a VxVM root disk and mirror
Using vxdiskadd to place a disk under control of VxVM
VxVM root disk volume restrictions
Booting root volumes
136
Taking a disk offline 118
119
133
185
167
207
217
Setting default values for vxassist 241
215
Displaying subdisk information 216
274
252
275
290
Moving volumes from a VM disk 290
288
289
356
Chapter Creating and administering volume sets
390
386
387
388
441
434
512
Setup tasks after installation 511
463
507
521 531
Understanding Veritas Volume Manager
Veritas Enterprise Administrator User’s Guide
How data is stored
VxVM and the operating system
Physical objects-physical disks
How VxVM handles storage management
Disk arrays
Device discovery
Multipathed disk arrays
Operating system
Enclosure-based naming
Fibre Channel hub or switch
C2t99d0
Provide redundant loop access
Combining virtual objects in VxVM
Virtual objects
Virtual objects in VxVM include the following
Connection between objects in VxVM
VM disks
Disk groups
Subdisks
Plexes
Volumes
How VxVM handles storage management
Disk01-01 vol06-01
Implementation of non-layered volumes
Volume layouts in VxVM
Implementation of layered volumes
Concatenation and spanning
Layout methods
Following sections describe each layout method
12 Example of concatenation Data Disk01-01
+1 n+2
Example of spanning Data
Striping RAID-0
Su1 Su2 Su3
Devname3
Devname2
Striping plus mirroring mirrored-stripe or RAID-0+1
Mirroring RAID-1
Mirroring plus striping striped-mirror, RAID-1+0 or RAID-10
17 Mirrored-stripe volume laid out on six disks
Column Mirror Striped plex
You need a full license to use this feature
Mirror volumes
RAID-5 striping with parity
Traditional RAID-5 arrays
Data Parity
Veritas Volume Manager RAID-5 arrays
Left-symmetric layout
22 Veritas Volume Manager RAID-5 array Stripe
Understanding Veritas Volume Manager
RAID-5 logging
Layered volumes
Example of a striped-mirror layered volume
Understanding Veritas Volume Manager
How online relayout works
Online relayout
Three columns of length 5L/3
Striped volume
Limitations of online relayout
Transformations and volume length
Transformation characteristics
Dirty flags
Volume resynchronization
Resynchronization process
Dirty region logs
Dirty region logging
Sequential DRL
Log subdisks and plexes
Data volume configuration
SmartSync recovery accelerator
Redo log volume configuration
Volume snapshots
Original
Space than original Volume
Comparison of snapshot features
Use on creation Requires less storage
FastResync
Non-persistent FastResync
FastResync enhancements
Persistent FastResync
DCO volume versioning
How non-persistent FastResync works with snapshots
Version 20 DCO volume layout
Version 0 DCO volume layout
How persistent FastResync works with snapshots
Where the size of each map in bytes is
DCO
FastResync
Effect of growing a volume on the FastResync map
FastResync limitations
Hot-relocation
Volume sets
Volume sets
Disk devices
Administering disks
Operating system-based naming
Disk device naming in VxVM
There are two different methods of naming disk devices
# vxdisk path egrep diskname
Private and public disk regions
Format for boot, root or swap disks, for mirrors or
Configures disk access records for them automatically
To boot the system.Typically, most disks on a system are
Configured as this disk type. However, it is not a suitable
Next example discovers fabric devices
Discovering and configuring newly added disk devices
Following command scans for the devices c1t1d0 and c2t2d0
Partial device discovery
Disk categories
Discovering disks and dynamically adding disk arrays
For more information, see the vxdisk1M manual
Removing support for a disk array
Third-party driver coexistence
Adding support for a new disk array
Enabling discovery of new devices
# vxddladm listsupport libname=libraryname.sl
Administering the Device Discovery Layer
Listing details of supported disk arrays
# vxddladm listsupport all
Re-including support for an excluded disk array library
Excluding support for a disk array library
Listing excluded disk arrays
Adding unsupported disk arrays to the Disks category
Listing supported disks in the Disks category
# vxdisk list
Length=serialnolength policy=ap
# vxddladm addjbod vid=SEAGATE pid=ST318404LSUN18G
# vxdmpadm listenclosure all
Adding foreign devices
Removing disks from the Disks category
# vxddladm rmjbod vid=SEAGATE
See Migrating between DMP and HP-UX native multipathing on
Placing disks under VxVM control
Legacy
Mode Format of output from VxVM command
Default
Changing the disk-naming scheme
# vxdmpadm getlungroup dmpnodename=disk25
VxVM vxdmpadm Error V-5-1-10910 Invalid da-name
Regenerating persistent device names
Restart the VxVM configuration demon
This regenerates the persistent name database
# vxdmpadm setattr enclosure enclosure tpdmode=nativepseudo
Changing device naming for TPD-controlled enclosures
# vxdmpadm getsubpaths dmpnodename=enclosure-basedname
# vxconfigd -kr reset
Persistent simple or nopriv disks in the boot disk group
Persistent simple or nopriv disks in non-boot disk groups
Installing and formatting disks
Re-import the disk group using the following command
Add or initialize disks Menu VolumeManager/Disk/AddDisks
Displaying and changing default disk layout attributes
Adding a disk to VxVM
C3t0d0 c3t1d0 c3t2d0 c3t3d0
Add site tag to disks? y,n,q,? default n
Continue operation? y,n,q,? default y y
Use default disk names for the disks? y,n,q,? default y
Exclude disks from hot-relocation use? y,n,q,? default n n
Following disks
Continue with operation? y,n,q,? default y y
Enter the desired format cdsdisk,hpdisk,q,? default cdsdisk
Enter desired private region length privlen,q,? default
Vxdiskadm then proceeds to add the disks
Using vxdiskadd to place a disk under control of VxVM
Add or initialize other disks? y,n,q,? default n
Reinitializing a disk
Rootability
Root disk mirrors
VxVM root disk volume restrictions
Setting up a VxVM root disk and mirror
Booting root volumes
# /etc/vx/bin/vxcplvmroot -b c0t4d0
# /etc/vx/bin/vxcplvmroot -m c1t1d0 -R 30 -v -b c0t4d0
# /etc/vx/bin/vxcplvmroot -R 30 -v -b c0t4d0
Creating an LVM root disk from a VxVM root disk
Display the initial crash dump configuration
Adding swap volumes to a VxVM rootable system
Adding persistent dump volumes to a VxVM rootable system
View the changed swap configuration
Dynamic LUN expansion
Removing a persistent dump volume
You can now remove the volume if required
# crashconf -ds /dev/vx/dsk/bootdg/dumpvol
109
Removing disks
# vxvol -g diskgroup stop volume1 volume2
VxVM Info V-5-2-268 Removal of disk mydg01 is complete
Continue with operation? y,n,q,? default y
Remove another disk? y,n,q,? default n
Removing a disk with subdisks
Removing a disk with no subdisks
# /usr/lib/vxvm/bin/vxdiskunsetup c#t#d#
Removing a disk from VxVM control
Removing and replacing disks
To replace a disk
Are you sure you want do this? y,n,q,? default n
# vxassist move mkting !mydg02
Removing and replacing disks
Following devices are available as replacements c0t1d0
Replacing a failed or removed disk
Following devices are available as replacements
Replace another disk? y,n,q,? default n
# vxreattach -r accesname
Then run the following command on the master node
Select a disk device to enable address,list,q,? c0t2d0
Enabling a disk
Disable another device? y,n,q,? default n
Enable another device? y,n,q,? default n
Taking a disk offline
Renaming a disk
You would use the following command to rename the disk
Reserving disks
See the vxedit1M manual page for more information
Displaying disk information
VxVM returns a display similar to the following
Enter disk device or all address,all,q,? default all
List disk information Menu VolumeManager/Disk/ListDisk
Displaying disk information with vxdiskadm
Displaying the Pfto values
Controlling Powerfail Timeout
Setting the Pfto values
To set the Pfto value on a disk, use the following command
Enabling or disabling Pfto
For example, to disable Pfto on the disk c5t0d6
Controlling Powerfail Timeout
How DMP works
Administering dynamic multipathing DMP
How DMP works
Enc00
Path failover mechanism
How DMP monitors I/O on paths
Throttling
Load balancing
# vxvol -g diskgroup stopall
Use the following commands to initiate the migration
DMP coexistence with HP-UX native multipathing
Migrating between DMP and HP-UX native multipathing
# vxvol -g diskgroup startall
Restart all the volumes in each disk group
Under the new naming scheme as
DMP in a clustered environment
Disabling multipathing and making devices invisible to VxVM
Disabling and enabling multipathing for specific devices
Enabling or disabling controllers with shared disk groups
Enabling multipathing and making devices visible to VxVM
135
Administering dynamic multipathing DMP
Displaying DMP database information
Typical output from the vxdisk list command is as follows
Displaying the paths to a disk
C1t0d3 state=enabled Type=secondary
Disabled Config
Disabled Log
Disabled Lockrgn
Retrieving information about a DMP node
Administering DMP using vxdmpadm
# vxdmpadm getdmpnode nodename=c3t2d1
# vxdmpadm getsubpaths dmpnodename=c2t66d0
Displaying the members of a LUN group
# vxdmpadm getdmpnode enclosure=enc0
# vxdmpadm getlungroup dmpnodename=c11t0d10
# vxdmpadm listctlr all
Displaying information about controllers
Displaying information about enclosures
Following is example output from this command
Displaying information about array ports
Displaying information about TPD-controlled devices
# vxdmpadm getsubpaths tpdnodename=emcpower10
# vxdmpadm gettpdnode nodename=c7t0d10
To reset the I/O counters to zero, use this command
Examples of using the vxdmpadm iostat command
Gathering and displaying I/O statistics
# vxdmpadm iostat show enclosure=Disk
# vxdmpadm iostat start memory=4096
# vxdmpadm iostat show pathname=c3t115d0
# vxdmpadm iostat show dmpnodename=c0t0d0
# vxdmpadm setattr path c1t20d0 pathtype=nopreferred
Setting the attributes of the paths to an enclosure
# vxdmpadm setattr path c2t10d0 pathtype=active
# vxdmpadm setattr path c3t10d0 pathtype=nomanual
Specifying the I/O policy
Displaying the I/O policy
Adaptiveminq
Following policies may be set
Adaptive
# vxdmpadm setattr enclosure enc1 iopolicy=adaptive
048
# vxdmpadm setattr arraytype A/A iopolicy=round-robin
This is the default I/O policy for Active/Active A/A arrays
# vxdmpadm setattr enclosure Disk iopolicy=minimumq
# vxdmpadm setattr arrayname Sena iopolicy=priority
# vxdisk list c3t2d15
Default setting for this attribute is useallpaths=no
Example of applying load balancing in a SAN
# vxdmpadm setattr arrayname Disk iopolicy=singleactive
# dd if=/dev/vx/rdsk/mydg/myvol1 of=/dev/null
DMP statistics are now reset
# vxdmpadm getattr enclosure ENC0 iopolicy
# vxdmpadm setattr enclosure ENC0 iopolicy=singleactive
# vxdmpadm -c-fdisable path=pathname
# vxdmpadm -c-fdisable ctlr=ctlrname
Disabling I/O for paths, controllers or array ports
Upgrading disk controller firmware
# vxdmpadm enable path=pathname
# vxdmpadm enable ctlr=ctlrname
Enabling I/O for paths, controllers or array ports
For the other controller on the HBA, enter
Re-enable the plex associated with the device
Renaming an enclosure
Stop I/O to all disks through one controller of the HBA
# vxdmpadm getattr enclosure enc0 recoveryoption
Configuring the response to I/O failures
# vxdmpadm setattr \
# vxdmpadm setattr arraytype A/A recoveryoption=default
Configuring the I/O throttling mechanism
# vxdmpadm setattr enclosure enc0 recoveryoption=nothrottle
# vxdmpadm getattr enclosure HDS9500-ALUA0 recoveryoption
Displaying recoveryoption values
HDS9500-ALUA0 Error-Retry
# vxdmpadm start restore interval=seconds policy=checkall
Configuring DMP path restoration policies
Displaying the status of the DMP path restoration thread
Stopping the DMP path restoration thread
This produces output such as the following
One daemon should be shown as running
Displaying information about the DMP error-handling thread
Configuring array policy modules
To add and configure an APM, use the following command
# vxdmpadm -r cfgapm modulename
Administering DMP using vxdmpadm
Creating and administering disk groups
Creating and administering disk groups
To nodg
Specifying a disk group to commands
Block special device corresponding to this volume is
System-wide reserved disk groups
See the vxdg1M manual page for more information
Rules for determining the default disk group
Displaying the system-wide boot disk group
# vxdg bootdg
# vxdisk -s list devicename
Displaying disk group information
# vxdg list
# vxdg list diskgroup
Displaying free space in a disk group
Creating a disk group
Following is example output
# vxdg -g diskgroup set cds=onoff
Adding a disk to a disk group
# vxdiskadd c1t0d0
# vxdg init mktdg mktdg01=c1t0d0
# vxdg -g diskgroup rmdisk diskname
# vxdiskunsetup devicename
# vxdiskunsetup c1t0d0
Removing a disk from a disk group
Deporting a disk group
Newdg
Importing a disk group
# vxdisk -s list
VxVM Info V-5-2-374 The import of newdg was successful
Select disk group to import group,list,q,? default list
Select another disk group? y,n,q,? default n
Handling disks with duplicated identifiers
Writing a new Udid to a disk
Option to the vxdg import command, as shown in this
# vxdisk -f-g diskgroup updateudid disk
# vxdisk updateudid c2t66d0 c2t67d0
# vxdisk listtag
# vxdg -o useclonedev=on -o updateid import mydg
# vxdisk -g diskgroup settag tagname disk
# vxdisk settag mytaggeddisks c2t66d0 c2t67d0
# vxdg -o useclonedev=on -o tag=mytaggeddisks import mydg
Enabling configuration database copies on tagged disks
Sample cases of operations on cloned disks
# vxdg -q listmeta diskgroup
Importing cloned disks without tags
# vxdg -g mydg set tagmeta=on tag=t1 nconfig=all nlog=all
# vxdisk -o alldgs list
To import only the cloned disks into the mydg disk group
Symmir command is used to split off the BCV device
# /usr/symcli/bin/symmir -g mydg split DEV001
Disks are tagged as follows
# vxdg -n newdg -o useclonedev=on -o updateid import mydg
Importing cloned disks with tags
State of the cloned disk is now shown as online clonedisk
# vxdisk set EMC08 clone=off # vxdisk -o alldgs list
Renaming a disk group
This command results in output such as the following
Dgid 774226267.1025.tweety
Moving disks between disk groups
# vxdg -tC -n newdg import diskgroup
Moving disk groups between systems
# vxrecover -g diskgroup -sb
# vxdg -C import diskgroup
Handling errors when importing disks
To clear the locks during import, use the following command
# vxdisk clearimport devicename
Reserving minor numbers for disk groups
Following error message indicates a recoverable error
# vxdg -f import diskgroup
# vxprint -g mydg reminor
# xvdg init newdg minor=30000 c1d0t0 c1t1d0
# vxdg -g diskgroup set maxdev=4079
Compatibility of disk groups between platforms
Example of a serial split brain condition in a cluster
Handling conflicting configuration copies
191 Typical arrangement of a 2-node campus cluster
Expected a = Expected B =
Automatically
Imported on host Y
# vxsplitlines -g newdg
Correcting conflicting configuration information
Reorganizing the contents of disk groups
Reorganizing the contents of disk groups
197
Disk group join operation
Limitations of disk group split and join
Listing objects potentially affected by a move
Moving DCO volumes between disk groups
201
Split Snapshot
Moving objects between disk groups
# vxdg -o expand move mydg rootdg mydg01
# vxprint
Splitting disk groups
Following commands would also achieve the same result
# vxdg -o expand split rootdg mydg rootdg07 rootdg08
# vxdg -o overrideverify join sourcedg targetdg
Joining disk groups
Mydg06 C1t98d0
Following command joins disk group mydg to rootdg
Disabling a disk group
Mydg Mydg05 C1t96d0
Use the disk group ID to import the disk group
Recovering a destroyed disk group
Destroying a disk group
Upgrading a disk group
Upgrading a disk group
Features supported by disk group versions
# vxdg upgrade dgname
To list the version of a disk group, use this command
# vxdg list dgname
# vxdg -T 120 init newdg newdg01=c0t3d0
Managing the configuration daemon in VxVM
# vxnotify -s
Backing up and restoring disk group configuration data
Using vxnotify to monitor configuration changes
# vxnotify -f
Using vxnotify to monitor configuration changes
# vxmake -g mydg sd mydg02-01 mydg02,0,8000
Creating subdisks
# vxprint -g diskgroup -l subdisk
This command provides the following output
Displaying subdisk information
# vxprint -st
Splitting subdisks
Moving subdisks
# vxsd -g mydg mv mydg03-01 mydg12-01 mydg12-02
Associating subdisks with plexes
Joining subdisks
# vxmake -g diskgroup plex plex sd=subdisk
# vxsd -g diskgroup assoc plex subdisk10 ... subdiskMN-1
# vxsd -g mydg assoc home-1 mydg02-01 mydg02-00 mydg02-01
# vxsd -g mydg -l 4096b assoc vol10-01 mydg15-01
# vxsd -g mydg -l 1 assoc vol02-01 mydg11-01
# vxsd -g mydg aslog vol01-02 mydg02-01
# vxassist -g diskgroup addlog volume disk
Associating log subdisks
# vxsd -g diskgroup aslog plex subdisk
Changing subdisk attributes
To remove a subdisk, use the following command
Dissociating subdisks from plexes
Removing subdisks
# vxedit -g mydg set putil0=DO-NOT-USE mydg02-01
# vxedit -g mydg set comment=subdisk comment mydg02-01
# vxmake -g mydg plex vol01-02 sd=mydg02-01,mydg02-02
Creating plexes
Displaying plex information
Creating a striped plex
Plex states
Active plex state
Clean plex state
Dcosnp plex state
Offline plex state
Empty plex state
Iofail plex state
LOG plex state
Temprm plex state
Snaptmp plex state
Stale plex state
Temp plex state
Plex condition flags
Plex kernel states
Disabled plex kernel state
Enabled plex kernel state
Attaching and associating plexes
# vxmend -g diskgroup off plex
Taking plexes offline
# vxmend -g mydg off vol01-02 vol02-02
Reattaching plexes
Detaching plexes
Moving plexes
Start the volume using the following command
# vxmend -g diskgroup fix clean plex
# vxvol -g diskgroup start volume
# vxplex -g diskgroup -o rm dis plex
Copying volumes to plexes
# vxplex -g diskgroup cp volume newplex
Dissociating and removing plexes
# vxedit -g mydg set putil0=DO-NOT-USE vol01-02
Changing plex attributes
# vxedit -g diskgroup set attribute=value ... plex
# vxedit -g mydg set comment=plex comment tutil2=u vol01-02
Creating volumes
RAID-5
Types of volume layouts
Mirror and concatenated-mirror volumes
Supported volume logs and maps
Advanced approach
Creating a volume
Assisted approach
Using vxassist
Vxassist
# vxassist options make volume length attributes
Following is a sample vxassist defaults file
Setting default values for vxassist
# vxassist -g dgrp maxsize layout=raid5 nlog=2
Discovering the maximum size of a volume
Disk group alignment constraints on volumes
# vxassist -g diskgroup maxsize layout=layout attributes
# vxprint -g diskgroup -G -F %align
Creating a volume on any disk
# vxassist -b -g diskgroup make volume length
# vxassist -b make voldefault 10g
# vxassist -b -g mydg make volspec 5g mydg03 mydg04
Creating a volume on specific disks
# vxassist -b -g mydg make volspec 5g ctlrc1 !targetc1t5
Specifying ordered allocation of storage to volumes
Mydg01 mydg02 mydg03 mydg04 mydg05 mydg06 mydg07 mydg08
Stripe volume
Volume across controllers
# vxassist -b -g mydg make volmir 5g layout=mirror
Creating a mirrored volume
Creating a mirrored-concatenated volume
Creating a concatenated-mirror volume
Creating a volume with a version 0 DCO volume
# vxvol -g diskgroup set logtype=drldrlseq volume
# vxdg -T 90 upgrade diskgroup
For more information, see Upgrading a disk group on
Creating a volume with dirty region logging enabled
Creating a volume with a version 20 DCO volume
# vxdg upgrade diskgroup
# vxassist -b -g diskgroup make volume length layout=stripe
Creating a striped volume
# vxassist -b -g mydg make volzebra 10g layout=stripe
Creating a striped-mirror volume
Creating a mirrored-stripe volume
Mirroring across targets, controllers or enclosures
# vxassist -b -g mydg make volraid 10g layout=raid5 nlog=2
Creating a RAID-5 volume
# vxassist -g diskgroup listtag volume
Creating tagged volumes
# vxassist -g diskgroup list tag=tagname volume
Creating a volume using vxmake
# vxmake -g diskgroup -d descriptionfile
Creating a volume using a vxmake description file
# vxassist -b -g diskgroup make volume length layout=mirror
Initializing and starting a volume
Mydg04-021/8000,mydg04-031/16000
# vxvol -g diskgroup init zero volume
Initializing and starting a volume created using vxmake
# vxvol -g diskgroup init enable volume
# vxvol -g diskgroup init active volume
Accessing a volume
Administering volumes
Vxprint command can also be applied to a single disk group
Displaying volume information
This is example output from this command
Empty volume state
Volume states
Active volume state
Clean volume state
Replay volume state
Volume kernel states
Invalid volume state
Needsync volume state
Monitoring and controlling tasks
Detached volume kernel state
Disabled volume kernel state
Enabled volume kernel state
Vxtask operations
Managing tasks with vxtask
Using the vxtask command
Putting a volume in maintenance mode
Stopping a volume
Adding a mirror to a volume
Starting a volume
To start all Disabled volumes, enter
# /etc/vx/bin/vxmirror -d yes
Mirroring all volumes
Mirroring volumes on a VM disk
# /etc/vx/bin/vxmirror -g diskgroup -a
At the following prompt, press Return to make the mirror
Mirror volumes on another disk? y,n,q,? default n
# vxassist -gdiskgroup remove mirror volume
Removing a mirror
Adding logs and maps to volumes
Preparing a volume for DRL and instant snapshots
Specifying storage for version 20 DCO plexes
# vxprint -g diskgroup -F%version $DCONAME
Using a DCO and DCO volume with a RAID-5 volume
# DCONAME=‘vxprint -g diskgroup -F%dconame volume‘
Determining the DCO version number
Disabling and re-enabling DRL
Determining if DRL is enabled on a volume
Determining if DRL logging is active on a volume
This command returns on if DRL logging is enabled
To re-enable DRL on a volume, enter this command
Upgrading existing volumes to use version 20 DCOs
To re-enable sequential DRL on a volume, enter
Use the following command on the volume to upgrade it
Adding traditional DRL logging to a mirrored volume
# vxassist -g mydg addlog vol03 logtype=drl
To remove a DRL log, use the vxassist command as follows
# vxassist -g mydg addlog volume logtype=drlseq nlog=n
Removing a traditional DRL log
# vxassist -g mydg addlog volraid
Adding a RAID-5 log using vxplex
# vxassist -b -g diskgroup addlog volume loglen=length
Adding a RAID-5 log
Removing a RAID-5 log
Resizing a volume
# vxprint -g diskgroup -ht volume
# vxassist -g diskgroup maxgrow volume
Unmounted File System
Resizing volumes using vxresize
Online JFS Full Base JFS Lite
VxFS Mounted File System
Extending to a given length
Resizing volumes using vxassist
Extending by a given length
Shrinking to a given length
Resizing volumes using vxvol
Shrinking by a given length
# vxassist -g diskgroup removetag volume tagname
Setting tags on volumes
Changing the read policy for mirrored volumes
Moving volumes from a VM disk
Removing a volume
To set the read policy to select, use the following command
VxVM Info V-5-2-188 Evacuation of disk mydg02 is complete
To move volumes from a disk
Move volumes from another disk? y,n,q,? default n
VxVM vxevac Info
# vxvol -g diskgroup set fastresync=on volume
Enabling FastResync on a volume
Disabling FastResync
Checking whether FastResync is enabled on a volume
# vxprint -g diskgroup -F%fastresync volume
# vxprint -g diskgroup -F%hasdcolog volume
# vxassist -g mydg relayout vol02 layout=stripe
Performing online relayout
Permitted relayout transformations
Mirror-concat Mirror-stripe
Supported relayout transformations for RAID-5 volumes
Relayout to From raid5 Concat
Relayout to From mirror-concat Concat
Relayout to From stripe or stripe-mirror Concat
Layered striped-mirror volumes
Relayout to From mirror-stripe Concat
Supported relayout transformations for unmirrored stripe
# vxassist -g fsgrp relayout vol04 layout=raid5 ncol=4
Specifying a non-default layout
Specifying a plex for relayout
Tagging a relayout operation
Viewing the status of a relayout
To resume the operation, use the vxtask command
Controlling the progress of a relayout
# vxassist -g mydg relayout vol1 ncol=5
Converting between layered and non-layered volumes
# vxrelayout -g mydg -o bg,slow=1000,iosize=10m start vol04
# vxrelayout -g mydg -o bg reverse vol04
# vxassist -g mydg convert vol1 layout=mirror-stripe
Converting between layered and non-layered volumes
Administering volume snapshots
Administering volume snapshots
Traditional third-mirror break-off snapshots
Independent Volume Vxassist snapclear
Traditional third-mirror break-off snapshots
Full-sized instant snapshots
308 Administering volume snapshots
Cycle Start Vxsnap make Vxsnap refresh Vxsnap prepare
Space-optimized instant snapshots
Emulation of third-mirror break-off snapshots
Linked break-off snapshot volumes
Cascaded snapshots
Cascaded snapshots
Creating a snapshot of a snapshot
Creating a snapshot of a snapshot
Create instant snapshot S2 of S1 Vxsnap make source=S1
Vxsnap dis S2
Creating multiple snapshots
Restoring the original volume from a snapshot
Restoring the original volume from a snapshot
Creating instant snapshots
Creating instant snapshots
Preparing to create instant and break-off snapshots
Creating a shared cache object
# RSZ=‘vxprint -g diskgroup -F%regionsz $DCONAME‘
# LEN=‘vxprint -g diskgroup -F%len volume‘
For example to start the cache object, cobjmydg
# vxcache -g mydg start cobjmydg
Creating and managing space-optimized instant snapshots
Creating instant snapshots
# fsck -F vxfs /dev/vx/rdsk/diskgroup/snapshot
# vxsnap -g mydg syncwait snap2myvol
Creating and managing full-sized instant snapshots
# vxsnap -g mydg make source=myvol/snapvol=snap1myvol
# vxsnap -g diskgroup syncwait snapvol
# vxprint -gdiskgroup -F%incompletesnapvol
# vxsnap -g mydg addmir vol1 nmirror=2 alloc=mydg10,mydg11
Creating and managing third-mirror break-off snapshots
# vxsnap -g mydg snapwait vol1 nmirror=2
# vxsnap -g diskgroup -b addmir volume mirvol=snapvol \
Creating and managing linked break-off snapshot volumes
Mirdg=snapdg
# vxsnap -g diskgroup make source=vol1/snapvol=snapvol1 \
Reattach the snapshot volume with the original volume. See
Creating multiple instant snapshots
# vxsnap -g diskgroup make \
# vxvset -g mydg list vset1
Creating instant snapshots of volume sets
# vxvset -g mydg list snapvset1
Svol1 409600
Svol0 204800
Svol2 614400
Removing a snapshot mirror
Adding snapshot mirrors to a volume
# vxsnap -g mydg rmmir vol1
# vxsnap -g mydg rmmir vol1 mirvol=prepsnap mirdg=mysnapdg
Removing a linked break-off snapshot volume
Adding a snapshot to a cascaded snapshot hierarchy
Refreshing an instant snapshot
# vxsnap -g mydg reattach snapmyvol source=myvol nmirror=1
Reattaching an instant snapshot
# vxsnap -g mydg snapwait myvol nmirror=1
Reattaching a linked break-off snapshot volume
# vxsnap -g snapdg snapwait myvol mirvol=prepsnap
Restoring a volume from an instant snapshot
# vxsnap -g mydg restore myvol source=snap3myvol
Dissociating an instant snapshot
# vxedit -g mydg -r rm snap2myvol
Removing an instant snapshot
Splitting an instant snapshot hierarchy
# vxsnap -g mydg dis snap2myvol
# vxsnap -g mydg print
Displaying instant snapshot information
# vxsnap -g mydg split snap2myvol
# vxsnap -g diskgroup print vol
# vxsnap -g dg -vx list
# vxsnap -g diskgroup -l -v -x list vol
Vxsnap -g diskgroup syncstop vol volset
Controlling instant snapshot synchronization
Vxsnap -g diskgroup syncresume \
Vxsnap -b -g diskgroup syncstart \
Improving the performance of snapshot synchronization
Listing the snapshots created on a cache
# vxcache -g diskgroup listvol cacheobject
# vxcache -g mydg set highwatermark=60 cobjmydg
Tuning the autogrow attributes of a cache
Growing and shrinking a cache
Finally, remove the cache object and its cache volume
Removing a cache
Creating traditional third-mirror break-off snapshots
# vxassist -g diskgroup snapwait volume
# vxassist -b -g diskgroup snapstart nmirror=N volume
# vxassist -g diskgroup snapstart voldef
# vxedit -g diskgroup -rf rm snapshot
Create a snapshot volume using the following command
# vxassist -g diskgroup snapshot nmirror=N volume snapshot
# vxassist -g diskgroup snapshot voldef snapvol
Converting a plex into a snapshot plex
# vxassist -g diskgroup -o allvols snapshot
Reattaching a snapshot volume
Creating multiple snapshots
# vxplex -g diskgroup convert state=SNAPDONE plex
# vxassist -g diskgroup -o allplexes snapback snapshot
Adding plexes to a snapshot volume
# vxassist -g diskgroup snapback nmirror=number snapshot
# vxprint -g diskgroup -F%rid $DCOVOL
Dissociating a snapshot volume
# vxassist snapclear snapshot
# vxassist -g mydg snapprint
Output from this command is shown in the following examples
# vxassist snapprint volume
Displaying snapshot information
Adding a version 0 DCO and DCO volume
Specifying storage for version 0 DCO plexes
# vxdco -g diskgroup -o rm dis dcoobj
Removing a version 0 DCO and DCO volume
# vxdco -g mydg dis myvoldco
For more information, see the vxdco1M manual
Reattaching a version 0 DCO and DCO volume
# vxdco -g mydg att myvol myvoldco
Adding a version 0 DCO and DCO volume
Creating and administering volume sets
Adding a volume to a volume set
Creating a volume set
Stopping and starting volume sets
Listing details of volume sets
# vxvset -g mydg start set1 # vxvset -g mydg list set1
Removing a volume from a volume set
Raw device node access to component volumes
# vxvset -g diskgroup -f rmvol volset volume
Enabling raw device access when creating a volume set
Controlling raw device access for an existing volume set
Displaying the raw device access settings for a volume set
# vxvset -g diskgroup -fset makedev=onoff vset
# vxvset -g mydg set makedev=on myvset2
# vxvset -g mydg set compvolaccess=read-write myvset2
# vxvset -g mydg set makedev=off myvset2
Raw device node access to component volumes
Configuring off-host processing
Example implementation of off-host processing
Implementing off-host processing solutions
# vxprint -g volumedg -F%instant volume
Implementing off-host online backup
# vxvol -g volumedg set fastresync=on volume
# vxdg import snapvoldg
# vxdg deport snapvoldg
# fsck -F vxfs /dev/vx/rdsk/snapvoldg/snapvol
# vxprint -g volumedg -F%instantvolume
# vxsnap -g volumedg snapwait volume mirvol=snapvol
Implementing decision support
375
# mount -F vxfs /dev/vx/dsk/snapvoldg/snapvol \ mountpoint
You can then resume the procedure from on
Implementing off-host processing solutions
Administering hot-relocation
How hot-relocation works
381
Mydg05
Sd mydg01-04 Sd mydg01-06 Sd mydg02-03 Sd mydg02-04
# vxrecover -b -g mydg home src
Partial disk failure mail messages
# vxstat -g mydg -s -ff home-02 src-02
How space is chosen for relocation
Complete disk failure mail messages
Failing disks mydg02
Configuring a system for hot-relocation
# vxdg -g diskgroup spare
Displaying spare disk information
Mydg mydg02 C0t2d0 658007
Marking a disk as a hot-relocation spare
Mark another disk as a spare? y,n,q,? default n
# vxedit -g mydg set spare=on mydg01
Following confirmation is displayed
Where diskname is the disk media name
Removing a disk from use as a hot-relocation spare
Excluding a disk from hot-relocation use
To use vxdiskadm to exclude a disk from hot-relocation use
Making a disk available for hot-relocation use
# vxedit -g diskgroup set nohotuse=off diskname
To root Subject Attempting VxVM relocation on host teal
Configuring hot-relocation to use only spare disks
Moving and unrelocating subdisks
Spare=only
Enter the original disk name disk,list,q,?
Moving and unrelocating subdisks using vxdiskadm
Unrelocate to a new disk y,n,q,? default n
Status message is displayed at the end of the operation
Moving and unrelocating subdisks using vxunreloc
Moving and unrelocating subdisks using vxassist
# vxassist -g mydg move home !mydg05 mydg02
Moving hot-relocated subdisks back to a different disk
Moving hot-relocated subdisks back to their original disk
Forcing hot-relocated subdisks to accept different offsets
Examining which subdisks were hot-relocated from a disk
Restarting vxunreloc after errors
# vxprint -g mydg -se sdorigdmname=mydg01
Nohup vxrelocd root user1 user2
Nohup vxrelocd -o slow=IOdelay root
Modifying the behavior of hot-relocation
# nohup vxrelocd root
See the vxrelocd1M manual page for more information
Alternatively, you can use the following command
# nohup /etc/vx/bin/vxrelocd root user1 user2
Administering cluster functionality
Overview of cluster volume management
399
Example of a 4-node cluster
Two types of disk groups are defined
Private and shared disk groups
Sharedread sr
Activation modes of shared disk groups
Exclusivewrite ew
Readonly ro
Sharedwrite sw
Activation mode Description
Enableactivation=true Defaultactivationmode=activation-mode
Activation modes for shared disk groups
Connectivity policy of shared disk groups
Global detach policy
Local detach policy
Disk group failure policy
Guidelines for choosing detach and failure policies
Limitations of shared disk groups
Effect of disk connectivity on cluster reconfiguration
Cluster reconfiguration
Cluster initialization and configuration
Reason user initiated stop
Vxclustadm utility
Various reasons that may be given are shown in Table
# /etc/vx/bin/vxclustadm nodestate state out of cluster
Reason Description
Node abort messages
Volume reconfiguration
Vxconfigd daemon recovery
Vxconfigd daemon
# hagrp -unfreeze group
# hagrp -freeze group
Node shutdown
Node abort
Multiple host failover configurations
Cluster shutdown
Failover
Import lock
Where the reason can describe errors such as
Corruption of disk group configuration
See the vxdctl1M manual page for more information
Administering VxVM in cluster environments
# vxdctl -c mode
Requesting node status and discovering the master node
Listing shared disk groups
Example output from this command is displayed here
Example output from this command is as follows
Determining if a disk is shareable
# vxdg -s init diskgroup diskname=devicename
Creating a shared disk group
# vxdg -s -f import diskgroup
Importing disk groups as shared
Forcibly importing a disk group
# vxdg -s import diskgroup
Converting a disk group from shared to private
# vxdg -g diskgroup set activation=mode
Changing the activation mode on a shared disk group
Setting the disk detach policy on a shared disk group
Default disk detach policy is global
# vxvol -g dskgrp set exclusive=on volmir
Creating volumes with exclusive open access by a node
Setting exclusive open access to a volume by a node
# vxdg -g diskgroup set dgfailpolicy=dgdisableleave
Displaying the supported cluster protocol version range
This command produces output similar to the following
This command produces out put similar to the following
Displaying the cluster protocol version
Upgrading the cluster protocol version
Recovering volumes in shared disk groups
Obtaining cluster performance statistics
# vxstat -b
Vol Vol1 2421 600000 99.0
Administering VxVM in cluster environments
Administering Sites and remote mirrors
Site-consistent volume with two plexes at each of two sites
Example of a two-site configuration with remote storage only
Configuring site-based allocation on a disk group
Configuring sites for hosts and disks
To remove the site name from a host, use this command
# vxdg -g diskgroup set siteconsistent=off
Configuring site consistency on a disk group
Configuring site consistency on a volume
# vxdg -g diskgroup set siteconsistent=on
Site-based allocation of storage to volumes
Setting the siteread policy on a volume
RAID-5 volumes in a site-consistent disk group
# vxassist -g diskgroup make volume size mirror=site
Command Description
Examples of storage allocation using sites
Turn on site consistency for the disk group
Turn on site consistency for each volume in the disk group
Making an existing disk group site consistent
Register a site record for each site with the disk group
Simulating site failure
Fire drill testing the configuration
Recovery from simulated site failure
Automatic site reattachment
# ps -afe
Failure scenarios and recovery procedures
# kill -9 PID
Failure scenario Recovery technique
Recovery from a loss of site connectivity
Recovery from host failure
Recovery from storage failure
Recovery from site failure
Failure scenarios and recovery procedures
See the vxse1M manual
About Storage Expert
One of the following keywords must be specified
Before using Storage Expert
How Storage Expert works
Running Storage Expert
# vxsestripes2 info
Displaying rule attributes and their default values
Discovering what a rule does
Running a rule
Rule result types
Setting rule attributes
Recovery time
Identifying configuration problems using Storage Expert
Checking for non-mirrored RAID-5 logs vxseraid5log3
Checking minimum and maximum RAID-5 log sizes vxseraid5log2
Vxseraid5log1
Checking the version number of disk groups vxsedg4
Checking disk group configuration copies and logs vxsedg2
Checking on disk config size vxsedg3
Disk groups
Checking states of plexes and volumes vxsevolplex
Checking volume redundancy vxseredundancy
Checking for non-imported disk groups vxsedg6
Volumes needing recovery
Checking the number of columns in RAID-5 volumes vxseraid5
Disk striping
System name
Disk sparing and relocation management
Hardware failures
Rootability
Checking the system name vxsehost
Rule definitions and attributes
See Running a rule on
Rule Description
Rule Attribute Default Description Value
R5maxsize
Nsdthreshold
Vxseraid5 Toonarrowraid5
Toowideraid5
Vxsespares
Vxseredundancy Volumeredundancy
Vxsestripes1 Default stripeunit
Vxserootmir
Vxsevolplex
Rule definitions and attributes
Data assignment
Performance guidelines
Mirroring
Striping
RAID-5
Combining mirroring and striping
Volume read policies
Performance monitoring
Setting performance priorities
Obtaining performance data
Printing volume statistics
Tracing volume operations
By vxtrace
Using I/O statistics
Using performance data
# vxassist -g mydg move archive !mydg03 destdisk
Following is an extract from typical output
# vxprint -g mydg -tvh archive
Mydg03-03 Archive-01 40960 C1t2d0
471
General tuning guidelines
Tuning VxVM
Using I/O tracing
Number of configuration copies for a disk group
Tuning guidelines for large systems
# vxdmpadm gettune dmptunable
# vxedit set nconfig=5 bigdg
Changing the values of tunables
# vxdmpadm settune dmptunable=value
Tunable parameters
Dmppathage
Dmppathswitchblksshift
Dmphealthtime
Dmploglevel
Dmpqueuedepth
Dmprestorecycles
Dmprestoreinterval
Dmpprobeidlelun
Dmpretrycount
Dmprestorepolicy
Dmpretrytimeout
Dmpscsitimeout
Volfmrlogsz
Volcheckptdefault
Voldefaultiodelay
Dmpstatinterval
Volmaxio
Volmaxvol
Volmaxioctl
Volmaxspecialio
Volmaxparallelio
Volsubdisknum
Voldrlminregionsz
Volcvmsmartsync
Voldrlmaxdrtregs
Voldrlmaxseqdirty
Voliotiobuflimit
Voliotiobufdefault
Voliomemmaxpoolsz
Volioterrbufdflt
Voliotmaxopen
Voliotiobufmax
Volpagemodmaxmemsz
Volraidrsrtransmax
Volraidminpoolsz
Tuning VxVM
If you are using a C shell csh or tcsh, use the commands
Commands summary
# vxdg list mydg
Vxdisk -g diskgroup list diskname
# vxdisk -g mydg list
Vxdg list diskgroup
Table A-2 Administering disks Command Description
Reserve=on mydg02
# vxedit -g mydg rename \
Mydg03 mydg02
# vxedit -g mydg set \
# vxdisk offline c0t1d0
# vxdiskunsetup c0t3d0
Spare=on mydg04
Spare=off mydg04
# vxdg -n newdg deport mydg
# vxdg init mydg \
Mydg01=c0t1d0
# vxsplitlines -g mydg
Newdg myvol2 myvol3
# vxdg -o expand listmove \
Mydg newdg myvol1
Newdg myvol1
Mydg02-01 mydg02,0,8000
# vxrecover -g mydg -sb
# vxdg destroy mydg
# vxmake -g mydg sd \
Mydg02-01
# vxsd -g mydg assoc \
Vol01-01 mydg10-010 \
Mydg11-011 mydg12-012
Sd=mydg02-01,mydg02-02
# vxsd -g mydg -o rm dis \
# vxmake -g mydg plex \
Vol01-02 \
# vxplex -g mydg mv \
Vol01-02
Vol02-02 vol02-03
Vol02-02
Table A-6 Creating volumes Command Description
# vxplex -g mydg cp vol02 \
Vol03-01
Mysvol 20g layout=stripe \
Volume length layout=striperaid5 \
Vxassist -b -g diskgroup make \
Stripeunit=W ncol=N attributes
Raidlog2
# vxmake -g mydg -Uraid5\
Vol r5vol \
Plex=raidplex,raidlog1,\
Table A-7 Administering volumes Command Description
# vxsnap -gmydg prepare \ myvol drl=on
Drl=onsequentialoff
Cachevolname=cvol
# vxassist -g mydg make \
Cvol 1g layout=mirror \
Init=active mydg16 mydg17
Stripeunit=16 ncol=4
# vxsnap -g mydg unprepare \ myvol
# vxassist -gmydg relayout \ vol2 layout=stripe
Vol3 layout=raid5 \
# vxtask -h -g mydg list
# vxrecover -g mydg \
# vxassist -g mydg remove \ myvol
Mytask -b mydg05
# vxtask abort mytask
# vxtask pause mytask
# vxtask -p -g mydg list
# vxtask resume mytask
Online manual pages
Administrative commands
Table A-9 Manual pages Name Description
Vxnotify
Vxrecover
Vxmend
Vxmirror
Device driver interfaces
File formats
Setup tasks after installation
Configuring Veritas Volume Manager
Adding foreign devices
Adding unsupported disk arrays as JBODs
Adding disks to disk groups
Guidelines for configuring storage
Mirroring guidelines
Striping guidelines
Dirty region logging guidelines
Hot-relocation guidelines
RAID-5 guidelines
517
Accessing volume devices
Configuring cluster support
Controlling VxVM’s view of multipathed devices
Converting existing VxVM disk groups to shared disk groups
Configuring shared disk groups
Reconfiguration tasks
Glossary
Single copy of a configuration database
Page
See disk enclosure
Jbod
Page
RAID
Page
Page
Page
Index
CDS
Page
CVM
Page
Page
DMP
Empty
Page
Page
Page
Page
Iofail 228 Nodarec 228 Nodevice 228 Recover 228 Removed
Page
Page
Page
Page
Page
TPD
Page
Detached Disabled Enabled
Page
Page
Page
Page
Page
