Veritas Volume Manager Administrator’s Guide
Legal Notices
Page
Contents
VxVM root disk volume restrictions
Using vxdiskadd to place a disk under control of VxVM
Booting root volumes
Setting up a VxVM root disk and mirror
119
Taking a disk offline 118
133
136
207
167
185
215
Setting default values for vxassist 241
Displaying subdisk information 216
217
275
252
274
288
Moving volumes from a VM disk 290
289
290
356
Chapter Creating and administering volume sets
387
386
388
390
441
434
463
Setup tasks after installation 511
507
512
521 531
Understanding Veritas Volume Manager
Veritas Enterprise Administrator User’s Guide
How data is stored
VxVM and the operating system
Physical objects-physical disks
How VxVM handles storage management
Disk arrays
Operating system
Multipathed disk arrays
Device discovery
Enclosure-based naming
Fibre Channel hub or switch
C2t99d0
Provide redundant loop access
Virtual objects in VxVM include the following
Virtual objects
Combining virtual objects in VxVM
Connection between objects in VxVM
VM disks
Disk groups
Subdisks
Plexes
Volumes
How VxVM handles storage management
Disk01-01 vol06-01
Implementation of layered volumes
Volume layouts in VxVM
Implementation of non-layered volumes
Following sections describe each layout method
Layout methods
Concatenation and spanning
12 Example of concatenation Data Disk01-01
+1 n+2
Example of spanning Data
Striping RAID-0
Su1 Su2 Su3
Devname3
Devname2
Striping plus mirroring mirrored-stripe or RAID-0+1
Mirroring RAID-1
Mirroring plus striping striped-mirror, RAID-1+0 or RAID-10
17 Mirrored-stripe volume laid out on six disks
Column Mirror Striped plex
RAID-5 striping with parity
Mirror volumes
You need a full license to use this feature
Traditional RAID-5 arrays
Data Parity
Veritas Volume Manager RAID-5 arrays
Left-symmetric layout
22 Veritas Volume Manager RAID-5 array Stripe
Understanding Veritas Volume Manager
RAID-5 logging
Layered volumes
Example of a striped-mirror layered volume
Understanding Veritas Volume Manager
How online relayout works
Online relayout
Three columns of length 5L/3
Striped volume
Limitations of online relayout
Transformations and volume length
Transformation characteristics
Resynchronization process
Volume resynchronization
Dirty flags
Dirty region logs
Dirty region logging
Sequential DRL
Log subdisks and plexes
Data volume configuration
SmartSync recovery accelerator
Redo log volume configuration
Volume snapshots
Original
Use on creation Requires less storage
Comparison of snapshot features
Space than original Volume
FastResync
Non-persistent FastResync
FastResync enhancements
How non-persistent FastResync works with snapshots
DCO volume versioning
Persistent FastResync
Version 20 DCO volume layout
Version 0 DCO volume layout
How persistent FastResync works with snapshots
Where the size of each map in bytes is
DCO
FastResync
Effect of growing a volume on the FastResync map
FastResync limitations
Hot-relocation
Volume sets
Volume sets
Disk devices
Administering disks
There are two different methods of naming disk devices
Disk device naming in VxVM
Operating system-based naming
# vxdisk path egrep diskname
Private and public disk regions
To boot the system.Typically, most disks on a system are
Configures disk access records for them automatically
Configured as this disk type. However, it is not a suitable
Format for boot, root or swap disks, for mirrors or
Following command scans for the devices c1t1d0 and c2t2d0
Discovering and configuring newly added disk devices
Partial device discovery
Next example discovers fabric devices
For more information, see the vxdisk1M manual
Discovering disks and dynamically adding disk arrays
Disk categories
Adding support for a new disk array
Third-party driver coexistence
Enabling discovery of new devices
Removing support for a disk array
Listing details of supported disk arrays
Administering the Device Discovery Layer
# vxddladm listsupport all
# vxddladm listsupport libname=libraryname.sl
Listing excluded disk arrays
Excluding support for a disk array library
Re-including support for an excluded disk array library
Adding unsupported disk arrays to the Disks category
Listing supported disks in the Disks category
# vxddladm addjbod vid=SEAGATE pid=ST318404LSUN18G
Length=serialnolength policy=ap
# vxdmpadm listenclosure all
# vxdisk list
# vxddladm rmjbod vid=SEAGATE
Removing disks from the Disks category
Adding foreign devices
See Migrating between DMP and HP-UX native multipathing on
Placing disks under VxVM control
Default
Mode Format of output from VxVM command
Changing the disk-naming scheme
Legacy
# vxdmpadm getlungroup dmpnodename=disk25
VxVM vxdmpadm Error V-5-1-10910 Invalid da-name
This regenerates the persistent name database
Restart the VxVM configuration demon
Regenerating persistent device names
# vxdmpadm getsubpaths dmpnodename=enclosure-basedname
Changing device naming for TPD-controlled enclosures
# vxdmpadm setattr enclosure enclosure tpdmode=nativepseudo
# vxconfigd -kr reset
Persistent simple or nopriv disks in the boot disk group
Re-import the disk group using the following command
Installing and formatting disks
Persistent simple or nopriv disks in non-boot disk groups
Adding a disk to VxVM
Displaying and changing default disk layout attributes
Add or initialize disks Menu VolumeManager/Disk/AddDisks
C3t0d0 c3t1d0 c3t2d0 c3t3d0
Use default disk names for the disks? y,n,q,? default y
Continue operation? y,n,q,? default y y
Exclude disks from hot-relocation use? y,n,q,? default n n
Add site tag to disks? y,n,q,? default n
Enter the desired format cdsdisk,hpdisk,q,? default cdsdisk
Continue with operation? y,n,q,? default y y
Enter desired private region length privlen,q,? default
Following disks
Add or initialize other disks? y,n,q,? default n
Using vxdiskadd to place a disk under control of VxVM
Reinitializing a disk
Vxdiskadm then proceeds to add the disks
Rootability
Root disk mirrors
VxVM root disk volume restrictions
# /etc/vx/bin/vxcplvmroot -b c0t4d0
Booting root volumes
Setting up a VxVM root disk and mirror
# /etc/vx/bin/vxcplvmroot -m c1t1d0 -R 30 -v -b c0t4d0
# /etc/vx/bin/vxcplvmroot -R 30 -v -b c0t4d0
Creating an LVM root disk from a VxVM root disk
Adding persistent dump volumes to a VxVM rootable system
Adding swap volumes to a VxVM rootable system
View the changed swap configuration
Display the initial crash dump configuration
You can now remove the volume if required
Removing a persistent dump volume
# crashconf -ds /dev/vx/dsk/bootdg/dumpvol
Dynamic LUN expansion
109
Removing disks
# vxvol -g diskgroup stop volume1 volume2
Remove another disk? y,n,q,? default n
Continue with operation? y,n,q,? default y
Removing a disk with subdisks
VxVM Info V-5-2-268 Removal of disk mydg01 is complete
Removing a disk from VxVM control
# /usr/lib/vxvm/bin/vxdiskunsetup c#t#d#
Removing and replacing disks
Removing a disk with no subdisks
# vxassist move mkting !mydg02
Are you sure you want do this? y,n,q,? default n
To replace a disk
Removing and replacing disks
Following devices are available as replacements c0t1d0
Replacing a failed or removed disk
Following devices are available as replacements
Replace another disk? y,n,q,? default n
Select a disk device to enable address,list,q,? c0t2d0
Then run the following command on the master node
Enabling a disk
# vxreattach -r accesname
Taking a disk offline
Enable another device? y,n,q,? default n
Disable another device? y,n,q,? default n
Reserving disks
You would use the following command to rename the disk
Renaming a disk
VxVM returns a display similar to the following
Displaying disk information
See the vxedit1M manual page for more information
Displaying disk information with vxdiskadm
List disk information Menu VolumeManager/Disk/ListDisk
Enter disk device or all address,all,q,? default all
Setting the Pfto values
Controlling Powerfail Timeout
To set the Pfto value on a disk, use the following command
Displaying the Pfto values
Enabling or disabling Pfto
For example, to disable Pfto on the disk c5t0d6
Controlling Powerfail Timeout
How DMP works
Administering dynamic multipathing DMP
How DMP works
Enc00
Path failover mechanism
How DMP monitors I/O on paths
Throttling
Load balancing
DMP coexistence with HP-UX native multipathing
Use the following commands to initiate the migration
Migrating between DMP and HP-UX native multipathing
# vxvol -g diskgroup stopall
# vxvol -g diskgroup startall
Restart all the volumes in each disk group
Under the new naming scheme as
DMP in a clustered environment
Enabling or disabling controllers with shared disk groups
Disabling and enabling multipathing for specific devices
Disabling multipathing and making devices invisible to VxVM
Enabling multipathing and making devices visible to VxVM
135
Administering dynamic multipathing DMP
Displaying the paths to a disk
Typical output from the vxdisk list command is as follows
Displaying DMP database information
Disabled Log
Disabled Config
Disabled Lockrgn
C1t0d3 state=enabled Type=secondary
# vxdmpadm getdmpnode nodename=c3t2d1
Administering DMP using vxdmpadm
Retrieving information about a DMP node
# vxdmpadm getdmpnode enclosure=enc0
Displaying the members of a LUN group
# vxdmpadm getlungroup dmpnodename=c11t0d10
# vxdmpadm getsubpaths dmpnodename=c2t66d0
# vxdmpadm listctlr all
Displaying information about controllers
Displaying information about array ports
Following is example output from this command
Displaying information about enclosures
# vxdmpadm gettpdnode nodename=c7t0d10
# vxdmpadm getsubpaths tpdnodename=emcpower10
Displaying information about TPD-controlled devices
Gathering and displaying I/O statistics
Examples of using the vxdmpadm iostat command
To reset the I/O counters to zero, use this command
# vxdmpadm iostat show pathname=c3t115d0
# vxdmpadm iostat start memory=4096
# vxdmpadm iostat show dmpnodename=c0t0d0
# vxdmpadm iostat show enclosure=Disk
# vxdmpadm setattr path c2t10d0 pathtype=active
Setting the attributes of the paths to an enclosure
# vxdmpadm setattr path c3t10d0 pathtype=nomanual
# vxdmpadm setattr path c1t20d0 pathtype=nopreferred
Specifying the I/O policy
Displaying the I/O policy
Adaptive
Following policies may be set
# vxdmpadm setattr enclosure enc1 iopolicy=adaptive
Adaptiveminq
048
# vxdmpadm setattr enclosure Disk iopolicy=minimumq
This is the default I/O policy for Active/Active A/A arrays
# vxdmpadm setattr arrayname Sena iopolicy=priority
# vxdmpadm setattr arraytype A/A iopolicy=round-robin
Example of applying load balancing in a SAN
Default setting for this attribute is useallpaths=no
# vxdmpadm setattr arrayname Disk iopolicy=singleactive
# vxdisk list c3t2d15
# vxdmpadm getattr enclosure ENC0 iopolicy
DMP statistics are now reset
# dd if=/dev/vx/rdsk/mydg/myvol1 of=/dev/null
# vxdmpadm -c-fdisable ctlr=ctlrname
# vxdmpadm -c-fdisable path=pathname
Disabling I/O for paths, controllers or array ports
# vxdmpadm setattr enclosure ENC0 iopolicy=singleactive
# vxdmpadm enable ctlr=ctlrname
# vxdmpadm enable path=pathname
Enabling I/O for paths, controllers or array ports
Upgrading disk controller firmware
Renaming an enclosure
Re-enable the plex associated with the device
Stop I/O to all disks through one controller of the HBA
For the other controller on the HBA, enter
# vxdmpadm setattr \
Configuring the response to I/O failures
# vxdmpadm getattr enclosure enc0 recoveryoption
# vxdmpadm setattr arraytype A/A recoveryoption=default
Configuring the I/O throttling mechanism
# vxdmpadm setattr enclosure enc0 recoveryoption=nothrottle
HDS9500-ALUA0 Error-Retry
Displaying recoveryoption values
# vxdmpadm getattr enclosure HDS9500-ALUA0 recoveryoption
# vxdmpadm start restore interval=seconds policy=checkall
Configuring DMP path restoration policies
This produces output such as the following
Stopping the DMP path restoration thread
Displaying the status of the DMP path restoration thread
Configuring array policy modules
Displaying information about the DMP error-handling thread
To add and configure an APM, use the following command
One daemon should be shown as running
# vxdmpadm -r cfgapm modulename
Administering DMP using vxdmpadm
Creating and administering disk groups
Creating and administering disk groups
Block special device corresponding to this volume is
Specifying a disk group to commands
System-wide reserved disk groups
To nodg
Displaying the system-wide boot disk group
Rules for determining the default disk group
# vxdg bootdg
See the vxdg1M manual page for more information
# vxdg list
Displaying disk group information
# vxdg list diskgroup
# vxdisk -s list devicename
Following is example output
Creating a disk group
Displaying free space in a disk group
# vxdiskadd c1t0d0
Adding a disk to a disk group
# vxdg init mktdg mktdg01=c1t0d0
# vxdg -g diskgroup set cds=onoff
# vxdiskunsetup c1t0d0
# vxdiskunsetup devicename
Removing a disk from a disk group
# vxdg -g diskgroup rmdisk diskname
Deporting a disk group
# vxdisk -s list
Importing a disk group
Newdg
Select another disk group? y,n,q,? default n
Select disk group to import group,list,q,? default list
Handling disks with duplicated identifiers
VxVM Info V-5-2-374 The import of newdg was successful
# vxdisk -f-g diskgroup updateudid disk
Option to the vxdg import command, as shown in this
# vxdisk updateudid c2t66d0 c2t67d0
Writing a new Udid to a disk
# vxdisk -g diskgroup settag tagname disk
# vxdg -o useclonedev=on -o updateid import mydg
# vxdisk settag mytaggeddisks c2t66d0 c2t67d0
# vxdisk listtag
Sample cases of operations on cloned disks
Enabling configuration database copies on tagged disks
# vxdg -q listmeta diskgroup
# vxdg -o useclonedev=on -o tag=mytaggeddisks import mydg
# vxdisk -o alldgs list
# vxdg -g mydg set tagmeta=on tag=t1 nconfig=all nlog=all
Importing cloned disks without tags
# /usr/symcli/bin/symmir -g mydg split DEV001
Symmir command is used to split off the BCV device
To import only the cloned disks into the mydg disk group
Importing cloned disks with tags
# vxdg -n newdg -o useclonedev=on -o updateid import mydg
State of the cloned disk is now shown as online clonedisk
Disks are tagged as follows
# vxdisk set EMC08 clone=off # vxdisk -o alldgs list
Renaming a disk group
This command results in output such as the following
# vxdg -tC -n newdg import diskgroup
Moving disks between disk groups
Dgid 774226267.1025.tweety
Moving disk groups between systems
# vxrecover -g diskgroup -sb
To clear the locks during import, use the following command
Handling errors when importing disks
# vxdisk clearimport devicename
# vxdg -C import diskgroup
# vxdg -f import diskgroup
Following error message indicates a recoverable error
Reserving minor numbers for disk groups
# vxprint -g mydg reminor
# xvdg init newdg minor=30000 c1d0t0 c1t1d0
# vxdg -g diskgroup set maxdev=4079
Compatibility of disk groups between platforms
Example of a serial split brain condition in a cluster
Handling conflicting configuration copies
191 Typical arrangement of a 2-node campus cluster
Expected a = Expected B =
Automatically
Imported on host Y
# vxsplitlines -g newdg
Correcting conflicting configuration information
Reorganizing the contents of disk groups
Reorganizing the contents of disk groups
197
Disk group join operation
Limitations of disk group split and join
Listing objects potentially affected by a move
Moving DCO volumes between disk groups
201
Split Snapshot
Moving objects between disk groups
# vxdg -o expand move mydg rootdg mydg01
# vxprint
# vxdg -o expand split rootdg mydg rootdg07 rootdg08
Following commands would also achieve the same result
Splitting disk groups
# vxdg -o overrideverify join sourcedg targetdg
Joining disk groups
Disabling a disk group
Following command joins disk group mydg to rootdg
Mydg Mydg05 C1t96d0
Mydg06 C1t98d0
Destroying a disk group
Recovering a destroyed disk group
Upgrading a disk group
Use the disk group ID to import the disk group
Upgrading a disk group
Features supported by disk group versions
# vxdg list dgname
To list the version of a disk group, use this command
# vxdg upgrade dgname
# vxdg -T 120 init newdg newdg01=c0t3d0
Managing the configuration daemon in VxVM
Using vxnotify to monitor configuration changes
Backing up and restoring disk group configuration data
# vxnotify -f
# vxnotify -s
Using vxnotify to monitor configuration changes
# vxmake -g mydg sd mydg02-01 mydg02,0,8000
Creating subdisks
Displaying subdisk information
This command provides the following output
# vxprint -st
# vxprint -g diskgroup -l subdisk
# vxsd -g mydg mv mydg03-01 mydg12-01 mydg12-02
Moving subdisks
Splitting subdisks
# vxmake -g diskgroup plex plex sd=subdisk
Joining subdisks
Associating subdisks with plexes
# vxsd -g mydg -l 4096b assoc vol10-01 mydg15-01
# vxsd -g mydg assoc home-1 mydg02-01 mydg02-00 mydg02-01
# vxsd -g mydg -l 1 assoc vol02-01 mydg11-01
# vxsd -g diskgroup assoc plex subdisk10 ... subdiskMN-1
Associating log subdisks
# vxassist -g diskgroup addlog volume disk
# vxsd -g diskgroup aslog plex subdisk
# vxsd -g mydg aslog vol01-02 mydg02-01
Dissociating subdisks from plexes
To remove a subdisk, use the following command
Removing subdisks
Changing subdisk attributes
# vxedit -g mydg set putil0=DO-NOT-USE mydg02-01
# vxedit -g mydg set comment=subdisk comment mydg02-01
# vxmake -g mydg plex vol01-02 sd=mydg02-01,mydg02-02
Creating plexes
Plex states
Creating a striped plex
Displaying plex information
Dcosnp plex state
Clean plex state
Active plex state
Iofail plex state
Empty plex state
LOG plex state
Offline plex state
Stale plex state
Snaptmp plex state
Temp plex state
Temprm plex state
Plex condition flags
Enabled plex kernel state
Disabled plex kernel state
Attaching and associating plexes
Plex kernel states
# vxmend -g mydg off vol01-02 vol02-02
Taking plexes offline
# vxmend -g diskgroup off plex
Reattaching plexes
Detaching plexes
# vxmend -g diskgroup fix clean plex
Start the volume using the following command
# vxvol -g diskgroup start volume
Moving plexes
# vxplex -g diskgroup cp volume newplex
Copying volumes to plexes
Dissociating and removing plexes
# vxplex -g diskgroup -o rm dis plex
# vxedit -g diskgroup set attribute=value ... plex
Changing plex attributes
# vxedit -g mydg set comment=plex comment tutil2=u vol01-02
# vxedit -g mydg set putil0=DO-NOT-USE vol01-02
Creating volumes
RAID-5
Types of volume layouts
Mirror and concatenated-mirror volumes
Supported volume logs and maps
Advanced approach
Creating a volume
Assisted approach
Using vxassist
Vxassist
# vxassist options make volume length attributes
Following is a sample vxassist defaults file
Setting default values for vxassist
Disk group alignment constraints on volumes
Discovering the maximum size of a volume
# vxassist -g diskgroup maxsize layout=layout attributes
# vxassist -g dgrp maxsize layout=raid5 nlog=2
# vxassist -b -g diskgroup make volume length
Creating a volume on any disk
# vxassist -b make voldefault 10g
# vxprint -g diskgroup -G -F %align
# vxassist -b -g mydg make volspec 5g ctlrc1 !targetc1t5
Creating a volume on specific disks
# vxassist -b -g mydg make volspec 5g mydg03 mydg04
Specifying ordered allocation of storage to volumes
Mydg01 mydg02 mydg03 mydg04 mydg05 mydg06 mydg07 mydg08
Stripe volume
Volume across controllers
Creating a mirrored-concatenated volume
Creating a mirrored volume
Creating a concatenated-mirror volume
# vxassist -b -g mydg make volmir 5g layout=mirror
Creating a volume with a version 0 DCO volume
For more information, see Upgrading a disk group on
# vxdg -T 90 upgrade diskgroup
# vxvol -g diskgroup set logtype=drldrlseq volume
# vxdg upgrade diskgroup
Creating a volume with a version 20 DCO volume
Creating a volume with dirty region logging enabled
# vxassist -b -g mydg make volzebra 10g layout=stripe
Creating a striped volume
# vxassist -b -g diskgroup make volume length layout=stripe
Creating a striped-mirror volume
Creating a mirrored-stripe volume
Mirroring across targets, controllers or enclosures
# vxassist -b -g mydg make volraid 10g layout=raid5 nlog=2
Creating a RAID-5 volume
# vxassist -g diskgroup list tag=tagname volume
Creating tagged volumes
# vxassist -g diskgroup listtag volume
Creating a volume using vxmake
# vxmake -g diskgroup -d descriptionfile
Creating a volume using a vxmake description file
Mydg04-021/8000,mydg04-031/16000
Initializing and starting a volume
# vxassist -b -g diskgroup make volume length layout=mirror
# vxvol -g diskgroup init enable volume
Initializing and starting a volume created using vxmake
# vxvol -g diskgroup init active volume
# vxvol -g diskgroup init zero volume
Accessing a volume
Administering volumes
This is example output from this command
Displaying volume information
Vxprint command can also be applied to a single disk group
Active volume state
Volume states
Clean volume state
Empty volume state
Invalid volume state
Volume kernel states
Needsync volume state
Replay volume state
Disabled volume kernel state
Detached volume kernel state
Enabled volume kernel state
Monitoring and controlling tasks
Vxtask operations
Managing tasks with vxtask
Using the vxtask command
Putting a volume in maintenance mode
Stopping a volume
To start all Disabled volumes, enter
Starting a volume
Adding a mirror to a volume
Mirroring volumes on a VM disk
Mirroring all volumes
# /etc/vx/bin/vxmirror -g diskgroup -a
# /etc/vx/bin/vxmirror -d yes
# vxassist -gdiskgroup remove mirror volume
Mirror volumes on another disk? y,n,q,? default n
Removing a mirror
At the following prompt, press Return to make the mirror
Adding logs and maps to volumes
Preparing a volume for DRL and instant snapshots
Specifying storage for version 20 DCO plexes
# DCONAME=‘vxprint -g diskgroup -F%dconame volume‘
Using a DCO and DCO volume with a RAID-5 volume
Determining the DCO version number
# vxprint -g diskgroup -F%version $DCONAME
Determining if DRL logging is active on a volume
Determining if DRL is enabled on a volume
This command returns on if DRL logging is enabled
Disabling and re-enabling DRL
To re-enable sequential DRL on a volume, enter
Upgrading existing volumes to use version 20 DCOs
To re-enable DRL on a volume, enter this command
Use the following command on the volume to upgrade it
Adding traditional DRL logging to a mirrored volume
# vxassist -g mydg addlog volume logtype=drlseq nlog=n
To remove a DRL log, use the vxassist command as follows
Removing a traditional DRL log
# vxassist -g mydg addlog vol03 logtype=drl
# vxassist -b -g diskgroup addlog volume loglen=length
Adding a RAID-5 log using vxplex
Adding a RAID-5 log
# vxassist -g mydg addlog volraid
# vxprint -g diskgroup -ht volume
Resizing a volume
# vxassist -g diskgroup maxgrow volume
Removing a RAID-5 log
Online JFS Full Base JFS Lite
Resizing volumes using vxresize
VxFS Mounted File System
Unmounted File System
Extending by a given length
Resizing volumes using vxassist
Extending to a given length
Shrinking by a given length
Resizing volumes using vxvol
Shrinking to a given length
# vxassist -g diskgroup removetag volume tagname
Setting tags on volumes
Changing the read policy for mirrored volumes
To set the read policy to select, use the following command
Removing a volume
Moving volumes from a VM disk
Move volumes from another disk? y,n,q,? default n
To move volumes from a disk
VxVM vxevac Info
VxVM Info V-5-2-188 Evacuation of disk mydg02 is complete
# vxvol -g diskgroup set fastresync=on volume
Enabling FastResync on a volume
# vxprint -g diskgroup -F%fastresync volume
Checking whether FastResync is enabled on a volume
# vxprint -g diskgroup -F%hasdcolog volume
Disabling FastResync
# vxassist -g mydg relayout vol02 layout=stripe
Performing online relayout
Permitted relayout transformations
Relayout to From raid5 Concat
Supported relayout transformations for RAID-5 volumes
Relayout to From mirror-concat Concat
Mirror-concat Mirror-stripe
Relayout to From mirror-stripe Concat
Layered striped-mirror volumes
Supported relayout transformations for unmirrored stripe
Relayout to From stripe or stripe-mirror Concat
Specifying a plex for relayout
Specifying a non-default layout
Tagging a relayout operation
# vxassist -g fsgrp relayout vol04 layout=raid5 ncol=4
Controlling the progress of a relayout
To resume the operation, use the vxtask command
Viewing the status of a relayout
# vxrelayout -g mydg -o bg,slow=1000,iosize=10m start vol04
Converting between layered and non-layered volumes
# vxrelayout -g mydg -o bg reverse vol04
# vxassist -g mydg relayout vol1 ncol=5
# vxassist -g mydg convert vol1 layout=mirror-stripe
Converting between layered and non-layered volumes
Administering volume snapshots
Administering volume snapshots
Traditional third-mirror break-off snapshots
Independent Volume Vxassist snapclear
Traditional third-mirror break-off snapshots
Full-sized instant snapshots
308 Administering volume snapshots
Cycle Start Vxsnap make Vxsnap refresh Vxsnap prepare
Space-optimized instant snapshots
Emulation of third-mirror break-off snapshots
Linked break-off snapshot volumes
Cascaded snapshots
Cascaded snapshots
Creating a snapshot of a snapshot
Creating a snapshot of a snapshot
Create instant snapshot S2 of S1 Vxsnap make source=S1
Vxsnap dis S2
Creating multiple snapshots
Restoring the original volume from a snapshot
Restoring the original volume from a snapshot
Creating instant snapshots
Creating instant snapshots
Preparing to create instant and break-off snapshots
Creating a shared cache object
For example to start the cache object, cobjmydg
# LEN=‘vxprint -g diskgroup -F%len volume‘
# vxcache -g mydg start cobjmydg
# RSZ=‘vxprint -g diskgroup -F%regionsz $DCONAME‘
Creating and managing space-optimized instant snapshots
Creating instant snapshots
# fsck -F vxfs /dev/vx/rdsk/diskgroup/snapshot
# vxsnap -g mydg make source=myvol/snapvol=snap1myvol
Creating and managing full-sized instant snapshots
# vxsnap -g diskgroup syncwait snapvol
# vxsnap -g mydg syncwait snap2myvol
# vxprint -gdiskgroup -F%incompletesnapvol
# vxsnap -g mydg addmir vol1 nmirror=2 alloc=mydg10,mydg11
Creating and managing third-mirror break-off snapshots
# vxsnap -g mydg snapwait vol1 nmirror=2
# vxsnap -g diskgroup -b addmir volume mirvol=snapvol \
Creating and managing linked break-off snapshot volumes
Mirdg=snapdg
Creating multiple instant snapshots
Reattach the snapshot volume with the original volume. See
# vxsnap -g diskgroup make \
# vxsnap -g diskgroup make source=vol1/snapvol=snapvol1 \
# vxvset -g mydg list snapvset1
Creating instant snapshots of volume sets
# vxvset -g mydg list vset1
Svol2 614400
Svol0 204800
Svol1 409600
# vxsnap -g mydg rmmir vol1
Adding snapshot mirrors to a volume
Removing a snapshot mirror
Adding a snapshot to a cascaded snapshot hierarchy
Removing a linked break-off snapshot volume
Refreshing an instant snapshot
# vxsnap -g mydg rmmir vol1 mirvol=prepsnap mirdg=mysnapdg
# vxsnap -g mydg reattach snapmyvol source=myvol nmirror=1
Reattaching an instant snapshot
# vxsnap -g mydg snapwait myvol nmirror=1
Reattaching a linked break-off snapshot volume
# vxsnap -g mydg restore myvol source=snap3myvol
Restoring a volume from an instant snapshot
Dissociating an instant snapshot
# vxsnap -g snapdg snapwait myvol mirvol=prepsnap
Splitting an instant snapshot hierarchy
Removing an instant snapshot
# vxsnap -g mydg dis snap2myvol
# vxedit -g mydg -r rm snap2myvol
# vxsnap -g mydg split snap2myvol
Displaying instant snapshot information
# vxsnap -g diskgroup print vol
# vxsnap -g mydg print
# vxsnap -g dg -vx list
# vxsnap -g diskgroup -l -v -x list vol
Vxsnap -g diskgroup syncresume \
Controlling instant snapshot synchronization
Vxsnap -b -g diskgroup syncstart \
Vxsnap -g diskgroup syncstop vol volset
# vxcache -g diskgroup listvol cacheobject
Listing the snapshots created on a cache
Improving the performance of snapshot synchronization
# vxcache -g mydg set highwatermark=60 cobjmydg
Tuning the autogrow attributes of a cache
Removing a cache
Finally, remove the cache object and its cache volume
Growing and shrinking a cache
Creating traditional third-mirror break-off snapshots
# vxassist -g diskgroup snapstart voldef
# vxassist -b -g diskgroup snapstart nmirror=N volume
# vxassist -g diskgroup snapwait volume
# vxassist -g diskgroup snapshot nmirror=N volume snapshot
Create a snapshot volume using the following command
# vxassist -g diskgroup snapshot voldef snapvol
# vxedit -g diskgroup -rf rm snapshot
Converting a plex into a snapshot plex
Creating multiple snapshots
Reattaching a snapshot volume
# vxplex -g diskgroup convert state=SNAPDONE plex
# vxassist -g diskgroup -o allvols snapshot
# vxassist -g diskgroup snapback nmirror=number snapshot
Adding plexes to a snapshot volume
# vxassist -g diskgroup -o allplexes snapback snapshot
# vxassist snapclear snapshot
Dissociating a snapshot volume
# vxprint -g diskgroup -F%rid $DCOVOL
# vxassist snapprint volume
Output from this command is shown in the following examples
Displaying snapshot information
# vxassist -g mydg snapprint
Adding a version 0 DCO and DCO volume
Specifying storage for version 0 DCO plexes
# vxdco -g mydg dis myvoldco
Removing a version 0 DCO and DCO volume
# vxdco -g diskgroup -o rm dis dcoobj
# vxdco -g mydg att myvol myvoldco
Reattaching a version 0 DCO and DCO volume
For more information, see the vxdco1M manual
Adding a version 0 DCO and DCO volume
Creating and administering volume sets
Adding a volume to a volume set
Creating a volume set
Stopping and starting volume sets
Listing details of volume sets
Raw device node access to component volumes
Removing a volume from a volume set
# vxvset -g diskgroup -f rmvol volset volume
# vxvset -g mydg start set1 # vxvset -g mydg list set1
Enabling raw device access when creating a volume set
# vxvset -g diskgroup -fset makedev=onoff vset
Displaying the raw device access settings for a volume set
Controlling raw device access for an existing volume set
# vxvset -g mydg set makedev=off myvset2
# vxvset -g mydg set compvolaccess=read-write myvset2
# vxvset -g mydg set makedev=on myvset2
Raw device node access to component volumes
Configuring off-host processing
Example implementation of off-host processing
Implementing off-host processing solutions
# vxprint -g volumedg -F%instant volume
Implementing off-host online backup
# vxvol -g volumedg set fastresync=on volume
# fsck -F vxfs /dev/vx/rdsk/snapvoldg/snapvol
# vxdg deport snapvoldg
# vxdg import snapvoldg
Implementing decision support
# vxsnap -g volumedg snapwait volume mirvol=snapvol
# vxprint -g volumedg -F%instantvolume
375
# mount -F vxfs /dev/vx/dsk/snapvoldg/snapvol \ mountpoint
You can then resume the procedure from on
Implementing off-host processing solutions
Administering hot-relocation
How hot-relocation works
381
Mydg05
Partial disk failure mail messages
# vxrecover -b -g mydg home src
# vxstat -g mydg -s -ff home-02 src-02
Sd mydg01-04 Sd mydg01-06 Sd mydg02-03 Sd mydg02-04
Failing disks mydg02
Complete disk failure mail messages
How space is chosen for relocation
Configuring a system for hot-relocation
Mydg mydg02 C0t2d0 658007
Displaying spare disk information
# vxdg -g diskgroup spare
# vxedit -g mydg set spare=on mydg01
Mark another disk as a spare? y,n,q,? default n
Marking a disk as a hot-relocation spare
Removing a disk from use as a hot-relocation spare
Where diskname is the disk media name
Excluding a disk from hot-relocation use
Following confirmation is displayed
# vxedit -g diskgroup set nohotuse=off diskname
Making a disk available for hot-relocation use
To use vxdiskadm to exclude a disk from hot-relocation use
Moving and unrelocating subdisks
Configuring hot-relocation to use only spare disks
Spare=only
To root Subject Attempting VxVM relocation on host teal
Unrelocate to a new disk y,n,q,? default n
Moving and unrelocating subdisks using vxdiskadm
Status message is displayed at the end of the operation
Enter the original disk name disk,list,q,?
# vxassist -g mydg move home !mydg05 mydg02
Moving and unrelocating subdisks using vxassist
Moving and unrelocating subdisks using vxunreloc
Forcing hot-relocated subdisks to accept different offsets
Moving hot-relocated subdisks back to their original disk
Moving hot-relocated subdisks back to a different disk
# vxprint -g mydg -se sdorigdmname=mydg01
Restarting vxunreloc after errors
Examining which subdisks were hot-relocated from a disk
Modifying the behavior of hot-relocation
Nohup vxrelocd -o slow=IOdelay root
# nohup vxrelocd root
Nohup vxrelocd root user1 user2
# nohup /etc/vx/bin/vxrelocd root user1 user2
Alternatively, you can use the following command
See the vxrelocd1M manual page for more information
Administering cluster functionality
Overview of cluster volume management
399
Example of a 4-node cluster
Two types of disk groups are defined
Private and shared disk groups
Exclusivewrite ew
Activation modes of shared disk groups
Readonly ro
Sharedread sr
Enableactivation=true Defaultactivationmode=activation-mode
Activation mode Description
Activation modes for shared disk groups
Sharedwrite sw
Connectivity policy of shared disk groups
Global detach policy
Local detach policy
Disk group failure policy
Guidelines for choosing detach and failure policies
Limitations of shared disk groups
Effect of disk connectivity on cluster reconfiguration
Cluster reconfiguration
Cluster initialization and configuration
Various reasons that may be given are shown in Table
Vxclustadm utility
# /etc/vx/bin/vxclustadm nodestate state out of cluster
Reason user initiated stop
Reason Description
Node abort messages
Volume reconfiguration
Vxconfigd daemon recovery
Vxconfigd daemon
# hagrp -unfreeze group
# hagrp -freeze group
Node shutdown
Cluster shutdown
Multiple host failover configurations
Node abort
Failover
Import lock
Where the reason can describe errors such as
Corruption of disk group configuration
# vxdctl -c mode
Administering VxVM in cluster environments
Requesting node status and discovering the master node
See the vxdctl1M manual page for more information
Example output from this command is as follows
Example output from this command is displayed here
Determining if a disk is shareable
Listing shared disk groups
# vxdg -s init diskgroup diskname=devicename
Creating a shared disk group
Forcibly importing a disk group
Importing disk groups as shared
# vxdg -s import diskgroup
# vxdg -s -f import diskgroup
Converting a disk group from shared to private
Setting the disk detach policy on a shared disk group
Changing the activation mode on a shared disk group
Default disk detach policy is global
# vxdg -g diskgroup set activation=mode
Setting exclusive open access to a volume by a node
Creating volumes with exclusive open access by a node
# vxdg -g diskgroup set dgfailpolicy=dgdisableleave
# vxvol -g dskgrp set exclusive=on volmir
This command produces out put similar to the following
This command produces output similar to the following
Displaying the cluster protocol version
Displaying the supported cluster protocol version range
Obtaining cluster performance statistics
Recovering volumes in shared disk groups
Upgrading the cluster protocol version
# vxstat -b
Vol Vol1 2421 600000 99.0
Administering VxVM in cluster environments
Administering Sites and remote mirrors
Site-consistent volume with two plexes at each of two sites
Example of a two-site configuration with remote storage only
To remove the site name from a host, use this command
Configuring sites for hosts and disks
Configuring site-based allocation on a disk group
Configuring site consistency on a volume
Configuring site consistency on a disk group
# vxdg -g diskgroup set siteconsistent=on
# vxdg -g diskgroup set siteconsistent=off
RAID-5 volumes in a site-consistent disk group
Setting the siteread policy on a volume
Site-based allocation of storage to volumes
# vxassist -g diskgroup make volume size mirror=site
Command Description
Examples of storage allocation using sites
Making an existing disk group site consistent
Turn on site consistency for each volume in the disk group
Register a site record for each site with the disk group
Turn on site consistency for the disk group
Recovery from simulated site failure
Fire drill testing the configuration
Automatic site reattachment
Simulating site failure
# kill -9 PID
Failure scenarios and recovery procedures
# ps -afe
Recovery from host failure
Recovery from a loss of site connectivity
Recovery from storage failure
Failure scenario Recovery technique
Recovery from site failure
Failure scenarios and recovery procedures
See the vxse1M manual
About Storage Expert
How Storage Expert works
Before using Storage Expert
Running Storage Expert
One of the following keywords must be specified
Discovering what a rule does
Displaying rule attributes and their default values
Running a rule
# vxsestripes2 info
Rule result types
Setting rule attributes
Recovery time
Identifying configuration problems using Storage Expert
Vxseraid5log1
Checking minimum and maximum RAID-5 log sizes vxseraid5log2
Checking for non-mirrored RAID-5 logs vxseraid5log3
Checking on disk config size vxsedg3
Checking disk group configuration copies and logs vxsedg2
Disk groups
Checking the version number of disk groups vxsedg4
Checking for non-imported disk groups vxsedg6
Checking volume redundancy vxseredundancy
Checking states of plexes and volumes vxsevolplex
Disk striping
Checking the number of columns in RAID-5 volumes vxseraid5
Volumes needing recovery
Hardware failures
Disk sparing and relocation management
Rootability
System name
Checking the system name vxsehost
Rule definitions and attributes
See Running a rule on
Rule Description
Rule Attribute Default Description Value
Vxseraid5 Toonarrowraid5
Nsdthreshold
Toowideraid5
R5maxsize
Vxsestripes1 Default stripeunit
Vxseredundancy Volumeredundancy
Vxserootmir
Vxsespares
Vxsevolplex
Rule definitions and attributes
Data assignment
Performance guidelines
Mirroring
Striping
RAID-5
Combining mirroring and striping
Volume read policies
Obtaining performance data
Setting performance priorities
Performance monitoring
By vxtrace
Tracing volume operations
Printing volume statistics
Using I/O statistics
Using performance data
# vxprint -g mydg -tvh archive
Following is an extract from typical output
Mydg03-03 Archive-01 40960 C1t2d0
# vxassist -g mydg move archive !mydg03 destdisk
471
Using I/O tracing
Tuning VxVM
General tuning guidelines
Number of configuration copies for a disk group
Tuning guidelines for large systems
Changing the values of tunables
# vxedit set nconfig=5 bigdg
# vxdmpadm settune dmptunable=value
# vxdmpadm gettune dmptunable
Tunable parameters
Dmphealthtime
Dmppathswitchblksshift
Dmploglevel
Dmppathage
Dmprestoreinterval
Dmprestorecycles
Dmpprobeidlelun
Dmpqueuedepth
Dmpretrytimeout
Dmprestorepolicy
Dmpscsitimeout
Dmpretrycount
Voldefaultiodelay
Volcheckptdefault
Dmpstatinterval
Volfmrlogsz
Volmaxioctl
Volmaxvol
Volmaxio
Volsubdisknum
Volmaxparallelio
Volmaxspecialio
Voldrlmaxdrtregs
Volcvmsmartsync
Voldrlmaxseqdirty
Voldrlminregionsz
Voliomemmaxpoolsz
Voliotiobufdefault
Volioterrbufdflt
Voliotiobuflimit
Volpagemodmaxmemsz
Voliotiobufmax
Voliotmaxopen
Volraidrsrtransmax
Volraidminpoolsz
Tuning VxVM
If you are using a C shell csh or tcsh, use the commands
Commands summary
# vxdisk -g mydg list
Vxdisk -g diskgroup list diskname
Vxdg list diskgroup
# vxdg list mydg
Table A-2 Administering disks Command Description
Mydg03 mydg02
# vxedit -g mydg rename \
# vxedit -g mydg set \
Reserve=on mydg02
Spare=on mydg04
# vxdiskunsetup c0t3d0
Spare=off mydg04
# vxdisk offline c0t1d0
Mydg01=c0t1d0
# vxdg init mydg \
# vxsplitlines -g mydg
# vxdg -n newdg deport mydg
Mydg newdg myvol1
# vxdg -o expand listmove \
Newdg myvol1
Newdg myvol2 myvol3
# vxdg destroy mydg
# vxrecover -g mydg -sb
# vxmake -g mydg sd \
Mydg02-01 mydg02,0,8000
Vol01-01 mydg10-010 \
# vxsd -g mydg assoc \
Mydg11-011 mydg12-012
Mydg02-01
# vxmake -g mydg plex \
# vxsd -g mydg -o rm dis \
Vol01-02 \
Sd=mydg02-01,mydg02-02
Vol02-02 vol02-03
Vol01-02
# vxplex -g mydg mv \
# vxplex -g mydg cp vol02 \
Table A-6 Creating volumes Command Description
Vol03-01
Vol02-02
Vxassist -b -g diskgroup make \
Volume length layout=striperaid5 \
Stripeunit=W ncol=N attributes
Mysvol 20g layout=stripe \
Vol r5vol \
# vxmake -g mydg -Uraid5\
Plex=raidplex,raidlog1,\
Raidlog2
Table A-7 Administering volumes Command Description
# vxsnap -gmydg prepare \ myvol drl=on
Drl=onsequentialoff
Cvol 1g layout=mirror \
# vxassist -g mydg make \
Init=active mydg16 mydg17
Cachevolname=cvol
# vxassist -gmydg relayout \ vol2 layout=stripe
# vxsnap -g mydg unprepare \ myvol
Vol3 layout=raid5 \
Stripeunit=16 ncol=4
# vxassist -g mydg remove \ myvol
# vxrecover -g mydg \
Mytask -b mydg05
# vxtask -h -g mydg list
# vxtask -p -g mydg list
# vxtask pause mytask
# vxtask resume mytask
# vxtask abort mytask
Online manual pages
Administrative commands
Table A-9 Manual pages Name Description
Vxmend
Vxrecover
Vxmirror
Vxnotify
Device driver interfaces
File formats
Setup tasks after installation
Configuring Veritas Volume Manager
Adding disks to disk groups
Adding unsupported disk arrays as JBODs
Adding foreign devices
Guidelines for configuring storage
Mirroring guidelines
Striping guidelines
Dirty region logging guidelines
Hot-relocation guidelines
RAID-5 guidelines
517
Controlling VxVM’s view of multipathed devices
Configuring cluster support
Accessing volume devices
Converting existing VxVM disk groups to shared disk groups
Configuring shared disk groups
Reconfiguration tasks
Glossary
Single copy of a configuration database
Page
See disk enclosure
Jbod
Page
RAID
Page
Page
Page
Index
CDS
Page
CVM
Page
Page
DMP
Empty
Page
Page
Page
Page
Iofail 228 Nodarec 228 Nodevice 228 Recover 228 Removed
Page
Page
Page
Page
Page
TPD
Page
Detached Disabled Enabled
Page
Page
Page
Page
Page