Veritas Volume Manager Administrator’s Guide
Legal Notices
Page
Contents
VxVM root disk volume restrictions
Using vxdiskadd to place a disk under control of VxVM
Booting root volumes
Setting up a VxVM root disk and mirror
119
Taking a disk offline 118
133
136
185
167
207
215
Setting default values for vxassist 241
Displaying subdisk information 216
217
274
252
275
288
Moving volumes from a VM disk 290
289
290
356
Chapter Creating and administering volume sets
387
386
388
390
441
434
463
Setup tasks after installation 511
507
512
521 531
Understanding Veritas Volume Manager
Veritas Enterprise Administrator User’s Guide
How data is stored
VxVM and the operating system
Physical objects-physical disks
How VxVM handles storage management
Disk arrays
Device discovery
Multipathed disk arrays
Operating system
Enclosure-based naming
Fibre Channel hub or switch
C2t99d0
Provide redundant loop access
Combining virtual objects in VxVM
Virtual objects
Virtual objects in VxVM include the following
Connection between objects in VxVM
VM disks
Disk groups
Subdisks
Plexes
Volumes
How VxVM handles storage management
Disk01-01 vol06-01
Implementation of non-layered volumes
Volume layouts in VxVM
Implementation of layered volumes
Concatenation and spanning
Layout methods
Following sections describe each layout method
12 Example of concatenation Data Disk01-01
+1 n+2
Example of spanning Data
Striping RAID-0
Su1 Su2 Su3
Devname3
Devname2
Striping plus mirroring mirrored-stripe or RAID-0+1
Mirroring RAID-1
Mirroring plus striping striped-mirror, RAID-1+0 or RAID-10
17 Mirrored-stripe volume laid out on six disks
Column Mirror Striped plex
You need a full license to use this feature
Mirror volumes
RAID-5 striping with parity
Traditional RAID-5 arrays
Data Parity
Veritas Volume Manager RAID-5 arrays
Left-symmetric layout
22 Veritas Volume Manager RAID-5 array Stripe
Understanding Veritas Volume Manager
RAID-5 logging
Layered volumes
Example of a striped-mirror layered volume
Understanding Veritas Volume Manager
How online relayout works
Online relayout
Three columns of length 5L/3
Striped volume
Limitations of online relayout
Transformations and volume length
Transformation characteristics
Dirty flags
Volume resynchronization
Resynchronization process
Dirty region logs
Dirty region logging
Sequential DRL
Log subdisks and plexes
Data volume configuration
SmartSync recovery accelerator
Redo log volume configuration
Volume snapshots
Original
Space than original Volume
Comparison of snapshot features
Use on creation Requires less storage
FastResync
Non-persistent FastResync
FastResync enhancements
Persistent FastResync
DCO volume versioning
How non-persistent FastResync works with snapshots
Version 20 DCO volume layout
Version 0 DCO volume layout
How persistent FastResync works with snapshots
Where the size of each map in bytes is
DCO
FastResync
Effect of growing a volume on the FastResync map
FastResync limitations
Hot-relocation
Volume sets
Volume sets
Disk devices
Administering disks
Operating system-based naming
Disk device naming in VxVM
There are two different methods of naming disk devices
# vxdisk path egrep diskname
Private and public disk regions
To boot the system.Typically, most disks on a system are
Configures disk access records for them automatically
Configured as this disk type. However, it is not a suitable
Format for boot, root or swap disks, for mirrors or
Following command scans for the devices c1t1d0 and c2t2d0
Discovering and configuring newly added disk devices
Partial device discovery
Next example discovers fabric devices
Disk categories
Discovering disks and dynamically adding disk arrays
For more information, see the vxdisk1M manual
Adding support for a new disk array
Third-party driver coexistence
Enabling discovery of new devices
Removing support for a disk array
Listing details of supported disk arrays
Administering the Device Discovery Layer
# vxddladm listsupport all
# vxddladm listsupport libname=libraryname.sl
Re-including support for an excluded disk array library
Excluding support for a disk array library
Listing excluded disk arrays
Adding unsupported disk arrays to the Disks category
Listing supported disks in the Disks category
# vxddladm addjbod vid=SEAGATE pid=ST318404LSUN18G
Length=serialnolength policy=ap
# vxdmpadm listenclosure all
# vxdisk list
Adding foreign devices
Removing disks from the Disks category
# vxddladm rmjbod vid=SEAGATE
See Migrating between DMP and HP-UX native multipathing on
Placing disks under VxVM control
Default
Mode Format of output from VxVM command
Changing the disk-naming scheme
Legacy
# vxdmpadm getlungroup dmpnodename=disk25
VxVM vxdmpadm Error V-5-1-10910 Invalid da-name
Regenerating persistent device names
Restart the VxVM configuration demon
This regenerates the persistent name database
# vxdmpadm setattr enclosure enclosure tpdmode=nativepseudo
Changing device naming for TPD-controlled enclosures
# vxdmpadm getsubpaths dmpnodename=enclosure-basedname
# vxconfigd -kr reset
Persistent simple or nopriv disks in the boot disk group
Persistent simple or nopriv disks in non-boot disk groups
Installing and formatting disks
Re-import the disk group using the following command
Add or initialize disks Menu VolumeManager/Disk/AddDisks
Displaying and changing default disk layout attributes
Adding a disk to VxVM
C3t0d0 c3t1d0 c3t2d0 c3t3d0
Use default disk names for the disks? y,n,q,? default y
Continue operation? y,n,q,? default y y
Exclude disks from hot-relocation use? y,n,q,? default n n
Add site tag to disks? y,n,q,? default n
Enter the desired format cdsdisk,hpdisk,q,? default cdsdisk
Continue with operation? y,n,q,? default y y
Enter desired private region length privlen,q,? default
Following disks
Add or initialize other disks? y,n,q,? default n
Using vxdiskadd to place a disk under control of VxVM
Reinitializing a disk
Vxdiskadm then proceeds to add the disks
Rootability
Root disk mirrors
VxVM root disk volume restrictions
Setting up a VxVM root disk and mirror
Booting root volumes
# /etc/vx/bin/vxcplvmroot -b c0t4d0
# /etc/vx/bin/vxcplvmroot -m c1t1d0 -R 30 -v -b c0t4d0
# /etc/vx/bin/vxcplvmroot -R 30 -v -b c0t4d0
Creating an LVM root disk from a VxVM root disk
Adding persistent dump volumes to a VxVM rootable system
Adding swap volumes to a VxVM rootable system
View the changed swap configuration
Display the initial crash dump configuration
You can now remove the volume if required
Removing a persistent dump volume
# crashconf -ds /dev/vx/dsk/bootdg/dumpvol
Dynamic LUN expansion
109
Removing disks
# vxvol -g diskgroup stop volume1 volume2
Remove another disk? y,n,q,? default n
Continue with operation? y,n,q,? default y
Removing a disk with subdisks
VxVM Info V-5-2-268 Removal of disk mydg01 is complete
Removing a disk from VxVM control
# /usr/lib/vxvm/bin/vxdiskunsetup c#t#d#
Removing and replacing disks
Removing a disk with no subdisks
To replace a disk
Are you sure you want do this? y,n,q,? default n
# vxassist move mkting !mydg02
Removing and replacing disks
Following devices are available as replacements c0t1d0
Replacing a failed or removed disk
Following devices are available as replacements
Replace another disk? y,n,q,? default n
Select a disk device to enable address,list,q,? c0t2d0
Then run the following command on the master node
Enabling a disk
# vxreattach -r accesname
Disable another device? y,n,q,? default n
Enable another device? y,n,q,? default n
Taking a disk offline
Renaming a disk
You would use the following command to rename the disk
Reserving disks
See the vxedit1M manual page for more information
Displaying disk information
VxVM returns a display similar to the following
Enter disk device or all address,all,q,? default all
List disk information Menu VolumeManager/Disk/ListDisk
Displaying disk information with vxdiskadm
Setting the Pfto values
Controlling Powerfail Timeout
To set the Pfto value on a disk, use the following command
Displaying the Pfto values
Enabling or disabling Pfto
For example, to disable Pfto on the disk c5t0d6
Controlling Powerfail Timeout
How DMP works
Administering dynamic multipathing DMP
How DMP works
Enc00
Path failover mechanism
How DMP monitors I/O on paths
Throttling
Load balancing
DMP coexistence with HP-UX native multipathing
Use the following commands to initiate the migration
Migrating between DMP and HP-UX native multipathing
# vxvol -g diskgroup stopall
# vxvol -g diskgroup startall
Restart all the volumes in each disk group
Under the new naming scheme as
DMP in a clustered environment
Disabling multipathing and making devices invisible to VxVM
Disabling and enabling multipathing for specific devices
Enabling or disabling controllers with shared disk groups
Enabling multipathing and making devices visible to VxVM
135
Administering dynamic multipathing DMP
Displaying DMP database information
Typical output from the vxdisk list command is as follows
Displaying the paths to a disk
Disabled Log
Disabled Config
Disabled Lockrgn
C1t0d3 state=enabled Type=secondary
Retrieving information about a DMP node
Administering DMP using vxdmpadm
# vxdmpadm getdmpnode nodename=c3t2d1
# vxdmpadm getdmpnode enclosure=enc0
Displaying the members of a LUN group
# vxdmpadm getlungroup dmpnodename=c11t0d10
# vxdmpadm getsubpaths dmpnodename=c2t66d0
# vxdmpadm listctlr all
Displaying information about controllers
Displaying information about enclosures
Following is example output from this command
Displaying information about array ports
Displaying information about TPD-controlled devices
# vxdmpadm getsubpaths tpdnodename=emcpower10
# vxdmpadm gettpdnode nodename=c7t0d10
To reset the I/O counters to zero, use this command
Examples of using the vxdmpadm iostat command
Gathering and displaying I/O statistics
# vxdmpadm iostat show pathname=c3t115d0
# vxdmpadm iostat start memory=4096
# vxdmpadm iostat show dmpnodename=c0t0d0
# vxdmpadm iostat show enclosure=Disk
# vxdmpadm setattr path c2t10d0 pathtype=active
Setting the attributes of the paths to an enclosure
# vxdmpadm setattr path c3t10d0 pathtype=nomanual
# vxdmpadm setattr path c1t20d0 pathtype=nopreferred
Specifying the I/O policy
Displaying the I/O policy
Adaptive
Following policies may be set
# vxdmpadm setattr enclosure enc1 iopolicy=adaptive
Adaptiveminq
048
# vxdmpadm setattr enclosure Disk iopolicy=minimumq
This is the default I/O policy for Active/Active A/A arrays
# vxdmpadm setattr arrayname Sena iopolicy=priority
# vxdmpadm setattr arraytype A/A iopolicy=round-robin
Example of applying load balancing in a SAN
Default setting for this attribute is useallpaths=no
# vxdmpadm setattr arrayname Disk iopolicy=singleactive
# vxdisk list c3t2d15
# dd if=/dev/vx/rdsk/mydg/myvol1 of=/dev/null
DMP statistics are now reset
# vxdmpadm getattr enclosure ENC0 iopolicy
# vxdmpadm -c-fdisable ctlr=ctlrname
# vxdmpadm -c-fdisable path=pathname
Disabling I/O for paths, controllers or array ports
# vxdmpadm setattr enclosure ENC0 iopolicy=singleactive
# vxdmpadm enable ctlr=ctlrname
# vxdmpadm enable path=pathname
Enabling I/O for paths, controllers or array ports
Upgrading disk controller firmware
Renaming an enclosure
Re-enable the plex associated with the device
Stop I/O to all disks through one controller of the HBA
For the other controller on the HBA, enter
# vxdmpadm getattr enclosure enc0 recoveryoption
Configuring the response to I/O failures
# vxdmpadm setattr \
# vxdmpadm setattr arraytype A/A recoveryoption=default
Configuring the I/O throttling mechanism
# vxdmpadm setattr enclosure enc0 recoveryoption=nothrottle
# vxdmpadm getattr enclosure HDS9500-ALUA0 recoveryoption
Displaying recoveryoption values
HDS9500-ALUA0 Error-Retry
# vxdmpadm start restore interval=seconds policy=checkall
Configuring DMP path restoration policies
Displaying the status of the DMP path restoration thread
Stopping the DMP path restoration thread
This produces output such as the following
Configuring array policy modules
Displaying information about the DMP error-handling thread
To add and configure an APM, use the following command
One daemon should be shown as running
# vxdmpadm -r cfgapm modulename
Administering DMP using vxdmpadm
Creating and administering disk groups
Creating and administering disk groups
Block special device corresponding to this volume is
Specifying a disk group to commands
System-wide reserved disk groups
To nodg
Displaying the system-wide boot disk group
Rules for determining the default disk group
# vxdg bootdg
See the vxdg1M manual page for more information
# vxdg list
Displaying disk group information
# vxdg list diskgroup
# vxdisk -s list devicename
Displaying free space in a disk group
Creating a disk group
Following is example output
# vxdiskadd c1t0d0
Adding a disk to a disk group
# vxdg init mktdg mktdg01=c1t0d0
# vxdg -g diskgroup set cds=onoff
# vxdiskunsetup c1t0d0
# vxdiskunsetup devicename
Removing a disk from a disk group
# vxdg -g diskgroup rmdisk diskname
Deporting a disk group
Newdg
Importing a disk group
# vxdisk -s list
Select another disk group? y,n,q,? default n
Select disk group to import group,list,q,? default list
Handling disks with duplicated identifiers
VxVM Info V-5-2-374 The import of newdg was successful
# vxdisk -f-g diskgroup updateudid disk
Option to the vxdg import command, as shown in this
# vxdisk updateudid c2t66d0 c2t67d0
Writing a new Udid to a disk
# vxdisk -g diskgroup settag tagname disk
# vxdg -o useclonedev=on -o updateid import mydg
# vxdisk settag mytaggeddisks c2t66d0 c2t67d0
# vxdisk listtag
Sample cases of operations on cloned disks
Enabling configuration database copies on tagged disks
# vxdg -q listmeta diskgroup
# vxdg -o useclonedev=on -o tag=mytaggeddisks import mydg
Importing cloned disks without tags
# vxdg -g mydg set tagmeta=on tag=t1 nconfig=all nlog=all
# vxdisk -o alldgs list
To import only the cloned disks into the mydg disk group
Symmir command is used to split off the BCV device
# /usr/symcli/bin/symmir -g mydg split DEV001
Importing cloned disks with tags
# vxdg -n newdg -o useclonedev=on -o updateid import mydg
State of the cloned disk is now shown as online clonedisk
Disks are tagged as follows
# vxdisk set EMC08 clone=off # vxdisk -o alldgs list
Renaming a disk group
This command results in output such as the following
Dgid 774226267.1025.tweety
Moving disks between disk groups
# vxdg -tC -n newdg import diskgroup
Moving disk groups between systems
# vxrecover -g diskgroup -sb
To clear the locks during import, use the following command
Handling errors when importing disks
# vxdisk clearimport devicename
# vxdg -C import diskgroup
Reserving minor numbers for disk groups
Following error message indicates a recoverable error
# vxdg -f import diskgroup
# vxprint -g mydg reminor
# xvdg init newdg minor=30000 c1d0t0 c1t1d0
# vxdg -g diskgroup set maxdev=4079
Compatibility of disk groups between platforms
Example of a serial split brain condition in a cluster
Handling conflicting configuration copies
191 Typical arrangement of a 2-node campus cluster
Expected a = Expected B =
Automatically
Imported on host Y
# vxsplitlines -g newdg
Correcting conflicting configuration information
Reorganizing the contents of disk groups
Reorganizing the contents of disk groups
197
Disk group join operation
Limitations of disk group split and join
Listing objects potentially affected by a move
Moving DCO volumes between disk groups
201
Split Snapshot
Moving objects between disk groups
# vxdg -o expand move mydg rootdg mydg01
# vxprint
Splitting disk groups
Following commands would also achieve the same result
# vxdg -o expand split rootdg mydg rootdg07 rootdg08
# vxdg -o overrideverify join sourcedg targetdg
Joining disk groups
Disabling a disk group
Following command joins disk group mydg to rootdg
Mydg Mydg05 C1t96d0
Mydg06 C1t98d0
Destroying a disk group
Recovering a destroyed disk group
Upgrading a disk group
Use the disk group ID to import the disk group
Upgrading a disk group
Features supported by disk group versions
# vxdg upgrade dgname
To list the version of a disk group, use this command
# vxdg list dgname
# vxdg -T 120 init newdg newdg01=c0t3d0
Managing the configuration daemon in VxVM
Using vxnotify to monitor configuration changes
Backing up and restoring disk group configuration data
# vxnotify -f
# vxnotify -s
Using vxnotify to monitor configuration changes
# vxmake -g mydg sd mydg02-01 mydg02,0,8000
Creating subdisks
Displaying subdisk information
This command provides the following output
# vxprint -st
# vxprint -g diskgroup -l subdisk
Splitting subdisks
Moving subdisks
# vxsd -g mydg mv mydg03-01 mydg12-01 mydg12-02
Associating subdisks with plexes
Joining subdisks
# vxmake -g diskgroup plex plex sd=subdisk
# vxsd -g mydg -l 4096b assoc vol10-01 mydg15-01
# vxsd -g mydg assoc home-1 mydg02-01 mydg02-00 mydg02-01
# vxsd -g mydg -l 1 assoc vol02-01 mydg11-01
# vxsd -g diskgroup assoc plex subdisk10 ... subdiskMN-1
Associating log subdisks
# vxassist -g diskgroup addlog volume disk
# vxsd -g diskgroup aslog plex subdisk
# vxsd -g mydg aslog vol01-02 mydg02-01
Dissociating subdisks from plexes
To remove a subdisk, use the following command
Removing subdisks
Changing subdisk attributes
# vxedit -g mydg set putil0=DO-NOT-USE mydg02-01
# vxedit -g mydg set comment=subdisk comment mydg02-01
# vxmake -g mydg plex vol01-02 sd=mydg02-01,mydg02-02
Creating plexes
Displaying plex information
Creating a striped plex
Plex states
Active plex state
Clean plex state
Dcosnp plex state
Iofail plex state
Empty plex state
LOG plex state
Offline plex state
Stale plex state
Snaptmp plex state
Temp plex state
Temprm plex state
Plex condition flags
Enabled plex kernel state
Disabled plex kernel state
Attaching and associating plexes
Plex kernel states
# vxmend -g diskgroup off plex
Taking plexes offline
# vxmend -g mydg off vol01-02 vol02-02
Reattaching plexes
Detaching plexes
# vxmend -g diskgroup fix clean plex
Start the volume using the following command
# vxvol -g diskgroup start volume
Moving plexes
# vxplex -g diskgroup cp volume newplex
Copying volumes to plexes
Dissociating and removing plexes
# vxplex -g diskgroup -o rm dis plex
# vxedit -g diskgroup set attribute=value ... plex
Changing plex attributes
# vxedit -g mydg set comment=plex comment tutil2=u vol01-02
# vxedit -g mydg set putil0=DO-NOT-USE vol01-02
Creating volumes
RAID-5
Types of volume layouts
Mirror and concatenated-mirror volumes
Supported volume logs and maps
Advanced approach
Creating a volume
Assisted approach
Using vxassist
Vxassist
# vxassist options make volume length attributes
Following is a sample vxassist defaults file
Setting default values for vxassist
Disk group alignment constraints on volumes
Discovering the maximum size of a volume
# vxassist -g diskgroup maxsize layout=layout attributes
# vxassist -g dgrp maxsize layout=raid5 nlog=2
# vxassist -b -g diskgroup make volume length
Creating a volume on any disk
# vxassist -b make voldefault 10g
# vxprint -g diskgroup -G -F %align
# vxassist -b -g mydg make volspec 5g mydg03 mydg04
Creating a volume on specific disks
# vxassist -b -g mydg make volspec 5g ctlrc1 !targetc1t5
Specifying ordered allocation of storage to volumes
Mydg01 mydg02 mydg03 mydg04 mydg05 mydg06 mydg07 mydg08
Stripe volume
Volume across controllers
Creating a mirrored-concatenated volume
Creating a mirrored volume
Creating a concatenated-mirror volume
# vxassist -b -g mydg make volmir 5g layout=mirror
Creating a volume with a version 0 DCO volume
# vxvol -g diskgroup set logtype=drldrlseq volume
# vxdg -T 90 upgrade diskgroup
For more information, see Upgrading a disk group on
Creating a volume with dirty region logging enabled
Creating a volume with a version 20 DCO volume
# vxdg upgrade diskgroup
# vxassist -b -g diskgroup make volume length layout=stripe
Creating a striped volume
# vxassist -b -g mydg make volzebra 10g layout=stripe
Creating a striped-mirror volume
Creating a mirrored-stripe volume
Mirroring across targets, controllers or enclosures
# vxassist -b -g mydg make volraid 10g layout=raid5 nlog=2
Creating a RAID-5 volume
# vxassist -g diskgroup listtag volume
Creating tagged volumes
# vxassist -g diskgroup list tag=tagname volume
Creating a volume using vxmake
# vxmake -g diskgroup -d descriptionfile
Creating a volume using a vxmake description file
# vxassist -b -g diskgroup make volume length layout=mirror
Initializing and starting a volume
Mydg04-021/8000,mydg04-031/16000
# vxvol -g diskgroup init enable volume
Initializing and starting a volume created using vxmake
# vxvol -g diskgroup init active volume
# vxvol -g diskgroup init zero volume
Accessing a volume
Administering volumes
Vxprint command can also be applied to a single disk group
Displaying volume information
This is example output from this command
Active volume state
Volume states
Clean volume state
Empty volume state
Invalid volume state
Volume kernel states
Needsync volume state
Replay volume state
Disabled volume kernel state
Detached volume kernel state
Enabled volume kernel state
Monitoring and controlling tasks
Vxtask operations
Managing tasks with vxtask
Using the vxtask command
Putting a volume in maintenance mode
Stopping a volume
Adding a mirror to a volume
Starting a volume
To start all Disabled volumes, enter
Mirroring volumes on a VM disk
Mirroring all volumes
# /etc/vx/bin/vxmirror -g diskgroup -a
# /etc/vx/bin/vxmirror -d yes
# vxassist -gdiskgroup remove mirror volume
Mirror volumes on another disk? y,n,q,? default n
Removing a mirror
At the following prompt, press Return to make the mirror
Adding logs and maps to volumes
Preparing a volume for DRL and instant snapshots
Specifying storage for version 20 DCO plexes
# DCONAME=‘vxprint -g diskgroup -F%dconame volume‘
Using a DCO and DCO volume with a RAID-5 volume
Determining the DCO version number
# vxprint -g diskgroup -F%version $DCONAME
Determining if DRL logging is active on a volume
Determining if DRL is enabled on a volume
This command returns on if DRL logging is enabled
Disabling and re-enabling DRL
To re-enable DRL on a volume, enter this command
Upgrading existing volumes to use version 20 DCOs
To re-enable sequential DRL on a volume, enter
Use the following command on the volume to upgrade it
Adding traditional DRL logging to a mirrored volume
# vxassist -g mydg addlog volume logtype=drlseq nlog=n
To remove a DRL log, use the vxassist command as follows
Removing a traditional DRL log
# vxassist -g mydg addlog vol03 logtype=drl
# vxassist -b -g diskgroup addlog volume loglen=length
Adding a RAID-5 log using vxplex
Adding a RAID-5 log
# vxassist -g mydg addlog volraid
# vxprint -g diskgroup -ht volume
Resizing a volume
# vxassist -g diskgroup maxgrow volume
Removing a RAID-5 log
Online JFS Full Base JFS Lite
Resizing volumes using vxresize
VxFS Mounted File System
Unmounted File System
Extending to a given length
Resizing volumes using vxassist
Extending by a given length
Shrinking to a given length
Resizing volumes using vxvol
Shrinking by a given length
# vxassist -g diskgroup removetag volume tagname
Setting tags on volumes
Changing the read policy for mirrored volumes
Moving volumes from a VM disk
Removing a volume
To set the read policy to select, use the following command
Move volumes from another disk? y,n,q,? default n
To move volumes from a disk
VxVM vxevac Info
VxVM Info V-5-2-188 Evacuation of disk mydg02 is complete
# vxvol -g diskgroup set fastresync=on volume
Enabling FastResync on a volume
# vxprint -g diskgroup -F%fastresync volume
Checking whether FastResync is enabled on a volume
# vxprint -g diskgroup -F%hasdcolog volume
Disabling FastResync
# vxassist -g mydg relayout vol02 layout=stripe
Performing online relayout
Permitted relayout transformations
Relayout to From raid5 Concat
Supported relayout transformations for RAID-5 volumes
Relayout to From mirror-concat Concat
Mirror-concat Mirror-stripe
Relayout to From mirror-stripe Concat
Layered striped-mirror volumes
Supported relayout transformations for unmirrored stripe
Relayout to From stripe or stripe-mirror Concat
Specifying a plex for relayout
Specifying a non-default layout
Tagging a relayout operation
# vxassist -g fsgrp relayout vol04 layout=raid5 ncol=4
Viewing the status of a relayout
To resume the operation, use the vxtask command
Controlling the progress of a relayout
# vxrelayout -g mydg -o bg,slow=1000,iosize=10m start vol04
Converting between layered and non-layered volumes
# vxrelayout -g mydg -o bg reverse vol04
# vxassist -g mydg relayout vol1 ncol=5
# vxassist -g mydg convert vol1 layout=mirror-stripe
Converting between layered and non-layered volumes
Administering volume snapshots
Administering volume snapshots
Traditional third-mirror break-off snapshots
Independent Volume Vxassist snapclear
Traditional third-mirror break-off snapshots
Full-sized instant snapshots
308 Administering volume snapshots
Cycle Start Vxsnap make Vxsnap refresh Vxsnap prepare
Space-optimized instant snapshots
Emulation of third-mirror break-off snapshots
Linked break-off snapshot volumes
Cascaded snapshots
Cascaded snapshots
Creating a snapshot of a snapshot
Creating a snapshot of a snapshot
Create instant snapshot S2 of S1 Vxsnap make source=S1
Vxsnap dis S2
Creating multiple snapshots
Restoring the original volume from a snapshot
Restoring the original volume from a snapshot
Creating instant snapshots
Creating instant snapshots
Preparing to create instant and break-off snapshots
Creating a shared cache object
For example to start the cache object, cobjmydg
# LEN=‘vxprint -g diskgroup -F%len volume‘
# vxcache -g mydg start cobjmydg
# RSZ=‘vxprint -g diskgroup -F%regionsz $DCONAME‘
Creating and managing space-optimized instant snapshots
Creating instant snapshots
# fsck -F vxfs /dev/vx/rdsk/diskgroup/snapshot
# vxsnap -g mydg make source=myvol/snapvol=snap1myvol
Creating and managing full-sized instant snapshots
# vxsnap -g diskgroup syncwait snapvol
# vxsnap -g mydg syncwait snap2myvol
# vxprint -gdiskgroup -F%incompletesnapvol
# vxsnap -g mydg addmir vol1 nmirror=2 alloc=mydg10,mydg11
Creating and managing third-mirror break-off snapshots
# vxsnap -g mydg snapwait vol1 nmirror=2
# vxsnap -g diskgroup -b addmir volume mirvol=snapvol \
Creating and managing linked break-off snapshot volumes
Mirdg=snapdg
Creating multiple instant snapshots
Reattach the snapshot volume with the original volume. See
# vxsnap -g diskgroup make \
# vxsnap -g diskgroup make source=vol1/snapvol=snapvol1 \
# vxvset -g mydg list vset1
Creating instant snapshots of volume sets
# vxvset -g mydg list snapvset1
Svol1 409600
Svol0 204800
Svol2 614400
Removing a snapshot mirror
Adding snapshot mirrors to a volume
# vxsnap -g mydg rmmir vol1
Adding a snapshot to a cascaded snapshot hierarchy
Removing a linked break-off snapshot volume
Refreshing an instant snapshot
# vxsnap -g mydg rmmir vol1 mirvol=prepsnap mirdg=mysnapdg
# vxsnap -g mydg reattach snapmyvol source=myvol nmirror=1
Reattaching an instant snapshot
# vxsnap -g mydg snapwait myvol nmirror=1
Reattaching a linked break-off snapshot volume
# vxsnap -g mydg restore myvol source=snap3myvol
Restoring a volume from an instant snapshot
Dissociating an instant snapshot
# vxsnap -g snapdg snapwait myvol mirvol=prepsnap
Splitting an instant snapshot hierarchy
Removing an instant snapshot
# vxsnap -g mydg dis snap2myvol
# vxedit -g mydg -r rm snap2myvol
# vxsnap -g mydg split snap2myvol
Displaying instant snapshot information
# vxsnap -g diskgroup print vol
# vxsnap -g mydg print
# vxsnap -g dg -vx list
# vxsnap -g diskgroup -l -v -x list vol
Vxsnap -g diskgroup syncresume \
Controlling instant snapshot synchronization
Vxsnap -b -g diskgroup syncstart \
Vxsnap -g diskgroup syncstop vol volset
Improving the performance of snapshot synchronization
Listing the snapshots created on a cache
# vxcache -g diskgroup listvol cacheobject
# vxcache -g mydg set highwatermark=60 cobjmydg
Tuning the autogrow attributes of a cache
Growing and shrinking a cache
Finally, remove the cache object and its cache volume
Removing a cache
Creating traditional third-mirror break-off snapshots
# vxassist -g diskgroup snapwait volume
# vxassist -b -g diskgroup snapstart nmirror=N volume
# vxassist -g diskgroup snapstart voldef
# vxassist -g diskgroup snapshot nmirror=N volume snapshot
Create a snapshot volume using the following command
# vxassist -g diskgroup snapshot voldef snapvol
# vxedit -g diskgroup -rf rm snapshot
Converting a plex into a snapshot plex
Creating multiple snapshots
Reattaching a snapshot volume
# vxplex -g diskgroup convert state=SNAPDONE plex
# vxassist -g diskgroup -o allvols snapshot
# vxassist -g diskgroup -o allplexes snapback snapshot
Adding plexes to a snapshot volume
# vxassist -g diskgroup snapback nmirror=number snapshot
# vxprint -g diskgroup -F%rid $DCOVOL
Dissociating a snapshot volume
# vxassist snapclear snapshot
# vxassist snapprint volume
Output from this command is shown in the following examples
Displaying snapshot information
# vxassist -g mydg snapprint
Adding a version 0 DCO and DCO volume
Specifying storage for version 0 DCO plexes
# vxdco -g diskgroup -o rm dis dcoobj
Removing a version 0 DCO and DCO volume
# vxdco -g mydg dis myvoldco
For more information, see the vxdco1M manual
Reattaching a version 0 DCO and DCO volume
# vxdco -g mydg att myvol myvoldco
Adding a version 0 DCO and DCO volume
Creating and administering volume sets
Adding a volume to a volume set
Creating a volume set
Stopping and starting volume sets
Listing details of volume sets
Raw device node access to component volumes
Removing a volume from a volume set
# vxvset -g diskgroup -f rmvol volset volume
# vxvset -g mydg start set1 # vxvset -g mydg list set1
Enabling raw device access when creating a volume set
Controlling raw device access for an existing volume set
Displaying the raw device access settings for a volume set
# vxvset -g diskgroup -fset makedev=onoff vset
# vxvset -g mydg set makedev=on myvset2
# vxvset -g mydg set compvolaccess=read-write myvset2
# vxvset -g mydg set makedev=off myvset2
Raw device node access to component volumes
Configuring off-host processing
Example implementation of off-host processing
Implementing off-host processing solutions
# vxprint -g volumedg -F%instant volume
Implementing off-host online backup
# vxvol -g volumedg set fastresync=on volume
# vxdg import snapvoldg
# vxdg deport snapvoldg
# fsck -F vxfs /dev/vx/rdsk/snapvoldg/snapvol
# vxprint -g volumedg -F%instantvolume
# vxsnap -g volumedg snapwait volume mirvol=snapvol
Implementing decision support
375
# mount -F vxfs /dev/vx/dsk/snapvoldg/snapvol \ mountpoint
You can then resume the procedure from on
Implementing off-host processing solutions
Administering hot-relocation
How hot-relocation works
381
Mydg05
Partial disk failure mail messages
# vxrecover -b -g mydg home src
# vxstat -g mydg -s -ff home-02 src-02
Sd mydg01-04 Sd mydg01-06 Sd mydg02-03 Sd mydg02-04
How space is chosen for relocation
Complete disk failure mail messages
Failing disks mydg02
Configuring a system for hot-relocation
# vxdg -g diskgroup spare
Displaying spare disk information
Mydg mydg02 C0t2d0 658007
Marking a disk as a hot-relocation spare
Mark another disk as a spare? y,n,q,? default n
# vxedit -g mydg set spare=on mydg01
Removing a disk from use as a hot-relocation spare
Where diskname is the disk media name
Excluding a disk from hot-relocation use
Following confirmation is displayed
To use vxdiskadm to exclude a disk from hot-relocation use
Making a disk available for hot-relocation use
# vxedit -g diskgroup set nohotuse=off diskname
Moving and unrelocating subdisks
Configuring hot-relocation to use only spare disks
Spare=only
To root Subject Attempting VxVM relocation on host teal
Unrelocate to a new disk y,n,q,? default n
Moving and unrelocating subdisks using vxdiskadm
Status message is displayed at the end of the operation
Enter the original disk name disk,list,q,?
Moving and unrelocating subdisks using vxunreloc
Moving and unrelocating subdisks using vxassist
# vxassist -g mydg move home !mydg05 mydg02
Moving hot-relocated subdisks back to a different disk
Moving hot-relocated subdisks back to their original disk
Forcing hot-relocated subdisks to accept different offsets
Examining which subdisks were hot-relocated from a disk
Restarting vxunreloc after errors
# vxprint -g mydg -se sdorigdmname=mydg01
Modifying the behavior of hot-relocation
Nohup vxrelocd -o slow=IOdelay root
# nohup vxrelocd root
Nohup vxrelocd root user1 user2
See the vxrelocd1M manual page for more information
Alternatively, you can use the following command
# nohup /etc/vx/bin/vxrelocd root user1 user2
Administering cluster functionality
Overview of cluster volume management
399
Example of a 4-node cluster
Two types of disk groups are defined
Private and shared disk groups
Exclusivewrite ew
Activation modes of shared disk groups
Readonly ro
Sharedread sr
Enableactivation=true Defaultactivationmode=activation-mode
Activation mode Description
Activation modes for shared disk groups
Sharedwrite sw
Connectivity policy of shared disk groups
Global detach policy
Local detach policy
Disk group failure policy
Guidelines for choosing detach and failure policies
Limitations of shared disk groups
Effect of disk connectivity on cluster reconfiguration
Cluster reconfiguration
Cluster initialization and configuration
Various reasons that may be given are shown in Table
Vxclustadm utility
# /etc/vx/bin/vxclustadm nodestate state out of cluster
Reason user initiated stop
Reason Description
Node abort messages
Volume reconfiguration
Vxconfigd daemon recovery
Vxconfigd daemon
# hagrp -unfreeze group
# hagrp -freeze group
Node shutdown
Node abort
Multiple host failover configurations
Cluster shutdown
Failover
Import lock
Where the reason can describe errors such as
Corruption of disk group configuration
# vxdctl -c mode
Administering VxVM in cluster environments
Requesting node status and discovering the master node
See the vxdctl1M manual page for more information
Example output from this command is as follows
Example output from this command is displayed here
Determining if a disk is shareable
Listing shared disk groups
# vxdg -s init diskgroup diskname=devicename
Creating a shared disk group
Forcibly importing a disk group
Importing disk groups as shared
# vxdg -s import diskgroup
# vxdg -s -f import diskgroup
Converting a disk group from shared to private
Setting the disk detach policy on a shared disk group
Changing the activation mode on a shared disk group
Default disk detach policy is global
# vxdg -g diskgroup set activation=mode
Setting exclusive open access to a volume by a node
Creating volumes with exclusive open access by a node
# vxdg -g diskgroup set dgfailpolicy=dgdisableleave
# vxvol -g dskgrp set exclusive=on volmir
This command produces out put similar to the following
This command produces output similar to the following
Displaying the cluster protocol version
Displaying the supported cluster protocol version range
Upgrading the cluster protocol version
Recovering volumes in shared disk groups
Obtaining cluster performance statistics
# vxstat -b
Vol Vol1 2421 600000 99.0
Administering VxVM in cluster environments
Administering Sites and remote mirrors
Site-consistent volume with two plexes at each of two sites
Example of a two-site configuration with remote storage only
Configuring site-based allocation on a disk group
Configuring sites for hosts and disks
To remove the site name from a host, use this command
Configuring site consistency on a volume
Configuring site consistency on a disk group
# vxdg -g diskgroup set siteconsistent=on
# vxdg -g diskgroup set siteconsistent=off
Site-based allocation of storage to volumes
Setting the siteread policy on a volume
RAID-5 volumes in a site-consistent disk group
# vxassist -g diskgroup make volume size mirror=site
Command Description
Examples of storage allocation using sites
Making an existing disk group site consistent
Turn on site consistency for each volume in the disk group
Register a site record for each site with the disk group
Turn on site consistency for the disk group
Recovery from simulated site failure
Fire drill testing the configuration
Automatic site reattachment
Simulating site failure
# ps -afe
Failure scenarios and recovery procedures
# kill -9 PID
Recovery from host failure
Recovery from a loss of site connectivity
Recovery from storage failure
Failure scenario Recovery technique
Recovery from site failure
Failure scenarios and recovery procedures
See the vxse1M manual
About Storage Expert
How Storage Expert works
Before using Storage Expert
Running Storage Expert
One of the following keywords must be specified
Discovering what a rule does
Displaying rule attributes and their default values
Running a rule
# vxsestripes2 info
Rule result types
Setting rule attributes
Recovery time
Identifying configuration problems using Storage Expert
Checking for non-mirrored RAID-5 logs vxseraid5log3
Checking minimum and maximum RAID-5 log sizes vxseraid5log2
Vxseraid5log1
Checking on disk config size vxsedg3
Checking disk group configuration copies and logs vxsedg2
Disk groups
Checking the version number of disk groups vxsedg4
Checking states of plexes and volumes vxsevolplex
Checking volume redundancy vxseredundancy
Checking for non-imported disk groups vxsedg6
Volumes needing recovery
Checking the number of columns in RAID-5 volumes vxseraid5
Disk striping
Hardware failures
Disk sparing and relocation management
Rootability
System name
Checking the system name vxsehost
Rule definitions and attributes
See Running a rule on
Rule Description
Rule Attribute Default Description Value
Vxseraid5 Toonarrowraid5
Nsdthreshold
Toowideraid5
R5maxsize
Vxsestripes1 Default stripeunit
Vxseredundancy Volumeredundancy
Vxserootmir
Vxsespares
Vxsevolplex
Rule definitions and attributes
Data assignment
Performance guidelines
Mirroring
Striping
RAID-5
Combining mirroring and striping
Volume read policies
Performance monitoring
Setting performance priorities
Obtaining performance data
Printing volume statistics
Tracing volume operations
By vxtrace
Using I/O statistics
Using performance data
# vxprint -g mydg -tvh archive
Following is an extract from typical output
Mydg03-03 Archive-01 40960 C1t2d0
# vxassist -g mydg move archive !mydg03 destdisk
471
General tuning guidelines
Tuning VxVM
Using I/O tracing
Number of configuration copies for a disk group
Tuning guidelines for large systems
Changing the values of tunables
# vxedit set nconfig=5 bigdg
# vxdmpadm settune dmptunable=value
# vxdmpadm gettune dmptunable
Tunable parameters
Dmphealthtime
Dmppathswitchblksshift
Dmploglevel
Dmppathage
Dmprestoreinterval
Dmprestorecycles
Dmpprobeidlelun
Dmpqueuedepth
Dmpretrytimeout
Dmprestorepolicy
Dmpscsitimeout
Dmpretrycount
Voldefaultiodelay
Volcheckptdefault
Dmpstatinterval
Volfmrlogsz
Volmaxio
Volmaxvol
Volmaxioctl
Volmaxspecialio
Volmaxparallelio
Volsubdisknum
Voldrlmaxdrtregs
Volcvmsmartsync
Voldrlmaxseqdirty
Voldrlminregionsz
Voliomemmaxpoolsz
Voliotiobufdefault
Volioterrbufdflt
Voliotiobuflimit
Voliotmaxopen
Voliotiobufmax
Volpagemodmaxmemsz
Volraidrsrtransmax
Volraidminpoolsz
Tuning VxVM
If you are using a C shell csh or tcsh, use the commands
Commands summary
# vxdisk -g mydg list
Vxdisk -g diskgroup list diskname
Vxdg list diskgroup
# vxdg list mydg
Table A-2 Administering disks Command Description
Mydg03 mydg02
# vxedit -g mydg rename \
# vxedit -g mydg set \
Reserve=on mydg02
Spare=on mydg04
# vxdiskunsetup c0t3d0
Spare=off mydg04
# vxdisk offline c0t1d0
Mydg01=c0t1d0
# vxdg init mydg \
# vxsplitlines -g mydg
# vxdg -n newdg deport mydg
Mydg newdg myvol1
# vxdg -o expand listmove \
Newdg myvol1
Newdg myvol2 myvol3
# vxdg destroy mydg
# vxrecover -g mydg -sb
# vxmake -g mydg sd \
Mydg02-01 mydg02,0,8000
Vol01-01 mydg10-010 \
# vxsd -g mydg assoc \
Mydg11-011 mydg12-012
Mydg02-01
# vxmake -g mydg plex \
# vxsd -g mydg -o rm dis \
Vol01-02 \
Sd=mydg02-01,mydg02-02
# vxplex -g mydg mv \
Vol01-02
Vol02-02 vol02-03
# vxplex -g mydg cp vol02 \
Table A-6 Creating volumes Command Description
Vol03-01
Vol02-02
Vxassist -b -g diskgroup make \
Volume length layout=striperaid5 \
Stripeunit=W ncol=N attributes
Mysvol 20g layout=stripe \
Vol r5vol \
# vxmake -g mydg -Uraid5\
Plex=raidplex,raidlog1,\
Raidlog2
Table A-7 Administering volumes Command Description
# vxsnap -gmydg prepare \ myvol drl=on
Drl=onsequentialoff
Cvol 1g layout=mirror \
# vxassist -g mydg make \
Init=active mydg16 mydg17
Cachevolname=cvol
# vxassist -gmydg relayout \ vol2 layout=stripe
# vxsnap -g mydg unprepare \ myvol
Vol3 layout=raid5 \
Stripeunit=16 ncol=4
# vxassist -g mydg remove \ myvol
# vxrecover -g mydg \
Mytask -b mydg05
# vxtask -h -g mydg list
# vxtask -p -g mydg list
# vxtask pause mytask
# vxtask resume mytask
# vxtask abort mytask
Online manual pages
Administrative commands
Table A-9 Manual pages Name Description
Vxmend
Vxrecover
Vxmirror
Vxnotify
Device driver interfaces
File formats
Setup tasks after installation
Configuring Veritas Volume Manager
Adding foreign devices
Adding unsupported disk arrays as JBODs
Adding disks to disk groups
Guidelines for configuring storage
Mirroring guidelines
Striping guidelines
Dirty region logging guidelines
Hot-relocation guidelines
RAID-5 guidelines
517
Accessing volume devices
Configuring cluster support
Controlling VxVM’s view of multipathed devices
Converting existing VxVM disk groups to shared disk groups
Configuring shared disk groups
Reconfiguration tasks
Glossary
Single copy of a configuration database
Page
See disk enclosure
Jbod
Page
RAID
Page
Page
Page
Index
CDS
Page
CVM
Page
Page
DMP
Empty
Page
Page
Page
Page
Iofail 228 Nodarec 228 Nodevice 228 Recover 228 Removed
Page
Page
Page
Page
Page
TPD
Page
Detached Disabled Enabled
Page
Page
Page
Page
Page
