This chapter contains examples of common XVM administration procedures. After an overview of some things to keep in mind before you begin, it provides the following procedures:
Before configuring an XVM logical volume, you may need to assess the status of your disks and your system:
Before you can label a disk as an XVM disk, it must be formatted as a DVH/SGI disk or as a GTP disk:
To format the disk as a DVH/SGI disk under IRIX, use the fx command to initialize the disk if your disk has not been initialized during factory set-up.
For information on formatting a disk for use in XVM under Linux, see “Preparing to Configure XVM Volumes under Linux”.
For information on how the partitions must be configured on a GPT disk, see “XVM Partition Layout with GPT Disk Format” in Chapter 1
If you attempt to use the XVM Volume Manager to label a disk that is not a DVH/SGI disk or a GPT disk, you will get an error message indicating that the disk volume header partition is invalid.
When you run the xvm command under IRIX, you may get a message indicating that cluster services have not been enabled on the node and that you will only be able to manipulate local objects until cluster services are started. For information on starting cluster services, see CXFS Administration Guide for SGI InfiniteStorage .
Before beginning any of the procedures in this chapter, you may find it useful to execute an xvm show unlabeled/* command to view the names of the disks on the system that have not been assigned to the XVM Volume Manager.
You will not be able to label disks as XVM disks if they contain partitions currently in use as mounted filesystems. In a CXFS cluster, any XVM physical volumes that will be shared must be physically connected to all nodes in the cluster.
In general, you will find it useful to use the options of the show command to view your system configuration and status. For example, the show -v stripe0 command displays the stripe unit (in this case for stripe0).
To configure XVM logical volumes, you need to be logged in as root. However, you can display logical volume configuration information even if you do not have root privileges.
Note: As you configure an XVM logical volume, keep in mind that at any time you can view extended help information for an XVM command by entering the help command with the -v[erbose] option. For example, you can view the full help screen that includes the options for the slice command by entering the following:
xvm:cluster> help -v slice
These instructions only apply to standalone servers running SGI ProPack for Linux. For information on CXFS clients, see CXFS MultiOS Client-Only Guide for SGI InfiniteStorage. Before configuring logical volumes under Linux, you may need to run through the following checklist:
Under SGI ProPack for Linux, XVM is not installed by default. You should ensure that the xvm-cmds rpm is installed.
To use XVM under SGI ProPack for Linux, you must obtain and install the appropriate XVM license. XVM licenses are usually nodelocked and reside on the local machine in /etc/lk/keys.dat. You must have the XVM_STD_IPF feature. If you have the XVM_PLEX_IPF license only, XVM will report a license error. XVM_PLEX_IPF is required for the local mirroring feature. XVM_PLEX_CLUSTER is required for cluster mirror volumes.
Confirm that the xvm-standalone kernel module is loaded. You can use the lsmod command, as in the following example:
[root}# lsmod | grep xvm xvm-standalone 717208 0
You can determine what disks are potentially available on the system by executing
cat /proc/xscsi/dksc. You will not be able to configure the system disk as an XVM disk.You may need to determine whether any of the disks on the system are dual-ported, which means that each disk is connected to two controllers. If a disk is dual-ported, what appear to be two different disk paths could reference the same disk. You need to be careful that you don't mount the same disk device as a filesystem outside of XVM that you are also using as part of an XVM volume.
Use the ls command to get a list of the disks, as in the following example:
ls /dev/xscsi/pci*/target*/lun*/disc
You must format each disk you will use for the XVM volume as a DVH/SGI disk or as a GPT disk. To format the disk as a DVH disk, use the parted command as in the following example.
parted /dev/xscsi/pci05.01.0/target98/lun0/disc mklabel dvh
For information on how the partitions must be configured on a GPT disk, see “XVM Partition Layout with GPT Disk Format” in Chapter 1
If you are going to set up Command Tagged Queuing (CTQ) or if you are going to enable write-caching, you should do so at this point.
Once these requirements are met, configuring an XVM volume under Linux is the same as configuring an XVM volume under IRIX. After you are familiar with the general requirements for creating a simple XVM volume on Linux, you should be able to use the examples in this chapter to determine how to configure more complex volumes.
The following example shows the procedure for creating a simple logical volume that stripes data across three disks. In this example, the entire usable space of each disk is used for the slice.
Figure 7-1 shows the logical volume this example creates.
Assign disks to XVM to manage. This example assigns three disks to the XVM Volume Manager. You need to perform this procedure only once for each disk that you will be using to create XVM logical volumes.
# xvm xvm:cluster> label -name disk0 dks2d70 disk0 xvm:cluster> label -name disk1 dks2d71 disk1 xvm:cluster> label -name disk2 dks2d72 disk2
You may want to view all the disks that have been assigned to the XVM volume manager as XVM physical volumes to verify what you have labeled:
xvm:cluster> show phys/* phys/disk0 35542780 online phys/disk1 35542780 online phys/disk2 35542780 online
Create a slice that consists of all of the usable blocks of each of the XVM physical volumes:
xvm:cluster> slice -all disk* </dev/cxvm/disk0s0> slice/disk0s0 </dev/cxvm/disk1s0> slice/disk1s0 </dev/cxvm/disk2s0> slice/disk2s0
Create a stripe that consists of the three slices you have defined. In this example, the generated volume will be named stripedvol explicitly. A data subvolume will automatically be generated as well.
The following command names the generated volume stripedvol:
xvm:cluster> stripe -volname stripedvol slice/disk0s0 slice/disk1s0 slice/disk2s0 </dev/cxvm/stripedvol> stripe/stripe0
In this example:
/dev/rcxvm/stripedvol is the name of the volume on which you can execute the mkfs command
stripe/stripe0 is the name of the stripe object
In this example, the name of the stripe object is subject to change on subsequent boots but the name of the volume is not.
View the topology of the logical volume you have created:
xvm:cluster> show -top stripedvol vol/stripedvol 0 online subvol/stripedvol/data 106627968 online stripe/stripe0 106627968 online,tempname slice/disk0s0 35542780 online slice/disk1s0 35542780 online slice/disk2s0 35542780 onlineExit the xvm tool by typing exit (or quit or bye). You can then execute the mkfs command on the volume.
xvm:cluster> exit # mkfs /dev/cxvm/stripedvol meta-data=/dev/cxvm/stripedvol isize=256 agcount=51, agsize=261344 blks data = bsize=4096 blocks=13328496, imaxpct=25 = sunit=16 swidth=48 blks, unwritten=1 naming =version 1 bsize=4096 log =internal log bsize=4096 blocks=1168 realtime =none extsz=65536 blocks=0, rtextents=0You can now mount the filesystem. For a shared filesystem in a CXFS cluster, you mount the filesystem with the CXFS GUI or the cmgr(1M) command, as described in CXFS Version 2 Software Installation and Administration Guide.
For a local filesystem that is not part of a cluster, you can put a logical volume in the fstab file and use the mount command to mount the filesystem you created.
The following example shows the procedure for creating a stripe on the outer third of a disk. It also includes some advice on naming volume elements.
Figure 7-2 shows the logical volume this example creates.
Assign disks to XVM to manage. This example assigns four disks to XVM. Note that four separate controllers are chosen for better stripe performance.
xvm:cluster> label -name lucy dks21d0 lucy xvm:cluster> label -name ricky dks22d0 ricky xvm:cluster> label -name ethyl dks23d0 ethyl xvm:cluster> label -name fred dks24d0 fred
In this example, you use one-third of each disk for the stripe.
There are two ways to partition a disk into thirds. You can allocate the entire disk, but only use the last third. For example, for disk lucy you could do the following (and use slice/lucys2 for the stripe):
xvm:cluster> slice -equal 3 lucy </dev/cxvm/lucys0> slice/lucys0 </dev/cxvm/lucys1> slice/lucys1 </dev/cxvm/lucys2> slice/lucys2
Alternately, you can confine the block range explicitly to one-third of the disk. For example, you can do the following to allocate the last third of the other disks (ricky, ethyl, and fred):
xvm:cluster> slice -start 11852676 -length 5926340 ricky </dev/cxvm/rickys0> slice/rickys0 xvm:cluster> slice -start 11852676 -length 5926340 ethyl </dev/cxvm/ethyls0> slice/ethyls0 xvm:cluster> slice -start 11852676 -length 5926340 fred </dev/cxvm/freds0> slice/freds0
Verify the allocation.
The following example shows the allocation on lucy, the disk divided into three equal stripes:
The following example verifies the allocation on ricky, one of the disks that was allocated explicitly:
xvm:cluster> show -v ricky XVM physvol phys/ricky ========================= ... --------------------------------------------------- 0 11852676 (unused) 11852676 5926340 slice/rickys0 -------------------------------------------------------------------
Create the stripe. In this example, the generated volume is explicitly named I_Love_Lucy.
xvm:cluster> stripe -volname I_Love_Lucy -unit 128 slice/lucys2 \ slice/rickys0 slice/ethyls0 slice/freds0 </dev/cxvm/I_Love_Lucy> stripe/stripe0
Sometimes it may be useful to categorize portions of a complex volume by name. For example, you may want to name a portion of a volume faststripe so that a search can be done for volumes that have fast stripe objects. The following command names a stripe as well as the volume:
xvm:cluster> stripe -volname I_Love_Lucy -vename faststripe0 \ -unit 128 slice/lucys2 slice/rickys0 slice/ethys0 slice/freds0 </dev/cxvm/I_Love_Lucy> stripe/faststripe0
When you name the stripe as in the preceding example, you can use wildcards to show all fast stripes:
xvm:cluster> show -top stripe/fast* stripe/faststripe0 23705088 online slice/lucys2 5926340 online slice/rickys0 5926340 online slice/ethyls0 5926340 online slice/freds0 5926340 online
You can also use wildcards to show all objects starting with `I', as in the following example:
xvm:cluster> show I* vol/I_Love_Lucy 0 online
Exit the xvm tool by typing exit (or quit or bye). You can now execute the mkfs command on the volume.
xvm:cluster> exit hugh3 2# mkfs /dev/cxvm/I_Love_Lucy meta-data=/dev/rxvm/I_Love_Lucy isize=256 agcount=26, agsize=256416 blks data = bsize=4096 blocks=6666528, imaxpct=25 = sunit=16 swidth=48 blks, unwritten=1 naming =version 1 bsize=4096 log =internal log bsize=4096 blocks=1168 realtime =none extsz=65536 blocks=0, rtextents=0Mount the filesystem. For a shared filesystem in a CXFS cluster, you mount the filesystem with the CXFS GUI or the cmgr(1M) command, as described in CXFS Version 2 Software Installation and Administration Guide.
If your XVM volume is a local volume, you can put a logical volume in the fstab file and use the mount command to mount the filesystem you created.
The following example creates an XVM logical volume that includes both a data subvolume and a log subvolume. In this example, the data subvolume consists of all the usable space of two disks, and the log subvolume consists of all the usable space of a third disk.
Figure 7-3 shows the logical volume this example creates.
Assign three disks to XVM to manage.
# xvm xvm:cluster> label -name disk0 dks0d2 disk0 xvm:cluster> label -name disk1 dks0d3 disk1 xvm:cluster> label -name disk2 dks5d42 disk2
Create a slice that consists of all of the usable blocks of each of the XVM physical volumes you have created:
xvm:cluster> slice -all disk* </dev/xvm/disk0s0> slice/disk0s0 </dev/xvm/disk1s0> slice/disk1s0 </dev/xvm/disk2s0> slice/disk2s0
Combine two of the slices into a concat. In this example, the generated volume is named concatvol.
xvm:cluster> concat -volname concatvol slice/disk0s0 slice/disk1s0 </dev/cxvm/concatvol> concat/concat3
You can view the configuration of the volume you have defined that does not yet contain a log subvolume:
xvm:cluster> show -top vol/concatvol vol/concatvol 0 online subvol/concatvol/data 35554848 online concat/concat3 35554848 online,tempname slice/disk0s0 17777424 online slice/disk1s0 17777424 onlineCreate the log subvolume consisting of the third slice you created. Use the -tempname option to indicate that the system will generate a temporary name for the volume. You will not need to name this volume, as you will be attaching the log subvolume to the existing concatvol volume.
xvm:cluster> subvol -tempname -type log slice/disk2s0 </dev/cxvm/vol7_log> subvol/vol7/log
Attach the log subvolume to the existing concatvol volume.
xvm:cluster> attach subvol/vol7/log vol/concatvol vol/concatvol
Display the logical volume:
xvm:cluster> show -top vol/concatvol vol/concatvol 0 online subvol/concatvol/data 35554848 online concat/concat3 35554848 online,tempname slice/disk0s0 17777424 online slice/disk1s0 17777424 online subvol/concatvol/log 17779016 online slice/disk2s0 17779016 online
The following example creates an XVM logical volume that includes a data subvolume, a log subvolume, and a real-time subvolume. Two similar ways of performing this procedure are shown.
Figure 7-4 shows the logical volume this example creates.
This example assumes that you have already assigned disks to XVM to manage and that you have previously created the five slices you will use to build the logical volume:
slice/disk1s0
slice/disk2s0
slice/disk3s0
slice/disk4s0
slice/disk5s0.
Create the concat that will comprise the data subvolume:
xvm:cluster> concat -tempname slice/disk1s0 slice/disk2s0 </dev/cxvm/vol0> concat/concat0
Create the stripe that will comprise the real-time subvolume:
xvm:cluster> stripe -tempname slice/disk3s0 slice/disk4s0 </dev/cxvm/vol1> stripe/stripe1
Create the data subvolume:
xvm:cluster> subvolume -tempname -type data concat/concat0 </dev/cxvm/vol2> subvol/vol2/data
Create the real-time subvolume:
xvm:cluster> subvolume -tempname -type rt stripe/stripe1 </dev/cxvm/vol3_rt> subvol/vol3/rt
Create the log subvolume:
xvm:cluster> subvolume -tempname -type log slice/disk5s0 </dev/cxvm/vol4_log> subvol/vol4/log
Create the logical volume that contains the three subvolumes:
xvm:cluster> volume -volname myvol subvol/vol2/data \ subvol/vol4/log subvol/vol3/rt vol/myvol
Display the logical volume:
xvm:cluster> show -top myvol vol/myvol 0 online subvol/myvol/data 35558032 online concat/concat0 35558032 online,tempname slice/disk1s0 17779016 online slice/disk2s0 17779016 online subvol/myvol/log 8192 online slice/disk5s0 8192 online subvol/myvol/rt 35557888 online stripe/stripe1 35557888 online,tempname slice/disk3s0 17779016 online slice/disk4s0 17779016 online
The following sequence of commands generates the same volume, but with one less step since the volume name is established with the concat command. The log and real-time subvolumes are subsequently attached.
xvm:cluster> concat -volname myvol slice/disk1s0 slice/disk2s0
</dev/cxvm/myvol> concat/concat1
xvm:cluster> stripe -tempname slice/disk3s0 slice/disk4s0
</dev/cxvm/vol6> stripe/stripe2
xvm:cluster> subvolume -tempname -type rt stripe/stripe2
</dev/cxvm/vol7_rt> subvol/vol7/rt
xvm:cluster> subvolume -tempname -type log slice/disk5s0
</dev/cxvm/vol8_log> subvol/vol8/log
xvm:cluster> attach subvol/vol8/log subvol/vol7/rt myvol
vol/myvol
xvm:cluster> show -top myvol
vol/myvol 0 online
subvol/myvol/data 35558032 online
concat/concat1 35558032 online,tempname
slice/disk1s0 17779016 online
slice/disk2s0 17779016 online
subvol/myvol/log 8192 online
slice/disk5s0 8192 online
subvol/myvol/rt 35557888 online
stripe/stripe2 35557888 online,tempname
slice/disk3s0 17779016 online
slice/disk4s0 17779016 online
|
When you configure an XVM logical volume, you can create the volume's hierarchy from the bottom up or from the top down. The example in this section creates the same XVM logical volume as in the example in “Creating a Logical Volume with a Data and Log Subvolume” and shown in Figure 7-3, but it creates an empty volume first before attaching the child volume elements for that volume.
Assign three disks to XVM to manage:
# xvm xvm:cluster> label -name disk0 dks0d2 disk0 xvm:cluster> label -name disk1 dks0d3 disk1 xvm:cluster> label -name disk2 dks5d42 disk2
Create a slice that consists of all of the usable blocks of each of the XVM physical volumes you have created:
xvm:cluster> slice -all disk* </dev/cxvm/disk0s0> slice/disk0s0 </dev/cxvm/disk1s0> slice/disk1s0 </dev/cxvm/disk2s0> slice/disk2s0
Create an empty volume named topdownvol:
xvm:cluster> volume -volname topdownvol vol/topdownvol
Display the volume:
xvm:cluster> show -top vol/top* vol/topdownvol 0 offline (empty) * *Create an empty concat volume element and display the result:
xvm:cluster> concat -tempname </dev/cxvm/vol8> concat/concat5 xvm:cluster> show -top vol/vol8 vol/vol8 0 offline,tempname subvol/vol8/data 0 offline,pieceoffline concat/concat5 0 offline,tempname (empty) * *Attach the generated data subvolume that contains the concat to topdownvol and display the result:
xvm:cluster> attach subvol/vol8/data vol/topdownvol vol/topdownvol xvm:cluster> show -top vol/topdownvol vol/topdownvol 0 offline subvol/topdownvol/data 0 offline,pieceoffline concat/concat5 0 offline,tempname (empty) * *Attach two slices to fill the empty concat and display the result:
xvm:cluster> attach slice/disk0s0 slice/disk1s0 concat/concat5 </dev/cxvm/topdownvol> concat/concat5 xvm:cluster> show -top vol/topdownvol vol/topdownvol 0 online subvol/topdownvol/data 35554848 online concat/concat5 35554848 online,tempname slice/disk0s0 17777424 online slice/disk1s0 17777424 onlineCreate a log subvolume:
xvm:cluster> subvol -tempname -type log </dev/cxvm/vol9_log> subvol/vol9/log
Attach the log subvolume to topdownvol and display the result:
xvm:cluster> attach subvol/vol9/log vol/topdownvol vol/topdownvol xvm:cluster> show -top vol/topdownvol vol/topdownvol 0 offline subvol/topdownvol/data 35554848 online concat/concat5 35554848 online,tempname slice/disk0s0 17777424 online slice/disk1s0 17777424 online subvol/topdownvol/log 0 offline (empty) * *Attach the third slice to the log subvolume and display the results:
xvm:cluster> attach slice/disk2s0 subvol/topdownvol/log </dev/cxvm/topdownvol_log> subvol/topdownvol/log xvm:cluster> show -top vol/topdownvol vol/topdownvol 0 online subvol/topdownvol/data 35554848 online concat/concat5 35554848 online,tempname slice/disk0s0 17777424 online slice/disk1s0 17777424 online subvol/topdownvol/log 17779016 online slice/disk2s0 17779016 online
The following example creates a logical volume with striped mirrors. In this example the logical volume contains a stripe that consists of two mirrors, each mirroring a slice that contains all of the usable blocks of an XVM physical volume.
Figure 7-5 shows the logical volume this example creates.
Assign four disks to XVM to manage:
xvm:cluster> label -name disk0 dks2d70 disk0 xvm:cluster> label -name disk1 dks2d71 disk1 xvm:cluster> label -name disk2 dks2d72 disk2 xvm:cluster> label -name disk3 dks2d73 disk3
Create a slice out of all of the usable blocks on each XVM physical volume:
xvm:cluster> slice -all disk* </dev/cxvm/disk0s0> slice/disk0s0 </dev/cxvm/disk1s0> slice/disk1s0 </dev/cxvm/disk2s0> slice/disk2s0 </dev/cxvm/disk3s0> slice/disk3s0
Create two mirrors, each consisting of two of the slices you have defined. Since you are creating new mirrors that will be written to before they are read, you can specify the -clean option. This indicates that the mirrors do not need to be synchronized on creation.
If you do not specify the -clean option, executing this command initiates a mirror revive, which synchronizes the data on the slices. A message indicating that a revive has begun would be written to the SYSLOG, and another message would be written to the SYSLOG when the revive completes.
You will not need to define a persistent name for the volume that will be generated.
xvm:cluster> mirror -tempname -clean slice/disk0s0 slice/disk1s0 </dev/cxvm/vol2> mirror/mirror1 xvm:cluster> mirror -tempname -clean slice/disk2s0 slice/disk3s0 </dev/cxvm/vol3> mirror/mirror2
Create a stripe that consists of the two mirrors you have defined, naming the volume that will be generated to contain the stripe. This command attaches the mirrors to the stripe.
xvm:cluster> stripe -volname mirvol mirror/mirror1 mirror/mirror2 </dev/cxvm/mirvol> stripe/stripe2
Display the XVM logical volume:
xvm:cluster> show -top mirvol vol/mirvol 0 online subvol/mirvol/data 71085312 online stripe/stripe2 71085312 online,tempname mirror/mirror1 35542780 online,tempname slice/disk0s0 35542780 online slice/disk1s0 35542780 online mirror/mirror2 35542780 online,tempname slice/disk2s0 35542780 online slice/disk3s0 35542780 onlineYou can now execute the mkfs command on the volume.
xvm:cluster> exit 3# mkfs /dev/cxvm/mirvol meta-data=/dev/cxvm/mirvol isize=256 agcount=17, agsize=261440 blks data = bsize=4096 blocks=4444352, imaxpct=25 = sunit=16 swidth=32 blks, unwritten=1 naming =version 1 bsize=4096 log =internal log bsize=4096 blocks=1168 realtime =none extsz=65536 blocks=0, rtextents=0Mount the filesystem. For a filesystem in a CXFS cluster, you mount a filesystem with the CXFS GUI or the cmgr(1M) command, as described in CXFS Version 2 Software Installation and Administration Guide. For a local filesystem, you can put a logical volume in the fstab file and use the mount command.
This section describes the procedures for the following tasks:
The two mirroring procedures show two alternate ways of creating the same mirrored root and swap partitions.
In addition, this section describes the procedure for the following task:
“Creating a Mirrored XVM System Disk on a Running Root Disk”
The following procedure shows how to create an XVM system disk and then mirror the disk by using the -mirror option of the label command. The procedure described in “Mirroring a System Disk through Mirror Insertion” shows how to create the same mirrored system disk by inserting mirrors into the logical volume and then attaching slices to the empty mirror leg.
Label a disk as an XVM disk of type root.
The following command labels dks0d3 and names the physvol root_1:
xvm:local> label -type root -name root_1 dks0d3 root_1
Executing this command creates the physvol root_1 with two slices, as shown in Figure 7-6.
You can see the layout of the slices on the physvol root_1 with the show -v command:
xvm:local> show -v phys/roo* XVM physvol phys/root_1 ========================= ... Physvol Usage: Start Length Name --------------------------------------------------- 0 262144 slice/root_1s1 262144 17515280 slice/root_1s0 ...
In addition to creating the slices, executing this command creates the logical volume root_1_root0 and the logical volume root_1_swap1, as shown in Figure 7-7.
You can display the layout of the logical volumes with the show -top command:
xvm:local> show -top vol/root* vol/root_1_root0 0 online subvol/root_1_root0/data 17515280 online slice/root_1s0 17515280 online vol/root_1_swap1 0 online subvol/root_1_swap1/data 262144 online slice/root_1s1 262144 onlineInstall the operating system on the new system disk.Before installing, you must do the following:
Exit from the XVM Volume Manager
xvm:local> quit
Execute the mkfs command on the root filesystem
hugh2 4# mkfs /dev/lxvm/root_1_root0 meta-data=/dev/lxvm/root_1_root0 isize=256 agcount=9, agsize=243268 blks data = bsize=4096 blocks=2189410, imaxpct=25 = sunit=0 swidth=0 blks, unwritten=1 naming =version 1 bsize=4096 log =internal log bsize=4096 blocks=1168 realtime =none extsz=65536 blocks=0, rtextents=0Mount the root filesystem:
hugh2 3# mkdir /mnt hugh2 5# mount /dev/lxvm/root_1_root0 /mnt
You can now install the operating system on /mnt.
There are four environment variables that specify the location of root and swap that you change to indicate a new root or swap partition: root, OSLoadPartition, SystemPartition, and swap. For information on the environment variables, see IRIX Admin: System Configuration and Operation.
Reboot the operating system:
xvm:local> quit hugh2 2# /etc/reboot [...] The system is ready hugh2 1#
Label the disk to use as a mirror of the system disk.
The following example labels disk dks0d4 as an XVM physvol of type root name root_2:
xvm:local> label -type root -name root_2 -mirror root_1 dks0d4 </dev/lxvm/root_1_root0> mirror/mirror2 </dev/lxvm/root_1_swap1> mirror/mirror3 root_2
Executing this command creates the physvol root_2, which contains two slices, as shown in Figure 7-8.
In addition to creating the slices, executing this command mirrors the logical volume root_1_root0 and the logical volume root_1_swap1, as shown in Figure 7-9.
You can see the layout of the mirrored root and swap logical volumes with the show -top command:
xvm:local> show -top vol/roo* vol/root_1_root0 0 online subvol/root_1_root0/data 17516872 online mirror/mirror2 17516872 online,tempname slice/root_1s0 17516872 online slice/root_2s0 17516872 online vol/root_1_swap1 0 online subvol/root_1_swap1/data 262144 online mirror/mirror3 262144 online,tempname slice/root_1s1 262144 online slice/root_2s1 262144 online
The following procedure shows how to create the same mirrored system disk that was created in the procedure described in “Mirroring a System Disk with the label -mirror Command”. In this procedure, however, the mirrored disk is created by inserting mirrors into the logical volume and then attaching slices to the empty mirror leg.
Label a disk as an XVM disk of type root.
The following command labels dks0d3 and names the physvol root_1:
xvm:local> label -type root -name root_1 dks0d3 root_1
Executing this command creates the physvol root_1 with two slices, which is the same configuration that is shown in Figure 7-6.
This command also creates the logical volume root_1_root0 and the logical volume root_1_swap1, as is shown in Figure 7-7.
Install the operating system on the new system disk, as described in step 2 of “Mirroring a System Disk with the label -mirror Command”.
Reboot the operating system, as described in step 3 of “Mirroring a System Disk with the label -mirror Command”.
Insert mirrors into the root and swap logical volumes, above the slices that make up the root and swap partitions on those volumes.
The following commands insert mirrors into the logical volumes root_1_root0 and root_1_swap1, above the slices root_1s1 and root_1s0:
xvm:local> insert mirror slice/root_1s0 </dev/lxvm/root_1_root0> mirror/mirror5 xvm:local> insert mirror slice/root_1s1 </dev/lxvm/root_1_swap1> mirror/mirror6
After you have inserted the mirrors, the logical volumes root_1_root and root_1_swap are configured as shown in Figure 7-10.
You can view the logical volume configuration after the insertion of the mirrors with the show -top command:
xvm:local> show -top vol/root_1* vol/root_1_root0 0 online subvol/root_1_root0/data 17516872 online mirror/mirror5 17516872 online,tempname slice/root_1s0 17516872 online vol/root_1_swap1 0 online subvol/root_1_swap1/data 262144 online mirror/mirror6 262144 online,tempname slice/root_1s1 262144 onlineCreate a second system disk of type root:
xvm:local> label -type root -name root_2 dks5d7 root_2
Executing this command creates the physvol root_2, which contains two slices. This is the same configuration shown in Figure 7-8. The root and slice partitions on this second disk need to be at least as large as the root and swap partitions on the first disk, so that they can be mirrored.
Executing this command also generates logical volumes root_2_root0 and root_2_swap1. You can use the show -top command to see the logical volume configuration:
xvm:local> show -top vol/root_2* vol/root_2_root0 0 online subvol/root_2_root0/data 17516872 online slice/root_2s0 17516872 online vol/root_2_swap1 0 online subvol/root_2_swap1/data 262144 online slice/root_2s1 262144 onlineAttach the slices on root_2 to the mirrors that you inserted into the logical volumes root_1_root0 and root_1_swap1:
xvm:local> attach slice/root_2s0 mirror/mirror5 </dev/lxvm/root_1_root0> mirror/mirror5 xvm:local> attach slice/root_2s1 mirror/mirror6 </dev/lxvm/root_1_swap1> mirror/mirror6
The root and swap logical volumes are now configured as in Figure 7-9. You can use the show -top command to view the configuration:
xvm:local> show -top vol/root_1* vol/root_1_root0 0 online subvol/root_1_root0/data 17516872 online mirror/mirror5 17516872 online,tempname,reviving:53% slice/root_1s0 17516872 online slice/root_2s0 17516872 online vol/root_1_swap1 0 online subvol/root_1_swap1/data 262144 online mirror/mirror6 262144 online,tempname slice/root_1s1 262144 online slice/root_2s1 262144 onlineAttaching the slices on root_2 to the root_1_root0 and root_1_swap1 logical volumes leaves root_2_root0 and root_2_swap1 as empty logical volumes, as shown by the following command:
xvm:local> show -top vol/root_2* vol/root_2_root0 0 offline subvol/root_2_root0/data 17516872 offline,incomplete (empty) * * vol/root_2_swap1 0 offline subvol/root_2_swap1/data 262144 offline,incomplete (empty) * *These empty logical volumes will be deleted the next time you reboot, or you can delete them manually. The following command deletes the two empty volumes, as is verified by the show command that follows it.
xvm:local> delete -all vol/root_2_root0 vol/root_2_swap1 xvm:local> show vol/roo* vol/root_1_root0 0 online vol/root_1_swap1 0 online
The following procedure labels a running root disk as an XVM system disk and then, after rebooting the system, creates a three-way mirror of the system disk. Note that when you are creating an XVM system disk you must be in the local domain.
| Note: When you label an existing system disk as an XVM disk, the layout for partition 0 and partition 1 remains unchanged from the current layout. |
From the local domain, label the current running root disk as an XVM system disk.
The following command labels root disk dks0d1 as an XVM physvol of type root named xvmdisk:
xvm:local> label -nopartchk -type root -name xvmdisk dks0d1 xvmdisk
Executing this command creates the physvol xvmdisk with two slices, as shown in Figure 7-11.
In addition to creating the XVM slices for root and swap, executing this command creates the logical volume xvmdisk_root0 and the logical volume xvmdisk_swap1, as shown in Figure 7-12.
Reboot the operating system. This is necessary to ensure that the open volumes of the running root disk are closed and then opened to go through the XVM I/O path before the disks are mirrored.
xvm:local> quit hugh2 2# /etc/reboot [...] The system is ready hugh2 1#
Bring up the XVM Volume Manager in the local domain, since the system disk is generally used to boot only one node:
hugh2 1# xvm -domain local
Label the two disks you will use as mirrors for the XVM system disk.
The following command labels disks dks0d3 and dks0d4 as XVM disks of type root that will mirror xvmdisk. Note that since you are creating logical volumes in the local domain, the volumes are in the /dev/lxvm directory.
xvm:local> label -type root -mirror xvmdisk dks0d3 dks0d4 </dev/lxvm/xvmdisk_swap1> mirror/mirror0 </dev/lxvm/xvmdisk_root0> mirror/mirror1 dks0d3 dks0d4
Executing this command creates two physvols, named dks0d3 and dks0d4 by default. Each of these physvols contains two slices, as shown in Figure 7-13.
In addition to creating the slices, this command mirrors the logical volume xvmdisk_root0 and the logical volume xvmdisk_swap1 as a three-way mirror, as shown in Figure 7-14.
You can see the layout of the slices on xvmdisk, dks0d3, and dks0d4 with the show -v command.
xvm:local> show -v phys/* XVM physvol phys/dks0d3 ========================= ... Physvol Usage: Start Length Name --------------------------------------------------- 0 262144 slice/dks0d3s1 262144 17515280 slice/dks0d3s0 ... XVM physvol phys/dks0d4 ========================= ... Physvol Usage: Start Length Name --------------------------------------------------- 0 262144 slice/dks0d4s1 262144 17515280 slice/dks0d4s0 ... XVM physvol phys/xvmdisk ========================= ... Physvol Usage: Start Length Name --------------------------------------------------- 0 262144 slice/xvmdisks1 262144 17515280 slice/xvmdisks0 ...
You can see the layout of the mirrored root and swap logical volumes with the show -top command. In this example, the mirrors have just begun to revive.
xvm:local> show -top vol vol/xvmdisk_root0 0 online subvol/xvmdisk_root0/data 17515280 online,open mirror/mirror0 17515280 online,tempname,reviving:2%,open slice/xvmdisks0 17515280 online,open slice/dks0d3s0 17515280 online,open slice/dks0d4s0 17515280 online,open vol/xvmdisk_swap1 0 online subvol/xvmdisk_swap1/data 262144 online,open mirror/mirror1 262144 online,tempname,reviving:queued,open slice/xvmdisks1 262144 online,open slice/dks0d3s1 262144 online,open slice/dks0d4s1 262144 online,open
The following procedure shows how to configure an XVM system disk with a swap volume that includes a concat. This procedure does not relabel the currently running root disk.
This procedure creates a root logical volume that consists of one slice on an XVM system disk, and a swap logical volume that consists of two slices that make of a concat, as shown in Figure 7-15.
In this example, the two slices that make up the swap volume are on two different disks.
The following command labels disk dks1d2 as an XVM system disk named bootdisk. The -clrparts option of the XVM label command is used to override the existing partitioning scheme on the disk, if the disk already contains a partition 0.
xvm:local> label -clrparts -type root -name bootdisk dks1d2 bootdisk
Executing this command creates the physvol bootdisk with two slices, as shown in Figure 7-16.
In addition to creating the XVM slices for root and swap, executing this command creates the logical volumes bootdisk_root0 and bootdisk_swap1, as shown in Figure 7-17.
The show -top command shows the topology of the logical volumes, and indicates the size of the root and swap slices:
xvm:local> show -top vol/bootdis* vol/bootdisk_root0 0 online subvol/bootdisk_root0/data 17515280 online slice/bootdisks0 17515280 online vol/bootdisk_swap1 0 online subvol/bootdisk_swap1/data 262144 online slice/bootdisks1 262144 onlineThe following command labels a second disk, dks1d3, as an XVM system disk named moreswap. The -noparts option of the XVM label command is used because we do not want a root partition on this disk and we will be defining the swap volume on this disk manually. This example assumes that this second disk is not already a system disk containing a partition 0, or we would need to use the -clrparts option of the XVM label command.
xvm:local> label -noparts -type root -name moreswap dks1d3 moreswap
Create a swap slice on moreswap:
xvm:local> slice -type swap -start 0 -length 262144 moreswap </dev/lxvm/moreswaps0> slice/moreswaps0
Executing this command creates the swap slice on the physvol moreswap, as shown in Figure 7-18.
Create an empty two-piece concat, to which you will attach the two swap slices:
xvm:local> concat -tempname -pieces 2 </dev/lxvm/vol15> concat/concat3
Attach the two swap slices to the concat:
xvm:local> attach slice/bootdisks1 slice/moreswaps0 concat3 </dev/lxvm/vol15> concat/concat3
Attach the concat to the swap subvolume:
xvm:local> attach concat3 subvol/bootdisk_swap1/data </dev/lxvm/bootdisk_swap1> subvol/bootdisk_swap1/data
This creates the logical swap volume that is shown in Figure 7-15.
You can see the layout of the root and swap logical volumes you created with the show -top command:
xvm:local> show -t vol/bootdis* vol/bootdisk_root0 0 online subvol/bootdisk_root0/data 17515280 online slice/bootdisks0 17515280 online vol/bootdisk_swap1 0 online subvol/bootdisk_swap1/data 524288 online concat/concat3 524288 online,tempname slice/bootdisks1 262144 online slice/moreswaps0 262144 online
When running from an XVM root partition, changing the system name in /etc/sysid creates a conflict between the host name in the XVM volume header and the name of the system. This will result in a system panic at the next system reboot unless you first boot the miniroot and run the XVM give command to assign the system disk physical volume to the new system name,
To give away a physical volume from the miniroot, use the following procedure.
Reset the machine to bring up the System Maintenance menu.
Select Install System Software.
Choose remote or local install and enter the path and IP address of the system software, as appropriate for your system.
Start the system. This loads the Inst utility into the swap partition.
At the Inst prompt. enter sh to exit to a shell.
From the shell prompt in the miniroot, you can run the XVM give command:
Enter xvm
Use the give command to give the old physical volume name to the new sysid name:
xvm> give newhostname physvol
Exit from XVM.
Exit from the shell to return to the Inst menu.
Quit the Inst utility.
You should now be able to reboot the system.
The following procedure reconfigures a filesystem while the filesystem is online by mirroring the data in a new configuration, then detaching the original configuration. It is not necessary to unmount the filesystem to perform this procedure.
| Note: To use the mirroring feature of the XVM Volume Manager, you must purchase and install the appropriate FLEXlm license on IRIX or LK license on SGI ProPack 6 for Linux. |
Figure 7-19 shows the configuration of the original filesystem that has been built and mounted.
In the example, the original filesystem is a filesystem that consists of a single slice. It is named myfs, and is configured as follows:
xvm:cluster> show -top myfs
vol/myfs 0 online
subvol/myfs/data 102400 online,open
slice/disk5s0 102400 online,open
|
This procedure reconfigures this filesystem into one that consists of a four-way stripe.
Create the slices that will make up the four-way stripe. The stripe that you are creating should be the same size as the existing filesystem, so in this example each slice is one-quarter the size of the filesystem.
xvm:cluster> slice -length 25600 phys/disk[1234] </dev/cxvm/disk1s0> slice/disk1s0 </dev/cxvm/disk2s0> slice/disk2s0 </dev/cxvm/disk3s0> slice/disk3s0 </dev/cxvm/disk4s0> slice/disk4s0
Create the four-way stripe. This example does not specify a stripe unit, which accepts the default stripe unit of 128 blocks. In this case, using the default stripe unit uses all the blocks of each slice, since the slices are multiples of the stripe unit in size.
xvm:cluster> stripe -tempname slice/disk[1234]s0 </dev/cxvm/vol5> stripe/stripe5
Display the stripe configuration:
xvm:cluster> show -top stripe5 stripe/stripe5 102400 online,tempname slice/disk1s0 25600 online slice/disk2s0 25600 online slice/disk3s0 25600 online slice/disk4s0 25600 onlineInsert a temporary mirror above the point that will be reconfigured. In this example, that point is slice/disk5s0.
xvm:cluster> insert mirror slice/disk5s0 </dev/cxvm/myfs> mirror/mirror5
Display the logical volume:
xvm:cluster> show -top myfs vol/myfs 0 online subvol/myfs/data 102400 online,open mirror/mirror5 102400 online,tempname,open slice/disk5s0 102400 online,openFigure 7-20 shows the configuration of the filesystem myfs after the insertion of the mirror.
Attach the stripe to the mirror, which is mirror5 in this example. This will initiate a revive, which replicates the data of slice/disk5s0 on stripe5.
xvm:cluster> attach stripe/stripe5 mirror/mirror5 </dev/cxvm/myfs> mirror/mirror5
Display the logical volume:
xvm:cluster> show -top myfs vol/myfs 0 online subvol/myfs/data 102400 online,open mirror/mirror5 102400 online,tempname,reviving:26% slice/disk5s0 102400 online,open stripe/stripe5 102400 online,tempname,reviving slice/disk1s0 25600 online,open slice/disk2s0 25600 online,open slice/disk3s0 25600 online,open slice/disk4s0 25600 online,openFigure 7-21 shows the configuration of the filesystem myfs after the stripe has been attached to the mirror.
Detach slice/disk5s0 from the mirror. You must wait for the mirror revive to complete before you can do this, since you cannot detach the last valid piece of an open mirror (unless you use the -force option), and until the revive completes the slice is the only valid leg of the mirror.
xvm:cluster> detach slice/disk5s0 </dev/cxvm/disk5s0> slice/disk5s0
Figure 7-22 shows the configuration of the filesystem myfs after the original slice has been detached.
Remove the mirror layer from the tree by collapsing around the mirror:
xvm:cluster> collapse mirror/mirror5
The filesystem is now configured as a four-way stripe. Display the logical volume:
xvm:cluster> show -top myfs vol/myfs 0 online subvol/myfs/data 102400 online,open stripe/stripe5 102400 online,tempname,open slice/disk1s0 25600 online,open slice/disk2s0 25600 online,open slice/disk3s0 25600 online,open slice/disk4s0 25600 online,openFigure 7-23 shows the final configuration of the filesystem myfs.
The following sections describe the procedure for creating and modifying a logical volume. In this procedure, the modifications to the logical volume are made after you have made a filesystem on the logical volume and mounted the filesystem.
This procedure is divided into the following steps:
Creating the logical volume
Growing the logical volume
Mirroring data on the logical volume
Converting a concat to striped data using mirroring
Removing a mirror
Mirroring individual stripe members
These steps are described in the following sections.
| Note: To use the mirroring feature of the XVM Volume Manager, you must purchase and install the appropriate FLEXlm license on IRIX or LK license on SGI ProPack 6 for Linux. |
The following procedure creates a simple logical volume that contains a single slice. Figure 7-24 shows the original XVM logical volume this procedure creates.
Create a slice on the physvol pebble, naming the generated volume that contains the slice tinyvol:
xvm:cluster> slice -volname tinyvol -start 17601210 -length 177792 \ pebble </dev/cxvm/tinyvol> slice/pebbles0
Exit the XVM CLI by typing exit and then create a filesystem:
xvm:cluster> exit # mkfs /dev/cxvm/tinyvol meta-data=/dev/cxvm/tinyvol isize=256 agcount=5, agsize=4445 blks data = bsize=4096 blocks=22224, imaxpct=25 = sunit=0 swidth=0 blks, unwritten=1 naming =version 1 bsize=4096 log =internal log bsize=4096 blocks=1168 realtime =none extsz=65536 blocks=0, rtextents=0Mount the filesystem. For a shared filesystem in a CXFS cluster, you mount a filesystem with the CXFS GUI or the cmgr(1M) command, as described in CXFS Version 2 Software Installation and Administration Guide.
For a local filesystem, you can put a logical volume in the fstab file and use the mount command to mount the filesystem you created.
The following procedure grows the logical volume you have created. Figure 7-25 shows the logical volume after the insertion of a concat to grow the logical volume.
Display the logical volume tinyvol, showing the topology of the volume:
xvm:cluster> show -top tinyvol vol/tinyvol 0 online subvol/tinyvol/data 177792 online,open slice/pebbles0 177792 online,openChange the volume tinyvol to include a concat container:
xvm:cluster> insert concat slice/pebbles0 </dev/cxvm/tinyvol> concat/concat3
Display the results of the insert command:
xvm:cluster> show -top tinyvol vol/tinyvol 0 online subvol/tinyvol/data 177792 online,open concat/concat3 177792 online,tempname,open slice/pebbles0 177792 online,openFind or make a free slice on the physvol bambam:
xvm:cluster> slice -start 0 -length 177792 bambam </dev/xvm/bambams0> slice/bambams0
Attach the slice to tinyvol. There are two different ways to specify the concat volume element to which you are attaching the slice.
The following command attaches the slice by the relative location of the volume element:
xvm:cluster> attach slice/bambams0 tinyvol/data/0 </dev/cxvm/tinyvol> concat/concat3
The following command attaches the slice by referring to the object name of the volume element:
xvm:cluster> attach slice/bambams0 concat3
For information on referring to object names and relative locations in XVM commands, see “Object Names in XVM” in Chapter 3.
Display the results of the attach command:
xvm:cluster> show -top tinyvol vol/tinyvol 0 online subvol/tinyvol/data 355584 online,open concat/concat3 355584 online,tempname,open slice/pebbles0 177792 online,open slice/bambams0 177792 online,openExit the XVM CLI by typing exit and then grow the filesystem. Use the mount point where you mounted the filesystem with the CXFS GUI. In this example, the mount point is /clusterdisk:
xvm:cluster> exit # xfs_growfs /clusterdisk meta-data=/clusterdisk isize=256 agcount=5, agsize=4445 blks data = bsize=4096 blocks=22224, imaxpct=25 = sunit=0 swidth=0 blks, unwritten=1 naming =version 1 bsize=4096 log =internal bsize=4096 blocks=1168 realtime =none extsz=65536 blocks=0, rtextents=0 data blocks changed from 22224 to 44448
The following procedure creates a mirror for the data in the filesystem. Figure 7-26 shows the XVM logical volume after the insertion of the mirror.
Change tinyvol to include a mirror container:
# xvm xvm:cluster> insert mirror tinyvol/data/0 </dev/cxvm/tinyvol> mirror/mirror3
Display the results of the mirror insert:
xvm:cluster> show -top tinyvol vol/tinyvol 0 online subvol/tinyvol/data 355584 online,open mirror/mirror3 355584 online,tempname,open concat/concat3 355584 online,tempname,open slice/pebbles0 177792 online,open slice/bambams0 177792 online,openFind free space or make a new slice of the same size:
xvm:cluster> slice -start 0 -length 355584 wilma </dev/cxvm/wilmas0> slice/wilmas0
Attach the slice to the mirror:
xvm:cluster> attach slice/wilmas0 tinyvol/data/0 </dev/cxvm/tinyvol> mirror/mirror3
Display the results of the attach. In this example, the revive that was initiated when the slices were attached to the mirror has not yet completed:
xvm:cluster> show -top tinyvol vol/tinyvol 0 online subvol/tinyvol/data 355584 online,open mirror/mirror3 355584 online,tempname,open concat/concat3 355584 online,tempname,open slice/pebbles0 177792 online,open slice/bambams0 177792 online,open slice/wilmas0 355584 online,reviving:11%
The following procedure converts the previously created concat to a stripe that replaces the concat in the mirror. Figure 7-27 shows the resulting XVM logical volume.
Break the mirror:
xvm:cluster> detach -tempname mirror3/0 </dev/cxvm/vol6> concat/concat3
Delete the concat object, detaching and keeping the slices that make it up:
xvm:cluster> delete -nonslice concat3 </dev/cxvm/pebbles0> slice/pebbles0 </dev/cxvm/bambams0> slice/bambams0
Create a stripe using the slices:
xvm:cluster> stripe -tempname -unit 128 slice/pebbles0 slice/bambams0 </dev/cxvm/vol7> stripe/stripe0
Attach the stripe to the mirror:
xvm:cluster> attach stripe0 mirror3
Display the results of the attach. In this example, the revive that was initiated when the stripes were attached to the mirror has not yet completed.
xvm:cluster> show -top tinyvol vol/tinyvol 0 online subvol/tinyvol/data 355584 online,open mirror/mirror3 355584 online,tempname,open stripe/stripe0 355584 online,tempname,reviving:5% slice/pebbles0 177792 online,open slice/bambams0 177792 online,open slice/wilmas0 355584 online,open
The following procedure removes the mirror layer from the logical volume. Figure 7-28 shows the XVM logical volume after the mirror has been removed.
Detach the slice on which the data is mirrored:
xvm:cluster> detach -tempname slice/wilmas0 </dev/cxvm/wilmas0> slice/wilmas0
Remove the mirror layer:
xvm:cluster> collapse mirror3
Display the results of the collapse command:
xvm:cluster> show -top tinyvol vol/tinyvol 0 online subvol/tinyvol/data 355584 online,open stripe/stripe0 355584 online,tempname,open slice/pebbles0 177792 online,open slice/bambams0 177792 online,open
The following procedure mirrors the individual slices that make up the stripe. Figure 7-29 shows the XVM logical volume this example yields.
Place the slices within mirror containers. The following examples demonstrate alternate methods of specifying slices:
xvm:cluster> insert mirror tinyvol/data/0/0 </dev/cxvm/tinyvol> mirror/mirror4 xvm:cluster> insert mirror slice/bambams0 </dev/cxvm/tinyvol> mirror/mirror5
Display the results of the two insert commands:
xvm:cluster> show -top tinyvol vol/tinyvol 0 online subvol/tinyvol/data 355584 online,open stripe/stripe0 355584 online,tempname,open mirror/mirror4 177792 online,tempname,open slice/pebbles0 177792 online,open mirror/mirror5 177792 online,tempname,open slice/bambams0 177792 online,openFind some free space or reuse some unused slices:
xvm:cluster> slice -start 0 -length 177792 betty </dev/cxvm/bettys0> slice/bettys0 xvm:cluster> show slice/wilmas0 slice/wilmas0 355584 online,autoname
Attach the slices to the mirrors. Note that wilmas0 is larger than pebbles0. The mirror will continue to use the smallest size.
xvm:cluster> attach slice/wilmas0 tinyvol/data/0/0 </dev/cxvm/tinyvol> mirror/mirror4 xvm:cluster> attach slice/bettys0 stripe0/1 </dev/cxvm/tinyvol> mirror/mirror4
Display the results of the attach:
xvm:cluster> show -top tinyvol vol/tinyvol 0 online subvol/tinyvol/data 355584 online,open stripe/stripe1 355584 online,tempname,open mirror/mirror4 177792 online,tempname,open slice/pebbles0 177792 online,open slice/wilmas0 355584 online,open mirror/mirror5 177792 online,tempname,open slice/bambams0 177792 online,open slice/bettys0 177792 online,open
SGI storage solutions (TP-9XXX and IS-XXX series) based on LSI RAID are designed to use Automatic Volume Transfer (AVT) failover. This means that as soon as you use one of the alternate controller's path, a failover of the LUN is initiated.
To prevent the LUN from doing controller ping-pong during boot time (when the kernel discovers the disk labels), LSI has created an exclusion zone of 8192 blocks at the beginning and the end of each LUN assigned with a Host Type of SGIAVT.
It is very important when planning your XVM volumes and GPT labels to use this exclusion zone to minimize path failover during boot time. Figure 7-30 shows the optimal way of creating XVM volumes and a GPT label on a LUN using LSI raid:
The xvm label command automatically lays out the LUN to meet these requirements.
To stop this automatic layout feature, you can use an option for the xvm label command: -use-exclusion-zones. Executing xvm label -use-exclusion-zones causes xvm to allocate the user data area without any consideration of the exclusion zones.
| Note: GPT label itself reserves blocks 0-34 for it own label and also 34 blocks at the end of the disk for a backup of the label. More information on the GPT label is available at: http://en.wikipedia.org/wiki/GUID_Partition_Table |
To make the GPT label, use the Linux parted(8) command. First, you need to create the label and then create one or two partitions. If there are two partitions, one partition will contain the XVM label and the other the XVM user data. If there is only one partition, it will contain first the XVM label data followed by the user data.
| Note: The first partition must start on or before block 63. XVM scans the first 64 blocks of the disk to see if there is an XVM label. If no label is found, XVM ignores the disk. |
Follow this guideline to make a single partition:
Create a partition starting at sector 34 and ending at (DISK_SIZE - 8192)
Or follow these guidelines to make two partitions:
Create a partition 1 starting at sector 34 and ending at sector 8191
Create a partition 2 starting at sector 8192 and ending at (DISK_SIZE - 8192)
An example using the parted command is, as follows:
[email protected] # parted /dev/xscsi/pci07.00.0/node201000a0b80f200a/port1/lun4/disc GNU Parted 1.6.25.1 Copyright (C) 1998 - 2005 Free Software Foundation, Inc. This program is free software, covered by the GNU General Public License. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. Using /dev/sdk (parted) mklabel gpt (parted) unit s (parted) print Disk geometry for /dev/sdk: 0s - 70083583s Disk label type: gpt Number Start End Size File system Name Flags (parted) |
The example parted command, above, creates have a LUN with a GPT label and you know that the LUN is 70083583s long.
Create a valid GPT label with the correct partitions:
(parted) unit s (parted) mkpart primary 34 4095 (parted) mkpart primary 4096 70079487 (parted) print Disk geometry for /dev/sdk: 0s - 70083583s Disk label type: gpt Number Start End Size File system Name Flags 1 34s 4095s 4062s 2 4096s 70079487s 70075392s xfs (parted) |
To make the XVM label and slices, perform the following steps:
Create the XVM label, as follows:
system-a:~/root # xvm xvm:local> show unlabeled unlabeled/dev/xscsi/pci-0002:00:01.0-sas-phy4-0x500a0b82fbd13004-lun0 * * unlabeled/dev/xscsi/pci-0002:00:01.0-sas-phy4-0x500a0b82fbd13004-lun1 * * unlabeled/dev/xscsi/pci0001:00:03.0-1/target1/lun0/disc * * unlabeled/dev/xscsi/pci0001:00:03.0-1/target2/lun0/disc * * unlabeled/dev/xscsi/pci0002:00:02.0/node20000011c61ddd5d/port1/lun0/disc * * unlabeled/dev/xscsi/pci0002:00:02.0/node20000011c61ddd5d/port2/lun0/disc * * unlabeled/dev/xscsi/pci0002:00:02.1/node20000011c61ddd5d/port1/lun0/disc * * unlabeled/dev/xscsi/pci0002:00:02.1/node20000011c61ddd5d/port2/lun0/disc * * xvm:local> label -name fs2_lun1 unlabeled/dev/xscsi/pci0002:00:02.1/node20000011c61ddd 5d/port2/lun0/disc fs2_lun1 xvm:local> show phys/fs2_lun1 phys/fs2_lun1 71670988 online,local,accessible
The next step is to make slices on the lun. Any number of slices may be made, but the usual case is just one slice.
Due to some performance issues in XVM, it is strongly recommended that the XVM slice be completely outside the exclusion zone. The label command by default places the user data area entirely outside of the exclusion zones, so you do not need to consider the exclusion zones in allocating slices. (The -use-exclusion-zones flag can be used on the label command if it is necessary to put user data in the exclusion zones.)
In some cases, especially where striped volumes are used, it is important for performance to allocate a slice that is aligned on certain boundaries of the underlying lun. The -align option of the slice command adjusts the slice or slices that are created to line up in this manner. For following example, it is assumed that the boundary of interest is every 768KB, which is 1536 sectors.
xvm:local> slice -align 1536 phys/fs2_lun1 </dev/lxvm/fs2_lun1s0> slice/fs2_lun1s0 xvm:local> show -v phys/fs2_lun1 XVM physvol phys/fs2_lun1 size: 71670988 blocks sectorsize: 512 bytes state: online,local,accessible uuid: b4677698-77e0-4434-9155-55e92b2ffa22 system physvol: no physical drive: /dev/xscsi/pci0002:00:02.1/node20000011c61ddd5d/port2/lun0/disc on host alta preferred path: unspecified available paths: /dev/xscsi/pci0002:00:02.1/node20000011c61ddd5d/port2/lun0/disc <dev 17280> affinity=0 <current path> alternate paths: none Disk has the following XVM label: Clusterid: 0 Host Name: alta Disk Name: fs2_lun1 Magic: 0x786c6162 (balx) Version 2 Uuid: b4677698-77e0-4434-9155-55e92b2ffa22 last update: Thu May 28 10:54:50 2009 state: 0x91<online,local,accessible> flags: 0x0<idle> secbytes: 512 label area: 8157 blocks starting at disk block 35 (10 used) user area: 71670988 blocks starting at disk block 8192 Physvol Usage: Start Length Name --------------------------------------------------- 0 1024 (unused) 1024 71668224 slice/fs2_lun1s0 71669248 1740 (unused) Local stats for phys/fs2_lun1 since being enabled or reset: ------------------------------------------------------------------------------ stats collection is not enabled for this physvol
The start of the user area is at the end of the exclusion zone at sector 8192, and 8192 + 1024 = 9216, which is exactly 6 * 1536.
A XVM labeled DVH labeled disk has the following format:
Table 7-1. XVM DVH disk layout
Block | Size | Contents |
|---|---|---|
0 | 1 | Label - partition info, etc |
1 | 3071 | Usually unused |
3072 | 1024 | XVM Metadata |
4096 | data_length | User Data |
| Unused_length | May be unused space at end due to 2MB rounding |
A GPT label for XVM has the following format. Partition 1 could start at block 34, but we start it at block 40 for a nice round number. The first GPT partition for the XVM metadata must start within the first 64 blocks of the start of the disk.
Table 7-2. XVM GPT disk layout
Block | Size | Contents |
|---|---|---|
0 | 1 | Master Boot Record |
1 | 1 | GUID Partition Table Header |
2 | 32 | GUID Partition Entries |
40 | 4056 | Partion 1 - XVM Metadata (~2MB) |
4096 | data_length | Partition 2 - User Data |
End - 33 | 32 | Backup GUID Partition Entries |
End - 1 | 1 | Backup GUID Partition Table Header |
The problem with converting at DVH label to a GPT labeled disk is that a DVH labeled disk can have data going all the way to the end of the disk, where a GPT labeled disk must have up to 33 blocks at the end of the disk: one block for the backup GUID partition table header and up to 32 blocks for the backup GUID partition entries. So you have to have almost 33 blocks (17KB if 512 bytes/block) free at the end of a disk to have room for the backup GPT.
In some cases, this will be available and the partitioning tools can place the backup GPT in the used space at the end of the disk and be done. This is not normally the case, but can happen especially if the disk is part of a stripe. Usually stripes will not align exactly with the end of a disk and so there may be enough unused space at the end to place the backup GPT header and backup partition entries.
To convert to GPT, from your DVM labeled disk you will need to remember:
start_of_userdata = the block number where the user data started
and
end_block_number = the ending block number of the user data.
Or
data_length = the number of blocks of user data.
For the case where there is room at the end of the disk your new GPT disk will be formatted as follows:
Table 7-3. XVM GPT disk layout example 1
Block | Size | Contents |
|---|---|---|
0 | 1 | Master Boot Record |
1 | 1 | GUID Partition Table Header |
2 | 32 | GUID Partition Entries |
40 | 4056 | Partion 1 - XVM Metadata (~2MB) |
4096 | data_length | Partition 2 - User Data |
End - 33 | 32 | Backup GUID Partition Entries |
End - 1 | 1 | Backup GUID Partition Table Header |
If you don't have 17KB free at the end of the disk, you can still convert, but it will be more complicated as you will first have to free up 17KB from the end of the disk. Since the DVH user data partition is using this space, we will pick up the last 512KB (or whatever size >17KB that you want) and move it into the first 4096 blocks of the DVH label that is not used, say starting at block 3072 to keep everything on nice boundaries,
So if your DVH labeled disk looks like the following:
Table 7-4. XVM DVH disk layout example 1
Block | Size | Contents |
|---|---|---|
0 | 1 | Label - partition info, etc |
1 | 3071 | Usually unused |
3072 | 1024 | XVM Metadata |
4096 | data_length | User Data |
You can convert to a GPT labeled disk that will look like the following:
Table 7-5. XVM GPT disk layout example 2
Block | Size | Contents |
|---|---|---|
0 | 1 | Master Boot Record |
1 | 1 | GUID Partition Table Header |
2 | 32 | GUID Partition Entries |
40 | 3032 | Partion 1 - XVM Metadata (~1.5 MB) |
3072 | data_length | Partition 2 - User data starts here |
3072 | 1024 | 512KB of data copied from end of disk (make this slice0) |
4096 | data_length - 1024 | Original data less 256KB at end that was moved (make this slice1) |
End - 33 | 33 | Backup GUID Partition Entries and header |
Now make a concat of slice1 (the original user data less 256KB at the end of the disk) followed by slice0 (the last 512KB of data now in the old DVH labels unused space) and the volume will be identical to the volume that was on the DVH labeled disk. These figures show a disk that has only a single slice. This will normally be the case, but in some cases the dvh labeled disk may have more than 1 slice. If this is the case, the XVM show command will still how you the the block information needed to rebuild the slices with the GPT partition commands. The offsets into the GPT slice will be identical to the offsets in the DVH slice if no data is needed to be copied to make room for the backup GPT label. If data needs to be copied and 1024 block is the amount that is copied, the Offsets will be +1024 from the DVH numbers, with blocks 0-1024 being the 1024 blocks copied from the end of the slice.
For an example, take a DVH labeled XVM disk with the phyvol name Volume_B4. The physvol data can be examined with the show -v command:
xvm:local> show -v phys/volume_B4
XVM physvol phys/volume_B4
=========================
size: 489172992 blocks sectorsize: 512 bytes state: online,local
uuid: 5f41ca2e-c319-1028-8cbe-080069055325
system physvol: no
physical drive: /dev/xscsi/pci01.02.1/node200600a0b8160a4a/port1/lun4/disc on host mvcxfs23
preferred path: unspecified
available paths:
/dev/xscsi/pci01.02.1/node200600a0b8160a4a/port1/lun4/disc <dev 16688> affinity=0
<current path>
alternate paths: none
Disk has the following XVM label:
Clusterid: 0
Host Name: mvcxfs23
Disk Name: volume_B4
Magic: 0x786c6162 (balx) Version 2
Uuid: 5f41ca2e-c319-1028-8cbe-080069055325
last update: Wed Apr 12 15:06:58 2006
state: 0x11<online,local> flags: 0x0<idle>
secbytes: 512
label area: 1024 blocks starting at disk block 3072 (10 used)
user area: 489172992 blocks starting at disk block 4096
Physvol Usage:
Start Length Name
---------------------------------------------------
0 489172960 slice/volume_B4s0
489172960 32 (unused)
Local stats for phys/volume_B4 since being enabled or reset:
------------------------------------------------------------------------------
stats collection is not enabled for this physvol
|
The output shows us that the user area (shown in bold in the example) starts at block 4096. From the Physvol Usage section we can see that the slice is 489172960 blocks long. So we know that the address of the first block after the data area will be 4096+ 489172960 = 489177056. We can also see that there are 32 unused blocks after slice/volume_B4s0. This would be 16K, which is not enough for the backup GPT. Thus we will need to copy some data from the end of the user area to free up some space. If we copy the last 512KB (1024 blocks), we would copy starting from the block at 489176032 (489177056 - 1024). We could place this in the space that used to belong to the DVH label starting at block 3072 (start of user area - 1024, or in this case 4096 - 1024 = 3072).
Make sure the volume is unmounted and use the dd command to copy the data.
xmvcxfs23:~ # dd if=/dev/xscsi/pci01.02.1/node200600a0b8160a4a/port1/lun4/disc of=/dev/xscsi/pci01.02.1/node200600a0b8160a4a/port1/lun4/disc skip=489176032 seek=3072 count=1024 1024+0 records in 1024+0 records out |
Skip is the number of blocks to skip to get to the start of the 1024 blocks that we will copy.
Calculate the skip number from the values from the show -v that is done on the DVH labeled disk:
Skip = “user data length” - “amount to be copied” + “start of user area”
= 489172960 1024 + 4096
= 489176032
The seek value is the starting block where the data will be copied to. Again using the values from the show -v command:
Count = “start of user area” - “amount to be copied”
= 4096 - 1024
= 3072
Now that the data is in the right place we can use parted to partition and label the disk with GPT. We want our first partition, the one for the XVM metadata to start right after the GPT at the beginning of the disk and go up to where the data will start. Since the starting block for the second partition will be the same as the Count value from the dd command the end of the first partition will be count-1. With the mkpartsect command used in parted, you specify the block number of the start of the partition and the block number of the end of the partition. So the first partition will end at block 3071. The second partition (for the data) will start at block 3072 (where the data was copied to) and run for the “user data length” from the show -v command above. So it starts at 3072 and ends at 3072 + “user data length” - 1 which is 3072 + 489172960 - 1 or 489176031.
mvcxfs23:~ # parted /dev/xscsi/pci01.02.1/node200600a0b8160a4a/port1/lun4/disc GNU Parted 1.6.21 Copyright (C) 1998 - 2004 Free Software Foundation, Inc. This program is free software, covered by the GNU General Public License. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. Using /dev/sdt (parted) mklabel gpt (parted) mkpartsect primary 40 3071 (parted) mkpartsect primary 3072 489176032 (parted) print Disk geometry for /dev/sdt: 0.000-238856.000 megabytes Disk label type: gpt Minor Start End Filesystem Name Flags 1 0.020 1.500 2 1.500 238855.484 (parted)quit |
Now go into XVM and make the slices. The first slice will be the copied data, or the first 1024 blocks in the user data. The second slice is the rest of the partition or the “user data length - 1024”. You then concat the copied data to the end of the original data (minus the copied data), to get a volume back that will exactly match the volume that you started out with.
mvcxfs23:~ # xvm
xvm:cluster> set domain local
xvm:local> label -name volume_B4 -use-exclusion-zones /unlabeleddev/xscsi/pci01.02.1/node200600a0b81
60a4a/port1/lun4/disc
volume_B4
xvm:local>
xvm:local> slice -start 0 -length 1024 phys/volume_B4
</dev/lxvm/volume_B4s0> slice/volume_B4s0
xvm:local> slice -start 1024 -length 489171936 phys/volume_B4
</dev/lxvm/volume_B4s1> slice/volume_B4s1
xvm:local>
xvm:local> concat -volname testvol slice/volume_B4s1 slice/volume_B4s0
</dev/lxvm/testvol> concat/concat0
xvm:local> show -t vol/testvol
vol/testvol 0 online
subvol/testvol/data 489172960 online
concat/concat0 489172960 online,tempname
slice/volume_B4s1 489171936 online
slice/volume_B4s0 1024 online
|
Looking at the “concat/concat0” line, we can see that our new volume “testvol” is the same size as it was with the DVH label.
This section describes some XVM tuning variables in /proc/sys/dev/xvm that can be modified using the sysctl(8) command.
The xvm_mr_daemon_max and xvm_mr_daemon_min variables define the number of mirror daemons started for processing mirror retries and mirror completions that cannot be done from completion threads. Adjusting the values can keep more threads alive in the pool rather than spawing and exiting threads which can cause performance issues.
name default minimum maximum xvm_mr_daemon_min 1 0 255 xvm_mr_daemon_max 25 0 255 |
Use the sysctl -a | grep xvm command to view additional variables, as follows:
sysctl -a | grep xvm dev.xvm.xvm_verbose_debug = 0 dev.xvm.xvm_mr_daemon_max = 25 dev.xvm.xvm_maxfs_revive = 1 dev.xvm.xvm_revive_win_factor = 2 dev.xvm.xvm_max_revive_rsc = 5 dev.xvm.xvm_max_revive_threads = 4 dev.xvm.xvm_revive_pri = 90 dev.xvm.xvm_io_pri = 90 dev.xvm.xvm_drl_pri = 90 dev.xvm.xvm_multicast_pri = 90 dev.xvm.xvm_distribution_size = 2097152 |