This chapter explains what BDSpro requires if it is to achieve its full potential and how to modify your current setup if you determine that changes are needed. The chapter contains these sections to help you prepare for running BDSpro:
BDSpro requires that IRIX version 6.2 (or higher) and NFS (version 2.0 or 3.0) are installed on client and server systems. It also requires that the applications that use BDSpro services perform page-aligned I/O and direct reads and writes (see “How BDS Works” in Chapter 1).
BDSpro software must be installed on both client and server systems. See the BDSpro Release Notes for instructions on software installation.
Silicon Graphics currently ships SCSI disks manufactured by Quantum and IBM. Quantum disks excel in random access and relatively small I/O operations, and IBM disks excel in large sequential access operations. Because of the operating characteristics of BDS, disks that excel in large sequential access operations offer better performance and are therefore a better choice.
Most BDS installations are likely to use fast-and-wide SCSI controllers with speeds of 20 MB per second. The IBM drives provided by SGI spin at about 7 MB per second on the outer zone and 5 MB per second on the inner zone. If we assume a transfer rate of 5 MB per second, it is apparent that 4 drives will completely saturate a single 20 MB per second controller.
Silicon Graphics provides SCSI boxes with an eight-drive capacity that are configured for one SCSI channel. Since channel speed is 20 MB per second and drive speed is approximately 5 MB per second, eight drives offer more bandwidth than is needed. Therefore it is possible to configure SCSI boxes with two SCSI channels per box, doubling the bandwidth for each box.
For maximum sequential performance with the minimum number of disks, purchase more controllers and use one controller for every four disks.
BDSpro performance is highly dependent on the local performance of XFS on the server system. In general, BDSpro requires a local XFS access speed that is twice as high as the speed at which BDSpro is to perform. For example, if BDSpro is to operate at a rate of 50 MB per second, the server XFS rate must be at least 100 MB per second. This XFS rate is required to offset the overhead that BDSpro protocols impose on network transactions.
To measure local XFS performance, use lmdd commands similar to those shown below.
This command creates testfile, a 500 MB file with a blocksize of 7 MB:
server# lmdd of=/export/bds/testfile bs=7m move=500m direct=1 497.00 MB in 5.83 secs, 85.29 MB/sec |
This command performs a direct read on testfile with a transfer size 7 MB; the output shows the XFS transfer rate:
server# lmdd if=/export/bds/testfile bs=7m move=500m direct=1 497.00 MB in 4.44 secs, 111.92 MB/sec |
Inadequate XFS performance can frequently be corrected by properly configuring disks and by setting the correct size for direct memory access operations. These topics are explained in the sections that follow.
XFS uses a logical volume manager, xlv, to stripe data across multiple disk drives. On striped disks, large XFS requests are split and sent to each disk in parallel. The high sequential performance of XFS is attributable to this parallelism. (See Chapter 7 of IRIX Admin: Disks and Filesystems for more information.)
The size of data transfers is an important consideration in planning logical volumes. For example, assume that a logical volume contains 10 disks and the track size is 64 KB. In this case, transfers of 640 KB or larger are required to get all drives running simultaneously. If the data transfer size is 320 KB, only five drives are active in an I/O operation. Because only half of the available disks are used, a transfer size of 320 KB is very inefficient, reducing the total performance by half. With proper striping of logical volumes, however, disk performance can be maximized.
The xlv_make utility is used to stripe the disks in a logical volume. By default, xlv_make divides the disk into tracks and uses one track from each disk in rotation to create a stripe. The amount of data that xlv_make allocates on a single drive before going to the next is called the stripe unit. The stripe unit and the number of disks in the logical volume determine the stripe width, or
stripe width = stripe unit × number of disks |
Figure 2-1 illustrates a logical volume containing four disks. Notice from this figure that the stripe unit is set to two tracks instead of one (the default stripe unit size). If we assume a track size of 100 KB (track size is set by disk manufacturers), the stripe width for this logical volume is 800 KB.
When you create a logical volume, you can specify a stripe size using the stripe_unit argument of xlv_make (see the xlv_make(1M) reference page). Specifying the proper size of the stripe unit is the key to optimizing I/O performance. In most cases, the objective in setting the stripe_unit is to achieve a particular bandwidth; but you might also need to adjust the stripe size to accommodate an application that uses a fixed transfer size.
The transfer size should be a multiple of the system's page size. The transfer size should also be a multiple of the stripe width (800 KB in Figure 2-1). If the application needs the bandwidth of all four disks but is reading with a transfer size of 400 KB, you could set the stripe unit to one track instead of two to achieve the required bandwidth with half the transfer size.
It is not always advisable to use the smallest possible stripe unit. While small requests can be effective with read transfers because of the read-ahead assistance that SCSI track buffers offer, small stripe units degrade write performance.
For example, consider what happens when data is written using the default stripe unit size of one track. The write is broken into tracks and each track is sent to a different disk. When the data arrives at the controller, the controller first waits for the disk head to move to the beginning of the track before it writes the data. This wait, commonly referred to as a rotational delay, occurs between each track that is written to the same disk; as a result, using a one-track stripe unit reduces the write performance to half of the read performance.
It is possible to achieve higher write performance by using larger stripe units. Table 2-1 shows the effects of increasing the size of stripe units on XFS write performance.
Table 2-1. Effects of Stripe Unit Size on XFS Write Performance
Stripe Unit | Request Size | Write Performance |
|---|---|---|
1 track = 100 KB[a] | 1 track × 4 disks = 400 KB | 1/2 read performance |
2 tracks = 200 KB | 2 tracks × 4 disk = 800 KB | 2/3 read performance |
3 tracks = 300 KB | 3 tracks × 4 disks = 1.2 MB | 3/4 read performance |
4 tracks = 400 KB | 4 tracks × 4 disks = 1.6 MB | 4/5 read performance |
[a] Default size used by the xlv_make command. | ||
IRIX imposes a limit on the maximum size of DMA operations. This limit affects XFS, since direct I/O is a DMA operation. In IRIX 6.2, the default maximum DMA size is 4 MB. Frequently, this limit must be increased on BDSpro servers to achieve optimum performance.
To change the maximum DMA size, reset the maxdmasz variable using systune (see the systune(1M) reference page).
The values of maxdmasz are expressed in pages, which are 16KB on 64-bit systems. Change these values to the size that you need, then reconfigure and reboot the server.
Table 2-2 shows the performance for BDSpro (version 1.0) using IBM drives with a 2 GB capacity and a HIPPI network. Three disks were configured on each controller; the transfer size was set to the stripe width size. Notice from Table 2-2 that BDS writes are slower than XFS writes. This is due to the synchronous nature of BDS write operations.
Table 2-2. Performance Results With Sample Configurations [a]
Disks | Stripe Unit | Stripe Width | XFS Read | XFS Write | BDS Read | BDS Write |
|---|---|---|---|---|---|---|
6 | 64KB | 384 KB | 29 | 14 | 26 | 10 |
6 | 128KB | 768 KB | 30 | 18 | 27 | 14 |
6 | 256KB | 1536 KB | 29 | 21 | 28 | 16 |
9 | 64KB | 576 KB | 42 | 20 | 38 | 15 |
9 | 128KB | 1152 KB | 43 | 26 | 37 | 19 |
9 | 256KB | 2304 KB | 42 | 31 | 38 | 22 |
12 | 64KB | 768 KB | 54 | 26 | 45 | 19 |
12 | 128KB | 1536 KB | 54 | 33 | 44 | 23 |
12 | 256KB | 3072 KB | 57 | 41 | 48 | 27 |
15 | 64KB | 960 KB | 65 | 32 | 52 | 21 |
15 | 128KB | 1920 KB | 64 | 41 | 53 | 26 |
15 | 256KB | 3840 KB | 68 | 50 | 52 | 30 |
18 | 64KB | 1152 KB | 75 | 37 | 58 | 24 |
18 | 128KB | 2304 KB | 73 | 47 | 57 | 29 |
18 | 256KB | 4608 KB | 80 | 57 | 62 | 33 |
21 | 64KB | 1344 KB | 84 | 43 | 64 | 26 |
21 | 128KB | 2688 KB | 84 | 54 | 52 | 31 |
21 | 256KB | 5376 KB | 90 | 66 | 68 | 31 |
24 | 64KB | 1536 KB | 92 | 48 | 68 | 29 |
24 | 128KB | 3072 KB | 92 | 60 | 63 | 33 |
24 | 256KB | 6144 KB | 100 | 73 | 71 | 30 |
27 | 64KB | 1728 KB | 101 | 53 | 71 | 30 |
27 | 128KB | 3456 KB | 99 | 64 | 72 | 35 |
27 | 256KB | 6912 KB | 108 | 81 | 72 | 34 |
30 | 64KB | 1920 KB | 108 | 58 | 70 | 32 |
30 | 128KB | 3840 KB | 108 | 71 | 72 | 37 |
30 | 256KB | 7680 KB | 118 | 86 | 72 | 40 |
[a] Read and write speed is expressed in MB per second. | ||||||