To add BDSpro to an NFS implementation, you mount that filesystems that BDS is to use and test BDSpro performance on the filesystems. The chapter contains these sections:
 | Note: Be sure to review the information in Chapter 2, “Preparing for BDSpro,”and follow the recommendations that it contains before proceeding with BDSpro setup. |
To make filesystems available to BDSpro, export them from the server with standard NFS exportfs command. No special arguments to exportfs are required, and all standard exportfs arguments are valid (see the exportfs(1M) reference page for details).
| Note: Only XFS type filesystems are supported with BDSpro. |
To mount filesystems on BDSpro clients, use the standard NFS mount command and the -o bds option (see the fstab(4) reference page for complete information). The example that follows illustrates a BDS entry in a client /etc/fstab file:
hip0-goliath:/ /bdsmnt -o bds, vers=3, rw 0 0 |
In this example, the root filesystem from server goliath is mounted to /bdsmnt on the client. The interface hip0-goliath on the server connects to the HIPPI network where the client is also attached.
BDSpro software is ready to run after it is installed. You can start the bds daemon on the server from the command line if it is not already running (see the bds(1M) reference page for a list of options).
| Note: If it is necessary to stop the server, stop all client applications first. |
When the bds daemon starts, it binds to port 2050 (NFS port + 1). Do not stop the daemon during an active connection. Doing so causes the socket to linger while it tries to process all remaining packets. If you attempt to restart the daemon during this period, TCP prevents the restart to allow time for packet processing to finish (approximately two minutes).
To test a BDSpro setup, run the BDSpro server in false disk mode. False disk mode is analogous to sending the data to /dev/null or receiving it from /dev/zero. In false disk mode, the BDSpro server simulates data movement to and from the disk; network data is unaffected. This mode is used to verify network performance.
Use this command to start the BDSpro server in false disk mode:
server# /usr/sbin/bds -devnull -devzero -touch -log |
The touch option tells BDSpro to touch all data before sending it, which simulates XFS overhead. (See the bds(1M) reference page for a description of all options).
On the client, first mount the server:
client# mkdir /mnt client# mount -o bds server:/ /mnt |
| Note: The mount will fail if you have not yet upgraded the client kernel to use BDSpro, but you can still test BDSpro with lmdd, since this command contains a user level implementation of the XBDS™ protocol (see “Debugging Without Kernel-Level BDSpro,” later in this chapter). |
After the filesystem is mounted, try reading a file using an lmdd command similar to this (remember that no data is actually read in false disk mode):
client# lmdd if=/mnt/unix direct=1 bs=1m move=100m 100 MB in 1.54 secs, 65.01 MB/sec |
| Note: If you do not include the move=100m argument, the lmdd command will not stop. |
If BDSpro performance is not what you expected, you can use verbose debugging by adding the -debug option to the bds command line. You can debug in either false or real disk mode. When you use real data, debugging prints timing data for the network transfer and the filesystem transfer. In false data mode, only network timing is displayed.
Use the following command on the client to generate debugging on the server:
client# lmdd of=debugfile bs=4m move=20m direct=1 |
The previous lmdd command writes data across the network to the BDS server using a block size of 4MB. The BDS transfer size is whatever the remote client is using for read or write requests.
The example that follows shows the bds debugging output on the server. Because data is read from the network and written to the disk, readn timing results are network times and write timing results are XFS times.
server# bds -debug (null): F xid=24512 uid=0 gid=0 fhandle len=68 Want file handle 68 bytes V3 filehandle: A xid=24512 uid=0 gid=0 bytes=0 V3 filehandle: W xid=24513 uid=0 gid=0 off=0 len=4.0M readn: 1000c000 4.0M @ 84.5M/sec V3 filehandle: lseek to 0 = 0 write: 4.0M @ 52.0M/sec V3 filehandle: A xid=24513 uid=0 gid=0 bytes=4.0M V3 filehandle: W xid=24514 uid=0 gid=0 off=4.0M len=4.0M readn: 10410000 4.0M @ 0.5G/sec V3 filehandle: lseek to 4.0M = 4.0M write: 4.0M @ 50.2M/sec V3 filehandle: A xid=24514 uid=0 gid=0 bytes=4.0M V3 filehandle: W xid=24515 uid=0 gid=0 off=8.0M len=4.0M readn: 1000c000 4.0M @ 83.2M/sec V3 filehandle: lseek to 8.0M = 8.0M write: 4.0M @ 56.5M/sec V3 filehandle: A xid=24515 uid=0 gid=0 bytes=4.0M V3 filehandle: W xid=24516 uid=0 gid=0 off=12.0M len=4.0M readn: 10410000 4.0M @ 88.6M/sec V3 filehandle: lseek to 12.0M = 12.0M write: 4.0M @ 55.7M/sec V3 filehandle: A xid=24516 uid=0 gid=0 bytes=4.0M V3 filehandle: W xid=24517 uid=0 gid=0 off=16.0M len=4.0M readn: 1000c000 4.0M @ 88.5M/sec V3 filehandle: lseek to 16.0M = 16.0M write: 4.0M @ 52.9M/sec V3 filehandle: A xid=24517 uid=0 gid=0 bytes=4.0M hip0-ebony.engr.sgi.com moved 20.00 MB in 0.75 secs, 26.64 MB/sec |
| Note: Network writes are frequently returned before data transfers are completed. Transfers are mapped into socket buffers and the write is returned, but the network continues to transfer data after these events take place. |
The lmdd debugging tool included with BDSpro has a user level implementation of the BDS protocol. lmdd tries to use the kernel level BDS protocol, but if that is not present (or not enabled), lmdd uses the user level protocol.
If you need to debug on the client system, you can do so by mounting the filesystem without the -o bds option. In this case, the filesystem is mounted with kernel level BDS disabled.
To see local debugging output, use lmdd with the debug=1 option, as shown in this example:
# lmdd of=debugfile bs=4m move=20m direct=1 debug=1 0.003 bds_open(3, debugfile, 32769, 5) restart(debugfile) open(hip0-mahogany:/export/bds1/debugfile) = 3 write at 0.051, rwnd=0 swnd=0 0.263 A xid=2 uid=0 gid=0 bytes=4.0M bds_write(3, 1000c000, 4194304) = 4194304 @ 18.72MB/sec in 1 writes write at 0.268, rwnd=0 swnd=0 0.430 A xid=3 uid=0 gid=0 bytes=4.0M bds_write(3, 1000c000, 4194304) = 4194304 @ 24.62MB/sec in 1 writes write at 0.435, rwnd=0 swnd=0 0.562 A xid=4 uid=0 gid=0 bytes=4.0M bds_write(3, 1000c000, 4194304) = 4194304 @ 31.38MB/sec in 1 writes write at 0.567, rwnd=0 swnd=0 0.693 A xid=5 uid=0 gid=0 bytes=4.0M bds_write(3, 1000c000, 4194304) = 4194304 @ 31.61MB/sec in 1 writes write at 0.697, rwnd=0 swnd=0 0.821 A xid=6 uid=0 gid=0 bytes=4.0M bds_write(3, 1000c000, 4194304) = 4194304 @ 32.23MB/sec in 1 writes 20.00 MB in 0.83 secs, 24.23 MB/sec |
BDSpro is typically used on HIPPI because of its high performance (consult the IRIS HIPPI Administrator's Guide for information on installing and configuring a HIPPI network). If BDSpro testing shows inadequate performance, network problems might be the cause. In this case, you can use the ttcp test program to verify that the network is functioning properly. ttcp is a client/server program that moves data between systems and reports performance results.
To use ttcp, enter this command on the client to start the test:
client% ttcp -s -r -l 524288 ttcp-r : buflen=524288, nbuf=2048,align=16384/0, port=5001 tcp ttcp-r : socket |
Then, enter the following command on the server to start ttcp and send data to the client. The output of ttcp shows transfer rates:
server% ttcp -s -t -T -l 524288 -n 200 hip-client ttcp-t: buflen=524288,nbuf=200, align=16384/0, port=5001 tcp -> hip-client ttcp-t: socket ttcp-t: connect ttcp-t: 104857600 bytes in 1.42 real seconds = 73843.38 KB/sec +++ |
The server should be sending data at approximately 65 to 70 MB per second. If you omit the -T option, which touches the data to measure caching effects, the rate should be approximately 90 MB per second. If the network is not performing at the expected level, determine the cause of the problem and correct it. The problem may be caused by one of these conditions:
The data is not being transferred on the HIPPI network.
By default, IRIX designates the Ethernet interface as the primary network interface and assigns the hostname (the name in the /etc/sys_id file) to this interface. To assign a hostname to the HIPPI interface, IRIX appends prefix hippi to the hostname. (For example, if the Ethernet interface is named frosty, the HIPPI interface is named hippi-frosty).
Check the hostname of the HIPPI client that you specified in the ttcp command to verify that it is the HIPPI hostname (and not the Ethernet hostname) for this client. Remember to specify a HIPPI interface when mounting the filesystem also (see “Mounting Filesystems for BDSpro,” earlier in this chapter).
The server is running a debugging or sema metering kernel.
Reboot with a non-debug kernel, which performs much faster.
The client or server is not running IRIX 6.2.
IRIX 5.3 (or earlier) does not offer the HIPPI performance that BDSpro requires. Upgrade to IRIX 6.2 with BDSpro support.
The connection is passing through a router.
Use netstat -i to determine whether the HIPPI interface on the server connects to the HIPPI network where the client resides. If the client and server are not on the same network, a high-performance router will be required to support HIPPI speeds.
If you try all of these measures and performance is still not adequate, contact your Silicon Graphics support provider for additional assistance.