CT-BLAST is a cluster-aware implementation of the NCBI BLAST program. CT-BLAST does not modify any NCBI BLAST code. Rather, it restructures the flow of input and output from multiple copies of the NCBI BLAST code running on cluster computing nodes. CT-BLAST allows the user to minimize the time to return results by load balancing the queries across the cluster computing nodes. Since CT-BLAST does not require a modification of the source code to NCBI BLAST, new versions of NCBI BLAST are trivial to incorporate.
After CT-BLAST has been installed and configured on the cluster head node, you can invoke it by using the following command:
# ctblastall [options] |
The usage syntax of the ctblastall command is a superset of the syntax of the NCBI blastall command . You can use your shell to alias the blastall command to the ctblastall command.
To use CT-BLAST successfully, use the following order of operations:
Move your database to the file server (/data on the head node).
Format your database by using the formatdb command.
Distribute the database to the compute nodes.
Move your input file to the head node and create a directory for the run.
Run the ctblastall command.
Gather the data from the directory.
If you do not need to move or format your data frequently, you may simply script steps 1 through 3 so that you need to run steps 4 through 6 only.
After moving the databases, you can use test data provided by SGI to run a comparison, as follows:
# ctblastall -i /usr/share/data/ctblast/small.fsa -o small.out -p blastx -d nr --verbose |
Another method for populating the configuration file is described in “Configuring the Code”.
To obtain a list of options for the ctblastall command, simply enter the command and specify no options, as follows:
# /usr/local/bin/ctblastall |
The resulting options list is identical to the options list of the blastall command on a single machine. That list should appear approximately as follows:
CT-BLAST: copyright 2000,2001 by SGI
blastall arguments:
-p Program Name [String]
-d Database [String]
default = nr
-i Query File [File In]
default = stdin
-e Expectation value (E) [Real]
default = 10.0
-m alignment view options:
0 = pairwise,
1 = query-anchored showing identities,
2 = query-anchored no identities,
3 = flat query-anchored, show identities,
4 = flat query-anchored, no identities,
5 = query-anchored no identities and blunt ends,
6 = flat query-anchored, no identities and blunt ends,
7 = XML Blast output [Integer]
default = 0
-o BLAST report Output File [File Out] Optional
default = stdout
-F Filter query sequence (DUST with blastn, SEG with others) [String]
default = T
-G Cost to open a gap (zero invokes default behavior) [Integer]
default = 0
-E Cost to extend a gap (zero invokes default behavior) [Integer]
default = 0
-X X dropoff value for gapped alignment (in bits) (zero invokes default
behavior) [Integer]
default = 0
-I Show GI?s in deflines [T/F]
default = F
-q Penalty for a nucleotide mismatch (blastn only) [Integer]
default = -3
-r Reward for a nucleotide match (blastn only) [Integer]
default = 1
-v Number of database sequences to show one-line descriptions for (V)
[Integer]
default = 500
-b Number of database sequence to show alignments for (B) [Integer]
default = 250
-f Threshold for extending hits, default if zero [Integer]
default = 0
-g Perfom gapped alignment (not available with tblastx) [T/F]
default = T
-Q Query Genetic code to use [Integer]
default = 1
-D DB Genetic code (for tblast[nx] only) [Integer]
default = 1
-a Number of processors to use [Integer]
default = 1
-O SeqAlign file [File Out] Optional
-J Believe the query defline [T/F]
default = F
-M Matrix [String]
default = BLOSUM62
-W Word size, default if zero [Integer]
default = 0
-z Effective length of the database (use zero for the real size) [Real]
default = 0
-K Number of best hits from a region to keep (off by default, if used a
value of 100 is recommended) [Integer]
default = 0
-P 0 for multiple hits 1-pass, 1 for single hit 1-pass, 2 for 2-pass
[Integer]
default = 0
-Y Effective length of the search space (use zero for the real size) [Real]
default = 0
-S Query strands to search against database (for blast[nx], and tblastx).
3 is both, 1 is top, 2 is bottom [Integer]
default = 3
-T Produce HTML output [T/F]
default = F
-l Restrict search of database to list of GI?s [String] Optional
-U Use lower case filtering of FASTA sequence [T/F] Optional
default = F
-y Dropoff (X) for blast extensions in bits (0.0 invokes default behavior)
[Real]
default = 0.0
-Z X dropoff value for final gapped alignment (in bits) [Integer]
default = 0
--- The following are CT specific flags
--verbose be very verbose about CT-BLAST settings, config, etc.
--debug turn debugging options on (turns on verbose)
--test do a test run. Create all files, just do not enque
jobs.
--loadbalance=method load balance the run using either the cousein or
mikhailov algorithms {String] Optional.
--chunk=size number of sequences per chunk [Integer] Optional |
CT-BLAST accepts five arguments in addition to the regular blastall argument structure. These arguments are specific to CT-BLAST operations. They have been designed specifically not to collide with or overlap the arguments in the NCBI blastall code. The CT-BLAST arguments are as follows:
| --verbose | |
Turns on verbose reporting of the work in progress. This argument is highly useful for problem detection. All of the output resulting from using this argument is directed to the standard error stream. The verbosity level is quite high, and the results are informative. Using this argument might be a preferred way to interactively run a job. | |
| --debug | |
Turns on verbose reporting and additional tests and analysis. This argument gives you information that you might not need for day-to-day runs, but it is essential for debugging the actions of CT-BLAST at a particular instant. | |
| --test | |
Turns on verbose reporting and does all of the work except for submitting the job for execution. This argument does not submit the computation, but it does collect the results and test all of the features. It is useful for discovering a problem without overloading the cluster. | |
| --loadbalance=method | |
Turns on the load balancer. The load balancer reorders the input to the jobs to get the best performance out of the cluster. The method option specifies a load balancing algorithm, either cousein or mikhailov. These algorithms give nearly identical performance for most work, but give somewhat different performance for pathological cases. You can also specify the load balancing algorithm in the configuration files by using the sortingmethod=method entry. | |
| --chunksize | |
Divides the input file into a size work quantum. There are algorithms within the code to handle this function automatically, so usually, there is no need to use this option. However, you can use it to provide more granularity in controlling how the work gets divided, or you can use it as a cluster stress test (by using small sizes and large inputs). | |
Configuring the code requires creating a world readable /etc/ctblast directory and, at a minimum, an /etc/ctblast/ctblast.config (global configuration) file. If you are missing this file or any needed contents, simply running the ctblastall command will inform you of what is missing and provide guidelines for setting it up. You can use more than one method to populate this file. One method is described at the beginning of this chapter. An easier method is to use the /usr/local/bin/configure.pl program to map a site configuration file into the global configuration file. To build the global configuration file, the root user can run the configure.pl program, as follows:
# /usr/local/bin/configure.pl -g |
This program generates the configuration file in the correct location. Each user can run the configure.pl program to generate the ~/.ctblastrc and .ctblastrc files, respectively, as follows:
# /usr/local/bin/configure.pl -u ; # generate user config file in ~/.ctblastrc
# /usr/local/bin/configure.pl -l ; # generate local config file in ${CWD}/.ctblastrc |
The configuration files are read in the following order: global, user, local. The last configuration file can override previous configuration files. The configure.pl program generates these files from a site file named /etc/ctblast/ctblast.parameters. This file is a simple, comma-delimited, flat file database that describes the parameter, specifies which configuration file to put it in, specifies the value of the parameter, indicates where to place the configuration file, and provides a comment about the parameter. You can (and should) customize this file to your needs.
The following example is a sample of an existing /etc/ctblast/ctblast.parameters file:
# this is a comment blastall,all,/usr/local/bin/blastall,all,"The absolute path to the blastall binary? #this is also a comment group,all,ctblast,submission_node,"The queue to use for job submission? sortingmethod,all,cousein,submission_node,"The load balancing method to use? . . . |
After running the configure.pl program, you will have a configuration file that looks approximately like the following example:
computenodetmp=/cache/run/ group=ctblast hostn=/bin/hostname killjob=/usr/local/bin/qdel submit=/usr/local/bin/qsub echo=/bin/echo dbpath=/cache/db status=qstat sortingmethod=cousein date=/bin/date pwd=/bin/pwd mkdir=/bin/mkdir tmppath=/tmp rcp=/usr/bin/rcp rm=/bin/rm statuspath=/etc/ctblast/ blastall=/usr/local/bin/blastall dbhost=sgibio08:/data scheduler=pbs |
You can also view these entries by using the verbose option or the debug option with the ctblastall command.
Following is a description of each file entry:
| computenodetmp=/cache/run/ | |
Temporary path on each of the cluster compute nodes. | |
| group=ctblast | |
Scheduling resource to use for allocating jobs. This resource may be a label to help the scheduler route the jobs properly or to help decide how resources are allocated based on group. Scheduling decisions are made by the scheduling entity. Therefore, the purpose of this entry is to pass this information on to the scheduler. | |
| hostn=/bin/hostname | |
Path to the hostname command for the compute nodes. | |
| killjob=/usr/local/bin/qdel | |
Path to the program that deletes jobs from a scheduler. | |
| submit=/usr/local/bin/qsub | |
Path to the program that adds jobs to a scheduler. | |
| echo=/bin/echo | |
Path to the echo command on the compute nodes. | |
| dbpath=/cache/db | |
Location on the compute nodes at which the databases will reside. | |
| status=qstat | |
Name of file in /etc/ctblast that contains a single line, detailing how to run a status inquiry against the current scheduler. | |
| sortingmethod=cousein | |
Method selected for load balancing. Valid options are cousein, mikhailov, or none. If you have a variety of sequence lengths, using this parameter will have a large positive impact on performance. Note: As a result of the way this entry works, the output results will not be in the same order as the input sequences. | |
| date=/bin/date | |
Path to the date command on the computational nodes. | |
| pwd=/bin/pwd | |
Path to the pwd command (print working directory) on the computational nodes. | |
| mkdir=/bin/mkdir | |
Path to the mkdir command on the computational nodes. | |
| tmppath=/tmp | |
Path to local temporary storage used for submitting and processing results. | |
| rcp=/bin/rcp | |
Path to the rcp command on the computational nodes. | |
| rm=/bin/rm | |
Path to the rm command on the computational nodes. | |
| statuspath=/etc/ctblast | |
Path to the location on the head node at which the status command resides (see status=qstat above). This means that there is a file named ${statuspath}/$status that contains a simple query command about the scheduler for job status. This file is written by the configure.pl program. | |
| blastall=/usr/local/bin/blastall | |
Path on the computational nodes to the blastall command. | |
| dbhost=sgibio08:/data | |
Name of the file server and the path to the location at which the database files are stored. | |
| scheduler=pbs | |
Name of the scheduling server being used. Currently supported values are pbs and lsf. | |