Setting up MineSet on IRIX systems requires modifying dm_config and the DataMover configuration file /usr/lib/MineSet/.datamove. The DataMover process runs on the server, and is not directly accessible to users. The DataMover provides access to databases and data stored in flat files, and transforms data for the mining and visualization tools. The configuration has three parts:

“Configuring the DataMover Server” describes these files and how they may be modified.

Configuring Systune Parameters

The systune parameters on IRIX machines determine the default limits on the available system resources. Table 2-1 lists the systune parameter values that SGI recommends.

Table 2-1. systune Parameters

Parameter	Definition	Recommended Value
rlimit_pthread_cur	Current limit on the number of threads	1024
rlimit_rss_cur	Current limit on memory usage	Amount of physical memory on your machine
rlimit_vmem_cur	Current limit on virtual memory usage	Size of logical swap space on machine, or about twice the physical memory
rlimit_nofile_cur	Current limit on number of open file	1024 or the limit on the number of threads

---

Note: You must reboot your machine after installing the new parameters.

---

For more information about these parameters, see the systune(1M) reference page.

To configure MineSet on a Linux server follow these steps:

The MineSet server uses a hybrid strategy to determine where on the server to store model, schema, and data files. At installation time, the Installer program prompts for a base directory for all users' server files. Each user is then assigned a subdirectory of this base directory, according to username. For example, if ServerFilesBase is left at its installation default of C:\MineSet Files, user guest will have MineSet server files located in C:\MineSet File\guest.

An individual may override this machine-level default by setting the registry entry HKEY_CURRENT_USER\SOFTWARE\SGI\MineSet\3.0\ServerFilesDirectory. (To set a registry entry, right-click the My Computer icon on your desktop, and choose Preferences.) If this entry exists, its value will set the server files directory for that user only.

The different versions of Windows support various filesystems, the most common of which are NTFS and FAT. In general, it is better to locate the server files directory on an NTFS filesystem, if possible. The MineSet server allows multiple instances of the MineSet client to connect to the same server files directory. This can be convenient, if one user wants to try several experiments in parallel, or if two or more individuals want to share data and model files. On FAT filesystems, the DataMover uses a coarse-grained locking mechanism to maintain consistency, which means that only one client can run a history at one time from within a given server files directory. Other users attempting to run histories will receive a Failed to get a lock error message. On NTFS, the DataMover's locking implementation is more finely grained, so this problem will seldom occur.

Finally, whichever way the server files directory is determined, the MineSet server will then look for a file .datamove in that directory, which can contain additional configuration information, as described in the next section. The format and entries for the /usr/lib/MineSet/.datamove file are the same on Windows and UNIX. However, the most common use of the .datamove file is for setting the location of server files, and this is already handled by the registry on Windows. Therefore, most users of the Windows MineSet server will not need to create a .datamove file.

The DataMover configuration file, .datamove, creates files on the server system for each user. This .datamove file lets you control where these user files are created and whether different classes of files are saved or discarded. On UNIX systems, the .datamove file is located on the server, in the user's home directory. A sample .datamove file called datamove.sample is located on the server in the following directories:

If the .datamove file is absent, or if a particular entry is not present in the .datamove file, the DataMover uses a default value for that entry.

Each entry in the DataMover's configuration file must be on a separate line. For example:

file_cache = directory_name

file_cache specifies the location in which the DataMover stores its output data files and models resulting from mining algorithms. If the file_cache directory does not exist, the DataMover attempts to create it, when first invoked. On Windows the default file_cache directory is determined by the registry (see “Configuring MineSet on Windows Systems”); on UNIX, the default file_cache directory is ./mineset_files/%U. The %U is a wildcard that is filled in with the user's login name on the client machine.

The file_cache should be a directory in a filesystem with sufficient space to hold all of a user's output and temporary files. DataMover creates this directory if it does not already exist. Intermediate files are deleted when the DataMover no longer needs them, unless you set one of the following keep options:

keep_client_upload
keep_client_download
keep_classifier_files
keep_mlc_input
use_ascii_mlc_input

Table 2-2 describes these and other options.

A file in the file_cache directory is the result of a successful operation. If an operation returns an error (that is, the Tool Manager reports a message that begins with fatal error on server) MineSet does not change anything in the file_cache directory. Three examples help illustrate the point:

On IRIX, the DataMover works with Oracle versions 7.2 or later, INFORMIX, and Sybase. On Windows, the DataMover supports ODBC and OLE-DB data sources and native connections to Oracle. On a Linux server, the DataMover works with Oracle 8i.

If you are using relational databases, you may need to configure the server to find these databases. ODBC and OLE-DB data sources are listed in the Windows system registry and not in the dm_config file. Therefore, a Windows server using only ODBC and OLE-DB data sources does not need a dm_config file. Windows users need to create a dm_config file only if they want to use a native connection to a local or remote Oracle database. Oracle may also be accessed with ODBC or OLE-DB, but the native connection generally gives better performance. On IRIX, the dm_config file may be omitted if the server will not access any database.

This section details the DataMover configuration file, dm_config, which is read by the DataMover server during startup. Windows users access the file from the directory in which MineSet is installed, under config\datamove\dm_config. UNIX users access the file from the directory in which mineset is installed, under datamove\dm_config.

This file is not created automatically during installation. The system administrator must create it and log in as root to edit it. It can be created using any simple text editor such as Notepad or Emacs. You can find an example file, dm_config.sample, in the datamove directory. The format of this file is as follows:

Oracle {
"ORACLE_SID", "ORACLE_HOME";
}
 
Oracle_Remote {
“SERVICE_NAME”, “ORACLE_HOME”;
}
Informix {
"INFORMIXSERVER", "INFORMIXDIR";
}
Sybase {
"DSQUERY", "SYBASE";
}

Replace “ORACLE_SID” and “ORACLE_HOME” with the specific information and repeat it once for each Oracle database to be accessed via the DataMover. ORACLE_SID and ORACLE_HOME are Oracle-specific parameters defining an Oracle instance.

The Oracle_Remote section is for accessing remote Oracle databases via Oracle Networking. The SERVICE_NAME entry is a logical name for the remote database. Such logical names are defined by editing an Oracle-defined configuration file (for example, tnsnames.ora), or, depending on the Oracle version and operating system, with an Oracle-supplied applet (for example, Net8 Easy Config). The second ORACLE_HOME parameter is the directory of your client installation. On IRIX systems only, you can often do without an Oracle client installation. In this case the second parameter is interpreted as the directory where DataMover searches for the tnsnames.ora file. This file is described in Oracle's networking documentation.

Each line in the Informix section defines a database server that, in turn, can contain several databases. The server is checked at runtime to determine which databases it contains, so there is no need to record the individual databases in the dm_config file. The first entry is the INFORMIX server (corresponding to the INFORMIXSERVER environment variable), and the second is the INFORMIX directory (corresponding to the INFORMIXDIR environment variable).

Each entry in the Sybase section defines a database server (or, in Sybase terminology, an SQL Server). The first entry is the Sybase SQL Server name (corresponding to the DSQUERY environment variable); the second is the Sybase home directory (corresponding to the SYBASE environment variable). This may be a remote database.

---

Note: A Windows configuration file defines different pathnames, but the structure follows conventions similar to the following example.

---

An IRIX example configuration file shows:

Oracle {
"v81", "/ora/app/oracle/product/8.1.5";
"wrhse", "/opt/oracle";
}
 
Oracle_Remote {
   “lifeseq”, “/usr/lib/MineSet2/datamove/”;
}

#native connections to Informix and Sybase supported on IRIX only
Informix {
"learn_online", "/u5/informix";
}
Sybase {
"MINESET", "/usr/sybase/11.5.1";
}
 
#  Following parameter is relevant only if your database is in a
#  non-English locale.  Default of TRUE allows database to perform
#  automatic translation of character encoding (if necessary) to
#  compensate for any mismatch in locale between the database and the
#  MineSet client.  Setting parameter to FALSE disables this
#  adjustment, which is sometimes necessary with very old or
#  misconfigured databases.
 
enable_database_locale_adjustments = TRUE;

The enable_database_locale_adjustments is common to both platforms.

This configuration file lets the DataMover access the following:

Each of the INFORMIX and Sybase servers can, in turn, contain multiple databases.

For Sybase, DataMover uses vendor-supplied shared libraries as its connection to the databases. One of the purposes of the dm_config file is to specify where DataMover must look for its shared libraries.

DataMover looks for the shared libraries libct.so, libcs.so, ibcomn.so, libintl.so, libtcl.so, and libinsck.so on IRIX systems in the $SYBASE/lib/ directory.

The IRIX DataMover is compiled as an n32 program and therefore needs to load the n32 versions of the Sybase client libraries. n32 client libraries are included with Sybase versions 11.5 or later. If MineSet will be used with an earlier version of Sybase, you will need to install a recent version of the Sybase client.

A Windows example of the dm_config file shows:

# Connection to a local Oracle database on Windows NT.  First entry is
# the Oracle sid, and second entry is the ORACLE_HOME.  Note that if
# '\' is used as a directory separator, it must be doubled.
 
Oracle{
        "mse", "F:\\Oracle\\Ora81";
}
 
#  3-tier access to remote Oracle databases via Oracle networking.
#  First argument is the name of the data source, as created by the
#  Net8 configuration assistant or listed in the tnsnames.ora file.
#  Second argument is the local ORACLE_HOME directory.  At least a
#  client install of Oracle is required to connect to remote Oracle
#  databases.
 
Oracle_Remote {
 
    "uci_megamine", "F:/Oracle/Ora81";
    "MSE_NET8.ENGR.SGI.COM",  "F:/Oracle/Ora81";
    "DEMO_MEGA", "F:/Oracle/Ora81";
    "mega_v72", "F:/Oracle/Ora81";
}

Sometimes it is convenient to use MineSet with data that is already stored as a file, but requires further processing before it can be mined or visualized. In this case, the data file can be made available (with a modest effort) to the Tool Manager/DataMover.

First, the data file must be in a tab-delimited format, with the same number of fields in each line. A numeric or string field with a single “?” character appearing between delimiters is loaded as a Null value. You can import data files automatically, using the MineSet Tool Manager as detailed in “Importing Data Files”. You can find a detailed example of the MineSet .schema file format in Chapter 6, “Flat File Support for MineSet.”

You must describe the contents of the data file to the Tool Manager/DataMover using a file with the .schema extension. The format of the .schema file is as follows:

#
#  A line beginning with a "#" is a comment
#
input {
 
#  The first line lists the data file which is described.  It 
#  must be a simple filename, not a path.  
 
     file "carmodels.data";
 
#  Fields are listed left to right in the line, legal 
#  types are float, double, int, string, date, fixedString and  
#  dataString
#  Be sure to end every line with a semicolon ";"
 
     float mpg;
     int cylinders;
     float cubicinches;
     int horsepower;
     int weightlbs;
     double timeaccelerate;
     date when_introduced;
     string origin;
     fixedString(3) manufacturer_code;
     dataString model;
}

The schema and data files must be located in the same directory. If you prepare a dataset this way on the client machine, you can open it with the Tool Manager's Find File dialog. If the file requires any additional processing, it is copied to the server. Sometimes this is not convenient, especially if the file already exists on the server, or is large. In this case, the .schema and .data files must be copied (or symbolically linked) into your file_cache directory on the server.

Alternatively, you can load, or reload, the sample data separately. Each data directory in DBexamples on the server contains files necessary to load the data into any of the supported databases. Windows users can find these files in the directory in which MineSet is installed, under \Examples\DBexamples. UNIX users can find them in /usr/lib/MineSet/DBexamples. These files are:

README (explains the data)

*.sql (sets up an Oracle table)
*.ctl (control file for loading into Oracle)

*_syb.sql (sets up a Sybase table)
*.bcf.fmt (Sybase format file)

*_inf.sql (sets up an INFORMIX table)
*_load.sql (loads the data into the INFORMIX table)

In the *.ctl file, the separator is declared in the following line:

" fields terminated by X'20'  "

The separator is specified in ASCII hexadecimal; therefore:

X'20' is used for ` '
X'2c' is used for `,'
X'09' is used for `\t'

Perform the following steps on the server with an Oracle database:

Perform the following steps on the server with a Sybase database:

Perform the following steps on the server with an INFORMIX database:

MineSet batch mode requires a session file, which consists of the following information:

The following is an annotated example of a session file designed to run a Decision Tree classifier:

MineSet 3.1 
# Starts with the Preferences options 
preferences {use_binary_files, parallelize, num_threads -1, max_attr_vals 1000} 

# Next the single data source file, RDBMS table or query 
datafile {localserv      "E:\\mineset_demos\\marketing\\mineset_mellon\\respond2.schema"} 

# Next one or more transformations 
transformations { 
remove_col { "owcustid"}; 
remove_col { "Cluster"}; 
} 

# Next single data destination (mining tool, viz tool or file) 
classify classify_and_error decision_tree "foo" "foo" 

# Finally, all the OPTION settings 
options {options_list "test_model_in:", "test_show_viz:0", "operation:predict",      "predictor:classifier", "name:respond2", "model_version:", .... } 

To create the saved_session.mineset file:

Running MineSet in batch mode on NT is similar to running it on IRIX (“Step-by-step Example for Running MineSet Batch on IRIX”). If you want to run the batch directly on the NT MineSet server, follow the steps in that example, except instead of creating a shell script, create a MS-DOS .bat script:

mineset_batch -s USER_NAME -d RDBMS_LOGIN_NAME saved_session.mineset 

For example:

mineset_batch -s NT_passwd -d Oracle_passwd saved_session.mineset 

To run MineSet in batch mode on IRIX or Linux, enter the following command at the command line:

MIndUtil_p30 saved_session.mineset session.schema 

The following shell script, batch.sh, runs the Column Importance, Evidence, Decision Tree, and Cluster analyses on the contract database:

#!/bin/sh 
# This is a UNIX shell script that runs and times MineSet batch. 
# This sample runs Column Importance, Evidence, Decision Tree, and
# Cluster on the "contract" dataset. 

# Save the timings and any errors in results.out 
RESULT="results.out" 
echo > $RESULT 

# Make sure you have named the Mineset session filess contract-import.mineset, 
# contract-evi.mineset contract-dt.mineset, and contract-cluster.mineset and transfered them
# to the IRIX MineSet server in the same directory as the datafiles. 

# Run the calculations using the MineSet analytics, /usr/sbin/MIndUtil_s (32-bit single
# threaded) or # /usr/sbin/MIndUtil_p30 (64-bit parallel), and print the run times to the
# results.out file. ("timex" is a UNIX utility to time how long the analysis took.) 
timex /usr/sbin/MIndUtil_p30 contract-import.mineset contract.schema >>$RESULT 2>>$RESULT 
timex /usr/sbin/MIndUtil_p30 contract-evi.mineset contract.schema >>$RESULT 2>>$RESULT 
timex /usr/sbin/MIndUtil_p30 contract-dt.mineset contract.schema >>$RESULT 2>>$RESULT 
timex /usr/sbin/MIndUtil_p30 contract-cluster.mineset contract.schema >>$RESULT 2>>$RESULT