Chapter 2. Setting Up MineSet

This chapter describes how to set up MineSet, which requires configuring the DataMover. The configuration has two parts:

The DataMover is a process that runs on the server, although it is not directly accessible to users. The DataMover provides access to databases and data stored in flat files, and transforms data for the mining and visualization tools.

Configuring the DataMover Server

In order to use the MineSet tools, two configuration files must be created on the server: one by you, the other by the system administrator.

The User Configuration File


Note: You must have a UNIX® account on every server you want to access.

The DataMover server creates files on the server machine on behalf of each user. The DataMover configuration file, .datamove, lets you control where these files are created and whether different classes of files are saved or discarded. This file is located on the server, in your home directory. A sample .datamove file is located on the server, in the /usr/lib/MineSet/datamove directory.

If the .datamove file is absent, or if a particular entry is not present in the .datamove file, the DataMover uses a default value for that entry.

Each entry in the DataMover's configuration file must be on a separate line. For example:

file_cache = directory_name
temp_dir = directory_name

where file_cache specifies the location to which the DataMover stores its query specification and output data files. The location to which the DataMover stores intermediate files for mining processing is the directory specified after temp_dir. If either of the file_cache or temp_dir directories do not exist, the DataMover attempts to create them on its first invocation. The default temp_dir is /usr/tmp and the default file_cache directory is ./mineset_dir/%U. The %U is a wild card that is filled in with the user name on the client machine. This is useful in reducing contention if many users want to log in to a common account on the server. If multiple sessions were simultaneously connected to the same file_cache directory, they could overwrite each other's server files, causing incorrect and unexpected results. To prevent this, DataMover maintains a lock at the file_cache directory level. The second and later attempts to connect to a particular file_cache directory result in failure and an error message.

Once a query result has been returned to the client machine, the DataMover has the option to delete the query result and specification. The DataMover's behavior is controlled by the following options:

keep_temp_files
keep_data_files
keep_query_files

Each of these entries must be on a separate line. All entries default to no. Adding the entry

keep_query_files = yes

to the .datamove file instructs the DataMover not to delete query specification files when it is finished processing them.

Some of the files created by the DataMover can be large. Thus, if you specify one of the keep_* options as yes, you should log on to the server periodically and clean out the file cache. The settings in the .datamove file can be overridden for each file with a Tool Manager option.

Using MineSet to create a classifier via the Tool Manager typically causes a classifier file and a classifier-options file to be created in the file_cache directory. You can delete or retain these files for further use by setting the following options in the .datamove file:

keep_classifier_files=yes
keep_classifier_options_files=no

As with other options in the .datamove files, these must also be on separate lines. It is worth noting that the predefined default for the keep_classifier_files option is “yes.”

Mandatory Configuration File

The MineSet DataMover server must be configured to find information in the databases. The DataMover works with Oracle® versions 7.2 or later, INFORMIX®, and Sybase®.

The DataMover server reads the /usr/lib/MineSet/datamove/dm_config file during start up. This file is not created by Inst during installation. It must be created by the system administrator, who must log in as root to edit this file. It can be created via an editor such as jot, vi, or Emacs. An example file can be found in /usr/lib/MineSet/datamove/dm_config.sample. The format of this file is

Oracle {
"ORACLE_SID", "ORACLE_HOME";
}
Informix {
"INFORMIXSERVER", "INFORMIXDIR";
}
Sybase {
"DSQUERY", "SYBASE";
}

Each entry is optional; describe those databases in use at your site. If your server is not running any databases, that is, you intended to use MineSet with ASCII files only, simply make an empty dm_config file or copy the sample.

The line "ORACLE_SID", "ORACLE_HOME" is filled in with the specific information and repeated once for each Oracle database to be accessed via the DataMover. ORACLE_SID and ORACLE_HOME are Oracle specific parameters defining an Oracle instance.

Each line in the INFORMIX section defines a database server that, in turn, can contain several databases. The server is interrogated at runtime to determine which databases it contains, so there is no need to record the individual databases in the dm_config file. The first entry is the INFORMIX server (corresponding to the INFORMIXSERVER environment variable), and the second is the INFORMIX directory (corresponding to the INFORMIXDIR environment variable).

Each entry in the Sybase section defines a database server (or, in Sybase terminology, an SQL Server). The first entry is the Sybase SQL Server name (corresponding to the DSQUERY environment variable); the second is the Sybase home directory (corresponding to the SYBASE environment variable).

An example configuration file might be as follows:

Oracle {
"v73", "/usr/people/oracle/v73";
"wrhse", "/opt/oracle";
}
Informix {
"learn_online", "/u5/informix";
}
Sybase {
"MINESET", "/usr/sybase/10.0.2.4";
}

This configuration file lets the DataMover access two Oracle databases, one named “v73” (installed in /usr/people/oracle/v73), and another named “wrhse” (installed in /opt/oracle); an INFORMIX Server; and Sybase SQL Server. Each of the INFORMIX and Sybase servers can, in turn, contain multiple databases.

For Oracle and Sybase, DataMover uses vendor-supplied shared libraries as its connection to the databases. One of the purposes of the dm_config file is to specify where DataMover must look for its shared libraries. For Oracle, DataMover looks for the file $ORACLE_HOME/lib/libclntsh.so. The Oracle installation manual describes how to create this shared library, if it is not already present. For Sybase, DataMover looks in the $SYBASE/lib/ directory for the following shared libraries: libct.so, libcs.so, ibcomn.so, libintl.so, libtcl.so, libinsck.so.

Using MineSet With Existing Data Files

Sometimes it is convenient to use MineSet with data that is already stored as a file, but requires further processing before it can be mined or visualized. In this case, the data file can be made available (with a modest effort) to the Tool Manager/DataMover.

First, the data file must be in a tab-delimited format, with the same number of fields in each line. A numeric or string field with a single “?” character appearing between delimiters is loaded as a Null value.

For a detailed discussion of null values, refer to Appendix G, “Nulls in MineSet.”

The contents of the data file must be described to Tool Manager/DataMover via a file with the .schema extension. The format of the .schema file is shown below:

#
#  A line beginning with a "#" is a comment
#
input {

#  The first line lists the data file which is described.  It 
#  must be a simple filename, not a path.  

     file "carmodels.data";

#  Fields are listed left to right in the line, legal 
#  types are float, double, int, string, and  dataString
#  Be sure to end every line with a semicolon ";"

     float mpg;
     int cylinders;
     float cubicinches;
     int horsepower;
     int weightlbs;
     double timeaccelerate;
     int year;
     string origin;
     dataString model;
}

The schema and data files must be located in the same directory. If you prepare a dataset in this fashion on the client machine, it can be opened with the Tool Manager's Find File dialog. If the file requires any additional processing, it is copied to the server. Sometimes this is not convenient, especially if the file already exists on the server, or is large. In this case, the .schema and .data files should be copied (or symbolically linked) into the your file_cache directory on the server. The directory used as the file cache is specified in your .datamove file; the default is ./mineset_dir/%U.

Using MineSet With Existing Data Files

Sometimes it may be not be feasible to install DataMover on the machine running the database server. In this situation, DataMover may be installed an intermediate server, and DataMover then can use the database vendor's networking facility to connect to the remote database. (This is sometimes referred to as a 3-tier architecture.)

Oracle

An Oracle installation is required on the intermediate server; this Oracle installation need not be running an active database, but is needed for access to the shared library, libclntsh.so, and the Oracle names file, tnsnames.ora.

On the intermediate server, there should be an entry for the local Oracle install, with ORACLE_HOME and ORACLE_SID, as usual. Add entries for any desired remote database to the $ORACLE_HOME/network/admin/tnsnames.ora file of the Oracle install on the intermediate server.

Then, when a user wishes to log in to user “system”, password “manager” at database “remotedb”, they should give the name of the intermediate server for the Tool Manager “Log on to server...” dialog and choose the intermediate server's Oracle database. When logging in to the database itself, they should give “system” for the database username, and “manager@remotedb” for the password. The added “@remotedb” tells Oracle to use SQL*Net to connect to the remote database, instead of using a local connection.

Operating across SQL*Net is substantially slower than a local connection, especially for queries which return a large amount of data. Hence it is better to install DataMover on the same machine as the Oracle server, if at all possible.

Sybase

A Sybase installation is required on the intermediate DataMover server; this Sybase installation need not be running an active database, but it is needed for access to the shared libraries and the interfaces file.

In order to access the Sybase SQL server running on the remote machine, the interfaces file on the DataMover server machine should have an entry for this Sybase SQL server. Please refer to your Sybase manuals for the procedure for creating such entries. Also, the name of this Sybase SQL server on the remote machine should be included in the dm_config file on the intermediate DataMover server machine.

Once this setup is done, access to the Sybase SQL server on the remote machine is handled transparently. The user can choose it and access data from it just like any other database source, using the panels from the Tool Manager.