Chapter 5. Data and Configuration File Basics

This chapter covers data and configuration file basics that are the same for all MineSet tools. The following topics are discussed:

Data Types

MineSet supports integer, floating-point number, string, and date data types, as well as arrays of these types. The following data types are supported:

  • int represents a 32-bit signed integer.

  • float represents a single-precision floating point number. The decimal point is optional. Numbers in exponential “e” notation are also accepted.

  • double represents a double-precision floating point number. The decimal point is optional when representing a floating point number. Numbers in exponential “e” notation are also accepted. The superior precision of double can be useful for accurately representing large numbers, because float can represent only seven or eight significant digits accurately. This superior accuracy, however, consumes twice the memory space of float.

  • dataString represents a string that is unlikely to appear multiple times. If it appears multiple times, several copies are made. A dataString can be used to store a memory address. Addresses are unlikely to be compared, and each record can have a different address.

  • string represents a string of characters that can appear multiple times in the data file. Unlike a dataString, only a single copy of a given string is stored in memory, no matter how many times it appears in the data. This saves memory for strings appearing many times.

    Comparing strings is also much quicker than comparing dataStrings. Reading in strings can be slower than reading in dataStrings because it is necessary to look for duplications. An example of string use is a division name that appears once for each department in the division. If you are unsure whether to use a string or a dataString, use a string.

  • fixed string represents a string of fixed length. Like a dataString, if a fixed string appears multiple times, multiple copies are made. In general, fixed strings are used internally for representations of data from data bases, and are generally better to use than strings or dataStrings.

  • date represents a date and time. In the data file, a date appears in the format MM/DD/YYYY HH:MM:SS. Output from MineSet always represents dates with four-digit years, although two-digit years are acceptable for input. MineSet follows the X-OPEN standard for interpreting two-digit years. Fields with values 69 or greater are considered to be from the 20th century (1969-1999), and values from 0 to 68 are considered to be from the 21st century (2000-2068).

Enumerations

The syntax for declaring an enumeration is:

enum type name { value, value...};

For example:

enum string state { 
   "Alabama",
   "Alaska", 
   ... 
   “Wyoming" 
}; 

The word “string” indicates that the enumeration maps integers to strings; they can also be mapped to other types.

Once you declare an enumeration, you can declare a column to be of that enumeration using the following syntax:

enum enumname columnname;

For example, the following declares st to be a variable of state enumeration:

enum state st;

The input file corresponding to this column must contain values from 0-49 (or “?” representing null); however, the output shows the state name.

You can also use Enumerations to declare enumerated arrays (see “Enumerated Arrays” on page 49).

Arrays

In MineSet, you can use one-dimensional or two-dimensional arrays of fixed or variable size.

In a fixed-sized array, all entries of the given type have the same number of values. For example, the budgets of the 50 states can be represented by a separate float column for each state, or by a single array with 50 floats.

A special form of a fixed array is an “enumerated array.” Like the normal fixed array, there are a fixed number of values in the array; however, the values are associated with an enumeration. For each value in the enumeration, there is a single entry in the array. For example, if an enumeration represents the 50 states, an enumerated array based on this enumeration has 50 values.

A variant of the enumerated array is the “null enumerated array,” which has an additional entry at the beginning for null (represented as a “?”). For example, with the enumeration of the 50 states, the null enumerated array has 51 values, one for NULL, and the remaining 50 for the 50 states. The null array element could be used for entries where the state is unknown.

The tree visualizer also supports variable length arrays (see Chapter 6, “Creating Data and Configuration Files for the Tree Visualizer,” for details).

As with the columns, arrays are represented as value separated by tabs or other separators. For a fixed-sized array, you can use the same separator for columns and for individual array elements (in which case, array elements are not visually distinguished from separate columns). You can also define a different separator. In the sales example, for instance, you can treat the location as a four-element array, rather than as four columns. It then could be represented like this:

Eastern:Maryland:Baltimore:1816 appliances 72 115 138

In this example, the array is separated by colons, and the columns are separated by tabs. (For clarity, the rest of this document uses this format.)

For a variable-length array, you must use different separators for the array and for the columns; otherwise, it is impossible to determine where the variable-length array ends and the other columns begin.

Any field or array element in the data file can also have the value “?” (question mark), indicating an unknown or null value (see the discussion of nulls in Chapter 12, “Nulls in MineSet.”)

Fixed Arrays

You can also declare arrays using data declarations. The simplest form is the fixed array. The declaration syntax is:

type name [ number ] ;

For example:

float revenue [50];

You can also override the separator by declaring it as follows:

type name [ number ] separator `char';

For example:

float revenue [50] separator `:';

If no separator is specified, the default column separator (usually a tab) is used.

Enumerated Arrays

To declare an enumerated array, first declare the enumeration (see “Enumerations” on page 47). Then declare the array using one of the following syntaxes:

type name [ enum keyname ];
type name [enum keyname ] separator `char';

For example:

float revenue [enum state]; 

As with the normal fixed array, you can also specify a separator. To declare a null enumerated array, use one of the following syntaxes:

type name [ null enum keyname ]; 
type name [ null enum keyname ] separator `char'; 

For example:

float revenue [null enum state];

This indicates that the array contains one additional value at the beginning, corresponding to null.

Variable Names

A variable name can appear in two formats:

  • In the first format, it is a letter followed by a number of letters, digits, or underscores. It cannot be a keyword, and should not be quoted.

  • In the alternate format, the variable name should be surrounded by back quotes (`). The variable name can match a keyword and can contain non-alphanumeric characters. This second format is primarily for .schema files generated automatically by the Tool Manager.

There is no scoping of variable names; a given variable name can only be declared once in the .schema file.

Strings and Characters

Strings and characters in the .schema file follow C-language conventions. Strings are in double quotation marks ("), and characters are in single quotation marks ('). All standard backslash conventions are followed (for example, \n represents a new line).

Comments

Comments begin with a pound (#) symbol at the beginning of a line; anything after this symbol is ignored, up to the end of the line.

MineSet Expression Language

The expression language used in the Filter and Add Column panels is similar to expressions in C, C++, and Java. The basic operators are the same, as listed in Table 5-2:

Table 5-1. Operators Used With Expressions

Operator

Description

+

Addition

-

Subtraction

*

Multiplication

/

Division

%

Modulus

==

Equals

!=

Not equals

>

Greater than

<

Less than

>=

Greater than or equal to

<=

Less than or equal to

&&

AND

||

OR

!

NOT

&

Bitwise AND

|

Bitwise OR

^

Bitwise XOR

A?B:C

If (A), then B else C

The expression language also provides the functions listed in Chapter 5, “  Expression Language Functions”:

Table 5-2. Expression Language Functions

Function

Description

isNull( )

Determines if the value in parentheses is null

if ( ) then ( ) else ( )

Standard if/then/else

( ) ? ( ) : ( )

C syntax if/then/else

divide( x, y, z )

Divide x by y, and give value z if y is 0

Also, the following functions are available:

  • modulus(x, y, z) is similar to divide, but for modulus.

  • hierarchy(string) is valid only within a hierarchy. It produces a string describing the components of the hierarchy, separated by string. For example:

    hierarchy(":") 
    

    This might produce:

    Western:California:Mountain View
    

    The hierarchy function is most useful in the execute statement, passing the hierarchy information to the command being executed.

  • isSummary() returns 1 if the expression is being applied to base information; otherwise, it returns zero. Often, this is useful with the ?: operator, particularly in message and execute statements.

Type handling is similar to that in C. Expressions using int and float promote both sides to float. Expressions using int and double, or float and double, promote both sides to double. The result of a relational expression (for example, ==, <) is always an int. Type casting is also supported.

Unlike in C, strings can be compared using relational expressions; the strings are compared lexicographically.

Keywords

The currently recognized keywords are listed in Table 5-1. Variables cannot have these names unless they are surrounded by back quotes (`). Tokens appearing only in option statements are not keywords, and can be used for variable names.

Table 5-3. Keywords for the Tree Visualizer

 

 

 

 

aggregate

disk

int

normalize

any

divide

isSummary

off

ascending

double

key

on

average

enum

label

options

back

execute

landscape

scale

base

expressions

legend

separator

buckets

file

levels

sort

color

filter

max

string

colors

float

message

sum

count

height

min

view

dataString

hierarchy

modulus

 

descending

input

none

 


Configuration File Basics

This section discusses the various parts that make up a MineSet configuration file.

Sections

The configuration file consists of a series of sections, each of which has this syntax:

sectionKeyword 
{
    statements...
}

sectionKeyword names the section. A semicolon (;) can follow the closing brace (}) but is not required. The order of the sections is significant, since sections can refer to variables defined in previous sections. The sections found in a treeviz configuration file are “Input Section,” “Expressions Section,” “Hierarchy Section,” and “View Section.”

Options Files

As each section is encountered, a special configuration file (referred to as an “options file”) is also read in. Options files have names in the following form:

sectionName.treeviz.options

Options files normally contain options statements. These files are read in the following order:

  1. The directory containing system defaults.

    • For Windows: from the directory in which MineSet is installed, under \config\treeviz.

    • For UNIX: /usr/lib/MineSet/treeviz.

  2. Your home directory, where you set up personal defaults.

  3. The current directory, which lets you set up defaults for each directory.

Files with the same name can appear in more than one of the above-named directories; in this case, the order given is the one in which the directories are read. If the same option is found in multiple files, the last option read is used. Note that the appropriate section in the configuration file is read after all the options files have been read in; thus, options in the configuration file override those in the options files.

Statements

A statement has the following syntax:

statementKeyword info;

statementKeyword defines the statement, and info varies according to the keyword. A statement can be another section (using the brace format defined under “Sections”).

Option Statements

Many sections have options statements, which have this syntax:

options key info, key info...; 

key defines the specific option, and info depends on the key. In some cases, the key can be more than one word. To maximize the number of allowable variable names, most option keys are meaningful only within the appropriate option statement; keys do not conflict with variable names. You can declare several options on the same line, separating them by commas or placing them in several options statements. For example, the following two examples are equivalent:

options home angle 30, shrinkage 10.0; 
options home angle 30; 
options shrinkage 10.0;

If two conflicting values for the same option appear, the last value is taken.

Include Statements

The configuration file can contain lines of the following form:

include "filename" 

These lines can appear anywhere in the configuration file, but each must be on its own line. The filename must be in quotation marks; anything after the closing quotation mark is ignored. Include statements can be nested. If a relative pathname (one not beginning with a slash) is specified, the file is first sought relative to the directory containing the current configuration file. (If include statements are present, this might not be the same as the initially loaded configuration file.) If it is not found in the directory containing the current configuration file, the include file is sought in the current directory. If the file is not found, an error message appears.

Sinclude Statements

A statement similar to an include is sinclude, which has the syntax:

sinclude "filename" 

This is identical to the include statement, except that no error appears if the file does not exist; instead, the sinclude statement is ignored.

Input Options

The input section of a data file has several options. All options statements begin with the word options and have one or more comma-separated options.

  • The separator option defines the separator between columns in the data file. The default separator is a tab. The syntax is:

    options separator `char';
    

    For example:

    options separator `:';
    


    Note: Arrays can override the separator.


  • The backslash option controls whether backslashes in the input data are treated specially or like other characters. The syntax is:

        options backslash off;
        options backslash on;
    

    The default is off. If backslash processing is on, separators in the input data preceded by backslashes are treated as regular characters rather than separators. Also, within strings, standard C-style backslash processing is done.