Chapter 6. Flat File Support for MineSet

This chapter describes the .schema and the .data files that are required to define the MineSet flat files. The following subjects are discussed:

The Tool Manager also generates .schema files to include the .schema files for Tree Visualizer, Map Visualizer, Scatter Visualizer, and Splat Visualizer.

Data File

In its simplest form, the data file consists of a list of lines, each containing a set of fields, each separated by one tab. (Other separators are also allowed but only one, per file, can separate each field.) All lines must contain the same fields. (The interpretation of the fields is specified by the .schema file, described in the next section.) For example, the first few lines of retail store data might look like this:

Eastern Maryland Baltimore 1816 appliances 72 115 138
Eastern Maryland Baltimore 1816 clothing 355 344 395
Eastern Maryland Baltimore 1816 electronics 156 182 209
Eastern Maryland Baltimore 1816 furniture 78 75 82
Eastern Massachusetts Boston 1331 appliances 48 68 81
Eastern Massachusetts Boston 1331 clothing 307 258 296
Eastern Massachusetts Boston 1331 electronics 38 183 210
Eastern Massachusetts Boston 1331 furniture 52 69 75
Eastern Massachusetts Boston 1220 appliances 37 63 75
Eastern Massachusetts Boston 1220 clothing 233 240 276
Eastern Massachusetts Boston 1220 electronics 175 208 239
Eastern Massachusetts Boston 1220 furniture 35 53 58

In this example, the first five columns are strings: region, state, city, store ID, and product. These are followed by three numbers, representing current sales, last year's sales, and the sales target.

The data file cannot contain blank lines or comments. Missing or extra data on a line causes an error.


Note: One tab (the default separator) separates each field. Do not insert multiple tabs to line up the fields visually; this generates blank fields. You can use other characters, such as a colon (:), as a separator. In this case, the first line appears as follows:


Eastern:Maryland:Baltimore:1816:appliances:72:115:138

The order of the columns must match the format of the .schema file. For some visual tools, the order of the rows can affect the layout of the final graphic. See the tool-specific chapters for details.

Any field in the data can also be a “?”, indicating that the data is null (unknown). See Chapter 13, “Nulls in MineSet.”


Note: MineSet also supports a binary format, which currently is not documented.


.schema File

The .schema file consists of an input section, which defines the name and format of the file. The .schema files generated by the Tool Manager can also contain a history section, which is a copy of the .mineset file. This section is used by drill through and would normally not be present in manually generated .schema files.

A typical input section might look like this:

input {
   file "store";
 
   string region; 
   string state;
   string city;
   string storeId;
   string product;
   float sales;
   float lastYear;
float target;
 
   options separator `:';
}

This example states that the input file is called store, and that there are eight fields: five of type string and three of type float.

File Statements

The file statement names the data file to be read. This statement is required. Its syntax is:

file "filename";

Filename must be in double quote marks. If it is a relative pathname (no leading slash), it is first sought in the directory containing the current .schema file. If it is not found in the current .schema file's directory, the file is sought in the current directory.

Data Statements

The data statements declare the columns in the data file. The columns must be declared in the order they appear in the data file. The format of most data statements is:

type name;

Type is int, float, double string, dataString, date, and fixedString(n), where n is an integer representing the width of the string, and name is the variable name. Unlike in C, only one variable can be declared per statement.

Other supported types include enumerations, fixed arrays, and enumerated arrays. You must declare these data types must inside the input section, before the declaration of the specific column.

You can also override the separator by declaring it as follows:

type name [ number ] separator `char';

For example:

float revenue [50] separator `:';

If no separator is specified, the default column separator (usually a tab) is used.

Exceptions

The following exceptions apply to the .schema and .data files:

  • The Tree Visualizer supports only one-dimensional arrays.

  • The Tree Visualizer supports variable-length arrays.

  • The Map Visualizer and the Scatter Visualizer support a special enum format for dates.

  • The Tree Visualizer and the Map Visualizer support the Monitor option.


Note: These exceptions are discussed in detail in the respective viz tool's chapters.