This appendix describes the .schema and the .data files that are required to define the MineSet flat files. The Tool Manager also generates .schema files to include the .schema files for Tree Visualizer, Map Visualizer, Scatter Visualizer, and Splat Visualizer.
This appendix first discusses the. data file (the data file), then the .schema file; the final section notes the exceptions to the descriptions for these files for some tools.
In its simplest form, the data file consists of a list of lines, each containing a set of fields, each separated by one tab. (Other separators are also allowed—see “Input Options”—but only one, per file, may can separate each field). All lines must contain the same fields. (The interpretation of the fields is specified by the .schema file, described in the next section.) For example, the first few lines of retail store data might look like this:
In this example, the first five columns are strings: region, state, city, store ID, and product. These are followed by three numbers, representing current sales, last year's sales, and the sales target.
The data file cannot contain blank lines or comments. Missing or extra data on a line causes an error.
Eastern:Maryland:Baltimore:1816:appliances:72:115:138 |
The order of the columns must match the format of the .schema file. For some visual tools, the order of the rows can affect the layout of the final graphic. See the tool-specific appendices for details.
Any field in the data can also be a “?”, indicating that the data is null (unknown). See Appendix J, “Nulls in MineSet.”
MineSet supports integer, floating-point number, and string data types, as well as arrays of these types. The following data types are supported:
float represents a single-precision floating point number. The decimal point is optional. Numbers in exponential “e” notation are also accepted.
double represents a double-precision floating point number. The decimal point is optional when representing a floating point number. Numbers in exponential “e” notation are also accepted. The superior precision of double can be useful for accurately representing large numbers, since float can represent only seven or eight significant digits accurately. This superior accuracy, however, consumes twice the memory space of float.
dataString represents a string that is unlikely to appear multiple times. If it appears multiple times, several copies are made. A dataString can be used to store a memory address. Addresses are unlikely to be compared, and each record can have a different address.
string represents a string of characters that can appear multiple times in the data file. Unlike a dataString, only a single copy of a given string is stored in memory, no matter how many times it appears in the data. This saves memory for strings appearing many times.
Comparing strings is also much quicker than comparing dataStrings. Reading in strings can be slower than reading in dataStrings because it is necessary to look for duplications. An example of string use is a division name that appears once for each department in the division. If you are unsure whether to use a string or a dataString, use a string.fixed string represents a string of fixed length. Like a dataString, if a fixed string appears multiple times, multiple copies are made. In general, fixed strings are used internally for representations of data from data bases, and are generally better to use than strings or dataStrings.
date represents a date and time. In the data file, a date appears with the format MM/DD/YYYY HH:MM:SS. Output from MineSet always represents dates with four-digit years, although two-digit years are acceptable for input. MineSet follows the X-OPEN standard for interpreting two-digit years. Fields with values 69 or greater are considered to be from the 20th century (1969-1999), and values from 0 to 68 are considered to be from the 21st century (2000 - 2068).
In MineSet, you can use one-dimensional or two-dimensional arrays of fixed or variable size.
In a fixed-sized array, all entries of the given type have the same number of values. For example, the budgets of the 50 United States, can be represented by a separate float column for each state, or by a single array with 50 floats.
A special form of a fixed array is an “enumerated array.” Like the normal fixed array, there are a fixed number of values in the array; however, the values are associated with an enumeration. For each value in the enumeration, there is a single entry in the array. For example, if there is an enumeration representing the 50 states, an enumerated array based on this enumeration has 50 values.
A variant of the “enumerated array” is the “null enumerated array,” which has an additional entry at the beginning for null (represented as a “?” ). For example, with the enumeration of the 50 states, the null enumerated array has 51 values, one for NULL, and the remaining 50 for the 50 states. The null array element could be used for entries where the state is unknown.
The tree visualizer also supports variable length arrays (see Appendix B, “Creating Data and Configuration Files for the Tree Visualizer,” for details).
As with the columns, arrays are represented as values separated by tabs or other separators. For a fixed-sized array, the same separator can be used for columns and for individual array elements (in which case, array elements are not visually distinguished from separate columns). You can also define a different separator. In the sales example (in “The Data File”), for example, you can treat the location as a four-element array, rather than as four columns. It then could be represented like this:
Eastern:Maryland:Baltimore:1816 appliances 72 115 138
Here, the array is separated by colons, and the columns are separated by tabs. (For clarity, the rest of this document uses tabs to separate columns, and colons to separate array elements.)
For a variable-length array, you must use different separators for the array and for the columns; otherwise, it is impossible to determine where the variable-length array ends and the other columns begin.
Any field or array element in the data file can also have the value “?” (question mark), indicating an unknown or null value (see the discussion of nulls in Appendix J).
The schema file consists of an input section, which defines the name and format of the file. (The .schema files generated by the Tool Manager can also contain a history section, which is a copy of the .mineset file. This section is used by drill-through and would normally not be present in manually generated .schema files.)
A typical input section might look like this:
input {
file "store";
string region;
string state;
string city;
string storeId;
string product;
float sales;
float lastYear;
float target;
options separator `:';
}
|
This example states that the input file is called store, and that there are eight fields: five of type string, three of type float.
A variable name can appear in two formats:
In the first format, it is a letter followed by a number of letters, digits, or underscores. It cannot be a keyword, and should not be quoted.
In the alternate form, the variable name should be surrounded by back quotes (`). In this form, the variable name can match a keyword, and can contain even non-alphanumeric characters. The primary purpose of this second form is for .schema files generated automatically by the Tool Manager.
There is no scoping of variable names; a given variable name can only be declared once in the .schema file.
Strings and characters in the .schema file follow C-language conventions. Strings are in double quotation marks ("), and characters are in single quotation marks ('). All standard backslash conventions are followed (for example, \n represents a new line).
Comments begin with a pound (#) symbol at the beginning of a line; anything after this symbol to the end of the line is ignored, up to the end of the line.
The file statement names the data file to be read. This statement is required. Its syntax is:
file "filename"; |
Filename must be in double quote marks. If it is a relative pathname (no leading slash), it is first sought in the directory containing the current .schema file. If it is not found in the current .schema file's directory, the file is sought in the current directory.
The data statements declare the columns in the data file. The columns must be declared in the order they appear in the data file. The format of most data statements is
type name; |
where type is int, float, double string, dataString, date, and fixedString(n), where n is an integer representing the width of the string; name is the variable name. Unlike in C, only one variable can be declared per statement.
The syntax for declaring an enumeration is:
enum type name { value, value...};
|
For example:
enum string state {
"Alabama",
"Alaska",
...
“Wyoming"
};
|
The word “string” indicates that the enumeration maps integers to strings; they can also be mapped to other types.
Note that for compatibility with MineSet 1.0, “enum” can be replaced with “key.”
Once an enumeration is declared, a column can be declared to be of that enumeration using the following syntax:
enum enumname columnname; |
For example:
enum state st; |
declares st to be a variable of state enumeration. The input file corresponding to this column must contain values from 0-49 (or “?” representing null); however, the output shows the state name.
Enumerations also can be used to declare enumerated arrays (see “Enumerated Arrays”).
Arrays are also declared using data declarations. The simplest form is the fixed array. The declaration syntax is
type name [ number ] ; |
For example:
float revenue [50]; |
You can also override the separator by declaring it as
type name [ number ] separator `char'; |
For example:
float revenue [50] separator `:'; |
If no separator is specified, the default column separator (usually a tab) is used.
To declare an enumerated array, first declare the enumeration (see the “Enumerations” subsection on page 527). Then declare the array using the following syntax:
type name [ enum keyname ]; |
or
type name [enum keyname ] separator `char'; |
For example:
float revenue [enum state]; |
As with the normal fixed array, you can also specify a separator. Note that for compatibility with MineSet 1.0, the word "enum" can be omitted from within the brackets. To declare a null enumerated array, use the syntax:
type name [ null enum keyname ]; |
or
type name [ null enum keyname ] separator `char'; |
For example:
float revenue [null enum state]; |
indicates that the array contains one additional value at the beginning, corresponding to null.
The input section of a data file has several options. All options statements begin with the word options and have one or more comma-separated options.
The separator option defines the separator between columns in the data file. The default separator is a tab. The syntax is:
options separator `char';
For example:
options separator `:';
Note: Arrays can override the separator. The backslash option controls whether backslashes in the input data are treated specially or like other characters. The syntax is:
options backslash off; options backslash on;The default is off. If backslash processing is on, separators in the input data preceded by backslashes are treated as regular characters rather than separators. Also, within strings, standard C-style backslash processing is done.
The following exceptions apply to the .schema and .data files:
The Tree Visualizer supports only one-dimensional arrays.
The Tree Visualizer supports variable-length arrays.
The Map Visualizer and the Scatter Visualizer support a special enum format for dates.
The Tree Visualizer and the Map Visualizer support the Monitor option.
| Note: These exceptions are discussed in detail in the respective tool's appendix. |