This chapter describes the data and configuration files that are required for the Splat Visualizer. You can also generate these files automatically by using the Tool Manager. The subjects discussed are:
Read the about the Splat Visualizer in the MineSet Enterprise Edition User's Guide before using this chapter.
In its simplest form, the data file consists of a list of lines, each containing a set of fields, each separated by one tab. (Other separators are also allowed, but only one per file can separate each field. See “Input Options”.) All lines must contain the same fields. The interpretation of the fields is specified by the configuration file, described in the next sections. These sections use examples from the adultJobs data file provided as part of the Splat Visualizer package. Windows users can find the file in the directory in which MineSet is installed, under Examples\splatviz\examples. UNIX users can find the file in /usr/lib/MineSet/splatviz/examples.
The first few lines of the input file appear as:
Bachelors Adm-clerical 3 3 51189.4869565217 115 Bachelors Exec-managerial 2 5 70722.6271186441 59 Bachelors Adm-clerical 2 3 37876.328358209 134 Bachelors Exec-managerial 3 0 34436.8 5 Bachelors Tech-support 1 2 37583.66667 3 Bachelors Tech-support 1 3 13711.33333 3 Bachelors Tech-support 1 4 29878.74193 31 |
In this sample file listing, each line consists of six fields, separated by tabs. The first field is a string that identifies level of education. The second field is a string which identifies occupation. The third field identifies the age bin. The fourth field identifies the number of hours per week worked bin. The fifth field quantifies the average gross income. The sixth field is the weight of records in the aggregate (i.e. the record count, unless record weighting has been used).
This data file was derived from adult94.data by performing Tool Manager operations (specifically binning and aggregation). Windows users can find the file in the directory in which MineSet was installed, under \Examples\data\adult94.data. UNIX users can find the file in /usr/lib/MineSet/data/adult94.data.
The data file cannot contain blank lines or comments. Missing or extra data on a line causes an error.
The Splat Visualizer supports data of types the int, float, double, string, dataString, fixed string, and date. See Chapter 5, “Data and Configuration File Basics,” for more information about data types.
The configuration file format is flexible. You must separate words in it with spaces, and the file is case-sensitive. Except for the include statement and text within quoted strings, spacing and line breaks are irrelevant.
As each section is encountered, a special configuration file (referred to as a defaults file) is also read in. Defaults files normally contain options statements. These files are read in the following order:
The splatviz directory which usually contains system defaults. Windows users can find this in the directory in which MineSet is installed, under \config\splatviz. UNIX users can find the directory in /usr/lib/MineSet/splatviz.
The ~/.MineSet directory in which you can set up personal defaults (the tilde, ~], indicates your home directory).
The current directory, which lets you set up defaults for each directory.
Files with the same name can appear in more than one of the above-named directories; in this case, the order given is the one in which the directories are read. If the same option is found in multiple files, the last option read is used. Note that the appropriate section in the configuration file is read after all the defaults files; thus, options in the configuration file override those in the defaults files.
The first section of a configuration file is normally the input section. It defines the name and format of the data file. A typical input section might appear as follows:
input {
file "adultJobs.data";
enum string `age_bin_k`{"- 20", "20-30", "30-40",
"40-50", "50-60", "60-70", "70+"};
enum string `hours_per_week_bin_k`{"- 20", "20-25", "25-30", "30-35", "35-40", "40-45", "45-50", "50-55", "55-60", "60-65", "65-70", "70+"};
string `education`;
string `occupation`;
enum `age_bin_k``age_bin`;
enum `hours_per_week_bin_k``hours_per_week_bin`;
double `avg_gross_income`;
int `count_gross_income`;
}
|
This example states that the input file is called adultJobs.data, and that there are six fields: education, occupation, age_bin, hours_per_week_bin, avg_gross_income, and count_gross_income. The education and occupation fields are of type string. The age_bin and hours_per_week_bin are of type enum, where the values of these enums are defined by age_bin_k and hours_per_week_bin_k respectively. The column avg_gross_income is of type double and the field count_gross_income is of type int.
When the input section is entered, the defaults file inputDefaults is read in.
The file statement names the data file to be read. This statement is required. Its form is as follows:
file "filename"; |
filename must be in double quotation marks. If it is a relative pathname (no leading slash), it is first sought in the directory containing the current configuration file. If include statements are present, this might not be the same as the initially loaded configuration file. If it is not found in the current configuration file's directory, the file is sought in the current directory.
Enumeration statements declare enumerations, or enums. The enum statement has three forms.
enum type name from value1 to value2 by increment;
This declares an enum with values starting at value1 and incremented by increment until they reach or exceed value2. For example, the following statement declares age as an enum with the values 20, 30, 40, 50, 60, and 70:
enum int age from 20 to 70 by 10;
Type must be a number type (int, float, or double) or date (see “Dates”).
enum type name from value1 to value2 across numberOfValues;
This declares an enum with values ranging from value1 to value2. The numberOfValues is an integer specifying the number of values. For example, the following statement declares age as an enum with the values 20, 30, 40, 50, 60, and 70:
enum int age from 20 to 70 across 6;
Type must be a number type (int, float, or double) or date (see “Dates”).
enum type name {value1, value2, ..., valueN};
This explicitly lists the enum values.
Type can be any type or date (see “Dates”).
The enum statement includes special support for a date type that handles date and time values from January 1, 1753 forward. The date type is valid only within enum statements. A date enum statement can have any if the following types of syntax:
enum date “format” name from “value1” to “value2” across numberOfValues;
enum date “format” name { value1, value2, ..., valueN };
enum date “format” name from “value1” to “value2” by “increment”;
|
The format string specifies the format of the values; it is useful for controlling how dates are displayed in the animation control panel. The syntax of the format string is similar to the scanf function in C. Various units of time are represented by special characters preceded by the percent symbol (%). For example:
enum date cq “Calendar Q%Q, %Y” from “Calendar Q1, 1980” to “Calendar Q3, 1985” by “1 quarter”; |
The “Calendar Q” in the format string matches the “Calendar Q” in value1 and value2. The %Q in the format string indicates that the next number in value1 and value2 is the calendar quarter. The comma and space in the format string match the commas and spaces in the values. Finally, the %Y in the format string specifies that the year values are next.
Table 10-1 lists the characters that can follow the percent symbol and the units of time they represent.
Table 10-1. Characters That Can Follow the Percent Symbol in the Format String
Character | Time Unit | Precision |
|---|---|---|
Y | 4 | |
Q | 1 | |
M | 2 | |
N | month name | >= 3 |
D | 2 | |
h | 2 | |
m | 2 | |
s | 2 |
With the exception of N, each character matches an integer of the specified precision. N matches 3 or more characters giving the English name of the month.
The from-to-by form of the enum statement includes an increment value. For dates, the increment is a quoted string containing an integer, an optional space, and one of the special characters in Table 10-1 or one of the symbols year, quarter, month, day, hour, minute, or second. The plural forms of these symbols are also accepted. These symbols are not keywords, because they have special meaning only in the increment string. The following are examples of valid increments:
“1 year” “7 days” “4h” |
The data statements declare the fields in the data file. You must declare the fields in the order they appear in the data file. The format of most data statements is:
type name; |
type is int, float, double string, dataString, date, or fixedString(n), where n is an integer representing the width of the string; name is the variable name. Unlike in C, you can declare only one variable per statement.
You can also base a data field on an enumeration. The syntax is:
enum enumName name; |
The field must contain ints corresponding to the values of the enums. For example, you could declare the enum ageGroup as follows:
enum string ageGroup {"below 30", "30-39", "40-49", "50-59", "60 or above"};
|
In this case, you can declare the field age as follows:
enum ageGroup age; |
The field should contain ints between 0 and 4, where 0 is displayed as “below 30,” 1 as “30-39,” and so forth.
You can declare only one variable per statement.
All options statements begin with the word “options” and have one or more comma-separated options.
The separator option defines the separator between fields in the data file. The default separator is a tab. The syntax is:
options separator `char';
For example:
options separator `:';
The backslash option controls whether backslashes in the input data are treated specially or like other characters. The syntax is:
options backslash off; options backslash on;
The default is off. If backslash processing is on, separators in the input data preceded by backslashes are treated as regular characters rather than as separators. Within strings, this causes standard C-style backslash processing.
The view section of a configuration file describes how the data is displayed, including the mapping of sizes, colors, axes, and so on. The default values for these options are in view.splatviz.options. The Windows options are in the directory in which MineSet is installed, under config\scatterviz\view.splatviz.options. The UNIX options are in /usr/lib/MineSet/splatviz/view.splatviz.options. The syntax is:
view
{
viewStatement;
...
}
|
A sample view section is as follows:
view {
slider `age_bin`;
opacity `count_gross_income`;
color `avg_gross_income`;
axis `education`, color "grey";
axis `occupation`, color "grey";
axis `hours_per_week_bin`, max 100, color "grey";
options grid size 0 0 0;
summary `count_gross_income`, color "red";
}
|
When entering the view section, the viewDefaults file is read in.
The slider statement identifies an enum column to be used as a slider dimension. Its syntax is one of the following:
slider columnName; |
Declare the columnName name in the input section. If this column contains nulls, the slider includes a beginning position corresponding to those null values.
There can be one, two, or no slider statements. The first slider statement applies to the horizontal slider, and the second applies to the vertical slider. If there is no slider statement, the resulting display does not include animation.
In the Splat Visualizer, the opacity is based on counts or, more generally, record weights. If you map a column to this requirement, it is used to weight each record (rather than using 1) when computing a value for the opacity. Therefore, if you had a column with values for population, density, or the result of a count aggregation, you might want to map this column to the opacity (weight) requirement. If you had no such column, you can leave the requirement unmapped, and a column of 1's is used by default.
The opacity statement describes how a field of data is mapped to the opacity of the splats. The opacity statement consists of a series of clauses, separated by commas:
opacity clause1, clause2,... |
Alternatively, you can put the clauses in separate opacity statements.
The first clause normally contains the name of a field to be mapped to opacity. The field must be of a number type, int, float, or double, of which float is the most efficient.
The max clause allows you to alter the initial opacity setting for the scene. The most opaque splat in the scene will match the value specified in this max clause. The default is 1. The max clause has the following form:
max float |
The legend clause defines the meaning of the opacity mapping. The legend clause has the following forms:
legend off
This turns off the opacity legend.
legend on
This turns on the opacity legend (this is the default). The default legend is:
opacity:count
count is a column that the tool has created by counting the number of records in each aggregate. If a column is mapped to opacity, the name of this column, prepended with “sum_”, is shown in the legend. This new column is computed by sum aggregating the column mapped to the opacity requirement.
legend label “string”
This turns on the legend and explicitly sets the legend string. If you use this form, legend on is unnecessary.
The color statement describes how values are mapped to colors. The format is similar to the opacity statement, consisting of several clauses that you can separate with commas, or enter as multiple statements. The syntax is:
color clause1, clause2,... |
Color names follow the conventions of the X Window System, except that the names must be in quotes. Examples of valid colors are “green,” “hot pink,” and “#77ff42.” The latter is in the form “#rrggbb,” in which the red, green, and blue components of the color are specified in hexadecimal value. Pure saturation is represented by ff, a lack of color by 00. For example, “#000000” is black, “#ffffff” is white, “#ff0000” is red, and “#00ffff” is cyan.
Windows users should only use the “#ff0000” form, as only some of the named colors are supported (for example, white, black, gray, red, yellow, green, and so on.)
UNIX users can find the color list file in /usr/lib/X11/rgb.txt.
As with opacity, you can also specify a column to be mapped to splat color. If the column is a number type, you can use the scale and buckets clauses described in the following paragraphs to map a range of colors to the values of the field.
The colors clause specifies the colors to be used. The syntax is:
colors "colorname" "colorname"... |
The format for colorname is described in “Color Naming”. There are no commas between the colors, because commas are used to separate clauses in the color statement. A sample colors clause is:
colors "red" "gray" "blue" |
Colors in the list are subsequently referred to by their index, starting at zero. In the above example, red is color 0, gray is color 1, and blue is color 2.
If there is no colors statement, colors are chosen randomly. If there is a colors statement, at least as many colors must be specified as will be mapped.
The scale clause allows assignment of values to a continuous range of colors. For example, when a percentage is displayed, red can be assigned to 0%, gray to 50%, and blue to 100%. Intermediate values are interpolated; for example 25% is pinkish, and 55% is a slightly bluish gray.
The syntax for the scale clause is:
scale float float ... |
The first value is mapped to color 0, the second to color 1, and so forth. The colors statement must contain at least as many colors as will be mapped to the largest index.
Values in this statement must be in increasing order. Any value less than the first color is assigned the value of the first color. Any value greater than the last value is assigned the last color. Intermediate values are interpolated.
For example, assume the pctFemale field indicates what percentage of the group is female, and you want to map a group that is 100% female to red, 100% male to blue, and 50% each to gray. The colors statement for this is:
colors pctFemale, colors "blue" "gray" "red", scale 0 50 100; |
Use the scale clause only in conjunction with a numeric color variable.
The buckets clause is similar to the scale clause without interpolation. All values are rounded down to the highest value in the clause, and that exact color is used. Values less than the first value use the first color.
The syntax for the buckets clause is:
buckets float float ... |
The syntax and assignment of colors is the same as for the scale clause.
If, in the above example, you used the buckets clause instead of the scale clause, the statement would be:
colors pctFemale, colors "blue" "gray" "red", buckets 0 50 100; |
All values greater than or equal to 100 are colored red. Values greater than, or equal to, 50, but less than 100, are gray. All other values are then blue.
Use the buckets clause only with a numeric color variable.
The legend clause creates a legend of the colors. The legend clause syntax can be any of the following:
legend off legend on legend "string" "string" ... legend label "string" |
The legend off clause turns off the legend. The legend on clause turns on the legend. You can omit it if you include other legend statements. Specifying only legend on generates the default legend.
The default legend includes a single label to the left (with the name of the field that is mapped to color), and a list of colored labels on the right (with values obtained from the scale clause, buckets clause, or from the field). To override the strings in the colored labels, specify the strings as:
legend "string" "string" |
To override the label on the left, specify it following the word label. To eliminate this label, specify an empty string as follows:
legend label "" |
The axis statement causes a variable to be mapped to an axis in the 3D landscape. The variable's values determine where the entities are positioned on the axis. There can be up to three axis statements. Like the size and color statements, the axis statement contains a series of comma-separated clauses, but you must specify all of them in a single statement.
axis clause1, clause2,... |
As with size and color, you can specify a field to be used as an axis. The field can be an array that is indexed by slider dimensions. If the field is an array, it must be of numeric type. If the field is not numeric, it is sorted, and each unique value is assigned a position along the axis.
The summary statement specifies aggregate information to be calculated for all data defined by the slider position. The summary is used to color the drawing window in the animation control panel. Like the opacity and color statements, the summary statement has several clauses that you can specify in one statement, separated by commas, or in separate statements as follows:
summary clause1, clause2,... |
You can specify the variable to be used in the summary. This variable must be of numeric type. If you do not specify a summary variable, sum of counts is used. If you do specify a variable, then the weighted average of that variable (for all the data at the slider location) is used.
The color clause specifies the color used to display the summary values in the drawing window. It has the following form:
color “colorname” |
Various shades of the color, from white to the specified color, are used to represent summary values. The minimum summary value is mapped to white, while the maximum summary value is mapped to the specified color. The default summary color is red. If you do not specify a slider variable, this statement has no effect.
The legend clause creates a legend of the summary colors. The legend clause syntax can be any of the following:
legend off legend on legend label "string" |
The legend off clause turns off the legend. The legend on clause turns on the legend. You can omit it if you include other legend statements. Specifying only legend on generates the default legend.
The legend includes a single label to the left (which defaults to the aggregation function and variable used in the summary), and two colored labels on the right (with the minimum and maximum summary values). To override the label on the left, specify it following the word label. To eliminate this label, specify an empty string; as follows:
legend label "" |
The view section of the configuration file has several options for controlling parameters of the display. These options can appear in a single options statement, separated by commas, or in separate options statements. The syntax of the options statement is:
options option, option,... |
The following options are available:
axis label size float
background color “colorname”
hide label distance float
Controls the distance at which axis labels become invisible. Smaller distances might improve performance, but the labels disappear more quickly.
grid color “colorname”
grid size float float float
Controls the spacing between grid lines. It applies the three values to grid lines along the x, y, and z axes, respectively.
orientation top right front float float float
Specifies the initial orientation of the scene. You may use three floating point values to specify an arbitrary orientation vector.
perspective on off
Specifies whether or not to use perspective initially. The default is to use it.
shape splatType
Specifies the type of splat used. The shapeName can be “constant,” “linear,” “gaussian,” “texture,” or “sphere.”