The act of aggregation (see below) in which data is combined in various ways.

The process of summarizing multiple rows of data using one or more operations over some specified set of columns. Aggregation operations are capable of producing summary statistics, transposing a table, or even producing arrays based on the original data.

A formal procedure for accomplishing some task. For example, an inducer algorithm is a set of machine instructions that tell the computer how to build a classifier or regressor.

A data type capable of storing multiple items. Array types are produced by aggregation and are used by the visualizers to store data which animates over time. MineSet supports one-dimensional arrays (also known as vectors), and two-dimensional arrays (also known as matrices).

A column used as input by a predictive model. When performing predictive modeling, columns in the data are divided into Attributes (which provide the inputs) and the Label (which represents the output of the model).

An algorithm which chooses bin thresholds (see binning) so that the distribution of labels within different bins are as different as possible. Formally, this algorithm minimizes the entropy (see Entropy) within each bin.

Backfitting applies the full dataset to a model built from a smaller sample. The resulting model retains its original structure. The purpose of backfitting is to allow a model's internal distributions to accurately reflect the full data, while maintaining the model's error estimate.

Binning partitions continuous data into discrete groups, transforming real data into categorical data. For example, a continuous range of ages can be binned as 0-18, 19-25, 26-35, and so on. These groups or bins are defined as regions of a continuum that do not overlap.

Segmented blocks displayed in a visualization showing a distribution of data. Similar to pie charts, but square.

The choice point is a value between zero and one which guides, for instance, the selection of the number of clusters; higher choice points suggest a larger number of clusters while lower choice points suggest smaller. A choice point of 1.0 will always pick the upper boundary. In clustering, if your boundaries are one and five clusters, a choice point of 0.4 might pick two clusters while a choice point of 0.8 will pick four. A choice point of 1.0 always picks five.

A predictive model that attempts to describe one column (the label) in terms of others (the attributes). A classifier is constructed from data where the label is known, and may be later applied to predict label values for new data where the label is unknown. Internally, a classifier is an algorithm or mathematical formula that predicts one discrete value for each input row. For example, a classifier built from a dataset of iris flowers could predict the type of a presented iris given the length and width of its petals and stamen). Classifiers may also produce probability estimates for each value of the label. For example, a classifier built from a dataset of cars could predict the probability that a specific car was built in the United States. Classifiers only work with discrete labels. A Regressor is the analog of a Classifier for continuous labels.

Clustering determines which elements in a dataset are similar. It works to group records together according to an algorithm or mathematical formula that attempts to find centroids, or centers, around which similar records gravitate. It is the process of dividing a dataset into mutually exclusive subgroups, without relying on predefined classes.

The probability of some event (A) given that some other event (B) has occurred. Written P(A|B) and read “probability of A given B. Conditional probability is shown by the rectangular charts in the Evidence Visualizer's left-hand window, which show the relative probability of each attribute value given (conditioned on) each label value. Conditional probability can be thought of as evidence for a given label value.

An attribute which can take on a full range of continuous values. In MineSet, columns of types int, double, and float may be considered continuous.

Confidence of an Association Rule (X --> Y), quantifies how often X and Y occur together as a fraction of the number of records in which X occurs. For example, if the confidence is 50%, X and Y occur together in 50% of the records in which X occurs. Thus, knowing that X occurs in a record, the probability that Y also occurs in that record is 50%.

A file that identifies how the data from the .data file is to be mapped to visualizer requirements. MineSet configuration files have the following possible suffixes: .treeviz, .scatterviz, .splatviz, .mapviz, .eviviz, .dtableviz, .statviz, and .clusterviz

A method of estimating predictive error. Cross validation splits that dataset into k equal-sized pieces called folds (typically 10). k predictive models are built, each tested on a distinct fold after being trained on the remaining folds. The process can be repeated multiple times to increase the reliability of the estimate.

Data cleaning is the process of improving the quality of the data by modifying its form or content, for example, by removing or correcting erroneous data values.

A data file for MineSet is a tab-separated flat file of original data from which the model or visualization is created. MineSet's data files use the .data suffix. They may have been in binary (smaller and faster) or ASCII (human readable) format.

A dataset is a table having rows and columns of data. The rows are sometimes referred to as records or instances. The MineSet CD includes several sample datasets, such as churn for people interested in telecommunications habits, cars which is of general interest, adult which is a dataset based on information culled from the US Census Bureau. A dataset can also be described as a schema and set of instances matching the schema. Generally, no ordering on instances is assumed.

A predictive model based on a branching series of tests. Each test examines the value of a single column in the data and uses it to determine the next test to apply. The results of all tests determine which label to predict.

A column with a finite number of distinct values. In MineSet, strings, binned values, and dates are considered discrete. Ints may also be considered discrete if used as the label for a classifier.

The act of retrieving the portion of the original data upon which a visualization is based.

A measure from Information Theory which refers to the degree of disorder of data. The higher the entropy of a dataset, the more diverse and mixed its values. Many of the mining algorithms in MineSet operate by dividing data so as to minimize entropy.

The rate of correct or incorrect predictions made by the model over the dataset. Error rate is usually determined by using an independent test set that was not used at any time during the learning process. More complex accuracy estimation techniques, such as cross-validation, are sometimes used, especially with datasets containing a small number of instances.

The degree to which a probability estimate affects an eventual prediction. The Evidence classifier predicts its label values based on the sum total of evidence from each attribute. Formally, evidence is the negative log of the normalized conditional probability.

Expected confidence is the frequency of occurrence of the RHS item in the dataset. So the difference between expected confidence and confidence is a measure of the change in predictive power due to the presence of the LHS item. Expected confidence gives an indication of what the confidence would be if there were no relationship between the items.

In MineSet, a filter usually a boolean expression used to select or manipulate column contents. For example, `m.p.g. < 25' means to select only those entries showing less than 25 miles per gallon (that is, those records for which this expression evaluates to true). A filter is a saved set of chosen criteria that specify a subset of records in a dataset.

The fraction of the dataset (commonly two-thirds) which is used to train a model so that the remainder may be set aside for testing. The holdout method is the simplest form of error estimation supported by MineSet.

An hypothesis is a proposed explanation whose validity can be tested. Hypothesis testing is a top-down approach that attempts to validate or disprove preconceived ideas. In creating an hypothesis, one thinks up a possible explanation for a remarked behavior. By extension, the hypothesis dictates the data selected to be analyzed.

An algorithm which builds a model from training data. MineSet supports four classification inducers (Decision Tree, Option Tree, Evidence, Decision Table), and one regression inducer (Regression Tree).

A term for a record or row.

Knowledge discovery is the process of finding novel, interesting, and useful patterns in data. Data mining is a subset of knowledge discovery. It lets the data suggest new hypotheses to test.

The single column which a predictive model is attempting to predict. The label must be selected in advance by the user. For example, “iris type” is a good candidate for the label in the iris dataset.

The ratio of confidence to expected confidence of a rule created by the Association Rules Generator. Generally, the greater this number, the more interesting the rule.

A method for assessing predictive accuracy of a model. The lift curve visually displays the advantage a classifier provides over random determination of the label.

The mean may be thought of as the number found by adding the values of data in a sample, then dividing by the number of pieces of data. The mean is usually denoted by x-bar, and is popularly called “average.” (For example, the mean of (1 +2 +3) /3 is 2.)

The middle number when numbers are arranged in order of size.

A description based on observed phenomena, often omitting certain details. A model may imply prediction. For example, if a direct mail model is formed it implies “This is what we think a direct mail customer looks like.” The learner can change values and observe the effects of the change on the operation of the system. See also Classifier.

An alternative term for Confidence.

An alternative term for Support.

The prior probability for a class label is the probability of seeing this label in the data for a randomly chosen record, ignoring all attribute values. Mathematically, this is the number of records with the class label divided by the total number of records. (See also Conditional Probability. Prevalence was called support in previous versions of MineSet.)

Purity in the Column Importance context is a measure of correctness of the label value distribution. The cumulative purity is a measure of how well the data is partitioned in reference to the label values. The data is partitioned using columns found as important in the same way data is partitioned in a Decision Tree. Each set in the partition has its own purity measure, and the purity measure within the partition is a combination of these individual measures. For a given set in the partition, the purity is 0 if each class has equal representation, and 100 if every record is of the same class. Similarly, the cumulative purity will be 0 if each set in the partition has an equal representation of classes, and 100 if each set in the partition contains record that all have the same class. In MineSet, purity is based on Entropy.

Random seed is a method of selecting a particular starting point for selecting a random sample data. When you want a different random sample, specify a different random seed. Using the same seed for different mining explorations of a dataset allows you to work with the same random sample each time. Change the seed when you want to test the stability of a particular mode discovered during exploration.

The difference between the largest and smallest possible value of a column.

A predictive model where the label takes on a continuous value. Regressors are very similar to classifiers.

Relational databases are at the heart of the data warehouse. Data and relations between them are organized into tables—collections of records with each entry containing the same fields. Some fields are designated as keys, so that searches indexed by specific values for key fields retrieve data rapidly. Records in different tables may be linked if they have the same value in a certain field. Examples are Sybase, Informix, OLEDB, SQL server, and Oracle.

Rules are prescribed patterns of behavior applied to the handling of data, such a rules of association, or rules of classification. It is also the unique path followed in a decision tree to get from root to leaf in order to classify the records. A rules-based system performs inferences by applying a set of “if-then” rules to a set of facts following a given procedure.

A financial term which measures the worth of a project by measuring what benefits (return) accrue from an investment. Also known as ROI.

A record in a relational table.

A description of all columns in a dataset. The schema includes a name and type information for each column. Schemas may be stored in files with a .schema suffix.

The lack of symmetry in a frequency distribution.

A measure of dispersion of data. It is defined as the square root of variance.

Given an Association Rule X --> Y (X implies Y), support quantifies how often X and Y occur together in the file as a fraction of the total number of records. For example, if the support is 1%, X and Y occur together in 1% of the total number of records. Support is the same as Prevalence.

A normalized relational dataset.

A test set consists of records in the dataset that have been set aside to test the error rate of a classifier after it has been induced. See also Training Set.

A training set in MineSet is a subset of the data, segmented off before major data manipulation, used to build a classifier or model. It consists of records in the database for which the label has been determined, based on descriptive attributes. It is used by the inducer to learn how to construct the classifier. See also Test Set.

The trimming fraction indicates the fraction of extreme values of the dataset to be excluded from the value range prior to binning. The default trimming fraction is 0.05. This excludes the 5% of the instances with the most extreme values (2.5% with the lowest values in the range, and 2.5% with the highest values in the range). Trimming tends to reduce the influence of outliers on the generation of thresholds.

Uniform range is a selection used in automatic binning of data in which the value range is divided into uniformly sized subintervals.

Uniform weight is a selection used in automatic binning of data in which the value range is divided into the specified number of bins so that each bin contains the same number of records. If record weighting is enabled, the range is divided so that each bin contains the same total weight.

MineSet supports two types of weights:

A record weight is a numerical value applied to each row in the data. The mining tools treats a row with a weight k as if there were k copies of that row. Non-integral weights are supported as well. The weight values are supplied by a user-selected column in the data.

An attribute weight is a user-specified measure of importance for a column. Attribute weights are used by the clustering algorithm to affect the distance metric.