This chapter discusses the features and capabilities of the Column Importance mining tool, and the relationship between column importance and the importance ranking in the other data mining tools. Because of the differences in representation for classification models, different attributes may be judged more important for different models. A sample file, provided with MineSet, is discussed at the end of this chapter.
Column Importance is run from the tab labelled Col. Imp. on the Mining Tools panel (Figure 17-1). It determines how important various columns are in discriminating the different values of the label column you choose. You might, for example, want to find the best three columns for discriminating the label good credit risk so you can choose them for mapping to axes the Scatter Visualizer. When you select the label and click Go!, a popup window appears with the three columns that are the best three discriminators. A measure called “purity” (a number from 0 to 100) informs you how well the columns discriminate the different labels. Adding more columns can only increase the purity.
Purity is a measure of the skewness of the label value distribution. The cumulative purity measure is a measure of the purity of partioning the data. The data is partioned using columns found as importand in the same way data is partitioned in a Decision Tree. Each set in the partition has its own purity measure, and the purity measure within the partion is a combination of these individual measures. For a given set in the partition, the purity is 0 if each class has equal representation, and 100 if every record is of the same class. Similarly, the cumulative purity will be 0 if each set in the partition has an equal representation of classes, and 100 if each set in the partition contains record that all have the same class.
There are two modes of Column Importance:
Simple Mode
To invoke the Simple mode, choose a discrete label from the popup menu, and specify the number of columns you want to see, then click Go!.
Advanced Mode
Advanced mode lets you control the choice of columns. To enter Advanced mode, click Advanced Mode in the Column Importance panel. A dialog box appears, as shown in Figure 17-2. As with Further Inducer Options (see Chapter 10, “MineSet Inducers and Classifiers”), you can select a weight attribute and decide whether it behaves as a regular attribute for determining importance. The dialog box contains two lists of column names: The left list contains the available attributes and the right list contains attributes chosen as important (by either the user or the Column Importance algorithm).
Advanced mode can work two different ways: finding several new important attributes or ranking available attributes.
Finding Several Important Attributes
To enter this submode, click the first of the two radio buttons in the middle of the dialog (...find [number] additional important attributes). If you click Go! with no further changes, the effect is the same as if you were in Simple mode, finding the specified number of important columns and automatically moving them to the right column. Near each column, the cumulative purity is given (that is, the purity of all the columns up to and including the one on the line.)
Alternatively, by moving column names from the left list to the right list, you can prespecify columns that you want included and let the system add more. For example, to select the cylinders column and let the system find three more columns, click the cylinders column name, then click the right arrow between the lists.
Clicking Go! lets you see the cumulative purity of each column, together with the previous ones in the list. A purity of 100 means that using the given columns, you can perfectly discriminate the different label values in the dataset.
Ranking Available Attributes
Advanced mode also lets you compute the change in purity that each column would add to all those that were already marked important, that is, they are in the list on the right. For example, you might move cylinders to the list on the right, and then ask the system to compute the incremental improvement in purity that each column remaining in the left column would yield. The cumulative purity is computed for columns on the right (already marked important).
To enter this submode, click the second of the two radio buttons at the bottom of the dialog (...compute improved purity for left columns, cumulative purity for right columns.). This submode permits fine control over the process. If two columns are ranked very closely, you might prefer one over the other (for example, because it is cheaper to gather, more reliable, or easier to understand).
The importance of a column depends on the columns previously marked as important. For example, while net-income might be a good column individually, it might not be as important together with salary because they are likely to be highly correlated. The best set of three columns is not necessarily composed of the columns that rank highest individually. If two columns give the income in dollars and in another currency, they are ranked equally alone; however, once one of them is chosen, the other adds no discriminatory power to the set of best features.
Column selection is useful for finding the best three axes for Scatter Visualizer and Splat Visualizer. It is also useful for finding a good discriminatory hierarchy (hierarchy that separates different label values) for the Tree Visualizer when you select the label to be the key used in the Tree Visualizer.
All floating point values (doubles or floats) are prediscretized using automatic discretization (see Chapter 3, “The Tool Manager”). If a column has no value given to it in the left list, the algorithm did not consider it; this is because it either had a single value (for example, when it is discretized into one interval), or the number of records that it would separate is not statistically significant.
This section describes the differences among Column Importance, the importance ranking chosen by the Evidence Inducer, and the splits chosen by the Decision Tree Inducer. As Column Importance uses all of the data, these descriptions assume that you are running the inducers in “Classifier Only” mode, so that the inducers are using all of the data as well.
The Column Importance algorithm and the Evidence Inducer discretize all continuous attributes using the automatic discretization algorithm (the same algorithm that is applied in automatic binning in the Tool Manager). The Decision Tree algorithm does not pre-discretize attributes (columns) and finds thresholds as the tree is built.
The main advantage of the automatic discretization is that it discretizes the continuous range into several intervals at once, while the Decision Tree makes only binary splits.
The main advantage of the Decision Tree algorithm is that it discretizes subsets of the data (those that reach a specific node where a test is done). Thus the discretization is “local” to those records as opposed to a “global” discretization.
The Evidence Inducer and Column Importance rank attributes based on “mutual information” as the purity measure. The Decision Tree Inducer, Option Tree Inducer, and Decision Table Inducer defaults to “normalized mutual information,” which penalizes multi-way splits (see the description of splitting criterion in Chapter 11, “Inducing and Visualizing the Decision Tree Classifier”). Thus, the Decision Tree Inducer prefers an attribute with few values over attributes with many values. The default for Decision Trees can be changed to “mutual information.”
The Evidence Inducer ranks each attribute independently. If several attributes are highly correlated, they have similar ranking. If you use the Advanced mode from Column Importance, and the “...compute improved purity” option without any attributes chosen as important (that is, moved to the list on the right), the attribute ranking shown matches the sort order chosen by the Evidence Inducer.
The Column Importance tool, the Decision Tree Inducer, and the Decision Table Inducer all provide more powerful importance capabilities than the Evidence Inducer. All choose an importance ranking with respect to other columns.
In Column Importance, columns are judged as important relative to the set of columns in the list on the right. If two columns are highly correlated and one is chosen, the other will probably never be chosen; each column is chosen for its ability to provide more information about the label than is already present.
In the Decision Table Inducer, the Suggest button provides an importance facility similar to the column importance tool. Columns are judged as important relative to the set of columns already present in the mapping box. There are three major differences, however, between the Decision Table Inducer's suggest mode and the column importance tool. First, the Decision Table Inducer penalizes multi-way splits. Second, the Decision Table Inducer is capable of performing a more exhaustive search to find a better column ordering. Finally, the Decision Table Inducer does not report any purity scores—it merely ranks the columns.
The Decision Tree Inducer provides a more flexible importance ranking because different columns can be selected at different subtrees. For example, one column can be chosen for the left child of the root and another for the right child of the root. While this is appropriate for a Decision Tree, it is inappropriate for choosing a small set of columns to show in the Scatter Visualizer or Splat Visualizer. For these cases, the Column Importance tool is superior because it builds an “oblivious” Decision Tree in which every level of the tree tests the same column across the nodes. With Column Importance, a single column must be chosen for all combinations of the previously chosen columns.
The following example shows a case in which Column Importance might be useful. This example is associated with a sample dataset provided with MineSet. It shows how to work with the Column Importance mining tool, and explains the different outcomes and options.
When customers change their phone carrier from one telecommunications company to another, this is termed “churning.” This is a common problem in the telecommunications industry. The data used to generate this example is in /usr/lib/MineSet/data/churn.schema. The Column Importance mining tool lets you look at properties of this file.
Running the simple Column Importance mode yields the following three attributes:
Total Day Charge.
Number of customer service calls.
State.
By running “compute improved purity” from the advanced mode, you can see that Total Day Charge and Total Day Minutes have the same purity ranking (48.67). By moving one of them to the right (for example, Total Day Minutes) and rerunning Compute Improved Purity, you can see that there is no value to the other (Total Day Charge). These two attributes are highly correlated.
Looking at the attributes when Total Day Minutes is on the right, we can see that the following are good:
International plan (4.1)
Number of Customer Service Calls (8.1).
State (4.7)
You can choose to move International Plan to the right, because this information is readily available and easy to measure.
The other two attributes (Number of Customer Service Calls and State) remain highly important (in fact, their importance increases), so they are apparently not correlated with the International Plan.
By looking at the importance of attributes this way, you can determine which ones can be substituted with others that are equally good (or almost as good), but are easier to measure or understand. By looking at the purity, you can determine how much the additional attributes help. For example, in the above scenario, state significantly improves the purity. In the iris dataset, the third attribute chosen (sepal length) raises the purity only slightly higher. In some cases, the simpler, two-dimensional scatterplot might be easier to understand.