Some datasets were taken from the UCI repository (Merz, C. J., and Murphy, P. M. (1996). UCI Repository of machine learning databases, Irvine, CA: University of California, Department of Information and Computer Science) found at http://www.ics.uci.edu/~mlearn/MLRepository.html.
A general and easy-to-read introduction to machine learning is Weiss, S. M., and C. A. Kulikowski. Computer Systems that Learn. San Mateo, CA: Morgan Kaufmann Publishers, Inc., 1991.
An easy-to-read introduction to decision tree induction is Quinlan, J. R. C4.5: Programs for Machine Learning. Los Altos, CA: Morgan Kaufmann Publishers, Inc., 1993.
An excellent book on decision trees from a statistical perspective is Breiman, L., J. H. Friedman, R. A. Olshen, and C.J. Stone. Classification and Regression Trees. Wadsworth International Group, 1984.
A good edited volume of machine learning techniques is Dietterich, T. G. and J. W. Shavlik (Eds). Readings in Machine Learning. Morgan Kaufmann Publishers, Inc., 1990.
A summary of accuracy estimation techniques is given in Kohavi, R. “A study of cross-validation and bootstrap for accuracy estimation and model selection.” In Proceedings of the 14th International Joint Conference on Artificial Intelligence, edited by C. S. Mellish. Morgan Kaufmann Publishers, Inc., 1995.
An excellent introduction to the Evidence Classifier (Naive-Bayes) is Kononenko, I. (1993). Inductive and bayesian learning in medical diagnosis. Applied Artificial Intelligence, pp. 7:317-337.
A good reference to a paper explaining that no classifier can be “best” is Schaffer, C. A conservation law for generalization performance. In Machine Learning: Proceedings of the Eleventh International Conference, 259-265. Morgan Kaufmann Publishers, Inc., 1994.
A general comparison of algorithms and descriptions is provided in Taylor, C., D. Michie, and D. Spiegalhalter. Machine Learning, Neural and Statistical Classification. Paramount Publishing International, 1994.
The following paper describes the wrapper method used to select the features for the Evidence Classifier:
Kohavi, R., Sommerfield, D. (1995). Feature Subset Selection Using the Wrapper Model: Overfitting and Dynamic Search Space Topology. The First International Conference on Knowledge Discovery and Data Mining, pp. 192-197.
The following paper describes the Laplace correction option:
Cestnik, B. (1990). Estimating Probabilities: A crucial Task in Machine Learning. Proceedings of the Ninth European Conference on Artificial Intelligence, pp. 147-149.
The following paper describes the Evidence Classifier (Naive-Bayes):
Langley, P., Iba, W., Thompson, K. (1992). An Analysis of Bayesian Classifiers. Proceedings of the Tenth National Conference on Artificial Intelligence, pp. 223-228.
The following books describe the Evidence Classifier:
Good, I. J. The Estimation of Probabilities: An Essay on Modern Bayesian Methods. MIT Press, 1965.
Duda, R., Hart, P. Pattern Classification and Scene Analysis, Wiley, 1973.
The following paper shows that while the conditional independence assumption can be violated, the classification accuracy of the evidence classifier (called Simple Bayes in this paper) can be good:
Domingos P., Pazzani M (1996). Beyond independence: conditions for the optimality of the simple Bayesian classifier. Machine learning, Proceedings of the 13th International Conference (ICML '96), pp. 105-112.
The iris database (described in Chapter 8, “MineSet Inducers and Classifiers”) was originally used in Fisher, R. A. 1936. The use of multiple measurements in taxonomic problems. Annals of Eugenics 7(1):179-188. It is a classical problem in many statistical texts.
The breast cancer database was obtained from Dr. William H. Wolberg, L. Mangasarian, and W. H. Wolberg. Cancer diagnosis via linear programming. SIAM News 23(5):1 & 18. University of Wisconsin Hospitals, Madison, September 1990.
The data for the mushroom sample file comes from: Audubon Society Field Guide to North American Mushrooms. New York: Alfred A. Knopf, 1981.
The data on congressional voting was taken from the Congressional Quarterly Almanac, 98th Congress, 2nd session 1984, Volume XL, Congressional Quarterly Inc.: Washington, D.C., 1985.