This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Aytug, H. Articles by He, L. Search for Related Content

Risk Minimization and Minimum Description for Linear Discriminant Functions

Information Systems and Operations Management, The Warrington College of Business Administration, University of Florida, Gainesville, Florida 32611
Information Systems and Operations Management, The Warrington College of Business Administration, University of Florida, Gainesville, Florida 32611
Information Systems and Operations Management, Cameron School of Business, University of North Carolina Wilmington, Wilmington, North Carolina 28403
aytugh{at}ufl.edu
koehler{at}ufl.edu
hel{at}uncw.edu

Statistical learning theory provides a formal criterion for learning a concept from examples. This theory addresses directly the trade-off in empirical fit and generalization. In practice, this leads to the structural risk-minimization principle where one minimizes a bound on the overall risk functional. For learning linear discriminant functions, this bound is impacted by the minimum of two terms—the dimension and the inverse of the margin. A popular and powerful learning mechanism, support vector machines, focuses on maximizing the margin. We compare this to methods that focus on minimizing the dimensionality, which, coincidentally, fulfills another useful criterion—the minimum description length principle.

Key words: linear discriminant functions; structural risk-minimization principle; the minimum description length principle; VC-dimension; margin; risk bound; empirical risk; genetic algorithm; support vector machines
History: received July 2003; revised April 2007; accepted October 2007.