Variable Selection and Feature Construction using methods related to Information Theory
Kari Torkkola, Motorola, Arizona, kari.torkkola@motorola.com

By using information theory, variable selection and feature construction can be viewed as problems in the areas of coding and distortion. Variables or features can be understood as a "noisy channel" that conveys information about the message. The aim would be to select or to construct features that provide as much information as possible about the "message". This talk gives a brief tutorial to the use of information-theoretic concepts as components of various variable selection and feature construction methods, with an emphasis in learning discriminative feature transforms using mutual information between class labels and transformed features as a criterion.