Variable Selection and Feature Construction using methods related to Information
Theory
Kari Torkkola, Motorola, Arizona, kari.torkkola@motorola.com
By using information theory, variable selection and feature construction
can be viewed as problems in the areas of coding and distortion. Variables
or features can be understood as a "noisy channel" that conveys information
about the message. The aim would be to select or to construct features that
provide as much information as possible about the "message". This talk gives
a brief tutorial to the use of information-theoretic concepts as components
of various variable selection and feature construction methods, with an emphasis
in learning discriminative feature transforms using mutual information between
class labels and transformed features as a criterion.