Feature selection using SVM and Random Forest -- some preliminary experiments

Yi-Wei Chen and Chih-Jen Lin

After checking some simple feature statistics, we select a subset and train it using random forest (RF).  RF can further select a feature subset which is then trained by SVM with full parameter selection. Radius margin bounds of SVM are also considered at this stage. We submit final results based on the best validation accuracy in the above procedure. Results vary on data sets. For example, on one or two sets, we direct use SVM after simple feature statistics.