Feature Selection for Machine Learning - Code Repository

Launched: February, 2018

Updated: October, 2024

Actively maintained.

Links

• Online Course

Table of Contents

1. Basic Selection Methods

   1. Removing Constant Features
   2. Removing Quasi-Constant Features
   3. Removing Duplicated Features

2. Correlation Feature Selection

   1. Removing Correlated Features
   2. Basic Selection Methods + Correlation - Pipeline

3. Filter Methods: Statistical Methods

   1. Mutual Information
   2. Chi-square distribution
   3. Anova
   4. Basic Selection Methods + Statistical Methods - Pipeline

4. Filter Methods: Other Methods and Metrics

   1. Univariate roc-auc, mse, etc
   2. Method used in a KDD competition - 2009

5. Wrapper Methods

   1. Step Forward Feature Selection
   2. Step Backward Feature Selection
   3. Exhaustive Feature Selection

6. Embedded Methods: Linear Model Coefficients

   1. Logistic Regression Coefficients
   2. Linear Regression Coefficients
   3. Effect of Regularization on Coefficients
   4. Basic Selection Methods + Correlation + Embedded - Pipeline

7. Embedded Methods: Lasso

   1. Lasso
   2. Basic Selection Methods + Correlation + Lasso - Pipeline

8. Embedded Methods: Tree Importance

   1. Random Forest derived Feature Importance
   2. Tree importance + Recursive Feature Elimination
   3. Basic Selection Methods + Correlation + Tree importance - Pipeline

9. Hybrid Feature Selection Methods

   1. Feature Shuffling
   2. Recursive Feature Elimination
   3. Recursive Feature Addition

Links

• Online Course