Recursive Feature Elimination (RFE)

YouTube search... ...Google search

• Notes on Feature Preprocessing: The What, the Why, and the How | Matthew Mayo - KDnuggets
• Automated Learning
• Principal Component Analysis (PCA)
• Data Science
  • Data Governance
    • Benchmarks
    • Data Preprocessing
      • Feature Exploration/Learning
      • Data Quality ...validity, accuracy, cleaning, completeness, consistency, encoding, padding, augmentation, labeling, auto-tagging, normalization, standardization, and imbalanced data
• Visualization
• Hyperparameters
• Evaluating Machine Learning Models
• Recursive Feature Elimination (RFE) for Feature Selection in Python | Jason Brownlee - Machine Learning Mastery

..recursive feature elimination (RFE, Guyon et al. (2002)) is basically a backward selection of the predictors. This technique begins by building a model on the entire set of predictors and computing an importance score for each predictor. The least important predictor(s) are then removed, the model is re-built, and importance scores are computed again. In practice, the analyst specifies the number of predictor subsets to evaluate as well as each subset’s size. Therefore, the subset size is a tuning parameter for RFE. The subset size that optimizes the performance criteria is used to select the predictors based on the importance rankings. The optimal subset is then used to train the final model. Feature Engineering and Selection: A Practical Approach for Predictive Models | Max Kuhn and Kjell Johnson

Recursive Feature Elimination with Cross-Validation (RFECV)

The main issue of RFE is that it can be expensive to run — so you should do anything you can to reduce the number of features beforehand. Removing correlated features is a great way to do so because as you probably know, you don’t want highly correlated features in your dataset because they provide the same information — one is enough. Feature Selection in Python — Recursive Feature Elimination | Dario Radečić - Towards Data Science

• Recursive — involving doing or saying the same thing several times in order to produce a particular result or effect. Cambridge dictionary

• Feature — individual measurable property or characteristic of a phenomenon being observed — attribute in your dataset Wikipedia

• Cross-Validation — a technique for evaluating ML models by training several ML models on subsets of the available input data and evaluating them on the complementary subset of the data. Use cross-validation to detect overfitting, ie, failing to generalize a pattern Developer Guide - Amazon AWS ML