Recursive Feature Elimination (RFE) - Revision history

BPeat at 00:08, 28 September 2020

2020-09-28T00:08:20Z

BPeat at 19:37, 27 September 2020

2020-09-27T19:37:09Z

BPeat at 23:17, 21 September 2020

2020-09-21T23:17:25Z

BPeat at 19:51, 21 September 2020

2020-09-21T19:51:26Z

BPeat at 00:32, 20 September 2020

2020-09-20T00:32:20Z

BPeat at 05:05, 19 September 2020

2020-09-19T05:05:55Z

BPeat at 11:45, 2 June 2020

2020-06-02T11:45:17Z

BPeat at 13:12, 30 May 2020

2020-05-30T13:12:06Z

BPeat: /* Recursive Feature Elimination with Cross-Validation (RFECV) */

2020-05-30T13:10:58Z

‎Recursive Feature Elimination with Cross-Validation (RFECV)

BPeat at 12:51, 30 May 2020

2020-05-30T12:51:40Z

@@ Line 9: / Line 9: @@
 * [http://www.kdnuggets.com/2018/10/notes-feature-preprocessing-what-why-how.html Notes on Feature Preprocessing: The What, the Why, and the How | Matthew Mayo - KDnuggets]
-* [[Automated Machine Learning (AML) - AutoML]]
+* [[Algorithm Administration#Automated Learning|Automated Learning]]
 * [[Principal Component Analysis (PCA)]]
 * [[Data Science]]

@@ Line 18: / Line 18: @@
 **** [[Data Quality]] ...[[AI Verification and Validation|validity]], [[Evaluation - Measures#Accuracy|accuracy]], [[Data Quality#Data Cleaning|cleaning]], [[Data Quality#Data Completeness|completeness]], [[Data Quality#Data Consistency|consistency]], [[Data Quality#Data Encoding|encoding]], [[Data Quality#Zero Padding|padding]], [[Data Quality#Data Augmentation, Data Labeling, and Auto-Tagging|augmentation, labeling, auto-tagging]], [[Data Quality#Batch Norm(alization) & Standardization| normalization, standardization]], and [[Data Quality#Imbalanced Data|imbalanced data]]
 * [[Visualization]]
-* [[Hyperparameter]]s
+* [[Algorithm Administration#Hyperparameter|Hyperparameter]]s
 * [[Evaluating Machine Learning Models]]
 * [http://machinelearningmastery.com/rfe-feature-selection-in-python/ Recursive Feature Elimination (RFE) for Feature Selection in Python | Jason Brownlee - Machine Learning Mastery]

@@ Line 15: / Line 15: @@
 *** [[Benchmarks]]
 *** [[Data Preprocessing]]
-**** [[Feature Exploration/Learning]] ...inspection, data profiling, selection
+**** [[Feature Exploration/Learning]]
 **** [[Data Quality]] ...[[AI Verification and Validation|validity]], [[Evaluation - Measures#Accuracy|accuracy]], [[Data Quality#Data Cleaning|cleaning]], [[Data Quality#Data Completeness|completeness]], [[Data Quality#Data Consistency|consistency]], [[Data Quality#Data Encoding|encoding]], [[Data Quality#Zero Padding|padding]], [[Data Quality#Data Augmentation, Data Labeling, and Auto-Tagging|augmentation, labeling, auto-tagging]], [[Data Quality#Batch Norm(alization) & Standardization| normalization, standardization]], and [[Data Quality#Imbalanced Data|imbalanced data]]
 * [[Visualization]]

@@ Line 16: / Line 16: @@
 *** [[Data Preprocessing]]
 **** [[Feature Exploration/Learning]] ...inspection, data profiling, selection
-**** [[Data Quality]] ...[[AI Verification and Validation|validity]], accuracy, [[Data Quality#Data Cleaning|cleaning]], completeness, consistency, uniformity, [[Data Quality#Data Encoding|encoding]], [[Data Quality#Zero Padding|padding]],   [[Data Quality#Data Augmentation, Data Labeling, and Auto-Tagging|augmentation, labeling, auto-tagging]], [[Data Quality#Batch Norm(alization) & Standardization| normalization, standardization]], and [[Data Quality#Imbalanced Data|imbalanced data]]
+**** [[Data Quality]] ...[[AI Verification and Validation|validity]], [[Evaluation - Measures#Accuracy|accuracy]], [[Data Quality#Data Cleaning|cleaning]], [[Data Quality#Data Completeness|completeness]], [[Data Quality#Data Consistency|consistency]], [[Data Quality#Data Encoding|encoding]], [[Data Quality#Zero Padding|padding]], [[Data Quality#Data Augmentation, Data Labeling, and Auto-Tagging|augmentation, labeling, auto-tagging]], [[Data Quality#Batch Norm(alization) & Standardization| normalization, standardization]], and [[Data Quality#Imbalanced Data|imbalanced data]]
 * [[Visualization]]
 * [[Hyperparameter]]s

@@ Line 11: / Line 11: @@
 * [[Automated Machine Learning (AML) - AutoML]]
 * [[Principal Component Analysis (PCA)]]
-* [[Datasets]]
+* [[Data Science]]
-* [[Batch Norm(alization) & Standardization]]
+** [[Data Governance]]
-* [[Data Preprocessing]]
+*** [[Benchmarks]]
 * [[Hyperparameter]]s
 * [[Evaluating Machine Learning Models]]
 * [http://machinelearningmastery.com/rfe-feature-selection-in-python/ Recursive Feature Elimination (RFE) for Feature Selection in Python | Jason Brownlee - Machine Learning Mastery]

← Older revision		Revision as of 13:10, 30 May 2020
Line 33:		Line 33:
	* Feature — individual measurable property or characteristic of a phenomenon being observed — attribute in your dataset [http://en.wikipedia.org/wiki/Feature_(machine_learning) Wikipedia]		* Feature — individual measurable property or characteristic of a phenomenon being observed — attribute in your dataset [http://en.wikipedia.org/wiki/Feature_(machine_learning) Wikipedia]

−	* Cross-Validation — a technique for evaluating ML models by training several ML models on subsets of the available input data and evaluating them on the complementary subset of the data. Use cross-validation to detect overfitting, ie, failing to generalize a pattern [http://docs.aws.amazon.com/machine-learning/latest/dg/cross-validation.html Developer Guide - Amazon AWS ML]	+	* [[Cross-Validation]] — a technique for evaluating ML models by training several ML models on subsets of the available input data and evaluating them on the complementary subset of the data. Use cross-validation to detect overfitting, ie, failing to generalize a pattern [http://docs.aws.amazon.com/machine-learning/latest/dg/cross-validation.html Developer Guide - Amazon AWS ML]

@@ Line 23: / Line 23: @@
 ..recursive feature elimination (RFE, Guyon et al. (2002)) is basically a backward selection of the predictors. This technique begins by building a model on the entire set of predictors and computing an importance score for each predictor. The least important predictor(s) are then removed, the model is re-built, and importance scores are computed again. In practice, the analyst specifies the number of predictor subsets to evaluate as well as each subset’s size. Therefore, the subset size is a tuning parameter for RFE. The subset size that optimizes the performance criteria is used to select the predictors based on the importance rankings. The optimal subset is then used to train the final model. [http://bookdown.org/max/FES/recursive-feature-elimination.html Feature Engineering and Selection: A Practical Approach for Predictive Models | Max Kuhn and Kjell Johnson]
-The main issue of RFE is that it can be expensive to run — so you should do anything you can to reduce the number of features beforehand. Removing correlated features is a great way to do so because as you probably know, you don’t want highly correlated features in your dataset because they provide the same information — one is enough.  [http://towardsdatascience.com/feature-selection-in-python-recursive-feature-elimination-19f1c39b8d15 Feature Selection in Python — Recursive Feature Elimination | Dario Radečić - Towards Data Science]
 == Recursive Feature Elimination with Cross-Validation (RFECV) ==
-Recursive — involving doing or saying the same thing several times in order to produce a particular result or effect. [http://dictionary.cambridge.org/dictionary/english/recursive Cambridge dictionary]
+The main issue of RFE is that it can be expensive to run — so you should do anything you can to reduce the number of features beforehand. Removing correlated features is a great way to do so because as you probably know, you don’t want highly correlated features in your dataset because they provide the same information — one is enough.  [http://towardsdatascience.com/feature-selection-in-python-recursive-feature-elimination-19f1c39b8d15 Feature Selection in Python — Recursive Feature Elimination | Dario Radečić - Towards Data Science]
-Feature — individual measurable property or characteristic of a phenomenon being observed — attribute in your dataset [http://en.wikipedia.org/wiki/Feature_(machine_learning) Wikipedia]
+* Feature — individual measurable property or characteristic of a phenomenon being observed — attribute in your dataset [http://en.wikipedia.org/wiki/Feature_(machine_learning) Wikipedia]
-Cross-Validation — a technique for evaluating ML models by training several ML models on subsets of the available input data and evaluating them on the complementary subset of the data. Use cross-validation to detect overfitting, ie, failing to generalize a pattern [http://docs.aws.amazon.com/machine-learning/latest/dg/cross-validation.html Developer Guide - Amazon AWS ML]
+* Cross-Validation — a technique for evaluating ML models by training several ML models on subsets of the available input data and evaluating them on the complementary subset of the data. Use cross-validation to detect overfitting, ie, failing to generalize a pattern [http://docs.aws.amazon.com/machine-learning/latest/dg/cross-validation.html Developer Guide - Amazon AWS ML]