Difference between revisions of "Data Preprocessing"
(→SQL Database Optimization) |
|||
| Line 35: | Line 35: | ||
[http://www.researchgate.net/publication/49849827_Comparison_of_Multivariate_Data_Analysis_Strategies_for_High-Content_Screening/figures?lo=1 Article] | [http://www.researchgate.net/publication/49849827_Comparison_of_Multivariate_Data_Analysis_Strategies_for_High-Content_Screening/figures?lo=1 Article] | ||
| − | |||
<youtube>cw2LvVkmtkQ</youtube> | <youtube>cw2LvVkmtkQ</youtube> | ||
<youtube>TK-2189UcKk</youtube> | <youtube>TK-2189UcKk</youtube> | ||
| Line 43: | Line 42: | ||
<youtube>UuktvBOKEcE</youtube> | <youtube>UuktvBOKEcE</youtube> | ||
<youtube>WeXXtBNtwxk</youtube> | <youtube>WeXXtBNtwxk</youtube> | ||
| + | <youtube>0xVqLJe9_CY</youtube> | ||
== Splitting Data - training and testing sets == | == Splitting Data - training and testing sets == | ||
| Line 63: | Line 63: | ||
http://azurecomcdn.azureedge.net/mediahandler/acomblog/media/Default/blog/578a09a1-f144-4a62-98cb-e6e3ed774817.png | http://azurecomcdn.azureedge.net/mediahandler/acomblog/media/Default/blog/578a09a1-f144-4a62-98cb-e6e3ed774817.png | ||
| + | |||
| + | == Categorical Variables == | ||
| + | * [https://towardsdatascience.com/all-about-categorical-variable-encoding-305f3361fd02 All about Categorical Variable Encoding | Baijayanta Roy - Towards Data Science] | ||
| + | |||
| + | Categorical variables require special attention in regression analysis because, unlike dichotomous or continuous variables, they cannot by entered into the regression equation just as they are. Instead, they need to be recoded into a series of variables which can then be entered into the regression model. There are a variety of coding systems that can be used when recoding categorical variables. [https://stats.idre.ucla.edu/spss/faq/coding-systems-for-categorical-variables-in-regression-analysis-2/#:~:text=Categorical%20variables%20require%20special%20attention,equation%20just%20as%20they%20are.&text=For%20example%2C%20you%20may%20want,(or%20any%20given%20level). Coding Systems for Categorical Variables In Regression Analysis | UCLA institute for Digital Research & Education Statistical Consulting] | ||
| + | |||
| + | <youtube>YCdKs61ClV4</youtube> | ||
| + | <youtube>7TKnHHMBTok</youtube> | ||
| + | |||
== SQL Database Optimization == | == SQL Database Optimization == | ||
Revision as of 09:06, 29 June 2020
YouTube search... ...Google search
- Data Cleaning
- Sklearn.preprocessing
- The Passenger Screening Kaggle challenge 1st place solution was won in part due to data preparation/generation.
- Data Pre Processing Techniques You Should Know | Maneesha Rajaratne - Towards Data Science
- Machine Learning(ML) — Data Preprocessing | Raji Adam Bifola
- Most Influential Data Preprocessing Algorithms | S. García, J. Luengo, F. Herrera
- How to fix an Unbalanced Dataset | Will Badr - Amazon Web Services
- Creating and Using Datasources | Amazon Web Services
- Datasets
- Imbalanced Data
- Data Encoding
- Batch Norm(alization) & Standardization
- Feature Exploration/Learning
- Hyperparameters
- Data Augmentation, Data Labeling, and Auto-Tagging
- Visualization
- Python
- Master Data Management (MDM) / Feature Store / Data Lineage / Data Catalog
- Jon Tupitza Famous Jupyter Notebooks:
- The COVID Tracking Project - software used
Contents
Splitting Data - training and testing sets
Time-Series Data
- Time-based Algorithms
- A Comparison of Time Series Databases and Netsil’s Use of Druid | Netsil
- Microsoft announces the general availability of Azure Time Series Insights | Ryan Waite - Microsoft
- Top 10 Time Series Databases | Outlyer
Categorical Variables
Categorical variables require special attention in regression analysis because, unlike dichotomous or continuous variables, they cannot by entered into the regression equation just as they are. Instead, they need to be recoded into a series of variables which can then be entered into the regression model. There are a variety of coding systems that can be used when recoding categorical variables. Coding Systems for Categorical Variables In Regression Analysis | UCLA institute for Digital Research & Education Statistical Consulting
SQL Database Optimization
