Difference between revisions of "Benchmarks"
| Line 16: | Line 16: | ||
* [http://www.datasciencecentral.com/group/resources/forum/topics/benchmarking-20-machine-learning-models-accuracy-and-speed Benchmarking 20 Machine Learning Models Accuracy and Speed | Marc Borowczak - Data Science Central] | * [http://www.datasciencecentral.com/group/resources/forum/topics/benchmarking-20-machine-learning-models-accuracy-and-speed Benchmarking 20 Machine Learning Models Accuracy and Speed | Marc Borowczak - Data Science Central] | ||
* [http://www.sciencedirect.com/science/article/pii/S1532046418300716 Benchmarking deep learning models on large healthcare datasets | S. Purushotham, C. Meng, Z. Chea, and Y. Liu] | * [http://www.sciencedirect.com/science/article/pii/S1532046418300716 Benchmarking deep learning models on large healthcare datasets | S. Purushotham, C. Meng, Z. Chea, and Y. Liu] | ||
| − | |||
<img src="http://www.researchgate.net/profile/Benoit_Gallix/publication/324457640/figure/fig1/AS:622298201595905@1525378861825/Graph-illustrating-the-impact-of-data-available-on-performance-of-traditional-machine.png" width="500" height="400"> | <img src="http://www.researchgate.net/profile/Benoit_Gallix/publication/324457640/figure/fig1/AS:622298201595905@1525378861825/Graph-illustrating-the-impact-of-data-available-on-performance-of-traditional-machine.png" width="500" height="400"> | ||
| Line 25: | Line 24: | ||
| − | == GLUE == | + | == <span id="GLUE"></span>GLUE == |
| − | * [http://gluebenchmark.com/ GLUE] | + | * [http://gluebenchmark.com/ General Language Understanding Evaluation (GLUE)] |
The General Language Understanding Evaluation (GLUE) benchmark is a collection of resources for training, evaluating, and analyzing natural language understanding systems. GLUE consists of: | The General Language Understanding Evaluation (GLUE) benchmark is a collection of resources for training, evaluating, and analyzing natural language understanding systems. GLUE consists of: | ||
| Line 36: | Line 35: | ||
| − | == MLPerf == | + | == <span id="MLPerf"></span>MLPerf == |
* [http://mlperf.org/ MLPerf] benchmarks for measuring training and inference performance of ML hardware, software, and services. | * [http://mlperf.org/ MLPerf] benchmarks for measuring training and inference performance of ML hardware, software, and services. | ||
| Line 43: | Line 42: | ||
<youtube>0Fuxjq1eiZ4</youtube> | <youtube>0Fuxjq1eiZ4</youtube> | ||
<youtube>sH03-InVba4</youtube> | <youtube>sH03-InVba4</youtube> | ||
| + | |||
| + | == <span id="SQuAD"></span>The Stanford Question Answering Dataset (SQuAD) == | ||
| + | * [http://rajpurkar.github.io/SQuAD-explorer/ The Stanford Question Answering Dataset (SQuAD)] | ||
| + | |||
| + | == <span id="RACE"></span>ReAding Comprehension (RACE) == | ||
| + | * [http://www.qizhexie.com/data/RACE_leaderboard.html ReAding Comprehension {RACE)] | ||
Revision as of 14:04, 24 December 2019
YouTube search... ...Google search
- Evaluation Measures - Classification Performance - Accuracy, Precision & Recall (Sensitivity), and Specificity
- Datasets
- Gaming
- Machine Learning Benchmarks and AI Self-Driving Cars | Lance Eliot - AItrends
- Benchmarking simple models with feature extraction against modern black-box methods | Martin Dittgen - Towards Data Science
- DAWNBench | Stanford - an End-to-End Deep Learning Benchmark and Competition
- Benchmarking 20 Machine Learning Models Accuracy and Speed | Marc Borowczak - Data Science Central
- Benchmarking deep learning models on large healthcare datasets | S. Purushotham, C. Meng, Z. Chea, and Y. Liu
Contents
GLUE
The General Language Understanding Evaluation (GLUE) benchmark is a collection of resources for training, evaluating, and analyzing natural language understanding systems. GLUE consists of: A benchmark of nine sentence- or sentence-pair language understanding tasks built on established existing datasets and selected to cover a diverse range of dataset sizes, text genres, and degrees of difficulty, A diagnostic dataset designed to evaluate and analyze model performance with respect to a wide range of linguistic phenomena found in natural language, and A public leaderboard for tracking performance on the benchmark and a dashboard for visualizing the performance of models on the diagnostic set.
MLPerf
- MLPerf benchmarks for measuring training and inference performance of ML hardware, software, and services.
