Difference between revisions of "Bias and Variances"

From
Jump to: navigation, search
m
(22 intermediate revisions by the same user not shown)
Line 8: Line 8:
 
[http://www.google.com/search?q=Bias+Variance+deep+machine+learning+ML+artificial+intelligence ...Google search]
 
[http://www.google.com/search?q=Bias+Variance+deep+machine+learning+ML+artificial+intelligence ...Google search]
  
 +
* [[Evaluation]]
 +
** [[Evaluation - Measures]]
 +
*** [[Evaluation - Measures#Accuracy|Accuracy]]
 +
*** [[Evaluation - Measures#Precision & Recall (Sensitivity)|Precision & Recall (Sensitivity)]]
 +
*** [[Evaluation - Measures#Specificity|Specificity]]
 +
** [[Benchmarks]]
 +
** [[Explainable / Interpretable AI]]
 +
** [[Train, Validate, and Test]]
 +
** [[Algorithm Administration#Model Monitoring|Model Monitoring]]
 
* [http://becominghuman.ai/a-basic-recipe-for-machine-learning-2fbebd3549f5 A Basic Recipe for Machine Learning | Hafidz Zulkifli]
 
* [http://becominghuman.ai/a-basic-recipe-for-machine-learning-2fbebd3549f5 A Basic Recipe for Machine Learning | Hafidz Zulkifli]
 
* [http://en.wikipedia.org/wiki/Inductive_bias Inductive Bias | Wikipedia]
 
* [http://en.wikipedia.org/wiki/Inductive_bias Inductive Bias | Wikipedia]
* [[Evaluation Measures - Classification Performance]]
+
* [[Overfitting Challenge]]
* [http://www.technologyreview.com/s/612876/this-is-how-ai-bias-really-happensand-why-its-so-hard-to-fix/ This is how AI bias really happens—and why it’s so hard to fix | Karen Hao]
+
* [[Data Science]]
* [http://www.analyticsindiamag.com/top-6-common-statistical-errors-made-data-scientists/ Top 6 most common statistical errors made by data scientists | Richa Bhatia - Analytics India]
+
 
 +
Two terms of <b>bias</b> in Artificial Intelligence:
 +
 
 +
# <b>Node Bias</b> - The activation of a node in a neural network is determined by the following: output = activation function (dot_product(weights, inputs) + bias)  This means when calculating the output of a node, the inputs are multiplied by weights, and a bias value is added to the result. The bias value allows the activation function to be shifted to the left or right, to better fit the data. Hence changes to the weights alter the steepness of the sigmoid curve, whilst the bias offsets it, shifting the entire curve so it fits better. Note also how the bias only influences the output values, it doesn’t interact with the actual input data.  [http://medium.com/deeper-learning/glossary-of-deep-learning-bias-cf49d9c895e2 Glossary of Deep Learning: Bias | Jaron Collis - Deeper Learning]
 +
# <b>Model Bias</b> - Bias can creep into algorithms in several ways. AI systems learn to make decisions based on training data, which can include biased human decisions or reflect historical or social inequities, even if sensitive variables such as gender, race, or sexual orientation are removed. [http://hbr.org/2019/10/what-do-we-do-about-the-biases-in-ai#:~:text=Bias%20can%20creep%20into%20algorithms,or%20sexual%20orientation%20are%20removed.&text=Bias%20is%20all%20of%20our%20responsibility. What Do We Do About the Biases in AI? | J. Manyika, J. Silberg, and B. Presten - Harvard Business Review]
 +
 
 +
 
 +
= Node Bias =
 +
 
 +
http://cdn-images-1.medium.com/max/800/1*RvLVTYBiRXogAr6Ap5jgLA.png
 +
http://cdn-images-1.medium.com/max/800/0*ml9CpRkTA8fY9AFT.png
  
https://cdn-images-1.medium.com/max/800/1*RvLVTYBiRXogAr6Ap5jgLA.png
 
https://cdn-images-1.medium.com/max/800/0*ml9CpRkTA8fY9AFT.png
 
  
 +
{|<!-- T -->
 +
| valign="top" |
 +
{| class="wikitable" style="width: 550px;"
 +
||
 
<youtube>SjQyLhQIXSM</youtube>
 
<youtube>SjQyLhQIXSM</youtube>
 +
<b>Bias/Variance (C2W1L02)
 +
</b><br>[[Creatives#Andrew Ng|Andrew Ng]]  Deeplearning.ai  Take the Deep Learning Specialization: http://bit.ly/3amgU4n  Check out all our courses: http://www.deeplearning.ai
 +
|}
 +
|<!-- M -->
 +
| valign="top" |
 +
{| class="wikitable" style="width: 550px;"
 +
||
 
<youtube>ewogYw5oCAI</youtube>
 
<youtube>ewogYw5oCAI</youtube>
 +
<b>Diagnosing Bias vs Variance
 +
</b><br>[[Creatives#Andrew Ng|Andrew Ng]]  Advice for Applying Machine Learning
 +
|}
 +
|}<!-- B -->
 +
{|<!-- T -->
 +
| valign="top" |
 +
{| class="wikitable" style="width: 550px;"
 +
||
 
<youtube>C1N_PDHuJ6Q</youtube>
 
<youtube>C1N_PDHuJ6Q</youtube>
 +
<b>Basic Recipe for Machine Learning (C2W1L03)
 +
</b><br>[[Creatives#Andrew Ng|Andrew Ng]]  Deeplearning.ai  Take the Deep Learning Specialization: http://bit.ly/3cpg1K9  Check out all our courses: http://www.deeplearning.ai
 +
|}
 +
|<!-- M -->
 +
| valign="top" |
 +
{| class="wikitable" style="width: 550px;"
 +
||
 
<youtube>yLwZEuybaqE</youtube>
 
<youtube>yLwZEuybaqE</youtube>
 +
<b>Learning To See [Part 9: Bias Variance Throwdown]
 +
</b><br>In this series, we'll explore the complex landscape of machine learning and artificial intelligence through one example from the field of computer vision: using a decision tree to count the number of fingers in an image. It's gonna be crazy.
 +
|}
 +
|}<!-- B -->
 +
{|<!-- T -->
 +
| valign="top" |
 +
{| class="wikitable" style="width: 550px;"
 +
||
 
<youtube>HetFihsXSys</youtube>
 
<youtube>HetFihsXSys</youtube>
 +
<b>Bias in an Artificial Neural Network explained | How bias impacts training
 +
</b><br>When reading up on artificial neural networks, you may have come across the term “bias.” It’s sometimes just referred to as bias. Other times you may see it referenced as bias nodes, bias neurons, or bias units within a neural network. We’re going to break this bias down and see what it’s all about. We’ll first start out by discussing the most obvious question of, well, what is bias in an artificial neural network? We’ll then see, within a network, how bias is implemented. Then, to hit the point home, we’ll explore a simple example to illustrate the impact that bias has when introduced to a neural network.
 +
|}
 +
|<!-- M -->
 +
| valign="top" |
 +
{| class="wikitable" style="width: 550px;"
 +
||
 
<youtube>lpkSGTT8uMg</youtube>
 
<youtube>lpkSGTT8uMg</youtube>
 +
<b>Bias Variance Trade off
 +
</b><br>Bias Variance Trade off is an important concept when it comes to choosing a machine learning algorithm for you problem. Bias is the expectation in error and variance is the variability in the model. In this tutorial we look at why we need bias variance trade off,  how it effects the accuracy and how to optimally select bias and variance and avoid over fitting and under fitting conditions.
 +
|}
 +
|}<!-- B -->
 +
 +
 +
* [http://ai.googleblog.com/2017/08/exploring-and-visualizing-open-global.html Exploring and Visualizing an Open Global Dataset | Reena Jana and Josh Lovejoy]
 +
 +
http://4.bp.blogspot.com/-KbGvGbxn8X4/WZ9MwE2GdOI/AAAAAAAAB-Y/zqkSZzRgBmcTnrxWE2_Zt4mBn7bng3poACLcBGAs/s640/chair-world-light-1000.gif
 +
 +
= Model Bias =
 +
* [[Ethics]]
 +
* [[Privacy]]
 +
* [http://developer.ibm.com/articles/machine-learning-and-bias/ Machine learning and bias | IBM] [http://aif360.mybluemix.net/?_ga=2.258555210.1353694320.1576870371-739966746.1575208440 AI Fairness 360 Open Source Toolkit] extensible open source toolkit can help you examine, report, and mitigate discrimination and bias in machine learning models throughout the AI application lifecycle. Containing over 70 fairness metrics and 10 state-of-the-art bias mitigation algorithms
 +
* [http://www.usatoday.com/story/tech/2020/07/22/uber-lyft-algorithms-discriminate-charge-more-non-white-areas/5481950002/ Was your Uber, Lyft fare high because of algorithm bias? | Coral Murphy - USA TODAY]
 +
* [http://research.google.com/bigpicture/attacking-discrimination-in-ml/ Attacking discrimination with smarter machine learning |] [[Creatives#Martin Wattenberg |Martin Wattenberg]], [[Creatives#Fernanda Viegas |Fernanda Viegas]], and Moritz Hardt
 +
* [http://www.technologyreview.com/s/612876/this-is-how-ai-bias-really-happensand-why-its-so-hard-to-fix/ This is how AI bias really happens—and why it’s so hard to fix | Karen Hao]
 +
 +
{|<!-- T -->
 +
| valign="top" |
 +
{| class="wikitable" style="width: 550px;"
 +
||
 +
<youtube>PWCtoVt1CJM</youtube>
 +
<b>How To Stop Artificial Intelligence From Marginalizing Communities? | Timnit Gebru | TEDxCollegePark
 +
</b><br>Timnit Gebru, Stanford Alum and Co-Founder of Black in AI, shares remarkable insights to show how artificial intelligence is influencing thinking and decision-making in ways we didn't imagine and must counter before it further marginalizes people.  Timnit works at Microsoft, New York in the Fairness Accountability Transparency and [[Ethics]] (FATE) Group where her team works on the complex social implications of AI, machine learning, data science, large-scale experimentation, and increasing automation. She previously worked at Stanford’s Artificial Intelligence Lab where she received her PhD, and is Co-Founder of Black in AI, an organization that aims to foster collaborations and discuss initiatives to increase the presence of Black people in the field of Artificial Intelligence. This talk was given at a TEDx event using the TED conference format but independently organized by a local community. Learn more at http://www.ted.com/tedx
 +
|}
 +
|<!-- M -->
 +
| valign="top" |
 +
{| class="wikitable" style="width: 550px;"
 +
||
 +
<youtube>g-z84_nRQhw</youtube>
 +
<b>The Emerging Theory of Algorithmic Fairness
 +
</b><br>As algorithms reach ever more deeply into our daily lives, increasing concern that they be “fair” has resulted in an explosion of research in the theory and machine learning communities. This talk surveys key results in both areas and traces the arc of the emerging theory of algorithmic fairness.
 +
|}
 +
|}<!-- B -->

Revision as of 09:50, 29 November 2020

Youtube search... ...Google search

Two terms of bias in Artificial Intelligence:

  1. Node Bias - The activation of a node in a neural network is determined by the following: output = activation function (dot_product(weights, inputs) + bias) This means when calculating the output of a node, the inputs are multiplied by weights, and a bias value is added to the result. The bias value allows the activation function to be shifted to the left or right, to better fit the data. Hence changes to the weights alter the steepness of the sigmoid curve, whilst the bias offsets it, shifting the entire curve so it fits better. Note also how the bias only influences the output values, it doesn’t interact with the actual input data. Glossary of Deep Learning: Bias | Jaron Collis - Deeper Learning
  2. Model Bias - Bias can creep into algorithms in several ways. AI systems learn to make decisions based on training data, which can include biased human decisions or reflect historical or social inequities, even if sensitive variables such as gender, race, or sexual orientation are removed. What Do We Do About the Biases in AI? | J. Manyika, J. Silberg, and B. Presten - Harvard Business Review


Node Bias

1*RvLVTYBiRXogAr6Ap5jgLA.png 0*ml9CpRkTA8fY9AFT.png


Bias/Variance (C2W1L02)
Andrew Ng Deeplearning.ai Take the Deep Learning Specialization: http://bit.ly/3amgU4n Check out all our courses: http://www.deeplearning.ai

Diagnosing Bias vs Variance
Andrew Ng Advice for Applying Machine Learning

Basic Recipe for Machine Learning (C2W1L03)
Andrew Ng Deeplearning.ai Take the Deep Learning Specialization: http://bit.ly/3cpg1K9 Check out all our courses: http://www.deeplearning.ai

Learning To See [Part 9: Bias Variance Throwdown]
In this series, we'll explore the complex landscape of machine learning and artificial intelligence through one example from the field of computer vision: using a decision tree to count the number of fingers in an image. It's gonna be crazy.

Bias in an Artificial Neural Network explained | How bias impacts training
When reading up on artificial neural networks, you may have come across the term “bias.” It’s sometimes just referred to as bias. Other times you may see it referenced as bias nodes, bias neurons, or bias units within a neural network. We’re going to break this bias down and see what it’s all about. We’ll first start out by discussing the most obvious question of, well, what is bias in an artificial neural network? We’ll then see, within a network, how bias is implemented. Then, to hit the point home, we’ll explore a simple example to illustrate the impact that bias has when introduced to a neural network.

Bias Variance Trade off
Bias Variance Trade off is an important concept when it comes to choosing a machine learning algorithm for you problem. Bias is the expectation in error and variance is the variability in the model. In this tutorial we look at why we need bias variance trade off, how it effects the accuracy and how to optimally select bias and variance and avoid over fitting and under fitting conditions.


chair-world-light-1000.gif

Model Bias

How To Stop Artificial Intelligence From Marginalizing Communities? | Timnit Gebru | TEDxCollegePark
Timnit Gebru, Stanford Alum and Co-Founder of Black in AI, shares remarkable insights to show how artificial intelligence is influencing thinking and decision-making in ways we didn't imagine and must counter before it further marginalizes people. Timnit works at Microsoft, New York in the Fairness Accountability Transparency and Ethics (FATE) Group where her team works on the complex social implications of AI, machine learning, data science, large-scale experimentation, and increasing automation. She previously worked at Stanford’s Artificial Intelligence Lab where she received her PhD, and is Co-Founder of Black in AI, an organization that aims to foster collaborations and discuss initiatives to increase the presence of Black people in the field of Artificial Intelligence. This talk was given at a TEDx event using the TED conference format but independently organized by a local community. Learn more at http://www.ted.com/tedx

The Emerging Theory of Algorithmic Fairness
As algorithms reach ever more deeply into our daily lives, increasing concern that they be “fair” has resulted in an explosion of research in the theory and machine learning communities. This talk surveys key results in both areas and traces the arc of the emerging theory of algorithmic fairness.