Bayes - Revision history

BPeat at 02:44, 6 March 2024

2024-03-06T02:44:54Z

BPeat at 13:28, 16 September 2023

2023-09-16T13:28:13Z

BPeat at 12:19, 16 September 2023

2023-09-16T12:19:16Z

BPeat at 14:19, 6 August 2023

2023-08-06T14:19:15Z

BPeat at 02:01, 11 July 2023

2023-07-11T02:01:23Z

BPeat at 11:58, 9 July 2023

2023-07-09T11:58:08Z

BPeat at 23:46, 5 July 2023

2023-07-05T23:46:49Z

BPeat at 17:59, 4 July 2023

2023-07-04T17:59:28Z

BPeat: /* Bayes' Theorem */

2023-07-03T16:33:31Z

‎Bayes' Theorem

BPeat: /* Bayes' Theorem */

2023-07-03T16:32:23Z

‎Bayes' Theorem

@@ Line 21: / Line 21: @@
 * [[Analytics]] ... [[Visualization]] ... [[Graphical Tools for Modeling AI Components|Graphical Tools]] ... [[Diagrams for Business Analysis|Diagrams]] & [[Generative AI for Business Analysis|Business Analysis]] ... [[Requirements Management|Requirements]] ... [[Loop]] ... [[Bayes]] ... [[Network Pattern]]
-* [[AI Solver]] ... [[Algorithms]] ... [[Algorithm Administration|Administration]] ... [[Model Search]] ... [[Discriminative vs. Generative]] ... [[Optimizer]] ... [[Train, Validate, and Test]]
+* [[AI Solver]] ... [[Algorithms]] ... [[Algorithm Administration|Administration]] ... [[Model Search]] ... [[Discriminative vs. Generative]] ... [[Train, Validate, and Test]]
 ** [[...predict categories]]
 * [[Math for Intelligence]] ... [[Finding Paul Revere]] ... [[Social Network Analysis (SNA)]] ... [[Dot Product]] ... [[Kernel Trick]]

@@ Line 142: / Line 142: @@
 [https://www.google.com/search?q=Bayes+Prior+Probability+artificial+intelligence+ai ...Google search]
-<b>Prior</b> probability distribution in Bayesian statistical inference often simply called the prior, of an uncertain quantity is the probability distribution that would express one's beliefs about this quantity before some evidence is taken into account. For example, the prior could be the probability distribution representing the relative proportions of voters who will vote for a particular politician in a future election. The unknown quantity may be a parameter of the model or a [[Latent]] variable rather than an observable variable. Bayes' theorem calculates the renormalized pointwise product of the prior and the likelihood function, to produce the posterior probability distribution, which is the conditional distribution of the uncertain quantity given the data. Similarly, the prior probability of a random event or an uncertain proposition is the unconditional probability that is assigned before any relevant evidence is taken into account. Priors can be created using a number of methods.[https://en.wikipedia.org/wiki/Prior_probability Prior Probability | Wikipedia]
+<b>Prior</b> probability distribution in Bayesian statistical inference often simply called the prior, of an uncertain quantity is the probability distribution that would express one's beliefs about this quantity before some evidence is taken into account. For example, the prior could be the probability distribution representing the relative proportions of voters who will vote for a particular politician in a future election. The unknown quantity may be a parameter of the model or a [[latent]] variable rather than an observable variable. Bayes' theorem calculates the renormalized pointwise product of the prior and the likelihood function, to produce the posterior probability distribution, which is the conditional distribution of the uncertain quantity given the data. Similarly, the prior probability of a random event or an uncertain proposition is the unconditional probability that is assigned before any relevant evidence is taken into account. Priors can be created using a number of methods.[https://en.wikipedia.org/wiki/Prior_probability Prior Probability | Wikipedia]
 <youtube>nCRTuwCdmP0</youtube>

@@ Line 142: / Line 142: @@
 [https://www.google.com/search?q=Bayes+Prior+Probability+artificial+intelligence+ai ...Google search]
-<b>Prior</b> probability distribution in Bayesian statistical inference often simply called the prior, of an uncertain quantity is the probability distribution that would express one's beliefs about this quantity before some evidence is taken into account. For example, the prior could be the probability distribution representing the relative proportions of voters who will vote for a particular politician in a future election. The unknown quantity may be a parameter of the model or a latent variable rather than an observable variable. Bayes' theorem calculates the renormalized pointwise product of the prior and the likelihood function, to produce the posterior probability distribution, which is the conditional distribution of the uncertain quantity given the data. Similarly, the prior probability of a random event or an uncertain proposition is the unconditional probability that is assigned before any relevant evidence is taken into account. Priors can be created using a number of methods.[https://en.wikipedia.org/wiki/Prior_probability Prior Probability | Wikipedia]
+<b>Prior</b> probability distribution in Bayesian statistical inference often simply called the prior, of an uncertain quantity is the probability distribution that would express one's beliefs about this quantity before some evidence is taken into account. For example, the prior could be the probability distribution representing the relative proportions of voters who will vote for a particular politician in a future election. The unknown quantity may be a parameter of the model or a [[Latent]] variable rather than an observable variable. Bayes' theorem calculates the renormalized pointwise product of the prior and the likelihood function, to produce the posterior probability distribution, which is the conditional distribution of the uncertain quantity given the data. Similarly, the prior probability of a random event or an uncertain proposition is the unconditional probability that is assigned before any relevant evidence is taken into account. Priors can be created using a number of methods.[https://en.wikipedia.org/wiki/Prior_probability Prior Probability | Wikipedia]
 <youtube>nCRTuwCdmP0</youtube>

@@ Line 235: / Line 235: @@
 <youtube>Nr6jyodaFDo</youtube>
-In the Bayesian viewpoint, we formulate linear regression using probability distributions rather than point estimates. The response, y, is not estimated as a single value, but is assumed to be drawn from a probability distribution. The output, y is generated from a normal (Gaussian) Distribution characterized by a mean and variance. The mean for linear regression is the transpose of the weight matrix multiplied by the predictor matrix. The variance is the square of the standard deviation σ (multiplied by the Identity matrix because this is a multi-dimensional formulation of the model).
+In the Bayesian viewpoint, we formulate linear regression using probability distributions rather than point estimates. The response, y, is not estimated as a single value, but is assumed to be drawn from a probability distribution. The output, y is generated from a normal (Gaussian) Distribution characterized by a mean and variance. The mean for linear regression is the transpose of the [[Activation Functions#Weights|weight]] matrix multiplied by the predictor matrix. The variance is the square of the standard deviation σ (multiplied by the Identity matrix because this is a multi-dimensional formulation of the model).
 Bayesian methods have a highly desirable quality: they avoid overfitting. They do this by making some assumptions beforehand about the likely distribution of the answer. Another byproduct of this approach is that they have very few parameters. Machine Learning has both Bayesian algorithms for both classification (Two-class Bayes' point machine) and regression (Bayesian linear regression). Note that these assume that the data can be split or fit with a straight line. - Dinesh Chandrasekar
@@ Line 378: / Line 378: @@
 [https://www.google.com/search?q=Bayesian+Model+Averaging+BMA+artificial+intelligence+ai ...Google search]
-Bayesian model averaging (BMA) makes predictions using an average over several models with weights given by the posterior probability of each model given the data.[15] BMA is known to generally give better answers than a single model, obtained, e.g., via stepwise regression, especially where very different models have nearly identical performance in the training set but may otherwise perform quite differently. The most obvious question with any technique that uses Bayes' theorem is the prior, i.e., a specification of the probability (subjective, perhaps) that each model is the best to use for a given purpose. Conceptually, BMA can be used with any prior. [https://en.wikipedia.org/wiki/Ensemble_learning#Bayesian_parameter_averaging Bayesian Model Averaging (BMA) | Wikipedia]
+Bayesian model averaging (BMA) makes predictions using an average over several models with [[Activation Functions#Weights|weights]] given by the posterior probability of each model given the data.[15] BMA is known to generally give better answers than a single model, obtained, e.g., via stepwise regression, especially where very different models have nearly identical performance in the training set but may otherwise perform quite differently. The most obvious question with any technique that uses Bayes' theorem is the prior, i.e., a specification of the probability (subjective, perhaps) that each model is the best to use for a given purpose. Conceptually, BMA can be used with any prior. [https://en.wikipedia.org/wiki/Ensemble_learning#Bayesian_parameter_averaging Bayesian Model Averaging (BMA) | Wikipedia]
 <youtube>2usN9Ba8uuk</youtube>
@@ Line 387: / Line 387: @@
 [https://www.google.com/search?q=Bayesian+Model+Combination+BMC+artificial+intelligence+ai ...Google search]
-Bayesian model combination (BMC) is an algorithmic correction to Bayesian model averaging (BMA). Instead of sampling each model in the ensemble individually, it samples from the space of possible ensembles (with model weightings drawn randomly from a Dirichlet distribution having uniform parameters). This modification overcomes the tendency of BMA to converge toward giving all of the weight to a single model. Although BMC is somewhat more computationally expensive than BMA, it tends to yield dramatically better results. The results from BMC have been shown to be better on average (with statistical significance) than BMA, and bagging. The use of Bayes' law to compute model weights necessitates computing the probability of the data given each model. Typically, none of the models in the ensemble are exactly the distribution from which the training data were generated, so all of them correctly receive a value close to zero for this term. This would work well if the ensemble were big enough to sample the entire model-space, but such is rarely possible. Consequently, each pattern in the training data will cause the ensemble weight to shift toward the model in the ensemble that is closest to the distribution of the training data. It essentially reduces to an unnecessarily complex method for doing model selection. The possible weightings for an ensemble can be visualized as lying on a simplex. At each vertex of the simplex, all of the weight is given to a single model in the ensemble. BMA converges toward the vertex that is closest to the distribution of the training data. By contrast, BMC converges toward the point where this distribution projects onto the simplex. In other words, instead of selecting the one model that is closest to the generating distribution, it seeks the combination of models that is closest to the generating distribution. The results from BMA can often be approximated by using cross-validation to select the best model from a bucket of models. Likewise, the results from BMC may be approximated by using cross-validation to select the best ensemble combination from a random sampling of possible weightings. [https://en.wikipedia.org/wiki/Ensemble_learning#Bayesian_model_combination Bayesian Model Combination (BMC) | Wikipedia]
+Bayesian model combination (BMC) is an algorithmic correction to Bayesian model averaging (BMA). Instead of sampling each model in the ensemble individually, it samples from the space of possible ensembles (with model [[Activation Functions#Weights|weightings]] drawn randomly from a Dirichlet distribution having uniform parameters). This modification overcomes the tendency of BMA to converge toward giving all of the [[Activation Functions#Weights|weight]] to a single model. Although BMC is somewhat more computationally expensive than BMA, it tends to yield dramatically better results. The results from BMC have been shown to be better on average (with statistical significance) than BMA, and bagging. The use of Bayes' law to compute model [[Activation Functions#Weights|weights]] necessitates computing the probability of the data given each model. Typically, none of the models in the ensemble are exactly the distribution from which the training data were generated, so all of them correctly receive a value close to zero for this term. This would work well if the ensemble were big enough to sample the entire model-space, but such is rarely possible. Consequently, each pattern in the training data will cause the ensemble [[Activation Functions#Weights|weight]] to shift toward the model in the ensemble that is closest to the distribution of the training data. It essentially reduces to an unnecessarily complex method for doing model selection. The possible [[Activation Functions#Weights|weightings]] for an ensemble can be visualized as lying on a simplex. At each vertex of the simplex, all of the [[Activation Functions#Weights|weight]] is given to a single model in the ensemble. BMA converges toward the vertex that is closest to the distribution of the training data. By contrast, BMC converges toward the point where this distribution projects onto the simplex. In other words, instead of selecting the one model that is closest to the generating distribution, it seeks the combination of models that is closest to the generating distribution. The results from BMA can often be approximated by using cross-validation to select the best model from a bucket of models. Likewise, the results from BMC may be approximated by using cross-validation to select the best ensemble combination from a random sampling of possible [[Activation Functions#Weights|weightings]]. [https://en.wikipedia.org/wiki/Ensemble_learning#Bayesian_model_combination Bayesian Model Combination (BMC) | Wikipedia]
 = <span id="Quantum Bayesianism"></span>Quantum Bayesianism =

@@ Line 21: / Line 21: @@
 * [[Analytics]] ... [[Visualization]] ... [[Graphical Tools for Modeling AI Components|Graphical Tools]] ... [[Diagrams for Business Analysis|Diagrams]] & [[Generative AI for Business Analysis|Business Analysis]] ... [[Requirements Management|Requirements]] ... [[Loop]] ... [[Bayes]] ... [[Network Pattern]]
-* [[AI Solver]]
+* [[AI Solver]] ... [[Algorithms]] ... [[Algorithm Administration|Administration]] ... [[Model Search]] ... [[Discriminative vs. Generative]] ... [[Optimizer]] ... [[Train, Validate, and Test]]
 ** [[...predict categories]]
 * [[Math for Intelligence]] ... [[Finding Paul Revere]] ... [[Social Network Analysis (SNA)]] ... [[Dot Product]] ... [[Kernel Trick]]

@@ Line 20: / Line 20: @@
 [https://www.bing.com/news/search?q=ai+bayes+bayesian&qft=interval%3d%228%22 ...Bing News]
-* [[Analytics]] ... [[Visualization]] ... [[Graphical Tools for Modeling AI Components|Graphical Tools]] ... [[Loop]] ... [[Diagrams for Business Analysis|Diagrams]] & [[Generative AI for Business Analysis|Business Analysis]] ... [[Bayes]] ... [[Network Pattern]]
+* [[Analytics]] ... [[Visualization]] ... [[Graphical Tools for Modeling AI Components|Graphical Tools]] ... [[Loop]] ... [[Diagrams for Business Analysis|Diagrams]] & [[Generative AI for Business Analysis|Business Analysis]] ... [[Requirements Management|Requirements]] ... [[Bayes]] ... [[Network Pattern]]
 * [[AI Solver]]
 ** [[...predict categories]]

@@ Line 42: / Line 42: @@
 * [https://en.wikipedia.org/wiki/Evidence_under_Bayes%27_theorem Evidence Under Bayes' Theorem | Wikipedia]
 * [https://www.mathsisfun.com/data/bayes-theorem.html Bayes' Theorem - Math is Fun]
-* [https://www.britannica.com/topic/Bayess-theorem (3) Bayes’s theorem | Definition & Example | Britannica]
+* [https://www.britannica.com/topic/Bayess-theorem Bayes Theorem | Definition & Example | Britannica]
 Bayes' theorem describes how to update the probabilities of hypotheses when given evidence. It follows simply from the axioms of conditional probability, but can be used to powerfully reason about a wide range of problems involving belief updates.

@@ Line 33: / Line 33: @@
 = <span id="Bayes' Theorem"></span>Bayes' Theorem =
-[https://www.youtube.com/results?search_query=bayes+Theorem+artificial+intelligence+ai Youtube search...]
+[https://www.youtube.com/results?search_query=bayes+Theorem+artificial+intelligence+ai YouTube]
 [https://www.google.com/search?q=bayes+Theorem+artificial+intelligence+ai ...Google search]
 * [https://en.wikipedia.org/wiki/Evidence_under_Bayes%27_theorem Evidence Under Bayes' Theorem | Wikipedia]
-The probability of an event, based on prior knowledge of conditions that might be related to the event. [https://en.wikipedia.org/wiki/Bayes%27_theorem Bayes' Theorem | Wikipedia]
+Bayes' theorem describes how to update the probabilities of hypotheses when given evidence. It follows simply from the axioms of conditional probability, but can be used to powerfully reason about a wide range of problems involving belief updates.
-Bayes' theorem is a formula that describes how to update the probabilities of hypotheses when given evidence. It follows simply from the axioms of conditional probability, but can be used to powerfully reason about a wide range of problems involving belief updates. ¹
+<hr><center><b><i>
-According to Wikipedia, Bayes' theorem is stated mathematically as the following equation: ²
+The probability of an event, based on prior knowledge of conditions that might be related to the event.</i></b> [https://en.wikipedia.org/wiki/Bayes%27_theorem Bayes' Theorem | Wikipedia]
-$$P(A|B) = \frac{P(B|A)P(A)}{P(B)}$$
+</center><hr>
-- $P(A|B)$ is a conditional probability: the likelihood of event $A$ occurring given that $B$ is true.
+To expand on Bayes' theorem, we can think of it as a way of combining prior knowledge or belief with new evidence or data. The prior probability represents what we already know about the hypothesis before seeing the evidence. The likelihood represents how consistent the evidence is with the hypothesis. The marginal likelihood represents how probable the evidence is under any hypothesis. The posterior probability represents how probable the hypothesis is after seeing the evidence.
 Bayes' theorem can be used for a variety of applications, such as:
@@ Line 62: / Line 63: @@
 - Estimating the parameters of a statistical model based on observed data
 - Updating beliefs about the state of the world based on new information