Objective vs. Cost vs. Loss vs. Error Function - Revision history

BPeat at 01:31, 6 March 2024

2024-03-06T01:31:37Z

BPeat at 02:43, 29 November 2023

2023-11-29T02:43:31Z

BPeat at 14:42, 31 August 2023

2023-08-31T14:42:19Z

BPeat at 14:39, 31 August 2023

2023-08-31T14:39:38Z

BPeat at 14:16, 6 August 2023

2023-08-06T14:16:19Z

BPeat at 14:15, 6 August 2023

2023-08-06T14:15:27Z

BPeat at 02:34, 12 July 2023

2023-07-12T02:34:31Z

BPeat at 01:58, 11 July 2023

2023-07-11T01:58:45Z

BPeat at 12:00, 5 March 2023

2023-03-05T12:00:00Z

BPeat at 01:53, 5 September 2020

2020-09-05T01:53:21Z

@@ Line 20: / Line 20: @@
 [https://www.bing.com/news/search?q=ai+objective+cost+loss+error+function&qft=interval%3d%228%22 ... Bing News]
 * [[Backpropagation]] ... [[Feed Forward Neural Network (FF or FFNN)|FFNN]] ... [[Forward-Forward]] ... [[Activation Functions]] ...[[Softmax]] ... [[Loss]] ... [[Boosting]] ... [[Gradient Descent Optimization & Challenges|Gradient Descent]] ... [[Algorithm Administration#Hyperparameter|Hyperparameter]] ... [[Manifold Hypothesis]] ... [[Principal Component Analysis (PCA)|PCA]]
-* [[AI Solver]] ... [[Algorithms]] ... [[Algorithm Administration|Administration]] ... [[Model Search]] ... [[Discriminative vs. Generative]] ... [[Optimizer]] ... [[Train, Validate, and Test]]
+* [[AI Solver]] ... [[Algorithms]] ... [[Algorithm Administration|Administration]] ... [[Model Search]] ... [[Discriminative vs. Generative]] ... [[Train, Validate, and Test]]
 * [http://towardsdatascience.com/error-analysis-to-your-rescue-773b401380ef Error Analysis to Your Rescue! | Kritika Jalan]
 * [http://heartbeat.fritz.ai/5-regression-loss-functions-all-machine-learners-should-know-4fb140e9d4b0 5 Regression Loss Functions All Machine Learners Should Know | Prince Grover]

← Older revision		Revision as of 02:43, 29 November 2023
Line 24:		Line 24:
	* [http://towardsdatascience.com/error-analysis-to-your-rescue-773b401380ef Error Analysis to Your Rescue! \| Kritika Jalan]		* [http://towardsdatascience.com/error-analysis-to-your-rescue-773b401380ef Error Analysis to Your Rescue! \| Kritika Jalan]
	* [http://heartbeat.fritz.ai/5-regression-loss-functions-all-machine-learners-should-know-4fb140e9d4b0 5 Regression Loss Functions All Machine Learners Should Know \| Prince Grover]		* [http://heartbeat.fritz.ai/5-regression-loss-functions-all-machine-learners-should-know-4fb140e9d4b0 5 Regression Loss Functions All Machine Learners Should Know \| Prince Grover]
		+	* [[Process Supervision]]

@@ Line 49: / Line 49: @@
 <youtube>h7iBpEHGVNc</youtube>
 <youtube>euhATa4wgzo</youtube>
 <youtube>AoJQS10Ewn4</youtube>

@@ Line 14: / Line 14: @@
 </script>
 }}
-[http://www.youtube.com/results?search_query=objective+function+cost+loss YouTube Search]
+[https://www.youtube.com/results?search_query=ai+objective+cost+loss+error+function YouTube]
-[http://www.google.com/search?q=objective+function+cost+loss+deep+machine+learning+ML+artificial+intelligence ...Google search]
+[https://www.quora.com/search?q=ai%20objective%20cost%20loss%20error%20function ... Quora]
 * [[Backpropagation]] ... [[Feed Forward Neural Network (FF or FFNN)|FFNN]] ... [[Forward-Forward]] ... [[Activation Functions]] ...[[Softmax]] ... [[Loss]] ... [[Boosting]] ... [[Gradient Descent Optimization & Challenges|Gradient Descent]] ... [[Algorithm Administration#Hyperparameter|Hyperparameter]] ... [[Manifold Hypothesis]] ... [[Principal Component Analysis (PCA)|PCA]]
@@ Line 26: / Line 29: @@
-"The function we want to minimize or maximize is called the objective function, or criterion. When we are minimizing it, we may also call it the cost function, loss function, or error function - these terms are synonymous. The cost function is used more in optimization problem and loss function is used in parameter estimation.
+"The function we want to minimize or maximize is called the objective function, or criterion. When we are minimizing it, we may also call it the cost function, [[Loss|loss function]], or error function - these terms are synonymous. The cost function is used more in optimization problem and [[Loss|loss function]] is used in parameter estimation.
-The loss function (or error) is for a single training example, while the cost function is over the entire training set (or mini-batch for mini-batch gradient descent). Therefore, a loss function is a part of a cost function which is a type of an objective function.  [http://stats.stackexchange.com/questions/179026/objective-function-cost-function-loss-function-are-they-the-same-thing Objective function, cost function, loss function: are they the same thing? | StackExchange]
+The [[Loss|loss function (or error)]] is for a single training example, while the cost function is over the entire training set (or mini-batch for mini-batch gradient descent). Therefore, a [[loss]] function is a part of a cost function which is a type of an objective function.  [http://stats.stackexchange.com/questions/179026/objective-function-cost-function-loss-function-are-they-the-same-thing Objective function, cost function, loss function: are they the same thing? | StackExchange]
-* <b>Loss function</b> is usually a function defined on a data point, prediction and label, and measures the penalty. For example:
+* <b>[[Loss|Loss function]]</b> is usually a function defined on a data point, prediction and label, and measures the penalty. For example:
 ** square loss l(f(xi|θ),yi)=(f(xi|θ)−yi)2, used in linear [[Regression]]
 ** hinge loss l(f(xi|θ),yi)=max(0,1−f(xi|θ)yi), used in SVM
 ** 0/1 loss l(f(xi|θ),yi)=1⟺f(xi|θ)≠yi, used in theoretical analysis and definition of accuracy
-* <b>Cost function</b> is usually more general. It might be a sum of loss functions over your training set plus some model complexity penalty (regularization). For example:
+* <b>Cost function</b> is usually more general. It might be a sum of [[Loss|loss functions]] over your training set plus some model complexity penalty (regularization). For example:
 ** Mean Squared Error MSE(θ)=1N∑Ni=1(f(xi|θ)−yi)2
 ** SVM cost function SVM(θ)=∥θ∥2+C∑Ni=1ξi (there are additional constraints connecting ξi with C and with training set)
-* <b>Objective function</b> is the most general term for any function that you optimize during training. For example, a probability of generating training set in maximum likelihood approach is a well defined objective function, but it is not a loss function nor cost function (however you could define an equivalent cost function). For example:
+* <b>Objective function</b> is the most general term for any function that you optimize during training. For example, a probability of generating training set in maximum likelihood approach is a well defined objective function, but it is not a [[Loss|loss function]] nor cost function (however you could define an equivalent cost function). For example:
 ** MLE is a type of objective function (which you maximize)
 **Divergence between classes can be an objective function but it is barely a cost function, unless you define something artificial, like 1-Divergence, and name it a cost
-* <b>Error function</b> - [[Backpropagation]]; or automatic differentiation, is commonly used by the gradient descent optimization algorithm to adjust the [[Activation Functions#Weights|weight]] of neurons by calculating the gradient of the loss function. This technique is also sometimes called backward propagation of errors, because the error is calculated at the output and distributed back through the network layers. [http://en.wikipedia.org/wiki/Backpropagation Backpropagation | Wikipedia]
+* <b>Error function</b> - [[Backpropagation]]; or automatic differentiation, is commonly used by the gradient descent optimization algorithm to adjust the [[Activation Functions#Weights|weight]] of neurons by calculating the gradient of the [[Loss|loss function]]. This technique is also sometimes called backward propagation of errors, because the error is calculated at the output and distributed back through the network layers. [http://en.wikipedia.org/wiki/Backpropagation Backpropagation | Wikipedia]

@@ Line 17: / Line 17: @@
 [http://www.google.com/search?q=objective+function+cost+loss+deep+machine+learning+ML+artificial+intelligence ...Google search]
 * [[AI Solver]] ... [[Algorithms]] ... [[Algorithm Administration|Administration]] ... [[Model Search]] ... [[Discriminative vs. Generative]] ... [[Optimizer]] ... [[Train, Validate, and Test]]
 * [http://towardsdatascience.com/error-analysis-to-your-rescue-773b401380ef Error Analysis to Your Rescue! | Kritika Jalan]
 * [http://heartbeat.fritz.ai/5-regression-loss-functions-all-machine-learners-should-know-4fb140e9d4b0 5 Regression Loss Functions All Machine Learners Should Know | Prince Grover]

← Older revision		Revision as of 14:15, 6 August 2023
Line 43:		Line 43:
	**Divergence between classes can be an objective function but it is barely a cost function, unless you define something artificial, like 1-Divergence, and name it a cost		**Divergence between classes can be an objective function but it is barely a cost function, unless you define something artificial, like 1-Divergence, and name it a cost

−	* <b>Error function</b> - [[Backpropagation]]; or automatic differentiation, is commonly used by the gradient descent optimization algorithm to adjust the weight of neurons by calculating the gradient of the loss function. This technique is also sometimes called backward propagation of errors, because the error is calculated at the output and distributed back through the network layers. [http://en.wikipedia.org/wiki/Backpropagation Backpropagation \| Wikipedia]	+	* <b>Error function</b> - [[Backpropagation]]; or automatic differentiation, is commonly used by the gradient descent optimization algorithm to adjust the [[Activation Functions#Weights\|weight]] of neurons by calculating the gradient of the loss function. This technique is also sometimes called backward propagation of errors, because the error is calculated at the output and distributed back through the network layers. [http://en.wikipedia.org/wiki/Backpropagation Backpropagation \| Wikipedia]

@@ Line 2: / Line 2: @@
 |title=PRIMO.ai
 |titlemode=append
-|keywords=artificial, intelligence, machine, learning, models, algorithms, data, singularity, moonshot, Tensorflow, Google, Nvidia, Microsoft, Azure, Amazon, AWS
+|keywords=ChatGPT, artificial, intelligence, machine, learning, GPT-4, GPT-5, NLP, NLG, NLC, NLU, models, data, singularity, moonshot, Sentience, AGI, Emergence, Moonshot, Explainable, TensorFlow, Google, Nvidia, Microsoft, Azure, Amazon, AWS, Hugging Face, OpenAI, Tensorflow, OpenAI, Google, Nvidia, Microsoft, Azure, Amazon, AWS, Meta, LLM, metaverse, assistants, agents, digital twin, IoT, Transhumanism, Immersive Reality, Generative AI, Conversational AI, Perplexity, Bing, You, Bard, Ernie, prompt Engineering LangChain, Video/Image, Vision, End-to-End Speech, Synthesize Speech, Speech Recognition, Stanford, MIT |description=Helpful resources for your journey with artificial intelligence; videos, articles, techniques, courses, profiles, and tools
 |description=Helpful resources for your journey with artificial intelligence; videos, articles, techniques, courses, profiles, and tools
 }}
 [http://www.youtube.com/results?search_query=objective+function+cost+loss YouTube Search]
 [http://www.google.com/search?q=objective+function+cost+loss+deep+machine+learning+ML+artificial+intelligence ...Google search]
-* [[Evaluation - Measures]]
+* [[AI Solver]] ... [[Algorithms]] ... [[Algorithm Administration|Administration]] ... [[Model Search]] ... [[Discriminative vs. Generative]] ... [[Optimizer]] ... [[Train, Validate, and Test]]
 * [http://towardsdatascience.com/error-analysis-to-your-rescue-773b401380ef Error Analysis to Your Rescue! | Kritika Jalan]
 * [[Loss]] Functions