Loss - Revision history

BPeat at 03:02, 6 March 2024

2024-03-06T03:02:27Z

BPeat at 02:15, 29 November 2023

2023-11-29T02:15:40Z

BPeat at 15:38, 13 September 2023

2023-09-13T15:38:29Z

BPeat at 00:39, 3 September 2023

2023-09-03T00:39:56Z

BPeat at 14:41, 31 August 2023

2023-08-31T14:41:09Z

BPeat at 14:40, 31 August 2023

2023-08-31T14:40:55Z

BPeat at 14:40, 31 August 2023

2023-08-31T14:40:18Z

BPeat at 14:32, 31 August 2023

2023-08-31T14:32:55Z

BPeat: /* Loss Curve */

2023-08-31T14:31:44Z

‎Loss Curve

BPeat: /* Loss Curve */

2023-08-31T14:22:57Z

‎Loss Curve

@@ Line 22: / Line 22: @@
 * [[Backpropagation]] ... [[Feed Forward Neural Network (FF or FFNN)|FFNN]] ... [[Forward-Forward]] ... [[Activation Functions]] ...[[Softmax]] ... [[Loss]] ... [[Boosting]] ... [[Gradient Descent Optimization & Challenges|Gradient Descent]] ... [[Algorithm Administration#Hyperparameter|Hyperparameter]] ... [[Manifold Hypothesis]] ... [[Principal Component Analysis (PCA)|PCA]]
 * [[Objective vs. Cost vs. Loss vs. Error Function]]
-* [[AI Solver]] ... [[Algorithms]] ... [[Algorithm Administration|Administration]] ... [[Model Search]] ... [[Discriminative vs. Generative]] ... [[Optimizer]] ... [[Train, Validate, and Test]]
+* [[AI Solver]] ... [[Algorithms]] ... [[Algorithm Administration|Administration]] ... [[Model Search]] ... [[Discriminative vs. Generative]] ... [[Train, Validate, and Test]]
 * [[Cross-Entropy Loss]]
 * [[Optimization Methods]]

@@ Line 25: / Line 25: @@
 * [[Cross-Entropy Loss]]
 * [[Optimization Methods]]
 * [[Large Language Model (LLM)]] ... [[Natural Language Processing (NLP)]] ... [[Natural Language Generation (NLG)|Generation]] ... [[Natural Language Classification (NLC)|Classification]] ... [[Natural Language Processing (NLP)#Natural Language Understanding (NLU)|Understanding]] ... [[Language Translation|Translation]] ... [[Natural Language Tools & Services|Tools & Services]]
 * [[What is Artificial Intelligence (AI)? | Artificial Intelligence (AI)]] ... [[Generative AI]] ... [[Machine Learning (ML)]] ... [[Deep Learning]] ... [[Neural Network]] ... [[Reinforcement Learning (RL)|Reinforcement]] ... [[Learning Techniques]]

@@ Line 27: / Line 27: @@
 * [[Large Language Model (LLM)]] ... [[Natural Language Processing (NLP)]] ... [[Natural Language Generation (NLG)|Generation]] ... [[Natural Language Classification (NLC)|Classification]] ... [[Natural Language Processing (NLP)#Natural Language Understanding (NLU)|Understanding]] ... [[Language Translation|Translation]] ... [[Natural Language Tools & Services|Tools & Services]]
 * [[What is Artificial Intelligence (AI)? | Artificial Intelligence (AI)]] ... [[Generative AI]] ... [[Machine Learning (ML)]] ... [[Deep Learning]] ... [[Neural Network]] ... [[Reinforcement Learning (RL)|Reinforcement]] ... [[Learning Techniques]]
-* [[Embedding]] ... [[Fine-tuning]] ... [[Agents#AI-Powered Search|Search]] ... [[Clustering]] ... [[Recommendation]] ... [[Anomaly Detection]] ... [[Classification]] ... [[Dimensional Reduction]] ... [[...find outliers]]
+* [[Embedding]] ... [[Fine-tuning]] ... [[Retrieval-Augmented Generation (RAG)|RAG]] ... [[Agents#AI-Powered Search|Search]] ... [[Clustering]] ... [[Recommendation]] ... [[Anomaly Detection]] ... [[Classification]] ... [[Dimensional Reduction]].  [[...find outliers]]
 * [[Attention]] Mechanism ... [[Transformer]] ... [[Generative Pre-trained Transformer (GPT)]] ... [[Generative Adversarial Network (GAN)|GAN]] ... [[Bidirectional Encoder Representations from Transformers (BERT)|BERT]]
 * [http://towardsdatascience.com/common-loss-functions-in-machine-learning-46af0ffc4d23 Common Loss functions in machine learning | Ravindra Parmar - Towards data Science]

@@ Line 26: / Line 26: @@
 * [[Optimization Methods]]
 * [[Large Language Model (LLM)]] ... [[Natural Language Processing (NLP)]] ... [[Natural Language Generation (NLG)|Generation]] ... [[Natural Language Classification (NLC)|Classification]] ... [[Natural Language Processing (NLP)#Natural Language Understanding (NLU)|Understanding]] ... [[Language Translation|Translation]] ... [[Natural Language Tools & Services|Tools & Services]]
 * [[Embedding]] ... [[Fine-tuning]] ... [[Agents#AI-Powered Search|Search]] ... [[Clustering]] ... [[Recommendation]] ... [[Anomaly Detection]] ... [[Classification]] ... [[Dimensional Reduction]] ... [[...find outliers]]
 * [[Attention]] Mechanism ... [[Transformer]] ... [[Generative Pre-trained Transformer (GPT)]] ... [[Generative Adversarial Network (GAN)|GAN]] ... [[Bidirectional Encoder Representations from Transformers (BERT)|BERT]]

@@ Line 19: / Line 19: @@
 [https://news.google.com/search?q=ai+Loss+Curve+Large+Language+Model+LLM+transformer ...Google News]
 [https://www.bing.com/news/search?q=ai+Loss+Curve+Large+Language+Model+LLM+transformer&qft=interval%3d%228%22 ...Bing News]
 * [[Backpropagation]] ... [[Feed Forward Neural Network (FF or FFNN)|FFNN]] ... [[Forward-Forward]] ... [[Activation Functions]] ...[[Softmax]] ... [[Loss]] ... [[Boosting]] ... [[Gradient Descent Optimization & Challenges|Gradient Descent]] ... [[Algorithm Administration#Hyperparameter|Hyperparameter]] ... [[Manifold Hypothesis]] ... [[Principal Component Analysis (PCA)|PCA]]

← Older revision		Revision as of 14:40, 31 August 2023
Line 55:		Line 55:


−
	<b>Loss Curve</b> or <b>Loss Graph</b> for a LLM is a plot of the model's loss on the training dataset and the validation dataset over the course of training. The loss is a measure of how well the model is performing, and a lower loss indicates better performance.		<b>Loss Curve</b> or <b>Loss Graph</b> for a LLM is a plot of the model's loss on the training dataset and the validation dataset over the course of training. The loss is a measure of how well the model is performing, and a lower loss indicates better performance.

@@ Line 33: / Line 33: @@
 * [http://ml-cheatsheet.readthedocs.io/en/latest/loss_functions.html Loss Functions | ML Cheatsheet]
-Loss (error) is a measure of how well an AI model can approximate the expected output from a given input. Loss is usually calculated by a loss function, which compares the model’s prediction with the actual output and returns a numerical value that represents the difference or error. A lower loss means that the model is more accurate, while a higher loss means that the model is less accurate. There are different types of loss functions, such as mean squared error, cross entropy, hinge loss, etc., depending on the type of problem and output that the model is trying to solve.
+Loss is a measure of how well an AI model can approximate the expected output from a given input. Loss is usually calculated by a loss function, which compares the model’s prediction with the actual output and returns a numerical value that represents the difference or error. A lower loss means that the model is more accurate, while a higher loss means that the model is less accurate. There are different types of loss functions, such as mean squared error, cross entropy, hinge loss, etc., depending on the type of problem and output that the model is trying to solve.
 There are many options for loss in Tensorflow (Keras). The actual optimized objective is the mean of the output array across all datapoints. A loss function gives a distance between a model's predictions to the ground truth labels. This is the distance (loss value) that the network aims to minimize; the lower this value, the better the current model describes our training data set.  [http://keras.io/losses/ Click here For a list of Keras loss functions.]  Loss is one of the two parameters required to compile a model...

@@ Line 109: / Line 109: @@
 == Loss Curve vs Learning Curve ==
-a loss curve and a learning curve are two different plots that are commonly used to monitor the performance of a model during training.
+* [https://www.baeldung.com/cs/learning-curve-ml What Is a Learning Curve in Machine Learning? | Baeldung]
 <b>Loss Curve</b>: shows the values of the model's loss over time. The loss measures how bad the model is doing, and it is typically minimized during training. The x-axis represents the measure of time or progress, such as the number of iterations/epochs, while the y-axis represents the measure of loss. A loss curve can help diagnose problems such as overfitting or underfitting
-<b>Learning Curve</b>: shows the progress of a specific metric related to learning during the training of a machine learning model. The x-axis represents the measure of time or progress, such as the number of training examples or the number of iterations/epochs, while the y-axis represents the measure of error or performance, such as the training and validation error or accuracy. A learning curve can help diagnose problems and optimize the prediction performance of a model
+<b>Learning Curve</b>: shows the progress of a specific metric related to learning during the training of a machine learning model. The x-axis represents the measure of time or progress, such as the number of training examples or the number of iterations/epochs, while the y-axis represents the measure of error or performance, such as the training and validation error or accuracy. Very popular learning curves are accuracy, precision, and recall. A learning curve can help diagnose problems and optimize the prediction performance of a model

← Older revision		Revision as of 14:22, 31 August 2023
Line 63:		Line 63:
	----		----

		+	----
		+	::<code> plt.plot(learn.recorder.lrs, learn.recorder.losses)</code>
		+	----