Difference between revisions of "Forward-Forward"
m |
m |
||
| Line 17: | Line 17: | ||
[https://www.google.com/search?q=Backpropagation+deep+machine+learning+ML ...Google search] | [https://www.google.com/search?q=Backpropagation+deep+machine+learning+ML ...Google search] | ||
| − | * [[Backpropagation]] ... [[Feed Forward Neural Network (FF or FFNN)|FFNN]] ... [[Forward-Forward]] ... [[Activation Functions]] ... [[Loss]] ... [[Boosting]] ... [[Gradient Descent Optimization & Challenges|Gradient Descent]] ... [[Algorithm Administration#Hyperparameter|Hyperparameter]] ... [[Manifold Hypothesis]] ... [[Principal Component Analysis (PCA)|PCA]] | + | * [[Backpropagation]] ... [[Feed Forward Neural Network (FF or FFNN)|FFNN]] ... [[Forward-Forward]] ... [[Activation Functions]] ...[[Softmax]] ... [[Loss]] ... [[Boosting]] ... [[Gradient Descent Optimization & Challenges|Gradient Descent]] ... [[Algorithm Administration#Hyperparameter|Hyperparameter]] ... [[Manifold Hypothesis]] ... [[Principal Component Analysis (PCA)|PCA]] |
* [https://www.cs.toronto.edu/~hinton/FFA13.pdf The Forward-Forward Algorithm: Some Prelininary Investigations |] [[Creatives#Geoffry Hinton | Geoffry Hinton]] | * [https://www.cs.toronto.edu/~hinton/FFA13.pdf The Forward-Forward Algorithm: Some Prelininary Investigations |] [[Creatives#Geoffry Hinton | Geoffry Hinton]] | ||
* [[Objective vs. Cost vs. Loss vs. Error Function]] | * [[Objective vs. Cost vs. Loss vs. Error Function]] | ||
Latest revision as of 01:02, 11 July 2023
Youtube search... ...Google search
- Backpropagation ... FFNN ... Forward-Forward ... Activation Functions ...Softmax ... Loss ... Boosting ... Gradient Descent ... Hyperparameter ... Manifold Hypothesis ... PCA
- The Forward-Forward Algorithm: Some Prelininary Investigations | Geoffry Hinton
- Objective vs. Cost vs. Loss vs. Error Function
- Wikipedia
- What is the “forward-forward” algorithm, Geoffrey Hinton’s new AI technique? | Ben Dickson - TechTalks
Introduction of a new learning procedure for neural networks and to demonstrate that it works well enough on a few small problems to be worth serious investigation. The Forward-Forward algorithm replaces the forward and backward passes of backpropagation by two forward passes, one with positive (i.e. real) data and the other with negative data which could be generated by the network itself. Each layer has its own objective function which is simply to have high goodness for positive data and low goodness for negative data. The sum of the squared activities in a layer can be used as the goodness but there are many other possibilities, including minus the sum of the squared activities. If the positive and negative passes can be separated in time, the negative passes can be done offline, which makes the learning much simpler in the positive pass and allows video to be pipelined through the network without ever storing activities or stopping to propagate derivatives.
