Hopfield Network (HN) - Revision history

BPeat at 03:38, 2 March 2024

2024-03-02T03:38:33Z

BPeat at 14:14, 6 August 2023

2023-08-06T14:14:01Z

BPeat at 10:55, 4 July 2023

2023-07-04T10:55:42Z

BPeat at 15:22, 3 May 2023

2023-05-03T15:22:02Z

BPeat: Text replacement - "http:" to "https:"

2023-03-28T22:10:33Z

Text replacement - "http:" to "https:"

BPeat at 15:10, 19 March 2023

2023-03-19T15:10:38Z

BPeat at 21:01, 11 October 2020

2020-10-11T21:01:56Z

BPeat at 20:36, 26 September 2020

2020-09-26T20:36:57Z

BPeat: /* John J. Hopfield */

2020-09-26T20:33:43Z

‎John J. Hopfield

BPeat at 20:27, 26 September 2020

2020-09-26T20:27:16Z

@@ Line 28: / Line 28: @@
 * [[Attention]] Mechanism  ...[[Transformer]] ...[[Generative Pre-trained Transformer (GPT)]] ... [[Generative Adversarial Network (GAN)|GAN]] ... [[Bidirectional Encoder Representations from Transformers (BERT)|BERT]]
-A Hopfield Network is a form (one particular type) of recurrent artificial neural network popularized by [[Creatives#John J. Hopfield|John Hopfield]] in 1982, but described earlier by Little in 1974. Hopfield networks serve as [https://en.wikipedia.org/wiki/Content-addressable_memory <b>content-addressable ("associative") memory</b>] systems with binary threshold nodes. They are guaranteed to converge to a local minimum and, therefore, may converge to a false pattern (wrong local minimum) rather than the stored pattern (expected local minimum). [https://en.wikipedia.org/wiki/Hopfield_network#:~:text=A%20Hopfield%20network%20is%20a,systems%20with%20binary%20threshold%20nodes. Wikipedia]
+A Hopfield Network is a form (one particular type) of recurrent artificial neural network popularized by [[Creatives#John J. Hopfield|John Hopfield]] in 1982, but described earlier by Little in 1974. Hopfield networks serve as [https://en.wikipedia.org/wiki/Content-addressable_memory <b>content-addressable ("associative") [[memory]]</b>] systems with binary threshold nodes. They are guaranteed to converge to a local minimum and, therefore, may converge to a false pattern (wrong local minimum) rather than the stored pattern (expected local minimum). [https://en.wikipedia.org/wiki/Hopfield_network#:~:text=A%20Hopfield%20network%20is%20a,systems%20with%20binary%20threshold%20nodes. Wikipedia]
 Hopfield Network (HN) the [[Activation Functions#Weights|weight]] from node to another and from the later to the former are the same (symmetric). The Hopfield Network (HN) is fully connected, so every neuron’s output is an input to all the other neurons. Another feature of the network is that updating of nodes happens in a binary way. These features allow for a particular feature of Hopfield's nets - they are guaranteed to converge to an attractor (stable state).   [https://stats.stackexchange.com/users/145543/oba2311 oba2311]
-Every neuron is connected to every other neuron; it is a completely entangled plate of spaghetti as even all the nodes function as everything. Each node is input before training, then hidden during training and output afterwards. The networks are trained by setting the value of the neurons to the desired pattern after which the [[Activation Functions#Weights|weights]] can be computed. The [[Activation Functions#Weights|weights]] do not change after this. Once trained for one or more patterns, the network will always converge to one of the learned patterns because the network is only stable in those states. Note that it does not always conform to the desired state (it’s not a magic black box sadly). It stabilizes in part due to the total “energy” or “temperature” of the network being reduced incrementally during training. Each neuron has an activation threshold which scales to this temperature, which if surpassed by summing the input causes the neuron to take the form of one of two states (usually -1 or 1, sometimes 0 or 1). Updating the network can be done synchronously or more commonly one by one. If updated one by one, a fair random sequence is created to organize which cells update in what order (fair random being all options (n) occurring exactly once every n items). This is so you can tell when the network is stable (done converging), once every cell has been updated and none of them changed, the network is stable (annealed). These networks are often called associative memory because the converge to the most similar state as the input; if humans see half a table we can image the other half, this network will converge to a table if presented with half noise and half a table. Hopfield, John J. “Neural networks and physical systems with emergent collective computational abilities.” Proceedings of the national academy of sciences 79.8 (1982): 2554-2558.
+Every neuron is connected to every other neuron; it is a completely entangled plate of spaghetti as even all the nodes function as everything. Each node is input before training, then hidden during training and output afterwards. The networks are trained by setting the value of the neurons to the desired pattern after which the [[Activation Functions#Weights|weights]] can be computed. The [[Activation Functions#Weights|weights]] do not change after this. Once trained for one or more patterns, the network will always converge to one of the learned patterns because the network is only stable in those states. Note that it does not always conform to the desired state (it’s not a magic black box sadly). It stabilizes in part due to the total “energy” or “temperature” of the network being reduced incrementally during training. Each neuron has an activation threshold which scales to this temperature, which if surpassed by summing the input causes the neuron to take the form of one of two states (usually -1 or 1, sometimes 0 or 1). Updating the network can be done synchronously or more commonly one by one. If updated one by one, a fair random sequence is created to organize which cells update in what order (fair random being all options (n) occurring exactly once every n items). This is so you can tell when the network is stable (done converging), once every cell has been updated and none of them changed, the network is stable (annealed). These networks are often called associative [[memory]] because the converge to the most similar state as the input; if humans see half a table we can image the other half, this network will converge to a table if presented with half noise and half a table. Hopfield, John J. “Neural networks and physical systems with emergent collective computational abilities.” Proceedings of the national academy of sciences 79.8 (1982): 2554-2558.
 https://www.asimovinstitute.org/wp-content/uploads/2016/09/hn.png

@@ Line 30: / Line 30: @@
 A Hopfield Network is a form (one particular type) of recurrent artificial neural network popularized by [[Creatives#John J. Hopfield|John Hopfield]] in 1982, but described earlier by Little in 1974. Hopfield networks serve as [https://en.wikipedia.org/wiki/Content-addressable_memory <b>content-addressable ("associative") memory</b>] systems with binary threshold nodes. They are guaranteed to converge to a local minimum and, therefore, may converge to a false pattern (wrong local minimum) rather than the stored pattern (expected local minimum). [https://en.wikipedia.org/wiki/Hopfield_network#:~:text=A%20Hopfield%20network%20is%20a,systems%20with%20binary%20threshold%20nodes. Wikipedia]
-Hopfield Network (HN) the weight from node to another and from the later to the former are the same (symmetric). The Hopfield Network (HN) is fully connected, so every neuron’s output is an input to all the other neurons. Another feature of the network is that updating of nodes happens in a binary way. These features allow for a particular feature of Hopfield's nets - they are guaranteed to converge to an attractor (stable state).   [https://stats.stackexchange.com/users/145543/oba2311 oba2311]
+Hopfield Network (HN) the [[Activation Functions#Weights|weight]] from node to another and from the later to the former are the same (symmetric). The Hopfield Network (HN) is fully connected, so every neuron’s output is an input to all the other neurons. Another feature of the network is that updating of nodes happens in a binary way. These features allow for a particular feature of Hopfield's nets - they are guaranteed to converge to an attractor (stable state).   [https://stats.stackexchange.com/users/145543/oba2311 oba2311]
-Every neuron is connected to every other neuron; it is a completely entangled plate of spaghetti as even all the nodes function as everything. Each node is input before training, then hidden during training and output afterwards. The networks are trained by setting the value of the neurons to the desired pattern after which the weights can be computed. The weights do not change after this. Once trained for one or more patterns, the network will always converge to one of the learned patterns because the network is only stable in those states. Note that it does not always conform to the desired state (it’s not a magic black box sadly). It stabilizes in part due to the total “energy” or “temperature” of the network being reduced incrementally during training. Each neuron has an activation threshold which scales to this temperature, which if surpassed by summing the input causes the neuron to take the form of one of two states (usually -1 or 1, sometimes 0 or 1). Updating the network can be done synchronously or more commonly one by one. If updated one by one, a fair random sequence is created to organize which cells update in what order (fair random being all options (n) occurring exactly once every n items). This is so you can tell when the network is stable (done converging), once every cell has been updated and none of them changed, the network is stable (annealed). These networks are often called associative memory because the converge to the most similar state as the input; if humans see half a table we can image the other half, this network will converge to a table if presented with half noise and half a table. Hopfield, John J. “Neural networks and physical systems with emergent collective computational abilities.” Proceedings of the national academy of sciences 79.8 (1982): 2554-2558.
+Every neuron is connected to every other neuron; it is a completely entangled plate of spaghetti as even all the nodes function as everything. Each node is input before training, then hidden during training and output afterwards. The networks are trained by setting the value of the neurons to the desired pattern after which the [[Activation Functions#Weights|weights]] can be computed. The [[Activation Functions#Weights|weights]] do not change after this. Once trained for one or more patterns, the network will always converge to one of the learned patterns because the network is only stable in those states. Note that it does not always conform to the desired state (it’s not a magic black box sadly). It stabilizes in part due to the total “energy” or “temperature” of the network being reduced incrementally during training. Each neuron has an activation threshold which scales to this temperature, which if surpassed by summing the input causes the neuron to take the form of one of two states (usually -1 or 1, sometimes 0 or 1). Updating the network can be done synchronously or more commonly one by one. If updated one by one, a fair random sequence is created to organize which cells update in what order (fair random being all options (n) occurring exactly once every n items). This is so you can tell when the network is stable (done converging), once every cell has been updated and none of them changed, the network is stable (annealed). These networks are often called associative memory because the converge to the most similar state as the input; if humans see half a table we can image the other half, this network will converge to a table if presented with half noise and half a table. Hopfield, John J. “Neural networks and physical systems with emergent collective computational abilities.” Proceedings of the national academy of sciences 79.8 (1982): 2554-2558.
 https://www.asimovinstitute.org/wp-content/uploads/2016/09/hn.png

@@ Line 2: / Line 2: @@
 |title=PRIMO.ai
 |titlemode=append
-|keywords=artificial, intelligence, machine, learning, models, algorithms, data, singularity, moonshot, Tensorflow, Google, Nvidia, Microsoft, Azure, Amazon, AWS
+|keywords=ChatGPT, artificial, intelligence, machine, learning, GPT-4, GPT-5, NLP, NLG, NLC, NLU, models, data, singularity, moonshot, Sentience, AGI, Emergence, Moonshot, Explainable, TensorFlow, Google, Nvidia, Microsoft, Azure, Amazon, AWS, Hugging Face, OpenAI, Tensorflow, OpenAI, Google, Nvidia, Microsoft, Azure, Amazon, AWS, Meta, LLM, metaverse, assistants, agents, digital twin, IoT, Transhumanism, Immersive Reality, Generative AI, Conversational AI, Perplexity, Bing, You, Bard, Ernie, prompt Engineering LangChain, Video/Image, Vision, End-to-End Speech, Synthesize Speech, Speech Recognition, Stanford, MIT |description=Helpful resources for your journey with artificial intelligence; videos, articles, techniques, courses, profiles, and tools
 |description=Helpful resources for your journey with artificial intelligence; videos, articles, techniques, courses, profiles, and tools
 }}
 [https://www.youtube.com/results?search_query=Hopfield+network+HN YouTube search...]

@@ Line 17: / Line 17: @@
 ** [[Average-Stochastic Gradient Descent (SGD) Weight-Dropped LSTM (AWD-LSTM)]]
 ** Hopfield Network (HN)
-* [[Attention]] Mechanism  ...[[Transformer]] Model   ...[[Generative Pre-trained Transformer (GPT)]]
+* [[Attention]] Mechanism  ...[[Transformer]] ...[[Generative Pre-trained Transformer (GPT)]] ... [[Generative Adversarial Network (GAN)|GAN]] ... [[Bidirectional Encoder Representations from Transformers (BERT)|BERT]]
 A Hopfield Network is a form (one particular type) of recurrent artificial neural network popularized by [[Creatives#John J. Hopfield|John Hopfield]] in 1982, but described earlier by Little in 1974. Hopfield networks serve as [https://en.wikipedia.org/wiki/Content-addressable_memory <b>content-addressable ("associative") memory</b>] systems with binary threshold nodes. They are guaranteed to converge to a local minimum and, therefore, may converge to a false pattern (wrong local minimum) rather than the stored pattern (expected local minimum). [https://en.wikipedia.org/wiki/Hopfield_network#:~:text=A%20Hopfield%20network%20is%20a,systems%20with%20binary%20threshold%20nodes. Wikipedia]

@@ Line 5: / Line 5: @@
 |description=Helpful resources for your journey with artificial intelligence; videos, articles, techniques, courses, profiles, and tools
 }}
-[http://www.youtube.com/results?search_query=Hopfield+network+HN YouTube search...]
+[https://www.youtube.com/results?search_query=Hopfield+network+HN YouTube search...]
-[http://www.google.com/search?q=Hopfield+network+HN+machine+learning+ML+artificial+intelligence ...Google search]
+[https://www.google.com/search?q=Hopfield+network+HN+machine+learning+ML+artificial+intelligence ...Google search]
-* [http://www.asimovinstitute.org/author/fjodorvanveen/ Neural Network Zoo | Fjodor Van Veen]
+* [https://www.asimovinstitute.org/author/fjodorvanveen/ Neural Network Zoo | Fjodor Van Veen]
-* [http://pathmind.com/wiki/hopfieldnetworks Hopfield Networks | Chris Nicholson - A.I. Wiki pathmind]
+* [https://pathmind.com/wiki/hopfieldnetworks Hopfield Networks | Chris Nicholson - A.I. Wiki pathmind]
 * [[Recurrent Neural Network (RNN)]] Variants:
 ** [[Long Short-Term Memory (LSTM)]]
@@ Line 19: / Line 19: @@
 * [[Attention]] Mechanism  ...[[Transformer]] Model   ...[[Generative Pre-trained Transformer (GPT)]]
-A Hopfield Network is a form (one particular type) of recurrent artificial neural network popularized by [[Creatives#John J. Hopfield|John Hopfield]] in 1982, but described earlier by Little in 1974. Hopfield networks serve as [http://en.wikipedia.org/wiki/Content-addressable_memory <b>content-addressable ("associative") memory</b>] systems with binary threshold nodes. They are guaranteed to converge to a local minimum and, therefore, may converge to a false pattern (wrong local minimum) rather than the stored pattern (expected local minimum). [http://en.wikipedia.org/wiki/Hopfield_network#:~:text=A%20Hopfield%20network%20is%20a,systems%20with%20binary%20threshold%20nodes. Wikipedia]
+A Hopfield Network is a form (one particular type) of recurrent artificial neural network popularized by [[Creatives#John J. Hopfield|John Hopfield]] in 1982, but described earlier by Little in 1974. Hopfield networks serve as [https://en.wikipedia.org/wiki/Content-addressable_memory <b>content-addressable ("associative") memory</b>] systems with binary threshold nodes. They are guaranteed to converge to a local minimum and, therefore, may converge to a false pattern (wrong local minimum) rather than the stored pattern (expected local minimum). [https://en.wikipedia.org/wiki/Hopfield_network#:~:text=A%20Hopfield%20network%20is%20a,systems%20with%20binary%20threshold%20nodes. Wikipedia]
-Hopfield Network (HN) the weight from node to another and from the later to the former are the same (symmetric). The Hopfield Network (HN) is fully connected, so every neuron’s output is an input to all the other neurons. Another feature of the network is that updating of nodes happens in a binary way. These features allow for a particular feature of Hopfield's nets - they are guaranteed to converge to an attractor (stable state).   [http://stats.stackexchange.com/users/145543/oba2311 oba2311]
+Hopfield Network (HN) the weight from node to another and from the later to the former are the same (symmetric). The Hopfield Network (HN) is fully connected, so every neuron’s output is an input to all the other neurons. Another feature of the network is that updating of nodes happens in a binary way. These features allow for a particular feature of Hopfield's nets - they are guaranteed to converge to an attractor (stable state).   [https://stats.stackexchange.com/users/145543/oba2311 oba2311]
 Every neuron is connected to every other neuron; it is a completely entangled plate of spaghetti as even all the nodes function as everything. Each node is input before training, then hidden during training and output afterwards. The networks are trained by setting the value of the neurons to the desired pattern after which the weights can be computed. The weights do not change after this. Once trained for one or more patterns, the network will always converge to one of the learned patterns because the network is only stable in those states. Note that it does not always conform to the desired state (it’s not a magic black box sadly). It stabilizes in part due to the total “energy” or “temperature” of the network being reduced incrementally during training. Each neuron has an activation threshold which scales to this temperature, which if surpassed by summing the input causes the neuron to take the form of one of two states (usually -1 or 1, sometimes 0 or 1). Updating the network can be done synchronously or more commonly one by one. If updated one by one, a fair random sequence is created to organize which cells update in what order (fair random being all options (n) occurring exactly once every n items). This is so you can tell when the network is stable (done converging), once every cell has been updated and none of them changed, the network is stable (annealed). These networks are often called associative memory because the converge to the most similar state as the input; if humans see half a table we can image the other half, this network will converge to a table if presented with half noise and half a table. Hopfield, John J. “Neural networks and physical systems with emergent collective computational abilities.” Proceedings of the national academy of sciences 79.8 (1982): 2554-2558.
-http://www.asimovinstitute.org/wp-content/uploads/2016/09/hn.png
+https://www.asimovinstitute.org/wp-content/uploads/2016/09/hn.png
 <youtube>vTurUGqD_Lo</youtube>
@@ Line 34: / Line 34: @@
 = John J. Hopfield =
-* [http://en.wikipedia.org/wiki/John_Hopfield John Hopfield | Wikipedia]
+* [https://en.wikipedia.org/wiki/John_Hopfield John Hopfield | Wikipedia]
 {|<!-- T -->
@@ Line 43: / Line 43: @@
 <b>[[Creatives#John J. Hopfield|John Hopfield]]: Physics View of the Mind and Neurobiology | [[Creatives#Lex Fridman|Lex Fridman]] Podcast #76
 </b><br>[[Creatives#John J. Hopfield|John Hopfield]] is professor at Princeton, whose life's work weaved beautifully through biology, chemistry, neuroscience, and physics. Most crucially, he saw the messy world of biology through the piercing eyes of a physicist. He is perhaps best known for his work on associate neural networks, now known as Hopfield Networks (HN) that were one of the early ideas that catalyzed the development of the modern field of deep learning.  EPISODE LINKS:
-Now What? article: http://bit.ly/3843LeU
+Now What? article: https://bit.ly/3843LeU
 |}
 |<!-- M -->
@@ Line 51: / Line 51: @@
 <youtube>c8JZwBRm4ys</youtube>
 <b>[[Creatives#John J. Hopfield|John Hopfield]]: Mind From Machine
-</b><br>The “machine learning” revolution that has brought us self-driving cars, facial recognition and robots who learn can be traced back to [[Creatives#John J. Hopfield|John Hopfield]], whose career is as fascinating as the technologies his ideas helped foster. [[Creatives#John J. Hopfield|John Hopfield]] received the 2019 Benjamin Franklin Medal in Physics. Learn more about his remarkable work: http://bit.ly/2CJyDEJ For 195 years, The Franklin Institute Awards have recognized scientists and engineers who changed the world. http://fi.edu/awards
+</b><br>The “machine learning” revolution that has brought us self-driving cars, facial recognition and robots who learn can be traced back to [[Creatives#John J. Hopfield|John Hopfield]], whose career is as fascinating as the technologies his ideas helped foster. [[Creatives#John J. Hopfield|John Hopfield]] received the 2019 Benjamin Franklin Medal in Physics. Learn more about his remarkable work: https://bit.ly/2CJyDEJ For 195 years, The Franklin Institute Awards have recognized scientists and engineers who changed the world. https://fi.edu/awards
 |}
 |}<!-- B -->
 == Polariton ==
-In physics, polaritons /pəˈlærɪtɒnz, poʊ-/[1] are quasiparticles resulting from strong coupling of electromagnetic waves with an electric or magnetic dipole-carrying excitation.[example needed] They are an expression of the common quantum phenomenon known as level repulsion, also known as the avoided crossing principle. Polaritons describe the crossing of the dispersion of light with any interacting resonance. To this extent polaritons can also be thought as the new normal modes of a given material or structure arising from the strong coupling of the bare modes, which are the photon and the dipolar oscillation. The polariton is a bosonic quasiparticle, and should not be confused with the polaron (a fermionic one), which is an electron plus an attached phonon cloud. [http://en.wikipedia.org/wiki/Polariton Polariton | Wikipedia]
+In physics, polaritons /pəˈlærɪtɒnz, poʊ-/[1] are quasiparticles resulting from strong coupling of electromagnetic waves with an electric or magnetic dipole-carrying excitation.[example needed] They are an expression of the common quantum phenomenon known as level repulsion, also known as the avoided crossing principle. Polaritons describe the crossing of the dispersion of light with any interacting resonance. To this extent polaritons can also be thought as the new normal modes of a given material or structure arising from the strong coupling of the bare modes, which are the photon and the dipolar oscillation. The polariton is a bosonic quasiparticle, and should not be confused with the polaron (a fermionic one), which is an electron plus an attached phonon cloud. [https://en.wikipedia.org/wiki/Polariton Polariton | Wikipedia]
 == Kinetic Proofreading ==
 Kinetic proofreading (or kinetic amplification) is a mechanism for error correction in biochemical reactions, proposed independently by John Hopfield (1974) and Jacques Ninio (1975). Kinetic proofreading allows enzymes to discriminate between two possible reaction pathways leading to correct or incorrect products with an accuracy higher than what one would predict based on the difference in the activation energy between these two pathways.
-Increased specificity is obtained by introducing an irreversible step exiting the pathway, with reaction intermediates leading to incorrect products more likely to prematurely exit the pathway than reaction intermediates leading to the correct product. If the exit step is fast relative to the next step in the pathway, the specificity can be increased by a factor of up to the ratio between the two exit rate constants. (If the next step is fast relative to the exit step, specificity will not be increased because there will not be enough time for exit to occur.) This can be repeated more than once to increase specificity further. [http://en.wikipedia.org/wiki/Kinetic_proofreading Kinetic proofreading | Wikipedia]
+Increased specificity is obtained by introducing an irreversible step exiting the pathway, with reaction intermediates leading to incorrect products more likely to prematurely exit the pathway than reaction intermediates leading to the correct product. If the exit step is fast relative to the next step in the pathway, the specificity can be increased by a factor of up to the ratio between the two exit rate constants. (If the next step is fast relative to the exit step, specificity will not be increased because there will not be enough time for exit to occur.) This can be repeated more than once to increase specificity further. [https://en.wikipedia.org/wiki/Kinetic_proofreading Kinetic proofreading | Wikipedia]

@@ Line 18: / Line 18: @@
 ** Hopfield Network (HN)
-A Hopfield Network is a form (one particular type) of recurrent artificial neural network popularized by John Hopfield in 1982, but described earlier by Little in 1974. Hopfield networks serve as content-addressable ("associative") memory systems with binary threshold nodes. They are guaranteed to converge to a local minimum and, therefore, may converge to a false pattern (wrong local minimum) rather than the stored pattern (expected local minimum). [http://en.wikipedia.org/wiki/Hopfield_network#:~:text=A%20Hopfield%20network%20is%20a,systems%20with%20binary%20threshold%20nodes. Wikipedia]
+A Hopfield Network is a form (one particular type) of recurrent artificial neural network popularized by [[Creatives#John J. Hopfield|John Hopfield]] in 1982, but described earlier by Little in 1974. Hopfield networks serve as <b>content-addressable ("associative") memory</b> systems with binary threshold nodes. They are guaranteed to converge to a local minimum and, therefore, may converge to a false pattern (wrong local minimum) rather than the stored pattern (expected local minimum). [http://en.wikipedia.org/wiki/Hopfield_network#:~:text=A%20Hopfield%20network%20is%20a,systems%20with%20binary%20threshold%20nodes. Wikipedia]
-Hopfield Network the weight from node to another and from the later to the former are the same (symmetric). The Hopfield Network is fully connected, so every neuron’s output is an input to all the other neurons. Another feature of the network is that updating of nodes happens in a binary way. These features allow for a particular feature of Hopfield's nets - they are guaranteed to converge to an attractor (stable state).   [http://stats.stackexchange.com/users/145543/oba2311 oba2311]
+Hopfield Network (HN) the weight from node to another and from the later to the former are the same (symmetric). The Hopfield Network (HN) is fully connected, so every neuron’s output is an input to all the other neurons. Another feature of the network is that updating of nodes happens in a binary way. These features allow for a particular feature of Hopfield's nets - they are guaranteed to converge to an attractor (stable state).   [http://stats.stackexchange.com/users/145543/oba2311 oba2311]
 Every neuron is connected to every other neuron; it is a completely entangled plate of spaghetti as even all the nodes function as everything. Each node is input before training, then hidden during training and output afterwards. The networks are trained by setting the value of the neurons to the desired pattern after which the weights can be computed. The weights do not change after this. Once trained for one or more patterns, the network will always converge to one of the learned patterns because the network is only stable in those states. Note that it does not always conform to the desired state (it’s not a magic black box sadly). It stabilizes in part due to the total “energy” or “temperature” of the network being reduced incrementally during training. Each neuron has an activation threshold which scales to this temperature, which if surpassed by summing the input causes the neuron to take the form of one of two states (usually -1 or 1, sometimes 0 or 1). Updating the network can be done synchronously or more commonly one by one. If updated one by one, a fair random sequence is created to organize which cells update in what order (fair random being all options (n) occurring exactly once every n items). This is so you can tell when the network is stable (done converging), once every cell has been updated and none of them changed, the network is stable (annealed). These networks are often called associative memory because the converge to the most similar state as the input; if humans see half a table we can image the other half, this network will converge to a table if presented with half noise and half a table. Hopfield, John J. “Neural networks and physical systems with emergent collective computational abilities.” Proceedings of the national academy of sciences 79.8 (1982): 2554-2558.