Memory Networks - Revision history

BPeat at 00:56, 29 April 2024

2024-04-29T00:56:20Z

BPeat at 03:25, 2 March 2024

2024-03-02T03:25:43Z

BPeat at 02:53, 2 March 2024

2024-03-02T02:53:03Z

BPeat at 00:08, 9 September 2023

2023-09-09T00:08:18Z

BPeat at 14:03, 18 August 2023

2023-08-18T14:03:52Z

BPeat: /* How are Memories Stored in Neural Networks? */

2023-05-22T10:31:45Z

‎How are Memories Stored in Neural Networks?

BPeat at 15:25, 20 May 2023

2023-05-20T15:25:18Z

BPeat: /* How are Memories Stored in Neural Networks? */

2023-05-20T15:19:27Z

‎How are Memories Stored in Neural Networks?

BPeat: /* How are Memories Stored in Neural Networks? */

2023-05-20T15:18:08Z

‎How are Memories Stored in Neural Networks?

BPeat: /* How are Memories Stored in Neural Networks? */

2023-05-20T15:13:58Z

‎How are Memories Stored in Neural Networks?

@@ Line 11: / Line 11: @@
 [https://www.bing.com/news/search?q=Memory+Network+attention+model+ai&qft=interval%3d%228%22 ...Bing News]
 * [[Attention]] Mechanism  ...[[Transformer]] ...[[Generative Pre-trained Transformer (GPT)]] ... [[Generative Adversarial Network (GAN)|GAN]] ... [[Bidirectional Encoder Representations from Transformers (BERT)|BERT]]
 * [[Feature Exploration/Learning]]
 * [[Architectures]]

@@ Line 48: / Line 48: @@
 In the context of artificial [[neural network]]s, memories are stored in the weights of the connections between neurons. These weights determine the degree to which a given neuron firing activates or inhibits each of the neurons it’s connected to. Through a process called training, the network adjusts these weights to improve its ability to recognize patterns and make predictions. The capacity of a neural network can be controlled by two aspects of the model: the number of nodes and the number of layers. A model with more nodes or more layers has a greater capacity and, in turn, is potentially capable of learning a larger set of mapping functions. A model with more layers and more hidden units per layer has higher representational capacity.
-Incorporating implicit memory into AI systems can potentially improve their ability to learn and adapt to new situations. For example, an AI system with implicit memory could use its past experiences to make better predictions or decisions without explicitly being told to do so.  One approach to incorporating implicit memory into AI systems is through the use of [[Recurrent Neural Network (RNN)]]s. RNNs are a type of [[neural network]] that can process sequential data and have been used to model various types of memory, including implicit memory. RNNs have been used in a variety of applications, including [[Natural Language Processing (NLP)]] and [[Speech Recognition]].
+Incorporating implicit [[memory]] into AI systems can potentially improve their ability to learn and adapt to new situations. For example, an AI system with implicit [[memory]] could use its past experiences to make better predictions or decisions without explicitly being told to do so.  One approach to incorporating implicit [[memory]] into AI systems is through the use of [[Recurrent Neural Network (RNN)]]s. RNNs are a type of [[neural network]] that can process sequential data and have been used to model various types of [[memory]], including implicit [[memory]]. RNNs have been used in a variety of applications, including [[Natural Language Processing (NLP)]] and [[Speech Recognition]].
-While GPT models do not have an explicit memory component like some other types of [[neural network]]s, they do have the ability to generate coherent and [[context]]ually relevant text by [[Attention|attending]] to the relevant parts of the input. In this sense, GPT models can be seen as having a form of implicit memory that allows them to generate coherent and [[context]]ually relevant text.
+While GPT models do not have an explicit [[memory]] component like some other types of [[neural network]]s, they do have the ability to generate coherent and [[context]]ually relevant text by [[Attention|attending]] to the relevant parts of the input. In this sense, GPT models can be seen as having a form of implicit [[memory]] that allows them to generate coherent and [[context]]ually relevant text.
 <img src="https://joostdefolter.info/images/games/asnn_deepmind_small.gif" width="800">
-There are several types of memory, including episodic, semantic, procedural, and associative memories:
+There are several types of [[memory]], including episodic, semantic, procedural, and associative memories:
-* <b>Episodic</b> memory is when a person recalls a particular event experienced in the past. This kind of long-term memory brings to attention details about anything from what one ate for breakfast to the emotions that were stirred up during a serious conversation with a romantic partner. The temporal component is inherent within episodic memory. The experiences conjured by episodic memory can be very recent or decades-old. A related concept is autobiographical memory, which is the memory of information that forms part of a person’s life story. However, while autobiographical memory includes memories of events in one’s life (such as one’s sixteenth birthday party), it can also encompass facts (such as one’s birth date) and other non-episodic forms of information.
+* <b>Episodic</b> [[memory]] is when a person recalls a particular event experienced in the past. This kind of long-term [[memory]] brings to attention details about anything from what one ate for breakfast to the emotions that were stirred up during a serious conversation with a romantic partner. The temporal component is inherent within episodic [[memory]]. The experiences conjured by episodic [[memory]] can be very recent or decades-old. A related concept is autobiographical [[memory]], which is the [[memory]] of information that forms part of a person’s life story. However, while autobiographical [[memory]] includes memories of events in one’s life (such as one’s sixteenth birthday party), it can also encompass facts (such as one’s birth date) and other non-episodic forms of information.
-* <b>Semantic</b> memory is someone’s long-term store of knowledge: It’s composed of pieces of information such as facts learned in school, what concepts mean and how they are related, or the definition of a particular word. The details that make up semantic memory can correspond to other forms of memory. One may remember factual details about a party, for instance—what time it started, at whose house it took place, how many people were there, all part of semantic memory—in addition to recalling the sounds heard and excitement felt. But semantic memory can also include facts and meanings related to people, places, or things one has no direct relation to.
+* <b>Semantic</b> [[memory]] is someone’s long-term store of knowledge: It’s composed of pieces of information such as facts learned in school, what concepts mean and how they are related, or the definition of a particular word. The details that make up semantic [[memory]] can correspond to other forms of [[memory]]. One may remember factual details about a party, for instance—what time it started, at whose house it took place, how many people were there, all part of semantic [[memory]]—in addition to recalling the sounds heard and excitement felt. But semantic [[memory]] can also include facts and meanings related to people, places, or things one has no direct relation to.
-* <b>Procedural</b> memory is the memory of how to perform certain actions or tasks. It is sometimes referred to as “muscle memory” because it often involves physical actions such as riding a bike or playing a musical instrument. Procedural memory is typically acquired through repetition and practice and is often resistant to forgetting.
+* <b>Procedural</b> [[memory]] is the [[memory]] of how to perform certain actions or tasks. It is sometimes referred to as “muscle [[memory]]” because it often involves physical actions such as riding a bike or playing a musical instrument. Procedural [[memory]] is typically acquired through repetition and practice and is often resistant to forgetting.
-* <b>Associative</b> memory specifically deals with remembering the relationship between different objects or concepts. For example, remembering the name of someone or the aroma of a particular perfume would be an example of associative memory. Associative Memory refers to the ability to remember relationships between concepts, and not just the individual concepts themselves. In humans, this relates to visual and verbal information, such as remembering how two words are related (e.g., man – woman), or seeing an object and its alternate name (e.g., a guitar). Associative memory is thought to be mediated by the medial temporal lobe of the brain.
+* <b>Associative</b> [[memory]] specifically deals with remembering the relationship between different objects or concepts. For example, remembering the name of someone or the aroma of a particular perfume would be an example of associative [[memory]]. Associative [[Memory]] refers to the ability to remember relationships between concepts, and not just the individual concepts themselves. In humans, this relates to visual and verbal information, such as remembering how two words are related (e.g., man – woman), or seeing an object and its alternate name (e.g., a guitar). Associative [[memory]] is thought to be mediated by the medial temporal lobe of the brain.
 <youtube>piF6D6CQxUw</youtube>
 <youtube>Q57rzaHHO0k</youtube>

@@ Line 12: / Line 12: @@
 * [[Attention]] Mechanism  ...[[Transformer]] ...[[Generative Pre-trained Transformer (GPT)]] ... [[Generative Adversarial Network (GAN)|GAN]] ... [[Bidirectional Encoder Representations from Transformers (BERT)|BERT]]
 * [[Feature Exploration/Learning]]
 * [[Architectures]]

@@ Line 17: / Line 17: @@
 * [https://journals.sagepub.com/doi/full/10.1177/1729881418775849 Memory-based reinforcement learning algorithm for autonomous exploration in unknown environment | Amir Ramezani Dooraki & Deok Jin Lee]
 * [[Deep Reinforcement Learning (DRL)]]
-* [[Singularity]] ... [[Artificial Consciousness / Sentience|Sentience]] ... [[Artificial General Intelligence (AGI)| AGI]] ... [[Inside Out - Curious Optimistic Reasoning| Curious Reasoning]] ... [[Emergence]] ... [[Moonshots]] ... [[Explainable / Interpretable AI|Explainable AI]] ...  [[Algorithm Administration#Automated Learning|Automated Learning]]
+* [[Artificial General Intelligence (AGI) to Singularity]] ... [[Inside Out - Curious Optimistic Reasoning| Curious Reasoning]] ... [[Emergence]] ... [[Moonshots]] ... [[Explainable / Interpretable AI|Explainable AI]] ...  [[Algorithm Administration#Automated Learning|Automated Learning]]
 ** [[Inside Out - Curious Optimistic Reasoning#MERLIN|MERLIN]]
 * [https://skymind.ai/wiki/attention-mechanism-memory-network A Beginner's Guide to Attention Mechanisms and Memory Networks | Skymind]

@@ Line 14: / Line 14: @@
 * [[Feature Exploration/Learning]]
 * [[Architectures]]
-* [https://arxiv.org/pdf/1605.09128.pdf Control of Memory, Active Perception, and Action in Minecraft | J. Oh, V. Chockalingam, S. Singh, & H. Lee]
+* [https://arxiv.org/pdf/1605.09128.pdf Control of Memory, Active Perception, and Action in Minecraft | J. Oh, V. Chockalingam, S. Singh, & H. Lee]  ... [[Minecraft]]
 * [https://journals.sagepub.com/doi/full/10.1177/1729881418775849 Memory-based reinforcement learning algorithm for autonomous exploration in unknown environment | Amir Ramezani Dooraki & Deok Jin Lee]
 * [[Deep Reinforcement Learning (DRL)]]

← Older revision		Revision as of 15:18, 20 May 2023
Line 47:		Line 47:

	Incorporating implicit memory into AI systems can potentially improve their ability to learn and adapt to new situations. For example, an AI system with implicit memory could use its past experiences to make better predictions or decisions without explicitly being told to do so. One approach to incorporating implicit memory into AI systems is through the use of [[Recurrent Neural Network (RNN)]]s. RNNs are a type of [[neural network]] that can process sequential data and have been used to model various types of memory, including implicit memory. RNNs have been used in a variety of applications, including [[Natural Language Processing (NLP)]] and [[Speech Recognition]].		Incorporating implicit memory into AI systems can potentially improve their ability to learn and adapt to new situations. For example, an AI system with implicit memory could use its past experiences to make better predictions or decisions without explicitly being told to do so. One approach to incorporating implicit memory into AI systems is through the use of [[Recurrent Neural Network (RNN)]]s. RNNs are a type of [[neural network]] that can process sequential data and have been used to model various types of memory, including implicit memory. RNNs have been used in a variety of applications, including [[Natural Language Processing (NLP)]] and [[Speech Recognition]].
		+
		+	While GPT models do not have an explicit memory component like some other types of [[neural network]]s, they do have the ability to generate coherent and [[context]]ually relevant text by [[Attention\|attending]] to the relevant parts of the input. In this sense, GPT models can be seen as having a form of implicit memory that allows them to generate coherent and [[context]]ually relevant text.

	<img src="https://joostdefolter.info/images/games/asnn_deepmind_small.gif" width="800">		<img src="https://joostdefolter.info/images/games/asnn_deepmind_small.gif" width="800">

@@ Line 44: / Line 44: @@
 * [https://joostdefolter.info/games/associative-neural-networks Associative Neural Networks | Joost de Folter]
-In the context of artificial [[neural network]]s, memories are stored in the weights of the connections between neurons. These weights determine the degree to which a given neuron firing activates or inhibits each of the neurons it’s connected to. Through a process called training, the network adjusts these weights to improve its ability to recognize patterns and make predictions. The capacity of a neural network can be controlled by two aspects of the model: the number of nodes and the number of layers. A model with more nodes or more layers has a greater capacity and, in turn, is potentially capable of learning a larger set of mapping functions. A model with more layers and more hidden units per layer has higher representational capacity .
+In the context of artificial [[neural network]]s, memories are stored in the weights of the connections between neurons. These weights determine the degree to which a given neuron firing activates or inhibits each of the neurons it’s connected to. Through a process called training, the network adjusts these weights to improve its ability to recognize patterns and make predictions. The capacity of a neural network can be controlled by two aspects of the model: the number of nodes and the number of layers. A model with more nodes or more layers has a greater capacity and, in turn, is potentially capable of learning a larger set of mapping functions. A model with more layers and more hidden units per layer has higher representational capacity.
 <img src="https://joostdefolter.info/images/games/asnn_deepmind_small.gif" width="800">