Memory Networks
YouTube ... Quora ...Google search ...Google News ...Bing News
- Memory ... Memory Networks ... Hierarchical Temporal Memory (HTM) ... Lifelong Learning
- Attention Mechanism ...Transformer ...Generative Pre-trained Transformer (GPT) ... GAN ... BERT
- Feature Exploration/Learning
- Architectures
- Control of Memory, Active Perception, and Action in Minecraft | J. Oh, V. Chockalingam, S. Singh, & H. Lee ... Minecraft
- Memory-based reinforcement learning algorithm for autonomous exploration in unknown environment | Amir Ramezani Dooraki & Deok Jin Lee
- Deep Reinforcement Learning (DRL)
- Artificial General Intelligence (AGI) to Singularity ... Curious Reasoning ... Emergence ... Moonshots ... Explainable AI ... Automated Learning
- A Beginner's Guide to Attention Mechanisms and Memory Networks | Skymind
- Task-Oriented Conversation Generation Using Heterogeneous Memory Networks | Z. Lin, X. Huang, F. Ji, H. Chen, & Y. Zhang
How are Memories Stored in Neural Networks?
YouTube ... Quora ...Google search ...Google News ...Bing News
- Attention Mechanism ...Transformer ...Generative Pre-trained Transformer (GPT) ... GAN ... BERT
- Associative Neural Networks | Joost de Folter
In the context of artificial neural networks, memories are stored in the weights of the connections between neurons. These weights determine the degree to which a given neuron firing activates or inhibits each of the neurons it’s connected to. Through a process called training, the network adjusts these weights to improve its ability to recognize patterns and make predictions. The capacity of a neural network can be controlled by two aspects of the model: the number of nodes and the number of layers. A model with more nodes or more layers has a greater capacity and, in turn, is potentially capable of learning a larger set of mapping functions. A model with more layers and more hidden units per layer has higher representational capacity.
Incorporating implicit memory into AI systems can potentially improve their ability to learn and adapt to new situations. For example, an AI system with implicit memory could use its past experiences to make better predictions or decisions without explicitly being told to do so. One approach to incorporating implicit memory into AI systems is through the use of Recurrent Neural Network (RNN)s. RNNs are a type of neural network that can process sequential data and have been used to model various types of memory, including implicit memory. RNNs have been used in a variety of applications, including Natural Language Processing (NLP) and Speech Recognition.
While GPT models do not have an explicit memory component like some other types of neural networks, they do have the ability to generate coherent and contextually relevant text by attending to the relevant parts of the input. In this sense, GPT models can be seen as having a form of implicit memory that allows them to generate coherent and contextually relevant text.
There are several types of memory, including episodic, semantic, procedural, and associative memories:
- Episodic memory is when a person recalls a particular event experienced in the past. This kind of long-term memory brings to attention details about anything from what one ate for breakfast to the emotions that were stirred up during a serious conversation with a romantic partner. The temporal component is inherent within episodic memory. The experiences conjured by episodic memory can be very recent or decades-old. A related concept is autobiographical memory, which is the memory of information that forms part of a person’s life story. However, while autobiographical memory includes memories of events in one’s life (such as one’s sixteenth birthday party), it can also encompass facts (such as one’s birth date) and other non-episodic forms of information.
- Semantic memory is someone’s long-term store of knowledge: It’s composed of pieces of information such as facts learned in school, what concepts mean and how they are related, or the definition of a particular word. The details that make up semantic memory can correspond to other forms of memory. One may remember factual details about a party, for instance—what time it started, at whose house it took place, how many people were there, all part of semantic memory—in addition to recalling the sounds heard and excitement felt. But semantic memory can also include facts and meanings related to people, places, or things one has no direct relation to.
- Procedural memory is the memory of how to perform certain actions or tasks. It is sometimes referred to as “muscle memory” because it often involves physical actions such as riding a bike or playing a musical instrument. Procedural memory is typically acquired through repetition and practice and is often resistant to forgetting.
- Associative memory specifically deals with remembering the relationship between different objects or concepts. For example, remembering the name of someone or the aroma of a particular perfume would be an example of associative memory. Associative Memory refers to the ability to remember relationships between concepts, and not just the individual concepts themselves. In humans, this relates to visual and verbal information, such as remembering how two words are related (e.g., man – woman), or seeing an object and its alternate name (e.g., a guitar). Associative memory is thought to be mediated by the medial temporal lobe of the brain.
