Memory - Revision history

BPeat at 20:25, 5 January 2026

2026-01-05T20:25:23Z

BPeat at 15:21, 28 May 2025

2025-05-28T15:21:09Z

BPeat at 00:53, 29 April 2024

2024-04-29T00:53:06Z

BPeat at 00:51, 29 April 2024

2024-04-29T00:51:45Z

BPeat at 00:50, 29 April 2024

2024-04-29T00:50:42Z

BPeat at 20:43, 28 April 2024

2024-04-28T20:43:44Z

BPeat at 14:06, 23 March 2024

2024-03-23T14:06:01Z

BPeat at 22:55, 3 March 2024

2024-03-03T22:55:31Z

BPeat at 22:54, 3 March 2024

2024-03-03T22:54:33Z

BPeat: /* Technological Workarounds for Memory Limitations */

2024-03-03T16:40:05Z

‎Technological Workarounds for Memory Limitations

← Older revision		Revision as of 20:25, 5 January 2026
Line 38:		Line 38:
	** [[Hopfield Network (HN)]]		** [[Hopfield Network (HN)]]
	* [[Decentralized: Federated & Distributed]]		* [[Decentralized: Federated & Distributed]]
		+	* [[Life~Meaning]] ... [[Consciousness]] ... [[Loop#Feedback Loop - Creating Consciousness\|Creating Consciousness]] ... [[Quantum#Quantum Biology\|Quantum Biology]] ... [[Orch-OR]] ... [[TAME]] ... [[Protein Folding & Discovery\|Proteins]]

@@ Line 2: / Line 2: @@
 |title=PRIMO.ai
 |titlemode=append
-|keywords=ChatGPT, artificial, intelligence, machine, learning, GPT-4, GPT-5, NLP, NLG, NLC, NLU, models, data, singularity, moonshot, Sentience, AGI, Emergence, Moonshot, Explainable, TensorFlow, Google, Nvidia, Microsoft, Azure, Amazon, AWS, Hugging Face, OpenAI, Tensorflow, OpenAI, Google, Nvidia, Microsoft, Azure, Amazon, AWS, Meta, LLM, metaverse, assistants, agents, digital twin, IoT, Transhumanism, Immersive Reality, Generative AI, Conversational AI, Perplexity, Bing, You, Bard, Ernie, prompt Engineering LangChain, Video/Image, Vision, End-to-End Speech, Synthesize Speech, Speech Recognition, Stanford, MIT |description=Helpful resources for your journey with artificial intelligence; videos, articles, techniques, courses, profiles, and tools
+|keywords=ChatGPT, artificial, intelligence, machine, learning, NLP, NLG, NLC, NLU, models, data, singularity, moonshot, Sentience, AGI, Emergence, Moonshot, Explainable, TensorFlow, Google, Nvidia, Microsoft, Azure, Amazon, AWS, Hugging Face, OpenAI, Tensorflow, OpenAI, Google, Nvidia, Microsoft, Azure, Amazon, AWS, Meta, LLM, metaverse, assistants, agents, digital twin, IoT, Transhumanism, Immersive Reality, Generative AI, Conversational AI, Perplexity, Bing, You, Bard, Ernie, prompt Engineering LangChain, Video/Image, Vision, End-to-End Speech, Synthesize Speech, Speech Recognition, Stanford, MIT |description=Helpful resources for your journey with artificial intelligence; videos, articles, techniques, courses, profiles, and tools
 <!-- Google tag (gtag.js) -->
@@ Line 28: / Line 28: @@
 * [[Excel]] ... [[LangChain#Documents|Documents]] ... [[Database|Database; Vector & Relational]] ... [[Graph]] ... [[LlamaIndex]]
 * [[Embedding]] ... [[Fine-tuning]] ... [[Retrieval-Augmented Generation (RAG)|RAG]] ... [[Agents#AI-Powered Search|Search]] ... [[Clustering]] ... [[Recommendation]] ... [[Anomaly Detection]] ... [[Classification]] ... [[Dimensional Reduction]].  [[...find outliers]]
-* [[Large Language Model (LLM)]] ... [[Large Language Model (LLM)#Multimodal|Multimodal]] ... [[Foundation Models (FM)]] ... [[Generative Pre-trained Transformer (GPT)|Generative Pre-trained]] ... [[Transformer]] ... [[GPT-4]] ... [[GPT-5]] ... [[Attention]] ... [[Generative Adversarial Network (GAN)|GAN]] ... [[Bidirectional Encoder Representations from Transformers (BERT)|BERT]]
+* [[Large Language Model (LLM)]] ... [[Large Language Model (LLM)#Multimodal|Multimodal]] ... [[Foundation Models (FM)]] ... [[Generative Pre-trained Transformer (GPT)|Generative Pre-trained]] ... [[Transformer]] ... [[Attention]] ... [[Generative Adversarial Network (GAN)|GAN]] ... [[Bidirectional Encoder Representations from Transformers (BERT)|BERT]]
 * [[Recurrent Neural Network (RNN)]] Variants:
 ** [[Long Short-Term Memory (LSTM)]]

@@ Line 20: / Line 20: @@
 [https://www.bing.com/news/search?q=memory+ai&qft=interval%3d%228%22 ...Bing News]
-* [[Memory]] ... [[Memory Networks]] ... [[Hierarchical Temporal Memory (HTM)]]
+* [[Memory]] ... [[Memory Networks]] ... [[Hierarchical Temporal Memory (HTM)]] ... [[Lifelong Learning]]
 * [[State Space Model (SSM)]] ... [[Mamba]] ... [[Sequence to Sequence (Seq2Seq)]] ... [[Recurrent Neural Network (RNN)]] ... [[(Deep) Convolutional Neural Network (DCNN/CNN)|Convolutional Neural Network (CNN)]]
 * [[Mixture-of-Experts (MoE)]]
@@ Line 37: / Line 37: @@
 ** [[Average-Stochastic Gradient Descent (SGD) Weight-Dropped LSTM (AWD-LSTM)]]
 ** [[Hopfield Network (HN)]]
 * [[Decentralized: Federated & Distributed]]

@@ Line 20: / Line 20: @@
 [https://www.bing.com/news/search?q=memory+ai&qft=interval%3d%228%22 ...Bing News]
-* [[Memory]] ... [[Memory Networks]]
+* [[Memory]] ... [[Memory Networks]] ... [[Hierarchical Temporal Memory (HTM)]]
 * [[State Space Model (SSM)]] ... [[Mamba]] ... [[Sequence to Sequence (Seq2Seq)]] ... [[Recurrent Neural Network (RNN)]] ... [[(Deep) Convolutional Neural Network (DCNN/CNN)|Convolutional Neural Network (CNN)]]
 * [[Mixture-of-Experts (MoE)]]
@@ Line 29: / Line 29: @@
 * [[Embedding]] ... [[Fine-tuning]] ... [[Retrieval-Augmented Generation (RAG)|RAG]] ... [[Agents#AI-Powered Search|Search]] ... [[Clustering]] ... [[Recommendation]] ... [[Anomaly Detection]] ... [[Classification]] ... [[Dimensional Reduction]].  [[...find outliers]]
 * [[Large Language Model (LLM)]] ... [[Large Language Model (LLM)#Multimodal|Multimodal]] ... [[Foundation Models (FM)]] ... [[Generative Pre-trained Transformer (GPT)|Generative Pre-trained]] ... [[Transformer]] ... [[GPT-4]] ... [[GPT-5]] ... [[Attention]] ... [[Generative Adversarial Network (GAN)|GAN]] ... [[Bidirectional Encoder Representations from Transformers (BERT)|BERT]]
 * [[Recurrent Neural Network (RNN)]] Variants:
 ** [[Long Short-Term Memory (LSTM)]]

@@ Line 20: / Line 20: @@
 [https://www.bing.com/news/search?q=memory+ai&qft=interval%3d%228%22 ...Bing News]
-* [[Memory Networks]]
+* [[Memory]] ... [[Memory Networks]]
 * [[State Space Model (SSM)]] ... [[Mamba]] ... [[Sequence to Sequence (Seq2Seq)]] ... [[Recurrent Neural Network (RNN)]] ... [[(Deep) Convolutional Neural Network (DCNN/CNN)|Convolutional Neural Network (CNN)]]
 * [[Perspective]] ... [[Context]] ... [[In-Context Learning (ICL)]] ... [[Transfer Learning]] ... [[Out-of-Distribution (OOD) Generalization]]
 * [[Causation vs. Correlation]] ... [[Autocorrelation]] ...[[Convolution vs. Cross-Correlation (Autocorrelation)]]

@@ Line 23: / Line 23: @@
 * [[State Space Model (SSM)]] ... [[Mamba]] ... [[Sequence to Sequence (Seq2Seq)]] ... [[Recurrent Neural Network (RNN)]] ... [[(Deep) Convolutional Neural Network (DCNN/CNN)|Convolutional Neural Network (CNN)]]
 * [[In-Context Learning (ICL)]] ... [[Context]] ... [[Causation vs. Correlation]] ... [[Autocorrelation]] ... [[Out-of-Distribution (OOD) Generalization]] ... [[Transfer Learning]]
-* [[Assistants]] ... [[Personal Companions]] ... [[Agents]]  ... [[Negotiation]] ... [[LangChain]]
+* [[Agents]] ... [[Robotic Process Automation (RPA)|Robotic Process Automation]] ... [[Assistants]] ... [[Personal Companions]] ... [[Personal Productivity|Productivity]] ... [[Email]] ... [[Negotiation]] ... [[LangChain]]
 * [[Excel]] ... [[LangChain#Documents|Documents]] ... [[Database|Database; Vector & Relational]] ... [[Graph]] ... [[LlamaIndex]]
 * [[Embedding]] ... [[Fine-tuning]] ... [[Retrieval-Augmented Generation (RAG)|RAG]] ... [[Agents#AI-Powered Search|Search]] ... [[Clustering]] ... [[Recommendation]] ... [[Anomaly Detection]] ... [[Classification]] ... [[Dimensional Reduction]].  [[...find outliers]]

← Older revision		Revision as of 22:55, 3 March 2024
Line 125:		Line 125:
	== Impact on the Field ==		== Impact on the Field ==
	The latest research and products in memory AI are reshaping the field by addressing the challenges of catastrophic forgetting and controlled forgetting. These advancements are crucial for the development of AI systems capable of lifelong learning, trustworthy AI, and personalized user experiences. The semiconductor industry is also adapting to these changes, with a focus on memory enhancements to support the growing needs of AI servers and applications.		The latest research and products in memory AI are reshaping the field by addressing the challenges of catastrophic forgetting and controlled forgetting. These advancements are crucial for the development of AI systems capable of lifelong learning, trustworthy AI, and personalized user experiences. The semiconductor industry is also adapting to these changes, with a focus on memory enhancements to support the growing needs of AI servers and applications.
−
−	~~--------------------------------------------------------~~
−
−	~~===Memory Networks===~~
−	'''State Space Models (SSMs)''' are used for modeling and analyzing dynamical systems, allowing for the representation of both the state and input/output relationships<ref>{{Cite book \|last=Ljung \|first=Lennart \|title=System Identification: Theory for the User \|date=1999 \|publisher=Prentice Hall PTR \|isbn=978-0138816545}}</ref>. '''Mamba''' is a sequence-to-sequence (seq2seq) framework for multimodal data<ref>{{Cite journal \|last1=Tsai \|first1=Yu-Hsiang \|last2=Bhushan \|first2=Salil \|last3=Yang \|first3=Yi-Ren \|last4=Xue \|first4=Douglas \|last5=Asai \|first5=Akiko \|last6=Misailovic \|first6=Sasa \|last7=Dmello \|first7=Rafael \|year=2020 \|title=MUNAL: Multimodal Abstraction for Conversational Question Answering over Real-World Multimodal Data \|journal=Findings of the Association for Computational Linguistics (ACL) \|url=https://aclanthology.org/2020.findings-emnlp.208.pdf}}</ref>. '''Recurrent Neural Networks (RNNs)''' are a type of neural network well-suited for processing sequential data, with feedback connections that allow the network to maintain an internal state<ref>{{Cite book \|last1=Goodfellow \|first1=Ian \|last2=Bengio \|first2=Yoshua \|last3=Courville \|first3=Aaron \|title=Deep Learning \|date=2016 \|publisher=MIT Press \|isbn=978-0262035613}}</ref>. '''Convolutional Neural Networks (CNNs)''' are a class of deep neural networks primarily used for analyzing visual imagery, leveraging the spatial and temporal dependencies in the data<ref>{{Cite journal \|last1=Krizhevsky \|first1=Alex \|last2=Sutskever \|first2=Ilya \|last3=Hinton \|first3=Geoffrey \|year=2017 \|title=ImageNet classification with deep convolutional neural networks \|journal=Communications of the ACM \|volume=60 \|issue=6 \|pages=84–90 \|doi=10.1145/3065386 \|url=https://dl.acm.org/doi/10.1145/3065386}}</ref>.
−
−	'''In-Context Learning (ICL)''' refers to the ability of large language models to adapt their behavior based on the context provided in the input prompt, without the need for explicit fine-tuning or training on additional data<ref>{{Cite journal \|last1=Wei \|first1=Jason \|last2=Tay \|first2=Yi \|last3=Bommasani \|first3=Rishi \|last4=Aziz \|first4=Aden \|last5=Hechtrespector \|first5=Ari \|last6=Plu \|first6=Juliana \|last7=Yu \|first7=Karan \|last8=Narasimhan \|first8=Kianywee \|year=2022 \|title=Emergent Abilities of Large Language Models \|journal=arXiv preprint arXiv:2206.07682 \|url=https://arxiv.org/abs/2206.07682}}</ref>. '''Causation vs. Correlation''' is a fundamental distinction in statistics, where causation implies a direct cause-and-effect relationship, while correlation simply indicates a statistical association between variables<ref>{{Cite book \|last=Pearl \|first=Judea \|title=Causality: Models, Reasoning, and Inference \|date=2009 \|publisher=Cambridge University Press \|isbn=978-0521895606}}</ref>. '''Autocorrelation''' refers to the correlation of a signal or time series with a delayed copy of itself, which can reveal patterns and dependencies in the data<ref>{{Cite book \|last1=Box \|first1=George E.P. \|last2=Jenkins \|first2=Gwilym M. \|last3=Reinsel \|first3=Gregory C. \|title=Time Series Analysis: Forecasting and Control \|date=2008 \|publisher=John Wiley & Sons \|isbn=978-0470272848}}</ref>. '''Out-of-Distribution (OOD) Generalization''' is the ability of a machine learning model to perform well on data that differs from the distribution of the training data, which is a significant challenge in deploying models in real-world scenarios<ref>{{Cite journal \|last1=Hendrycks \|first1=Dan \|last2=Basart \|first2=Steven \|last3=Mu \|first3=Norman \|last4=Kadavath \|first4=Saurav \|last5=Wang \|first5=Frank \|last6=Dorundo \|first6=Evan \|last7=Desai \|first7=Rahul \|last8=Zhu \|first8=Tyler \|last9=Parekh \|first9=Samyak \|last10=Maynez \|first10=Matt \|last11=Colombo \|first11=Gabriel \|year=2021 \|title=The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization \|journal=Proceedings of the IEEE/CVF International Conference on Computer Vision \|pages=8340–8349 \|url=https://arxiv.org/abs/2006.16241}}</ref>. '''Transfer Learning''' involves leveraging knowledge gained from one task to improve performance on a related but different task, which can be beneficial when working with limited data or computational resources<ref>{{Cite journal \|last1=Zhuang \|first1=Fuzhen \|last2=Qi \|first2=Zhiyuan \|last3=Duan \|first3=Keyu \|last4=Xi \|first4=Dongbo \|last5=Zhu \|first5=Yongchun \|last6=Zhu \|first6=Hengshu \|last7=Xiong \|first7=Hui \|last8=He \|first8=Qing \|year=2021 \|title=A Comprehensive Survey on Transfer Learning \|journal=Proceedings of the IEEE \|volume=109 \|issue=1 \|pages=43–76 \|doi=10.1109/JPROC.2020.3004555 \|url=https://arxiv.org/abs/1911.02685}}</ref>.
−
−	'''Assistants''', '''Personal Companions''', and '''Agents''' are terms used to describe AI systems designed to interact with humans and assist them in various tasks, often through natural language interfaces<ref>{{Cite journal \|last1=Luger \|first1=George F. \|last2=Stubblefield \|first2=William A. \|title=Artificial Intelligence: Structures and Strategies for Complex Problem Solving \|date=2004 \|publisher=Addison-Wesley \|isbn=978-0805327867}}</ref>. '''Negotiation''' is a process where parties with different interests attempt to reach an agreement through communication and compromise<ref>{{Cite book \|last=Raiffa \|first=Howard \|title=The Art and Science of Negotiation \|date=1982 \|publisher=Harvard University Press \|isbn=978-0674048546}}</ref>. '''LangChain''' is a Python library that allows developers to build applications with large language models, providing tools for tasks such as data retrieval, memory management, and prompt engineering<ref>{{Cite web \|url=https://python.langchain.com/en/latest/index.html \|title=LangChain Documentation \|website=Python.langchain.com \|access-date=2023-04-09}}</ref>.
−
−	'''Excel''' is a widely-used spreadsheet software developed by Microsoft, which allows for data analysis, visualization, and other computational tasks<ref>{{Cite web \|url=https://www.microsoft.com/en-us/microsoft-365/excel \|title=Microsoft Excel \|website=Microsoft.com \|access-date=2023-04-09}}</ref>. '''Databases''' can be categorized as '''Vector Databases''' (which store and index high-dimensional vector embeddings) or '''Relational Databases''' (which store structured data in tables)<ref>{{Cite journal \|last1=Babkin \|first1=Andrey \|last2=Likhomanenko \|first2=Tatiana \|last3=Paley \|first3=David \|last4=Babkin \|first4=Alexander \|last5=Boska \|first5=Alexey \|year=2021 \|title=A survey on vector databases and vector similarity search \|journal=Information Systems \|volume=97 \|pages=101694 \|doi=10.1016/j.is.2020.101694 \|url=https://arxiv.org/abs/2004.12701}}</ref>. '''Graphs''' are data structures that represent relationships between objects, and are widely used in applications such as social networks, recommendation systems, and knowledge representation<ref>{{Cite book \|last1=Bondy \|first1=J. Adrian \|last2=Murty \|first2=U.S.R. \|title=Graph Theory with Applications \|date=1976 \|publisher=Elsevier \|isbn=978-0444194527}}</ref>. '''LlamaIndex''' is an open-source library for building data-centric applications with large language models, providing tools for data structuring, querying, and indexing<ref>{{Cite web \|url=https://gpt-index.readthedocs.io/en/latest/index.html \|title=LlamaIndex Documentation \|website=Gpt-index.readthedocs.io \|access-date=2023-04-09}}</ref>.
−
−	'''Embeddings''' are vector representations of data that capture semantic and contextual information, allowing for efficient computation and comparison of complex data structures<ref>{{Cite journal \|last1=Mikolov \|first1=Tomas \|last2=Sutskever \|first2=Ilya \|last3=Chen \|first3=Kai \|last4=Corrado \|first4=Greg S. \|last5=Dean \|first5=Jeff \|year=2013 \|title=Distributed Representations of Words and Phrases and their Compositionality \|journal=Advances in Neural Information Processing Systems \|volume=26 \|pages=3111–3119 \|url=https://arxiv.org/abs/1310.4546}}</ref>. '''Fine-tuning''' is a technique in transfer learning where a pre-trained model is further trained on a specific task or dataset, allowing the model to adapt to the new domain while leveraging its previously learned knowledge<ref>{{Cite journal \|last1=Howard \|first1=Jeremy \|last2=Ruder \|first2=Sebastian \|year=2018 \|title=Universal Language Model Fine-tuning for Text Classification \|journal=Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) \|pages=328–339 \|doi=10.18653/v1/P18-1031 \|url=https://aclanthology.org/P18-1031/}}</ref>. '''Retrieval-Augmented Generation (RAG)''' is a framework that combines large language models with external knowledge sources, allowing the model to retrieve and incorporate relevant information during the generation process<ref>{{Cite journal \|last1=Lewis \|first1=Patrick \|last2=Perez \|first2=Ethan \|last3=Piktus \|first3=Aleksandra \|last4=Petroni \|first4=Fabio \|last5=Karpukhin \|first5=Vladimir \|last6=Goyal \|first6=Naman \|last7=Küttler \|first7=Heinrich \|last8=Lewis \|first8=Mike \|last9=Yih \|first9=Wen-tau \|last10=Rocktäschel \|first10=Tim \|last11=Riedel \|first11=Sebastian \|last12=Kiela \|first12=Douwe \|year=2020 \|title=Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks \|journal=Advances in Neural Information Processing Systems \|volume=33 \|pages=9459–9474 \|url=https://arxiv.org/abs/2005.11401}}</ref>. '''Search''' refers to the process of finding information or data within a larger dataset or knowledge base<ref>{{Cite book \|last=Baeza-Yates \|first=Ricardo \|last2=Ribeiro-Neto \|first2=Berthier \|title=Modern Information Retrieval: The Concepts and Technology behind Search \|date=2011 \|publisher=Addison-Wesley Professional \|isbn=978-0321416919}}</ref>. '''Clustering''' is an unsupervised machine learning technique that groups similar data points together, based on their inherent characteristics or features<ref>{{Cite book \|last1=Jain \|first1=Anil K. \|last2=Dubes \|first2=Richard C. \|title=Algorithms for Clustering Data \|date=1988 \|publisher=Prentice-Hall \|isbn=978-0130222787}}</ref>. '''Recommendation''' systems are algorithms that suggest relevant items (such as products, movies, or content) to users based on their preferences, behavior, or similarity to other users<ref>{{Cite journal \|last1=Ricci \|first1=Francesco \|last2=Rokach \|first2=Lior \|last3=Shapira \|first3=Bracha \|year=2015 \|title=Recommender Systems: Introduction and Challenges \|journal=Recommender Systems Handbook \|pages=1–34 \|doi=10.1007/978-1-4899-7637-6_1 \|url=https://link.springer.com/chapter/10.1007/978-1-4899-7637-6_1}}</ref>. '''Anomaly Detection''' is the process of identifying data points or patterns that deviate significantly from the expected or normal behavior, which can be useful for detecting fraud, system failures, or rare events<ref>{{Cite journal \|last1=Chandola \|first1=Varun \|last2=Banerjee \|first2=Arindam \|last3=Kumar \|first3=Vipin \|year=2009 \|title=Anomaly Detection: A Survey \|journal=ACM Computing Surveys \|volume=41 \|issue=3 \|pages=15:1–15:58 \|doi=10.1145/1541880.1541882 \|url=https://dl.acm.org/doi/10.1145/1541880.1541882}}</ref>. '''Classification''' is a supervised machine learning task that involves assigning data points to predefined categories or classes based on their features<ref>{{Cite book \|last=Alpaydin \|first=Ethem \|title=Introduction to Machine Learning \|date=2020 \|publisher=MIT Press \|isbn=978-0262043021}}</ref>. '''Dimensional Reduction''' techniques aim to reduce the number of features or variables in a dataset while preserving the most important information, which can improve computational efficiency and model performance<ref>{{Cite journal \|last1=Cunningham \|first1=J. Padraig \|last2=Ghahramani \|first2=Zoubin \|year=2015 \|title=Linear Dimensionality Reduction: Survey, Insights, and Generalizations \|journal=Journal of Machine Learning Research \|volume=16 \|issue=1 \|pages=2859–2900 \|url=http://jmlr.org/papers/v16/14-500.html}}</ref>. '''Outlier Detection''' is the process of identifying data points that significantly deviate from the norm or majority of the data, which can be useful for detecting anomalies, errors, or rare events<ref>{{Cite journal \|last1=Campos \|first1=Guilherme O. \|last2=Zimek \|first2=Arthur \|last3=Sander \|first3=Jörg \|last4=Campello \|first4=Ricardo J.G.B. \|last5=Micenková \|first5=Barbora \|last6=Schubert \|first6=Erich \|last7=Assunção \|first7=Ira \|last8=Houle \|first8=Michael E. \|year=2016 \|title=On the evaluation of unsupervised outlier detection: measures, datasets, and an empirical study \|journal=Data Mining and Knowledge Discovery \|volume=30 \|issue=4 \|pages=891–927 \|doi=10.1007/s10618-015-0444-8 \|url=https://link.springer.com/article/10.1007/s10618-015-0444-8}}</ref>.
−
−	'''Large Language Models (LLMs)''' are transformer-based neural networks trained on vast amounts of text data, allowing them to generate human-like language and perform a wide range of natural language processing tasks<ref>{{Cite journal \|last1=Brown \|first1=Tom B. \|last2=Mann \|

← Older revision		Revision as of 22:54, 3 March 2024
Line 125:		Line 125:
	== Impact on the Field ==		== Impact on the Field ==
	The latest research and products in memory AI are reshaping the field by addressing the challenges of catastrophic forgetting and controlled forgetting. These advancements are crucial for the development of AI systems capable of lifelong learning, trustworthy AI, and personalized user experiences. The semiconductor industry is also adapting to these changes, with a focus on memory enhancements to support the growing needs of AI servers and applications.		The latest research and products in memory AI are reshaping the field by addressing the challenges of catastrophic forgetting and controlled forgetting. These advancements are crucial for the development of AI systems capable of lifelong learning, trustworthy AI, and personalized user experiences. The semiconductor industry is also adapting to these changes, with a focus on memory enhancements to support the growing needs of AI servers and applications.
		+
		+	--------------------------------------------------------
		+
		+	===Memory Networks===
		+	'''State Space Models (SSMs)''' are used for modeling and analyzing dynamical systems, allowing for the representation of both the state and input/output relationships<ref>{{Cite book \|last=Ljung \|first=Lennart \|title=System Identification: Theory for the User \|date=1999 \|publisher=Prentice Hall PTR \|isbn=978-0138816545}}</ref>. '''Mamba''' is a sequence-to-sequence (seq2seq) framework for multimodal data<ref>{{Cite journal \|last1=Tsai \|first1=Yu-Hsiang \|last2=Bhushan \|first2=Salil \|last3=Yang \|first3=Yi-Ren \|last4=Xue \|first4=Douglas \|last5=Asai \|first5=Akiko \|last6=Misailovic \|first6=Sasa \|last7=Dmello \|first7=Rafael \|year=2020 \|title=MUNAL: Multimodal Abstraction for Conversational Question Answering over Real-World Multimodal Data \|journal=Findings of the Association for Computational Linguistics (ACL) \|url=https://aclanthology.org/2020.findings-emnlp.208.pdf}}</ref>. '''Recurrent Neural Networks (RNNs)''' are a type of neural network well-suited for processing sequential data, with feedback connections that allow the network to maintain an internal state<ref>{{Cite book \|last1=Goodfellow \|first1=Ian \|last2=Bengio \|first2=Yoshua \|last3=Courville \|first3=Aaron \|title=Deep Learning \|date=2016 \|publisher=MIT Press \|isbn=978-0262035613}}</ref>. '''Convolutional Neural Networks (CNNs)''' are a class of deep neural networks primarily used for analyzing visual imagery, leveraging the spatial and temporal dependencies in the data<ref>{{Cite journal \|last1=Krizhevsky \|first1=Alex \|last2=Sutskever \|first2=Ilya \|last3=Hinton \|first3=Geoffrey \|year=2017 \|title=ImageNet classification with deep convolutional neural networks \|journal=Communications of the ACM \|volume=60 \|issue=6 \|pages=84–90 \|doi=10.1145/3065386 \|url=https://dl.acm.org/doi/10.1145/3065386}}</ref>.
		+
		+	'''In-Context Learning (ICL)''' refers to the ability of large language models to adapt their behavior based on the context provided in the input prompt, without the need for explicit fine-tuning or training on additional data<ref>{{Cite journal \|last1=Wei \|first1=Jason \|last2=Tay \|first2=Yi \|last3=Bommasani \|first3=Rishi \|last4=Aziz \|first4=Aden \|last5=Hechtrespector \|first5=Ari \|last6=Plu \|first6=Juliana \|last7=Yu \|first7=Karan \|last8=Narasimhan \|first8=Kianywee \|year=2022 \|title=Emergent Abilities of Large Language Models \|journal=arXiv preprint arXiv:2206.07682 \|url=https://arxiv.org/abs/2206.07682}}</ref>. '''Causation vs. Correlation''' is a fundamental distinction in statistics, where causation implies a direct cause-and-effect relationship, while correlation simply indicates a statistical association between variables<ref>{{Cite book \|last=Pearl \|first=Judea \|title=Causality: Models, Reasoning, and Inference \|date=2009 \|publisher=Cambridge University Press \|isbn=978-0521895606}}</ref>. '''Autocorrelation''' refers to the correlation of a signal or time series with a delayed copy of itself, which can reveal patterns and dependencies in the data<ref>{{Cite book \|last1=Box \|first1=George E.P. \|last2=Jenkins \|first2=Gwilym M. \|last3=Reinsel \|first3=Gregory C. \|title=Time Series Analysis: Forecasting and Control \|date=2008 \|publisher=John Wiley & Sons \|isbn=978-0470272848}}</ref>. '''Out-of-Distribution (OOD) Generalization''' is the ability of a machine learning model to perform well on data that differs from the distribution of the training data, which is a significant challenge in deploying models in real-world scenarios<ref>{{Cite journal \|last1=Hendrycks \|first1=Dan \|last2=Basart \|first2=Steven \|last3=Mu \|first3=Norman \|last4=Kadavath \|first4=Saurav \|last5=Wang \|first5=Frank \|last6=Dorundo \|first6=Evan \|last7=Desai \|first7=Rahul \|last8=Zhu \|first8=Tyler \|last9=Parekh \|first9=Samyak \|last10=Maynez \|first10=Matt \|last11=Colombo \|first11=Gabriel \|year=2021 \|title=The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization \|journal=Proceedings of the IEEE/CVF International Conference on Computer Vision \|pages=8340–8349 \|url=https://arxiv.org/abs/2006.16241}}</ref>. '''Transfer Learning''' involves leveraging knowledge gained from one task to improve performance on a related but different task, which can be beneficial when working with limited data or computational resources<ref>{{Cite journal \|last1=Zhuang \|first1=Fuzhen \|last2=Qi \|first2=Zhiyuan \|last3=Duan \|first3=Keyu \|last4=Xi \|first4=Dongbo \|last5=Zhu \|first5=Yongchun \|last6=Zhu \|first6=Hengshu \|last7=Xiong \|first7=Hui \|last8=He \|first8=Qing \|year=2021 \|title=A Comprehensive Survey on Transfer Learning \|journal=Proceedings of the IEEE \|volume=109 \|issue=1 \|pages=43–76 \|doi=10.1109/JPROC.2020.3004555 \|url=https://arxiv.org/abs/1911.02685}}</ref>.
		+
		+	'''Assistants''', '''Personal Companions''', and '''Agents''' are terms used to describe AI systems designed to interact with humans and assist them in various tasks, often through natural language interfaces<ref>{{Cite journal \|last1=Luger \|first1=George F. \|last2=Stubblefield \|first2=William A. \|title=Artificial Intelligence: Structures and Strategies for Complex Problem Solving \|date=2004 \|publisher=Addison-Wesley \|isbn=978-0805327867}}</ref>. '''Negotiation''' is a process where parties with different interests attempt to reach an agreement through communication and compromise<ref>{{Cite book \|last=Raiffa \|first=Howard \|title=The Art and Science of Negotiation \|date=1982 \|publisher=Harvard University Press \|isbn=978-0674048546}}</ref>. '''LangChain''' is a Python library that allows developers to build applications with large language models, providing tools for tasks such as data retrieval, memory management, and prompt engineering<ref>{{Cite web \|url=https://python.langchain.com/en/latest/index.html \|title=LangChain Documentation \|website=Python.langchain.com \|access-date=2023-04-09}}</ref>.
		+
		+	'''Excel''' is a widely-used spreadsheet software developed by Microsoft, which allows for data analysis, visualization, and other computational tasks<ref>{{Cite web \|url=https://www.microsoft.com/en-us/microsoft-365/excel \|title=Microsoft Excel \|website=Microsoft.com \|access-date=2023-04-09}}</ref>. '''Databases''' can be categorized as '''Vector Databases''' (which store and index high-dimensional vector embeddings) or '''Relational Databases''' (which store structured data in tables)<ref>{{Cite journal \|last1=Babkin \|first1=Andrey \|last2=Likhomanenko \|first2=Tatiana \|last3=Paley \|first3=David \|last4=Babkin \|first4=Alexander \|last5=Boska \|first5=Alexey \|year=2021 \|title=A survey on vector databases and vector similarity search \|journal=Information Systems \|volume=97 \|pages=101694 \|doi=10.1016/j.is.2020.101694 \|url=https://arxiv.org/abs/2004.12701}}</ref>. '''Graphs''' are data structures that represent relationships between objects, and are widely used in applications such as social networks, recommendation systems, and knowledge representation<ref>{{Cite book \|last1=Bondy \|first1=J. Adrian \|last2=Murty \|first2=U.S.R. \|title=Graph Theory with Applications \|date=1976 \|publisher=Elsevier \|isbn=978-0444194527}}</ref>. '''LlamaIndex''' is an open-source library for building data-centric applications with large language models, providing tools for data structuring, querying, and indexing<ref>{{Cite web \|url=https://gpt-index.readthedocs.io/en/latest/index.html \|title=LlamaIndex Documentation \|website=Gpt-index.readthedocs.io \|access-date=2023-04-09}}</ref>.
		+
		+	'''Embeddings''' are vector representations of data that capture semantic and contextual information, allowing for efficient computation and comparison of complex data structures<ref>{{Cite journal \|last1=Mikolov \|first1=Tomas \|last2=Sutskever \|first2=Ilya \|last3=Chen \|first3=Kai \|last4=Corrado \|first4=Greg S. \|last5=Dean \|first5=Jeff \|year=2013 \|title=Distributed Representations of Words and Phrases and their Compositionality \|journal=Advances in Neural Information Processing Systems \|volume=26 \|pages=3111–3119 \|url=https://arxiv.org/abs/1310.4546}}</ref>. '''Fine-tuning''' is a technique in transfer learning where a pre-trained model is further trained on a specific task or dataset, allowing the model to adapt to the new domain while leveraging its previously learned knowledge<ref>{{Cite journal \|last1=Howard \|first1=Jeremy \|last2=Ruder \|first2=Sebastian \|year=2018 \|title=Universal Language Model Fine-tuning for Text Classification \|journal=Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) \|pages=328–339 \|doi=10.18653/v1/P18-1031 \|url=https://aclanthology.org/P18-1031/}}</ref>. '''Retrieval-Augmented Generation (RAG)''' is a framework that combines large language models with external knowledge sources, allowing the model to retrieve and incorporate relevant information during the generation process<ref>{{Cite journal \|last1=Lewis \|first1=Patrick \|last2=Perez \|first2=Ethan \|last3=Piktus \|first3=Aleksandra \|last4=Petroni \|first4=Fabio \|last5=Karpukhin \|first5=Vladimir \|last6=Goyal \|first6=Naman \|last7=Küttler \|first7=Heinrich \|last8=Lewis \|first8=Mike \|last9=Yih \|first9=Wen-tau \|last10=Rocktäschel \|first10=Tim \|last11=Riedel \|first11=Sebastian \|last12=Kiela \|first12=Douwe \|year=2020 \|title=Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks \|journal=Advances in Neural Information Processing Systems \|volume=33 \|pages=9459–9474 \|url=https://arxiv.org/abs/2005.11401}}</ref>. '''Search''' refers to the process of finding information or data within a larger dataset or knowledge base<ref>{{Cite book \|last=Baeza-Yates \|first=Ricardo \|last2=Ribeiro-Neto \|first2=Berthier \|title=Modern Information Retrieval: The Concepts and Technology behind Search \|date=2011 \|publisher=Addison-Wesley Professional \|isbn=978-0321416919}}</ref>. '''Clustering''' is an unsupervised machine learning technique that groups similar data points together, based on their inherent characteristics or features<ref>{{Cite book \|last1=Jain \|first1=Anil K. \|last2=Dubes \|first2=Richard C. \|title=Algorithms for Clustering Data \|date=1988 \|publisher=Prentice-Hall \|isbn=978-0130222787}}</ref>. '''Recommendation''' systems are algorithms that suggest relevant items (such as products, movies, or content) to users based on their preferences, behavior, or similarity to other users<ref>{{Cite journal \|last1=Ricci \|first1=Francesco \|last2=Rokach \|first2=Lior \|last3=Shapira \|first3=Bracha \|year=2015 \|title=Recommender Systems: Introduction and Challenges \|journal=Recommender Systems Handbook \|pages=1–34 \|doi=10.1007/978-1-4899-7637-6_1 \|url=https://link.springer.com/chapter/10.1007/978-1-4899-7637-6_1}}</ref>. '''Anomaly Detection''' is the process of identifying data points or patterns that deviate significantly from the expected or normal behavior, which can be useful for detecting fraud, system failures, or rare events<ref>{{Cite journal \|last1=Chandola \|first1=Varun \|last2=Banerjee \|first2=Arindam \|last3=Kumar \|first3=Vipin \|year=2009 \|title=Anomaly Detection: A Survey \|journal=ACM Computing Surveys \|volume=41 \|issue=3 \|pages=15:1–15:58 \|doi=10.1145/1541880.1541882 \|url=https://dl.acm.org/doi/10.1145/1541880.1541882}}</ref>. '''Classification''' is a supervised machine learning task that involves assigning data points to predefined categories or classes based on their features<ref>{{Cite book \|last=Alpaydin \|first=Ethem \|title=Introduction to Machine Learning \|date=2020 \|publisher=MIT Press \|isbn=978-0262043021}}</ref>. '''Dimensional Reduction''' techniques aim to reduce the number of features or variables in a dataset while preserving the most important information, which can improve computational efficiency and model performance<ref>{{Cite journal \|last1=Cunningham \|first1=J. Padraig \|last2=Ghahramani \|first2=Zoubin \|year=2015 \|title=Linear Dimensionality Reduction: Survey, Insights, and Generalizations \|journal=Journal of Machine Learning Research \|volume=16 \|issue=1 \|pages=2859–2900 \|url=http://jmlr.org/papers/v16/14-500.html}}</ref>. '''Outlier Detection''' is the process of identifying data points that significantly deviate from the norm or majority of the data, which can be useful for detecting anomalies, errors, or rare events<ref>{{Cite journal \|last1=Campos \|first1=Guilherme O. \|last2=Zimek \|first2=Arthur \|last3=Sander \|first3=Jörg \|last4=Campello \|first4=Ricardo J.G.B. \|last5=Micenková \|first5=Barbora \|last6=Schubert \|first6=Erich \|last7=Assunção \|first7=Ira \|last8=Houle \|first8=Michael E. \|year=2016 \|title=On the evaluation of unsupervised outlier detection: measures, datasets, and an empirical study \|journal=Data Mining and Knowledge Discovery \|volume=30 \|issue=4 \|pages=891–927 \|doi=10.1007/s10618-015-0444-8 \|url=https://link.springer.com/article/10.1007/s10618-015-0444-8}}</ref>.
		+
		+	'''Large Language Models (LLMs)''' are transformer-based neural networks trained on vast amounts of text data, allowing them to generate human-like language and perform a wide range of natural language processing tasks<ref>{{Cite journal \|last1=Brown \|first1=Tom B. \|last2=Mann \|

← Older revision		Revision as of 16:40, 3 March 2024
Line 85:		Line 85:

	As AI models, particularly [[Large Language Model (LLM) \| LLMs]], grow in complexity and size, they encounter significant memory limitations. These constraints hinder their ability to process extensive context windows, retain information over time, and manage the vast amounts of data they generate. This report examines various technological workarounds that have been developed to address these challenges. AI memory limitations present a significant challenge as models become larger and more complex. However, innovative technological workarounds like MemGPT, RAG, fine-tuning, new architectures, and persistent memory are paving the way for more capable and efficient AI systems. Advanced memory management in chatbots includes optimization techniques, dynamic skill redirection, and the ability to bypass natural language processing for free-text entries. Custom enrichment further enhances memory information, providing relevant details for accurate responses.		As AI models, particularly [[Large Language Model (LLM) \| LLMs]], grow in complexity and size, they encounter significant memory limitations. These constraints hinder their ability to process extensive context windows, retain information over time, and manage the vast amounts of data they generate. This report examines various technological workarounds that have been developed to address these challenges. AI memory limitations present a significant challenge as models become larger and more complex. However, innovative technological workarounds like MemGPT, RAG, fine-tuning, new architectures, and persistent memory are paving the way for more capable and efficient AI systems. Advanced memory management in chatbots includes optimization techniques, dynamic skill redirection, and the ability to bypass natural language processing for free-text entries. Custom enrichment further enhances memory information, providing relevant details for accurate responses.
−

	== MemGPT ==		== MemGPT ==