In-Context Learning (ICL)
YouTube ... Quora ...Google search ...Google News ...Bing News
- A Survey on In-context Learning | Q. Dong, L. Li, D. Dai, C. Zheng, Z. Wu, B. Chang, X. Sun, J. Xu, L. Li, Z. Sui - arXiv Cornell University
- Generative AI ... Conversational AI ... OpenAI's ChatGPT ... Perplexity ... Microsoft's Bing ... You ...Google's Bard ... Baidu's Ernie
- Capabilities
- Discriminative vs. Generative
- Assistants ... Agents ... Negotiation ... HuggingGPT ... LangChain
- Context-based learning | Wikipedia
In-context learning (ICL) is a new paradigm in natural language processing (NLP) where large language models (LLMs) make predictions based on contexts augmented with just a few training examples1. LLMs are able to extract patterns from the examples provided in the context and use them to perform many complex NLP tasks. - In-context Learning - A New Paradigm in NLP? | The Global NLP Lab ..
... Why does ICL work so well?
There have been a few studies aiming to uncover this in the literature.
- One factor that might play a role is the distribution of the training data. When training on a very large dataset, the ICL ability of LLMs seems to emerge when the data appears in clusters and there are a sufficient number of rare classes present.
- Another factor is that Transformer models might be learning to encode learning algorithms implicitly during the training process, due to the properties of their architecture. During inference, transformer LLMs might be performing an implicit finetuning using the provided examples in the context.
