Difference between revisions of "In-Context Learning (ICL)"
m |
m |
||
| Line 32: | Line 32: | ||
In-context learning works well because it allows users to quickly build models for a new use case without worrying about fine-tuning and storing new parameters for each task. It typically requires very few training examples to get a prototype working, and the natural language interface is intuitive even for non-experts. [https://ai.stanford.edu/blog/understanding-incontext/ How does in-context learning work? A framework for understanding the differences from traditional supervised learning | Sang Michael Xie and Sewon Min - The Stanford AI Lab] | In-context learning works well because it allows users to quickly build models for a new use case without worrying about fine-tuning and storing new parameters for each task. It typically requires very few training examples to get a prototype working, and the natural language interface is intuitive even for non-experts. [https://ai.stanford.edu/blog/understanding-incontext/ How does in-context learning work? A framework for understanding the differences from traditional supervised learning | Sang Michael Xie and Sewon Min - The Stanford AI Lab] | ||
| − | <img src="https:// | + | <img src="https://ai.stanford.edu/blog/assets/img/posts/2022-08-01-understanding-incontext/images/image13.gif" width="600"> |
Revision as of 21:32, 22 April 2023
YouTube ... Quora ...Google search ...Google News ...Bing News
- A Survey on In-context Learning | Q. Dong, L. Li, D. Dai, C. Zheng, Z. Wu, B. Chang, X. Sun, J. Xu, L. Li, Z. Sui - arXiv Cornell University
- Generative AI ... Conversational AI ... OpenAI's ChatGPT ... Perplexity ... Microsoft's Bing ... You ...Google's Bard ... Baidu's Ernie
- Capabilities
- Discriminative vs. Generative
- Assistants ... Agents ... Negotiation ... HuggingGPT ... LangChain
- Context-based learning | Wikipedia
In-context learning (ICL) is a new paradigm in natural language processing (NLP) where large language models (LLMs) make predictions based on contexts augmented with just a few training examples1. LLMs are able to extract patterns from the examples provided in the context and use them to perform many complex NLP tasks. - In-context Learning - A New Paradigm in NLP? | The Global NLP Lab ..
... Why does ICL work so well?
There have been a few studies aiming to uncover this in the literature.
- One factor that might play a role is the distribution of the training data. When training on a very large dataset, the ICL ability of LLMs seems to emerge when the data appears in clusters and there are a sufficient number of rare classes present.
- Another factor is that Transformer models might be learning to encode learning algorithms implicitly during the training process, due to the properties of their architecture. During inference, transformer LLMs might be performing an implicit finetuning using the provided examples in the context.
In-context learning works well because it allows users to quickly build models for a new use case without worrying about fine-tuning and storing new parameters for each task. It typically requires very few training examples to get a prototype working, and the natural language interface is intuitive even for non-experts. How does in-context learning work? A framework for understanding the differences from traditional supervised learning | Sang Michael Xie and Sewon Min - The Stanford AI Lab
