Difference between revisions of "Large Language Model (LLM)"
m |
m |
||
| Line 9: | Line 9: | ||
* Models: | * Models: | ||
| − | ** [[ | + | ** [https://opt.alpa.ai/ Alpa] ... serving large models like GPT-3 simple, affordable, accessible |
| + | ** [https://github.com/microsoft/BioGPT BioGPT] ... [[Microsoft]] language model trained for biomedical tasks | ||
| + | ** [https://bigscience.notion.site/BLOOM-BigScience-176B-Model-ad073ca07cdf479398d5f95d88e218c4 BLOOM] ... Big Science Language Open-science Open-access Multilingual ... 176B | ||
| + | ** [https://cedille.ai/ Cedille] ... open-source French language model | ||
** [[ChatGPT]] | [[OpenAI]] | ** [[ChatGPT]] | [[OpenAI]] | ||
*** [https://www.technologyreview.com/2023/02/08/1068068/chatgpt-is-everywhere-heres-where-it-came-from/ ChatGPT is everywhere. Here’s where it came from | Will Douglas Heaven - MIT Technology Review] | *** [https://www.technologyreview.com/2023/02/08/1068068/chatgpt-is-everywhere-heres-where-it-came-from/ ChatGPT is everywhere. Here’s where it came from | Will Douglas Heaven - MIT Technology Review] | ||
| Line 17: | Line 20: | ||
**** [[Supervised]] Learning | **** [[Supervised]] Learning | ||
**** [[Proximal Policy Optimization (PPO)]] | **** [[Proximal Policy Optimization (PPO)]] | ||
| − | |||
| − | |||
| − | |||
| − | |||
** [https://www.deepmind.com/publications/an-empirical-analysis-of-compute-optimal-large-language-model-training Chinchilla |] [[Google | DeepMind]] | ** [https://www.deepmind.com/publications/an-empirical-analysis-of-compute-optimal-large-language-model-training Chinchilla |] [[Google | DeepMind]] | ||
** [https://arxiv.org/abs/2203.15556 ctrl] ... a Conditional Transformer Language Model for Controllable Generation | Salesforce | ** [https://arxiv.org/abs/2203.15556 ctrl] ... a Conditional Transformer Language Model for Controllable Generation | Salesforce | ||
| Line 52: | Line 51: | ||
** [https://huggingface.co/bigscience/T0pp T0pp |] [[Hugging Face]] | ** [https://huggingface.co/bigscience/T0pp T0pp |] [[Hugging Face]] | ||
** [https://ai.facebook.com/blog/textless-nlp-generating-expressive-speech-from-raw-audio/ Textless NLP ... Generating expressive speech from raw audio] | ** [https://ai.facebook.com/blog/textless-nlp-generating-expressive-speech-from-raw-audio/ Textless NLP ... Generating expressive speech from raw audio] | ||
| + | ** [[Toolformer]] | [[Meta]] ... models can teach themselves to use tools and APIs | ||
** [https://github.com/allenai/unifiedqa UnifiedQA] ... single QA system | ** [https://github.com/allenai/unifiedqa UnifiedQA] ... single QA system | ||
** [https://openai.com/blog/webgpt/ WebGPT] ... GPT-3 version that can search the web | ** [https://openai.com/blog/webgpt/ WebGPT] ... GPT-3 version that can search the web | ||
Revision as of 23:57, 24 February 2023
YouTube search... ...Google search
- Models:
- Alpa ... serving large models like GPT-3 simple, affordable, accessible
- BioGPT ... Microsoft language model trained for biomedical tasks
- BLOOM ... Big Science Language Open-science Open-access Multilingual ... 176B
- Cedille ... open-source French language model
- ChatGPT | OpenAI
- Chinchilla | DeepMind
- ctrl ... a Conditional Transformer Language Model for Controllable Generation | Salesforce
- Gopher | DeepMind
- RETRO | DeepMind
- DialogGPT ...Microsoft Releases DialogGPT AI Conversation Model | Anthony Alford - InfoQ - trained on over 147M dialogs
- minGPT | Andrej Karpathy - GitHub
- GLM-130B ... Open Bilingual Pre-Trained Model
- OPT-175B...Facebook-owner Meta opens access to AI large language model | Elizabeth Culliford - Reuters ... Facebook 175-billion-parameter language model - Open Pretrained Transformer
- Bidirectional Encoder Representations from Transformers (BERT)
- GLaM | Google
- GShard | Google ... Scaling Giant Models with Conditional Computation and Automatic Sharding
- GPT-2 | OpenAI ... Generative Pre-trained Transformer 2 by OpenAI
- GPT-Neo ... Open-source GPT-3 by EleutherAI
- InstructGPT ... OpenAI 1.3B InstructGPT model over outputs from a 175B GPT-3 model
- Jurassic-1 Language Model ... huge 178B language model to rival OpenAI's GPT-3]
- LaMDA | Google ... experimental language model
- Macaw | AI2
- Med-PaLM ... aligned to the medical domain
- Turing-NLG | Microsoft
- Megatron NLG ... Monolithic Transformer Language NLP Model Triple the Size of OpenAI’s GPT-3
- Muse ... VLM-4, a set of natively trained large Language Models in French, Italian, Spanish, German, and English
- nanoGPT ... for training/finetuning medium-sized GPTs
- Codex | OpenAI ... translates natural language into code
- OpenGPT-X ... model for Europe
- Palmyra | Hugging Face ... a privacy-first LLM for enterprises
- Pathways Language Model (PaLM) ...scaling to 540 Billion Parameters
- PLATO-XL | Baidu ... 11B Parameter Chatbot
- Dataflow-as-a-Service | SambaNova
- [https://arxiv.org/abs/2101.03961 Switch Transformers | Google Brain ... trillion parameters
- T0pp | Hugging Face
- Textless NLP ... Generating expressive speech from raw audio
- Toolformer | Meta ... models can teach themselves to use tools and APIs
- UnifiedQA ... single QA system
- WebGPT ... GPT-3 version that can search the web
- Wu Dao 1.0 (Enlightment 1.0) ... China’s first homegrown super-scale intelligent model
- YaLM 100B ... Yandex YaLM 100B parameters
- Yuan 1.0 | Inspur ... 245B parameters
- OpenAI Blog | OpenAI
- Attention Mechanism/Transformer Model
- Generative Pre-trained Transformer (GPT)
- SambaNova Systems ... Dataflow-as-a-Service GPT
