Difference between revisions of "Large Language Model (LLM)"

Revision as of 22:29, 24 February 2023

ChatGPT | OpenAI
- ChatGPT is everywhere. Here’s where it came from | Will Douglas Heaven - MIT Technology Review
  - Transformer / Attention Mechanism
  - Generative Pre-trained Transformer (GPT)
  - Reinforcement Learning (RL) from Human Feedback (RLHF)
  - Supervised Learning
  - Proximal Policy Optimization (PPO)
Alpa ... serving large models like GPT-3 simple, affordable, accessible
BioGPT ... Microsoft language model trained for biomedical tasks
BLOOM ... Big Science Language Open-science Open-access Multilingual
Cedille ... open-source French language model
Chinchilla | DeepMind
ctrl ... a Conditional Transformer Language Model for Controllable Generation | Salesforce
Gopher | DeepMind
RETRO | DeepMind
DialogGPT ...Microsoft Releases DialogGPT AI Conversation Model | Anthony Alford - InfoQ - trained on over 147M dialogs
minGPT | Andrej Karpathy - GitHub
GLM-130B ... Open Bilingual Pre-Trained Model
OPT-175B
- Facebook-owner Meta opens access to AI large language model | Elizabeth Culliford - Reuters] ... Facebook 175-billion-parameter language model - Open Pretrained Transformer (OPT-175B)
OpenAI Blog | OpenAI
Attention Mechanism/Transformer Model
Generative Pre-trained Transformer (GPT)
SambaNova Systems ... Dataflow-as-a-Service GPT

@@ Line 25: / Line 25: @@
 * [https://www.infoq.com/news/2019/11/microsoft-ai-conversation/ DialogGPT]  ...Microsoft Releases DialogGPT AI Conversation Model | Anthony Alford - InfoQ - trained on over 147M dialogs
 * [https://github.com/karpathy/minGPT minGPT | Andrej Karpathy - GitHub]
 * [https://gpt3demo.com/apps/glm-130b GLM-130B]  ... Open Bilingual Pre-Trained Model
+* [https://www.reuters.com/technology/facebook-owner-meta-opens-access-ai-large-language-model-2022-05-03/ OPT-175B]
+** [[Meta|Facebook]]-owner Meta opens access to AI large language model | Elizabeth Culliford - Reuters] ... [[Meta|Facebook]] 175-billion-parameter language model - Open Pretrained Transformer (OPT-175B)
 * [https://openai.com/blog/gpt-2-6-month-follow-up/ OpenAI Blog] | [[OpenAI]]
 * [[Attention]] Mechanism/[[Transformer]] Model
 * [[Generative Pre-trained Transformer (GPT)]]
 * [https://sambanova.ai/solutions/gpt/ SambaNova Systems] ... Dataflow-as-a-Service GPT
-* [https://www.reuters.com/technology/facebook-owner-meta-opens-access-ai-large-language-model-2022-05-03/  [[Meta|Facebook]]-owner Meta opens access to AI large language model | Elizabeth Culliford - Reuters] ... [[Meta|Facebook]] 175-billion-parameter language model - Open Pretrained Transformer (OPT-175B)