Large Language Model (LLM)

YouTube search... ...Google search

• Models:
  • Toolformer ... models can teach themselves to use tools and APIs
  • ChatGPT | OpenAI
    • ChatGPT is everywhere. Here’s where it came from | Will Douglas Heaven - MIT Technology Review
      • Transformer / Attention Mechanism
      • Generative Pre-trained Transformer (GPT)
      • Reinforcement Learning (RL) from Human Feedback (RLHF)
      • Supervised Learning
      • Proximal Policy Optimization (PPO)
  • Alpa ... serving large models like GPT-3 simple, affordable, accessible
  • BioGPT ... Microsoft language model trained for biomedical tasks
  • BLOOM ... Big Science Language Open-science Open-access Multilingual ... 176B
  • Cedille ... open-source French language model
  • Chinchilla | DeepMind
  • ctrl ... a Conditional Transformer Language Model for Controllable Generation | Salesforce
  • Gopher | DeepMind
  • RETRO | DeepMind
  • DialogGPT ...Microsoft Releases DialogGPT AI Conversation Model | Anthony Alford - InfoQ - trained on over 147M dialogs
  • minGPT | Andrej Karpathy - GitHub
  • GLM-130B ... Open Bilingual Pre-Trained Model
  • OPT-175B...Facebook-owner Meta opens access to AI large language model | Elizabeth Culliford - Reuters ... Facebook 175-billion-parameter language model - Open Pretrained Transformer
  • Bidirectional Encoder Representations from Transformers (BERT)
  • GLaM | Google
  • GShard | Google ... Scaling Giant Models with Conditional Computation and Automatic Sharding
  • GPT-2 | OpenAI ... Generative Pre-trained Transformer 2 by OpenAI
  • GPT-Neo ... Open-source GPT-3 by EleutherAI
  • InstructGPT ... OpenAI 1.3B InstructGPT model over outputs from a 175B GPT-3 model
  • Jurassic-1 Language Model ... huge 178B language model to rival OpenAI's GPT-3]
  • LaMDA | Google ... experimental language model
  • Macaw | AI2
  • Med-PaLM ... aligned to the medical domain
  • Turing-NLG | Microsoft
  • Megatron NLG ... Monolithic Transformer Language NLP Model Triple the Size of OpenAI’s GPT-3
  • Muse ... VLM-4, a set of natively trained large Language Models in French, Italian, Spanish, German, and English
  • nanoGPT ... for training/finetuning medium-sized GPTs
  • Codex | OpenAI ... translates natural language into code
  • OpenGPT-X ... model for Europe
  • Palmyra | Hugging Face ... a privacy-first LLM for enterprises
  • Pathways Language Model (PaLM) ...scaling to 540 Billion Parameters
  • PLATO-XL | Baidu ... 11B Parameter Chatbot
  • Dataflow-as-a-Service | SambaNova
  • [https://arxiv.org/abs/2101.03961 Switch Transformers | Google Brain ... trillion parameters
  • T0pp | Hugging Face
  • Textless NLP ... Generating expressive speech from raw audio
• OpenAI Blog | OpenAI
• Attention Mechanism/Transformer Model
• Generative Pre-trained Transformer (GPT)
• SambaNova Systems ... Dataflow-as-a-Service GPT