Large Language Model (LLM)

YouTube search... ...Google search

• Case Studies
  • Writing
  • Publishing
  • Sequence to Sequence (Seq2Seq)
  • Recurrent Neural Network (RNN)
  • Long Short-Term Memory (LSTM)
  • ELMo
  • Bidirectional Encoder Representations from Transformers (BERT) ... a better model, but less investment than the larger OpenAI organization
• Large Language Models (LLMs)
  • ChatGPT | OpenAI
    • ChatGPT is everywhere. Here’s where it came from | Will Douglas Heaven - MIT Technology Review
      • Transformer / Attention Mechanism
      • Generative Pre-trained Transformer (GPT)
      • Reinforcement Learning (RL) from Human Feedback (RLHF)
      • Supervised Learning
      • Proximal Policy Optimization (PPO)
  • Alpa ... serving large models like GPT-3 simple, affordable, accessible
  • BioGPT ... Microsoft language model trained for biomedical tasks
  • BLOOM ... Big Science Language Open-science Open-access Multilingual
  • Cedille ... open-source French language model
  • Chinchilla | DeepMind
  • ctrl ... a Conditional Transformer Language Model for Controllable Generation | Salesforce
  • Gopher | DeepMind
  • RETRO | DeepMind
• OpenAI Blog | OpenAI
• Text Transfer Learning
• Natural Language Generation (NLG)
• Natural Language Tools & Services
• Generated Image
• SynthPub
• Language Models are Unsupervised Multitask Learners | Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever
• Neural Monkey | Jindřich Libovický, Jindřich Helcl, Tomáš Musil Byte Pair Encoding (BPE) enables NMT model translation on open-vocabulary by encoding rare and unknown words as sequences of subword units.
• Attention Mechanism/Transformer Model
• Language Models are Unsupervised Multitask Learners - GitHub
• Microsoft Releases DialogGPT AI Conversation Model | Anthony Alford - InfoQ - trained on over 147M dialogs
• minGPT | Andrej Karpathy - GitHub
• SambaNova Systems ... Dataflow-as-a-Service GPT
• Facebook-owner Meta opens access to AI large language model | Elizabeth Culliford - Reuters ... Facebook 175-billion-parameter language model - Open Pretrained Transformer (OPT-175B)