Facebook-owner Meta opens access to AI large language model | Elizabeth Culliford - Reuters ... Facebook 175-billion-parameter language model - Open Pretrained Transformer (OPT-175B)
What is GPT-3? Showcase, possibilities, and implications
What is going on in AI research lately? GPT-3 crashed the party, let’s see what it is and what it can do. Hoping we do not forget how problematic it might also become. GPT-3 Paper : Brown, Tom B., Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan et al. "Language models are few-shot learners." arXiv preprint arXiv:2005.14165 (2020). https://arxiv.org/pdf/2005.14165.pdf
14 Cool Apps Built on OpenAI's GPT-3 API
14 Cool applications just built on top of OpenAI's GPT-3 (generative predictive transformer) API (currently in private beta).
OpenAI Model Generates Python Code
This code completion engine can write an entire function from just the name! OpenAI demonstrates what happens when you learn a language model on thousands of GitHub Python repositories. Source Clip: https://youtu.be/fZSFNUT6iY8
Generative Python Code with GPT
In a quest to teach neural networks via transformers to write Python code. Project name: Generative Python Transformers!
GPT 3 Explanation And Demo Reaction | Should You Be Scared ? Techies Reaction To GPT3 AI OpenAI In this video we will look at some of the demos and reactions across social media on GPT-3. The links to the tweets and demo you see in this video have been linked below so please do react out to them if you have any questions.
Code 10x Faster With This CRAZY New AI Tool (GPT-3)
In this FREE LIVE training, Aaron and Naz will show you the new cutting edge machine learning AI, OpenAI's GPT-3.
Build 2 projects using GPT-3 in just a couple of minutes. Bare bones: branding generator, chat bot
Co-founded by Elon Musk OpenAI wants to make AI safe and accessible. A year ago the startup released GPT-2. That language model was at that time deemed too powerful to release. Eventually OpenAI made the model available. This year they've trained a much more powerful model at least 1 magnitude larger than GPT-2. I was one of the lucky 750+ people granted access as Beta testers by OpenAI to see what can be built using the GPT-3 API. The model costs millions of dollars to train which makes it out of reach for most organizations. This video skims the surface of what you can get done with this amazing new model.
Custom GPTs
Custom GPTs are personalized versions of AI models like ChatGPT that can be tailored for specific tasks or projects. They represent a significant advancement in AI implementation, allowing businesses and individuals to customize AI tools to meet unique challenges and operational needs.
OpenAI allows Plus and Enterprise users to create custom GPTs that can browse the web, create images, and run code. Users can upload knowledge files, modify the GPT's appearance, and define its actions
The OpenAI GPT Store provides a platform for users to create, share, and monetize their custom GPTs, expanding the capabilities and possibilities of AI assistants like ChatGPT. It allows users of ChatGPT Plus to create and share their own custom chatbots, known as GPTs (Generative Pre-trained Transformers). The GPT Store offers a platform for developers to monetize their custom GPTs and provides a wide range of AI tools and capabilities for users to explore and enhance their AI assistant capabilities
With the GPT Builder, users can tailor GPTs for specific tasks or topics by combining instructions, knowledge, and capabilities. It enables users to build AI Agents without the need for coding skills, making it accessible to a wide range of individuals, including educators, coaches, and anyone interested in building helpful tools.
To create a GPT using the GPT Builder, users can access the builder interface through the OpenAI platform at chat.openai.com/gpts/editor or by selecting "My GPTs" after logging in. The builder interface provides a split screen with a Create panel where users can enter prompts and instructions to build their chatbot, and a Preview panel that allows users to interact with the chatbot as they build it, making it easier to refine and customize the GPT.
The GPT Builder also offers features such as the ability to add images to the GPT, either by asking the builder to create an image or by uploading custom images. Additionally, GPTs can be granted access to web browsing, DALL-E (an image generation model), and OpenAI's Code Interpreter tool for writing and executing software. The builder interface also includes a Knowledge section where users can upload custom data to enhance the capabilities of their GPTs .
Let's build GPT: from scratch, in code, spelled out.
Andrej Karpathy ...We build a Generatively Pretrained Transformer (GPT), following the paper "Attention is All You Need" and OpenAI's GPT-2 / GPT-3. We talk about connections to ChatGPT, which has taken the world by storm. We watch GitHub Copilot, itself a GPT, help us write a GPT (meta :D!) . I recommend people watch the earlier makemore videos to get comfortable with the autoregressive language modeling framework and basics of tensors and PyTorch nn, which we take for granted in this video.
The GPU I'm training the model on is from Lambda GPU Cloud, I think the best and easiest way to spin up an on-demand GPU instance in the cloud that you can ssh to: https://lambdalabs.com . If you prefer to work in notebooks, I think the easiest path today is Google Colab.
Suggested exercises:
EX1: The n-dimensional tensor mastery challenge: Combine the `Head` and `MultiHeadAttention` into one class that processes all the heads in parallel, treating the heads as another batch dimension (answer is in nanoGPT).
EX2: Train the GPT on your own dataset of choice! What other data could be fun to blabber on about? (A fun advanced suggestion if you like: train a GPT to do addition of two numbers, i.e. a+b=c. You may find it helpful to predict the digits of c in reverse order, as the typical addition algorithm (that you're hoping it learns) would proceed right to left too. You may want to modify the data loader to simply serve random problems and skip the generation of train.bin, val.bin. You may want to mask out the loss at the input positions of a+b that just specify the problem using y=-1 in the targets (see CrossEntropyLoss ignore_index). Does your Transformer learn to add? Once you have this, swole doge project: build a calculator clone in GPT, for all of +-*/. Not an easy problem. You may need Chain of Thought traces.)
EX3: Find a dataset that is very large, so large that you can't see a gap between train and val loss. Pretrain the transformer on this data, then initialize with that model and finetune it on tiny shakespeare with a smaller number of steps and lower learning rate. Can you obtain a lower validation loss by the use of pretraining?
EX4: Read some transformer papers and implement one additional feature or change that people seem to use. Does it improve the performance of your GPT?
Transformers: The best idea in AI | Andrej Karpathy and Lex Fridman
Lex Fridman Podcast full episode
Please support this podcast by checking out our sponsors:
GUEST BIO:
Andrej Karpathy is a legendary AI researcher, engineer, and educator. He's the former director of AI at Tesla, a founding member of OpenAI, and an educator at Stanford.
