Difference between revisions of "Prompt Engineering (PE)"
m (→Prompting for Image Generation) |
m |
||
| Line 256: | Line 256: | ||
|}<!-- B --> | |}<!-- B --> | ||
| − | === Few-shot Learning | + | == <span id="Prompting for Image Generation"></span>Prompting for Image Generation == |
| + | [https://www.youtube.com/results?search_query=prompt+engineering+image+DALL+midjourney+Stable+Diffusion YouTube] | ||
| + | [https://www.quora.com/search?q=prompt%20engineering%20image%20DALL%20midjourney%20Stable%20Diffusion ... Quora] | ||
| + | [https://www.google.com/search?q=prompt+engineering+image+DALL+midjourney+Stable+Diffusion ...Google search] | ||
| + | [https://news.google.com/search?q=prompt+engineering+image+DALL+midjourney+Stable+Diffusion ...Google News] | ||
| + | [https://www.bing.com/news/search?q=prompt+engineering+image+DALL+midjourney+Stable+Diffusion%3d%228%22 ...Bing News] | ||
| + | |||
| + | * [[Video/Image#DALL-E | DALL-E]] | ||
| + | * [[Video/Image#Stable Diffusion | Stable Diffusion]] | ||
| + | * [[Video/Image#MidJourney | MidJourney]] | ||
| + | * [https://www.artbreeder.com/ ArtBreeder] ... a collaborative, machine learning-based art website. Using the models StyleGAN and BigGAN | ||
| + | * [https://www.whytryai.com/p/image-prompt-sites Here’s a Whole Bunch of Sites to Help You With Image Prompts | Daniel Nest - Why Try AI]: | ||
| + | ** [https://lexica.art/ Lexica] ... a massive database of images generated via Stable Diffusion, with prompts attached. You can search the library, explore styles related to a selected image, and easily copy text prompts to use elsewhere. You can even generate images directly on the site. | ||
| + | ** [https://prompthero.com/ PromptHero] ... not limited to just Stable Diffusion, organizes images into a few top-level categories like “Anime,” “Architecture,” “Landscapes,” etc. making it easier to browse images related to the same theme.] | ||
| + | ** [https://www.prompthunt.com/explore Prompt Hunt] ... filter by your preferred AI art generator | ||
| + | ** Prompt builders - | ||
| + | *** [https://prompterguide.com/ Prompter] ... use Midjourney, prompt builder, [https://www.google.com/sheets/about/ Google Sheet] that lets you customize all possible aspects of your Midjourney prompt. | ||
| + | *** [https://promptomania.com/prompt-builder/ promptoMANIA] ... all-in-one prompt builder; guides you through picking your subject, styles, artist tags, and other relevant descriptors. You also get nice visual references for the modifiers, giving you a decent idea of how your final image might turn out. | ||
| + | ** Automated prompt generators - | ||
| + | *** [https://huggingface.co/spaces/Gustavosta/MagicPrompt-Stable-Diffusion Stable Diffusion Prompt Generator] ... demo of the model series: “MagicPrompt” | ||
| + | *** [https://huggingface.co/spaces/doevent/prompt-generator Midjourney Prompt Generator] generates multiple prompts which often include Midjourney-specific tags like “--ar” (to set the aspect ratio). | ||
| + | ** [https://aigenprompt.vercel.app/ AI Prompt Generator] ... creates a narrative around your subject, describing the setting in great detail using natural language. You can then plop it into any text-to-image program. | ||
| + | ** Image-to-text tools - | ||
| + | *** [https://huggingface.co/spaces/pharma/CLIP-Interrogator CLIP Interrogator] ... features a neat “Analyze” tab that breaks down your prompt into underlying components like “Artist,” “Medium,” etc. | ||
| + | *** [https://replicate.com/methexis-inc/img2prompt Img2Prompt by Methexis] ... tweaked version of the CLIP Interrogator. | ||
| + | *** [https://huggingface.co/spaces/tomg-group-umd/pez-dispenser PEZ Dispenser] ... condense an existing text prompt into a shorter one | ||
| + | |||
| + | |||
| + | After I have a rough idea of what I want to accomplish, I try to narrow things down to people, places and things - the core actors or main drivers in the scene I’m trying to construct. I use the service to generate a few rough prompts to get a feel for what the scene might look like. I find it much easier to take something that works well and then add on to it rather than having to go back and remove things until it looks better. You start with the big important strokes and then work in the finer details. | ||
| + | ... I see prompt writing from the perspective of an artist, coder and engineer. I use my programming experience to help me understand how the service may interpret my prompt, which guides me to more effective tinkering with it to coax the results I’m after. Every word in a prompt has a weight associated with it, so trying to work out what works best and where becomes a core asset in the skillset. My background in software quality assurance is a pretty big driver in that “what happens if” style of thinking. - [https://www.theverge.com/2022/9/2/23326868/dalle-midjourney-ai-promptbase-prompt-market-sales-artist-interview Professional AI whisperers have launched a marketplace for DALL-E prompts | Justin Reckling interviewed by article author Adi Robertson - The Verge] | ||
| + | |||
| + | = Few-shot Learning = | ||
* [https://ai.plainenglish.io/chatgpt-prompt-engineering-lets-think-step-by-step-and-other-magic-phrases-f5c6e143a82a ChatGPT Prompt Engineering, “Let’s Think Step by Step”, and other Magic Phrases - Mr Newq - Medium] | * [https://ai.plainenglish.io/chatgpt-prompt-engineering-lets-think-step-by-step-and-other-magic-phrases-f5c6e143a82a ChatGPT Prompt Engineering, “Let’s Think Step by Step”, and other Magic Phrases - Mr Newq - Medium] | ||
| Line 262: | Line 293: | ||
| − | + | = writeGPT = | |
* [https://writegpt.ai/ WriteGPT | ][[OpenAI]] ... web extension to facilitate prompt writing | * [https://writegpt.ai/ WriteGPT | ][[OpenAI]] ... web extension to facilitate prompt writing | ||
| Line 290: | Line 321: | ||
|}<!-- B --> | |}<!-- B --> | ||
| − | + | = Recursively Criticizing and Improving the output (RCI) = | |
* [https://posgnu.github.io/rci-web/ Language Models can Solve Computer Tasks | Marktechpost] | * [https://posgnu.github.io/rci-web/ Language Models can Solve Computer Tasks | Marktechpost] | ||
| Line 296: | Line 327: | ||
| − | + | = PromptBase = | |
* [https://promptbase.com PromptBase] ... Find top prompts, produce better results, save on API costs, sell your own prompts. | * [https://promptbase.com PromptBase] ... Find top prompts, produce better results, save on API costs, sell your own prompts. | ||
** [https://promptbase.com/prompt/generator-for-chatgpt-prompts Generator For ChatGPT Prompts |] [https://promptbase.com/profile/bartnase Bart Nase - PromptBase] | ** [https://promptbase.com/prompt/generator-for-chatgpt-prompts Generator For ChatGPT Prompts |] [https://promptbase.com/profile/bartnase Bart Nase - PromptBase] | ||
| Line 343: | Line 374: | ||
Prompt templates are objects that are made up of a template with certain input variables. This object can then be called with .format(...) to format the input variables accordingly. | Prompt templates are objects that are made up of a template with certain input variables. This object can then be called with .format(...) to format the input variables accordingly. | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
Revision as of 22:04, 26 April 2023
YouTube ... Quora ...Google search ...Google News ...Bing News
- Assistants ... Agents ... Negotiation ... HuggingGPT ... LangChain
- Python ... Generative AI with Python ... Javascript ... Generative AI with Javascript ... Game Development with Generative AI
- Natural Language Processing (NLP) ...Generation ...LLM ...Tools & Services
- Attention Mechanism ...Transformer Model ...Generative Pre-trained Transformer (GPT)
- Generative AI for Business Analysis
- Generative AI ... Conversational AI ... OpenAI's ChatGPT ... Perplexity ... Microsoft's Bing ... You ...Google's Bard ... Baidu's Ernie
- Prompt Injection Attack
- Prompt Engineer | Anthropic ... Salary - The expected salary range for this position is $175k - $335k
- Prompt Engineering | Kris - All About AI ...Allabtai ... and other informative AI videos
- Lists:
- Prompt Engineering Guide | Elvis Saravia - dair.ai ... lecture, notebook, & slides
- Awesome ChatGPT Prompts
- ChatGPT cheatsheet | QuickRef.ME
- Learn Prompting ... a Free, Open Source Course on Communicating with Artificial Intelligence
Human-Machine Interaction (HMI) is used in engineering for asking questions or interacting with an artificial intelligence like myself is often referred to as Human-Computer Interaction (HCI) or Human-Machine Interaction (HMI). These terms describe the study and practice of designing, evaluating, and implementing interactive systems or devices that allow humans to communicate with machines, including artificial intelligence models like myself. This can involve designing user interfaces, voice and Speech Recognition, natural language processing, and other technologies that allow people to communicate effectively with machines. In general, engineers and designers in the field of HCI/HMI are focused on creating systems and devices that are intuitive, user-friendly, and effective at facilitating communication between humans and machines, like me, to achieve certain goals. As an AI language model, I am designed to assist you with your questions and concerns. Here are some tips on how to ask and converse with me effectively:
- Be clear and specific in your questions. The more precise your question, the easier it will be for me to provide you with a helpful answer.
- Use simple language. While I am designed to understand a wide range of language, it is best to use clear and concise language to avoid confusion.
- Provide context. Providing additional information and context around your question can help me to provide you with a more accurate and relevant answer.
- Be patient. Sometimes it may take me a moment to process your question and generate a response. Please be patient and allow me time to think.
- Follow up with additional questions. If you need more information or clarification, feel free to ask follow-up questions so that I can provide you with the most accurate and relevant information possible.
Remember, I am a language model designed to assist you, so feel free to ask any questions or have any conversations you would like. I'm always here to help! - written by ChatGPT | OpenAI
Contents
What is Prompt Engineering (PE)?
|
|
|
|
|
|
|
|
|
Prompting for Text vs Image Generation
Prompting for text generation and image generation are two different tasks, and as such, the prompts that are used for each task are also different.
- For text generation, the prompt is typically a short sentence or phrase that describes the desired output. For example, a prompt for generating a poem might be "A sonnet about love." The prompt should be specific enough to give the model a clear idea of what is desired, but it should also be open-ended enough to allow for creativity.
- For image generation, the prompt is typically a more detailed description of the desired image. This might include the subject of the image, the setting, the mood, and any other relevant details. For example, a prompt for generating an image of a cat sitting on a chair might be "A realistic image of a black cat sitting on a red chair in a sunny room." The prompt should be as detailed as possible to give the model a clear idea of what is desired.
Here are some tips for writing effective prompts for text generation and image generation:
- Be specific. The more specific the prompt, the better the model will be able to understand what you are asking for.
- Be open-ended. Don't be afraid to leave some room for creativity. The model will use its knowledge of the world to fill in the blanks.
- Use keywords. If you are not sure how to describe something, use keywords that are related to it. For example, if you want to generate an image of a cat, you could use the keywords "cat," "animal," "furry," and "meow."
- Use examples. If you can, provide examples of the kind of output you are looking for. This will help the model to understand what you are trying to achieve.
Prompting for Text Generation
- The Art of ChatGPT Prompting: A Guide to Crafting Clear and Effective Prompts | Faith Akin ...free e-book
|
|
|
|
|
|
Prompting for Image Generation
YouTube ... Quora ...Google search ...Google News ...Bing News
- DALL-E
- Stable Diffusion
- MidJourney
- ArtBreeder ... a collaborative, machine learning-based art website. Using the models StyleGAN and BigGAN
- Here’s a Whole Bunch of Sites to Help You With Image Prompts | Daniel Nest - Why Try AI:
- Lexica ... a massive database of images generated via Stable Diffusion, with prompts attached. You can search the library, explore styles related to a selected image, and easily copy text prompts to use elsewhere. You can even generate images directly on the site.
- PromptHero ... not limited to just Stable Diffusion, organizes images into a few top-level categories like “Anime,” “Architecture,” “Landscapes,” etc. making it easier to browse images related to the same theme.]
- Prompt Hunt ... filter by your preferred AI art generator
- Prompt builders -
- Prompter ... use Midjourney, prompt builder, Google Sheet that lets you customize all possible aspects of your Midjourney prompt.
- promptoMANIA ... all-in-one prompt builder; guides you through picking your subject, styles, artist tags, and other relevant descriptors. You also get nice visual references for the modifiers, giving you a decent idea of how your final image might turn out.
- Automated prompt generators -
- Stable Diffusion Prompt Generator ... demo of the model series: “MagicPrompt”
- Midjourney Prompt Generator generates multiple prompts which often include Midjourney-specific tags like “--ar” (to set the aspect ratio).
- AI Prompt Generator ... creates a narrative around your subject, describing the setting in great detail using natural language. You can then plop it into any text-to-image program.
- Image-to-text tools -
- CLIP Interrogator ... features a neat “Analyze” tab that breaks down your prompt into underlying components like “Artist,” “Medium,” etc.
- Img2Prompt by Methexis ... tweaked version of the CLIP Interrogator.
- PEZ Dispenser ... condense an existing text prompt into a shorter one
After I have a rough idea of what I want to accomplish, I try to narrow things down to people, places and things - the core actors or main drivers in the scene I’m trying to construct. I use the service to generate a few rough prompts to get a feel for what the scene might look like. I find it much easier to take something that works well and then add on to it rather than having to go back and remove things until it looks better. You start with the big important strokes and then work in the finer details.
... I see prompt writing from the perspective of an artist, coder and engineer. I use my programming experience to help me understand how the service may interpret my prompt, which guides me to more effective tinkering with it to coax the results I’m after. Every word in a prompt has a weight associated with it, so trying to work out what works best and where becomes a core asset in the skillset. My background in software quality assurance is a pretty big driver in that “what happens if” style of thinking. - Professional AI whisperers have launched a marketplace for DALL-E prompts | Justin Reckling interviewed by article author Adi Robertson - The Verge
Few-shot Learning
Few-shot learning uses only a small number of examples to guide the model in generating accurate outputs for new tasks like recognizing and classifying novel objects or concepts. Typically, fewer than a dozen examples are used.
writeGPT
- WriteGPT | OpenAI ... web extension to facilitate prompt writing
|
|
Recursively Criticizing and Improving the output (RCI)
The RCI approach significantly outperforms existing LLM methods for automating computer tasks and surpasses supervised learning (SL) and reinforcement learning (RL) approaches on the MiniWoB++ benchmark. RCI is competitive with the state-of-the-art SL+RL method, using only a handful of demonstrations per task rather than tens of thousands, and without a task-specific reward function. Furthermore, we demonstrate RCI prompting's effectiveness in enhancing LLMs' reasoning abilities on a suite of natural language reasoning tasks, outperforming chain of thought (CoT) prompting. We find that RCI combined with CoT performs better than either separately.
PromptBase
- PromptBase ... Find top prompts, produce better results, save on API costs, sell your own prompts.
- The wild world of PromptBase, the eBay for Generative AI prompts | Ryan Broderick - Fast Company ... There’s an art to writing the string of prompts that can produce what you want from ChatGPT, DALL-E 2, and Midjourney. Right now, there are more buyers than sellers.
|
|
|
|
Prompt Templates - LangChain
- Assistants ... Agents ... Negotiation ... HuggingGPT ... LangChain
- Prompt Templates | Langchain - GitHub
Prompt templates are objects that are made up of a template with certain input variables. This object can then be called with .format(...) to format the input variables accordingly.