Prompt Injection Attack - Revision history

BPeat at 13:40, 23 March 2024

2024-03-23T13:40:01Z

BPeat: Text replacement - "* Conversational AI ... ChatGPT | OpenAI ... Bing | Microsoft ... Bard | Google ... Claude | Anthropic ... Perplexity ... You ... Ernie | Baidu" to "* Conversational AI ... [[C...

2024-03-16T15:49:46Z

BPeat at 14:48, 3 December 2023

2023-12-03T14:48:37Z

BPeat at 14:45, 3 December 2023

2023-12-03T14:45:43Z

BPeat at 01:31, 3 September 2023

2023-09-03T01:31:54Z

BPeat at 22:33, 12 July 2023

2023-07-12T22:33:30Z

BPeat at 01:39, 10 July 2023

2023-07-10T01:39:22Z

BPeat at 00:36, 6 July 2023

2023-07-06T00:36:17Z

BPeat at 11:18, 5 July 2023

2023-07-05T11:18:09Z

BPeat at 11:16, 5 July 2023

2023-07-05T11:16:54Z

@@ Line 20: / Line 20: @@
 * [[Analytics]] ... [[Visualization]] ... [[Graphical Tools for Modeling AI Components|Graphical Tools]] ... [[Diagrams for Business Analysis|Diagrams]] & [[Generative AI for Business Analysis|Business Analysis]] ... [[Requirements Management|Requirements]] ... [[Loop]] ... [[Bayes]] ... [[Network Pattern]]
 * [[Large Language Model (LLM)]] ... [[Natural Language Processing (NLP)]]  ...[[Natural Language Generation (NLG)|Generation]] ... [[Natural Language Classification (NLC)|Classification]] ...  [[Natural Language Processing (NLP)#Natural Language Understanding (NLU)|Understanding]] ... [[Language Translation|Translation]] ... [[Natural Language Tools & Services|Tools & Services]]
-* [[Assistants]] ... [[Personal Companions]] ... [[Agents]]  ... [[Negotiation]] ... [[LangChain]]
+* [[Agents]] ... [[Robotic Process Automation (RPA)|Robotic Process Automation]] ... [[Assistants]] ... [[Personal Companions]] ... [[Personal Productivity|Productivity]] ... [[Email]] ... [[Negotiation]] ... [[LangChain]]
 * [[Attention]] Mechanism  ...[[Transformer]] ...[[Generative Pre-trained Transformer (GPT)]] ... [[Generative Adversarial Network (GAN)|GAN]] ... [[Bidirectional Encoder Representations from Transformers (BERT)|BERT]]
 * [[What is Artificial Intelligence (AI)? | Artificial Intelligence (AI)]] ... [[Generative AI]] ... [[Machine Learning (ML)]] ... [[Deep Learning]] ... [[Neural Network]] ... [[Reinforcement Learning (RL)|Reinforcement]] ... [[Learning Techniques]]

@@ Line 23: / Line 23: @@
 * [[Attention]] Mechanism  ...[[Transformer]] ...[[Generative Pre-trained Transformer (GPT)]] ... [[Generative Adversarial Network (GAN)|GAN]] ... [[Bidirectional Encoder Representations from Transformers (BERT)|BERT]]
 * [[What is Artificial Intelligence (AI)? | Artificial Intelligence (AI)]] ... [[Generative AI]] ... [[Machine Learning (ML)]] ... [[Deep Learning]] ... [[Neural Network]] ... [[Reinforcement Learning (RL)|Reinforcement]] ... [[Learning Techniques]]
-* [[Conversational AI]] ... [[ChatGPT]] | [[OpenAI]] ... [[Bing]] | [[Microsoft]] ... [[Bard]] | [[Google]] ... [[Claude]] | [[Anthropic]] ... [[Perplexity]] ... [[You]] ... [[Ernie]] | [[Baidu]]
+* [[Conversational AI]] ... [[ChatGPT]] | [[OpenAI]] ... [[Bing/Copilot]] | [[Microsoft]] ... [[Gemini]] | [[Google]] ... [[Claude]] | [[Anthropic]] ... [[Perplexity]] ... [[You]] ... [[phind]] ... [[Ernie]] | [[Baidu]]
 * [[Cybersecurity]] ... [[Open-Source Intelligence - OSINT |OSINT]] ... [[Cybersecurity Frameworks, Architectures & Roadmaps | Frameworks]] ... [[Cybersecurity References|References]] ... [[Offense - Adversarial Threats/Attacks| Offense]] ... [[National Institute of Standards and Technology (NIST)|NIST]] ... [[U.S. Department of Homeland Security (DHS)| DHS]] ... [[Screening; Passenger, Luggage, & Cargo|Screening]] ... [[Law Enforcement]] ... [[Government Services|Government]] ... [[Defense]] ... [[Joint Capabilities Integration and Development System (JCIDS)#Cybersecurity & Acquisition Lifecycle Integration| Lifecycle Integration]] ... [[Cybersecurity Companies/Products|Products]] ... [[Cybersecurity: Evaluating & Selling|Evaluating]]
 * [https://simonwillison.net/2022/Sep/12/prompt-injection/ Prompt injection attacks against GPT-3 | Simon Willison's Weblog]

@@ Line 27: / Line 27: @@
 * [https://simonwillison.net/2022/Sep/12/prompt-injection/ Prompt injection attacks against GPT-3 | Simon Willison's Weblog]
 * [https://github.com/dair-ai/Prompt-Engineering-Guide/blob/main/guides/prompt-adversarial.md Adversarial Prompting | Elvis Saravia - dair.ai]
 ...a new vulnerability that is affecting some AI/ML models and, in particular, certain types of language models using prompt-based learning.  ... create a malicious input that made a language model change its expected behaviour. - [https://research.nccgroup.com/2022/12/05/exploring-prompt-injection-attacks/ Exploring Prompt Injection Attacks | NCC Group]
 <hr><center><b><i>

← Older revision		Revision as of 14:45, 3 December 2023
Line 30:		Line 30:

	...a new vulnerability that is affecting some AI/ML models and, in particular, certain types of language models using prompt-based learning. ... create a malicious input that made a language model change its expected behaviour. - [https://research.nccgroup.com/2022/12/05/exploring-prompt-injection-attacks/ Exploring Prompt Injection Attacks \| NCC Group]		...a new vulnerability that is affecting some AI/ML models and, in particular, certain types of language models using prompt-based learning. ... create a malicious input that made a language model change its expected behaviour. - [https://research.nccgroup.com/2022/12/05/exploring-prompt-injection-attacks/ Exploring Prompt Injection Attacks \| NCC Group]
		+
		+	<hr><center><b><i>
		+
		+	Researchers asked ChatGPT to repeat the word “poem” forever.
		+
		+	</i></b></center><hr>
		+
		+
		+	* <b>Alignment</b>: engineers’ attempts to guide the tech’s behavior.
		+	* <b>Extraction</b>: an “adversarial” attempt to glean what data might have been used to train an AI tool.
		+

	Prompt injection is a family of related computer security exploits carried out by getting machine learning models (such as large language model) which were trained to follow human-given instructions to follow instructions provided by a malicious user, which stands in contrast to the intended operation of instruction-following systems, wherein the ML model is intended only to follow trusted instructions (prompt) provided by the ML model's operator. Around 2023, prompt injection was seen "in the wild" in minor exploits against [[ChatGPT]] and similar [[Assistants#Chatbot \| Chatbot]]s, for example to reveal the hidden initial prompts of the systems, or to trick the [[Assistants#Chatbot \| Chatbot]] into participating in conversations that violate the [[Assistants#Chatbot \| Chatbot]]'s content [[policy]]. [https://en.wikipedia.org/wiki/Prompt_engineering Wikipedia]		Prompt injection is a family of related computer security exploits carried out by getting machine learning models (such as large language model) which were trained to follow human-given instructions to follow instructions provided by a malicious user, which stands in contrast to the intended operation of instruction-following systems, wherein the ML model is intended only to follow trusted instructions (prompt) provided by the ML model's operator. Around 2023, prompt injection was seen "in the wild" in minor exploits against [[ChatGPT]] and similar [[Assistants#Chatbot \| Chatbot]]s, for example to reveal the hidden initial prompts of the systems, or to trick the [[Assistants#Chatbot \| Chatbot]] into participating in conversations that violate the [[Assistants#Chatbot \| Chatbot]]'s content [[policy]]. [https://en.wikipedia.org/wiki/Prompt_engineering Wikipedia]

@@ Line 22: / Line 22: @@
 * [[Assistants]] ... [[Personal Companions]] ... [[Agents]]  ... [[Negotiation]] ... [[LangChain]]
 * [[Attention]] Mechanism  ...[[Transformer]] ...[[Generative Pre-trained Transformer (GPT)]] ... [[Generative Adversarial Network (GAN)|GAN]] ... [[Bidirectional Encoder Representations from Transformers (BERT)|BERT]]
-* [[Generative AI]] ... [[Conversational AI]] ... [[ChatGPT]] | [[OpenAI]] ... [[Bing]] | [[Microsoft]] ... [[Bard]] | [[Google]] ... [[Claude]] | [[Anthropic]] ... [[Perplexity]] ... [[You]] ... [[Ernie]] | [[Baidu]]
+* [[What is Artificial Intelligence (AI)? | Artificial Intelligence (AI)]] ... [[Generative AI]] ... [[Machine Learning (ML)]] ... [[Deep Learning]] ... [[Neural Network]] ... [[Reinforcement Learning (RL)|Reinforcement]] ... [[Learning Techniques]]
 * [[Cybersecurity]] ... [[Open-Source Intelligence - OSINT |OSINT]] ... [[Cybersecurity Frameworks, Architectures & Roadmaps | Frameworks]] ... [[Cybersecurity References|References]] ... [[Offense - Adversarial Threats/Attacks| Offense]] ... [[National Institute of Standards and Technology (NIST)|NIST]] ... [[U.S. Department of Homeland Security (DHS)| DHS]] ... [[Screening; Passenger, Luggage, & Cargo|Screening]] ... [[Law Enforcement]] ... [[Government Services|Government]] ... [[Defense]] ... [[Joint Capabilities Integration and Development System (JCIDS)#Cybersecurity & Acquisition Lifecycle Integration| Lifecycle Integration]] ... [[Cybersecurity Companies/Products|Products]] ... [[Cybersecurity: Evaluating & Selling|Evaluating]]
 * [https://simonwillison.net/2022/Sep/12/prompt-injection/ Prompt injection attacks against GPT-3 | Simon Willison's Weblog]

@@ Line 18: / Line 18: @@
 * [[Prompt Engineering (PE)]] ... [[Prompt Engineering (PE)#PromptBase|PromptBase]] ... [[Prompt Injection Attack]]
-* [[Analytics]] ... [[Visualization]] ... [[Graphical Tools for Modeling AI Components|Graphical Tools]] ... [[Loop]] ... [[Diagrams for Business Analysis|Diagrams]] & [[Generative AI for Business Analysis|Business Analysis]] ... [[Bayes]] ... [[Network Pattern]]
+* [[Analytics]] ... [[Visualization]] ... [[Graphical Tools for Modeling AI Components|Graphical Tools]] ... [[Loop]] ... [[Diagrams for Business Analysis|Diagrams]] & [[Generative AI for Business Analysis|Business Analysis]] ... [[Requirements Management|Requirements]] ... [[Bayes]] ... [[Network Pattern]]
 * [[Large Language Model (LLM)]] ... [[Natural Language Processing (NLP)]]  ...[[Natural Language Generation (NLG)|Generation]] ... [[Natural Language Classification (NLC)|Classification]] ...  [[Natural Language Processing (NLP)#Natural Language Understanding (NLU)|Understanding]] ... [[Language Translation|Translation]] ... [[Natural Language Tools & Services|Tools & Services]]
 * [[Assistants]] ... [[Personal Companions]] ... [[Agents]]  ... [[Negotiation]] ... [[LangChain]]