Difference between revisions of "LangChain"

From
Jump to: navigation, search
m
m
 
(36 intermediate revisions by the same user not shown)
Line 2: Line 2:
 
|title=PRIMO.ai
 
|title=PRIMO.ai
 
|titlemode=append
 
|titlemode=append
|keywords=artificial, intelligence, machine, learning, models, algorithms, data, singularity, moonshot, TensorFlow, Facebook, Google, Nvidia, Microsoft, Azure, Amazon, AWS  
+
|keywords=ChatGPT, artificial, intelligence, machine, learning, GPT-4, GPT-5, NLP, NLG, NLC, NLU, models, data, singularity, moonshot, Sentience, AGI, Emergence, Moonshot, Explainable, TensorFlow, Google, Nvidia, Microsoft, Azure, Amazon, AWS, Hugging Face, OpenAI, Tensorflow, OpenAI, Google, Nvidia, Microsoft, Azure, Amazon, AWS, Meta, LLM, metaverse, assistants, agents, digital twin, IoT, Transhumanism, Immersive Reality, Generative AI, Conversational AI, Perplexity, Bing, You, Bard, Ernie, prompt Engineering LangChain, Video/Image, Vision, End-to-End Speech, Synthesize Speech, Speech Recognition, Stanford, MIT |description=Helpful resources for your journey with artificial intelligence; videos, articles, techniques, courses, profiles, and tools
|description=Helpful resources for your journey with artificial intelligence; Attention, GPT, chat, videos, articles, techniques, courses, profiles, and tools  
+
 
 +
<!-- Google tag (gtag.js) -->
 +
<script async src="https://www.googletagmanager.com/gtag/js?id=G-4GCWLBVJ7T"></script>
 +
<script>
 +
  window.dataLayer = window.dataLayer || [];
 +
  function gtag(){dataLayer.push(arguments);}
 +
  gtag('js', new Date());
 +
 
 +
  gtag('config', 'G-4GCWLBVJ7T');
 +
</script>
 
}}
 
}}
 
[https://www.youtube.com/results?search_query=LangChain+GPT+AI YouTube]
 
[https://www.youtube.com/results?search_query=LangChain+GPT+AI YouTube]
Line 11: Line 20:
 
[https://www.bing.com/news/search?q=LangChain+GPT+AI&qft=interval%3d%228%22 ...Bing News]
 
[https://www.bing.com/news/search?q=LangChain+GPT+AI&qft=interval%3d%228%22 ...Bing News]
  
* [[Assistants]] ... [[Personal Companions]] ... [[Agents]] ... [[Negotiation]] ... [[LangChain]]
+
* [[Agents]] ... [[Robotic Process Automation (RPA)|Robotic Process Automation]] ... [[Assistants]] ... [[Personal Companions]] ... [[Personal Productivity|Productivity]] ... [[Email]] ... [[Negotiation]] ... [[LangChain]]
* [[Excel]] ... [[LangChain#Documents|Documents]] ... [[Database]] ... [[Graph]] ... [[LlamaIndex]] ... [[Algorithm Administration#AIOps/MLOps|AIOps/MLOps]] ... [[Platforms: AI/Machine Learning as a Service (AIaaS/MLaaS)|AIaaS/MLaaS]]
+
* [[Excel]] ... [[LangChain#Documents|Documents]] ... [[Database|Database; Vector & Relational]] ... [[Graph]] ... [[LlamaIndex]]
* [[Immersive Reality]] ... [[Metaverse]] ... [[Digital Twin]] ... [[Internet of Things (IoT)]] ... [[Transhumanism]]
+
* [[Immersive Reality]] ... [[Metaverse]] ... [[Omniverse]] ... [[Transhumanism]] ... [[Religion]]
 +
* [[Telecommunications]] ... [[Computer Networks]] ... [[Telecommunications#5G|5G]] ... [[Satellite#Satellite Communications|Satellite Communications]] ... [[Quantum Communications]] ... [[Agents#Communication | Communication Agents]] ... [[Smart Cities]] ... [[Digital Twin]] ... [[Internet of Things (IoT)]]  
 
* [https://hwchase17.github.io/langchainjs/docs/overview/ LangChain | GitHub]
 
* [https://hwchase17.github.io/langchainjs/docs/overview/ LangChain | GitHub]
 +
** [https://docs.langchain.com/docs/ LangChain Docs]
 
** [https://hwchase17.github.io/langchainjs/docs/modules/prompts/prompt_template Prompt Templates |  Langchain - GitHub]
 
** [https://hwchase17.github.io/langchainjs/docs/modules/prompts/prompt_template Prompt Templates |  Langchain - GitHub]
 
** [https://www.linkedin.com/in/harrison-chase-961287118/ Harrison Chase | LinkedIn]  ... [https://github.com/hwchase17 GitHub]  ...  
 
** [https://www.linkedin.com/in/harrison-chase-961287118/ Harrison Chase | LinkedIn]  ... [https://github.com/hwchase17 GitHub]  ...  
* [[Excel]] ... [[LangChain#Documents|Documents]] ... [[Database]] ... [[Graph]] ... [[Algorithm Administration#AIOps/MLOps|AIOps/MLOps]] ... [[Platforms: AI/Machine Learning as a Service (AIaaS/MLaaS)|AIaaS/MLaaS]]
+
* [[Zapier]]
* [[Case Studies]]
+
* [[Python]] ... [[Generative AI with Python|GenAI w/ Python]] ... [[JavaScript]] ... [[Generative AI with JavaScript|GenAI w/ JavaScript]] ... [[TensorFlow]] ... [[PyTorch]]
** [[Writing / Publishing]]  
+
* [[Analytics]] ... [[Visualization]] ... [[Graphical Tools for Modeling AI Components|Graphical Tools]] ... [[Diagrams for Business Analysis|Diagrams]] & [[Generative AI for Business Analysis|Business Analysis]] ... [[Requirements Management|Requirements]] ... [[Loop]] ... [[Bayes]] ... [[Network Pattern]]
* [[Python]]   ... [[Generative AI with Python]] ... [[Javascript]] ... [[Generative AI with Javascript]]  
+
* [[Development]] ... [[Notebooks]] ... [[Development#AI Pair Programming Tools|AI Pair Programming]] ... [[Codeless Options, Code Generators, Drag n' Drop|Codeless]] ... [[Hugging Face]] ... [[Algorithm Administration#AIOps/MLOps|AIOps/MLOps]] ... [[Platforms: AI/Machine Learning as a Service (AIaaS/MLaaS)|AIaaS/MLaaS]]
* [[Development]] ...[[Development#AI Pair Programming Tools|AI Pair Programming Tools]] ... [[Analytics]] ... [[Visualization]] ... [[Diagrams for Business Analysis]]
+
* [[Gaming]] ... [[Game-Based Learning (GBL)]] ... [[Games - Security|Security]] ... [[Game Development with Generative AI|Generative AI]] ... [[Metaverse#Games - Metaverse|Games - Metaverse]] ... [[Games - Quantum Theme|Quantum]] ... [[Game Theory]] ... [[Game Design | Design]]
* [[Gaming]] ... [[Game-Based Learning (GBL)]] ... [[Games - Security|Security]] ... [[Game Development with Generative AI|Generative AI]] ... [[Metaverse#Games - Metaverse|Metaverse]] ... [[Games - Quantum Theme|Quantum]] ... [[Game Theory]]
 
 
* [[ChatGPT#Integration| ChatGPT Integration]][https://twitter.com/hwchase17?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor Twitter]
 
* [[ChatGPT#Integration| ChatGPT Integration]][https://twitter.com/hwchase17?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor Twitter]
* [[Generative AI]] ... [[Conversational AI]] ... [[OpenAI]]'s [[ChatGPT]] ... [[Perplexity]] ... [[Microsoft]]'s [[Bing]] ... [[You]] ...[[Google]]'s [[Bard]] ... [[Baidu]]'s [[Ernie]]
+
* [[What is Artificial Intelligence (AI)? | Artificial Intelligence (AI)]] ... [[Generative AI]] ... [[Machine Learning (ML)]] ... [[Deep Learning]] ... [[Neural Network]] ... [[Reinforcement Learning (RL)|Reinforcement]] ... [[Learning Techniques]]
 +
* [[Conversational AI]] ... [[ChatGPT]] | [[OpenAI]] ... [[Bing/Copilot]] | [[Microsoft]] ... [[Gemini]] | [[Google]] ... [[Claude]] | [[Anthropic]] ... [[Perplexity]] ... [[You]] ... [[phind]] ... [[Ernie]] | [[Baidu]]
 
* [[Large Language Model (LLM)]] ... [[Natural Language Processing (NLP)]]  ...[[Natural Language Generation (NLG)|Generation]] ... [[Natural Language Classification (NLC)|Classification]] ...  [[Natural Language Processing (NLP)#Natural Language Understanding (NLU)|Understanding]] ... [[Language Translation|Translation]] ... [[Natural Language Tools & Services|Tools & Services]]
 
* [[Large Language Model (LLM)]] ... [[Natural Language Processing (NLP)]]  ...[[Natural Language Generation (NLG)|Generation]] ... [[Natural Language Classification (NLC)|Classification]] ...  [[Natural Language Processing (NLP)#Natural Language Understanding (NLU)|Understanding]] ... [[Language Translation|Translation]] ... [[Natural Language Tools & Services|Tools & Services]]
 
* [[Prompt Engineering (PE)]] ...[[Prompt Engineering (PE)#PromptBase|PromptBase]] ... [[Prompt Injection Attack]]  
 
* [[Prompt Engineering (PE)]] ...[[Prompt Engineering (PE)#PromptBase|PromptBase]] ... [[Prompt Injection Attack]]  
Line 30: Line 41:
 
* [https://blog.langchain.dev/langchain-chat/ LangChain Chat]  
 
* [https://blog.langchain.dev/langchain-chat/ LangChain Chat]  
 
* [https://www.mlq.ai/gpt-3-document-assistant-langchain/ Building a GPT-3 Enabled Document Assistant with LangChain | Peter Foy - MLQ.ai]
 
* [https://www.mlq.ai/gpt-3-document-assistant-langchain/ Building a GPT-3 Enabled Document Assistant with LangChain | Peter Foy - MLQ.ai]
* [https://yoheinakajima.com/task-driven-autonomous-agent-utilizing-gpt-4-pinecone-and-langchain-for-diverse-applications/ Task-driven Autonomous Agent Utilizing GPT-4], [[Agents#Pinecone|Pinecone]], and [[LangChain]] for Diverse Applications | Yohei Nakajima
+
* [https://yoheinakajima.com/task-driven-autonomous-agent-utilizing-gpt-4-pinecone-and-langchain-for-diverse-applications/ Task-driven Autonomous Agent Utilizing GPT-4], [[Database#Pinecone|Pinecone]], and [[LangChain]] for Diverse Applications | Yohei Nakajima
 
** [[Generative Pre-trained Transformer (GPT)#Autonomous GPT-4|Autonomous GPT-4]]
 
** [[Generative Pre-trained Transformer (GPT)#Autonomous GPT-4|Autonomous GPT-4]]
 
* [https://vercel.com/templates/next.js/agent-gpt AgentGPT template | Vercel] ... Assemble, configure, and deploy autonomous AI Agents in your browser, using LangChain, [[OpenAI]], [[Agents#Auto-GPT|Auto-GPT]] and T3 Stack
 
* [https://vercel.com/templates/next.js/agent-gpt AgentGPT template | Vercel] ... Assemble, configure, and deploy autonomous AI Agents in your browser, using LangChain, [[OpenAI]], [[Agents#Auto-GPT|Auto-GPT]] and T3 Stack
 +
* [https://www.kdnuggets.com/6-problems-of-llms-that-langchain-is-trying-to-assess 6 Problems of LLMs That LangChain is Trying to Assess | Josep Ferrer - KDnuggets]
  
  
 
<b>LangChain</b> is a [[Python]] framework built around [[Large Language Model (LLM)]] that can be used for chatbots, Generative Question-Answering (GQA), summarization, and more. The core idea of the library is that we can “chain” together different components to create more advanced use cases around [[Large Language Model (LLM)|LLMs]]. [[Large Language Model (LLM)|LLMs]] are emerging as a transformative technology, enabling developers to build applications that they previously could not. But using these LLMs in isolation is often not enough to create a truly powerful app - the real power comes when you are able to combine them with other sources of computation or knowledge.
 
<b>LangChain</b> is a [[Python]] framework built around [[Large Language Model (LLM)]] that can be used for chatbots, Generative Question-Answering (GQA), summarization, and more. The core idea of the library is that we can “chain” together different components to create more advanced use cases around [[Large Language Model (LLM)|LLMs]]. [[Large Language Model (LLM)|LLMs]] are emerging as a transformative technology, enabling developers to build applications that they previously could not. But using these LLMs in isolation is often not enough to create a truly powerful app - the real power comes when you are able to combine them with other sources of computation or knowledge.
  
 +
LangChain offers a way to interact with and [[fine-tuning]] LLMs on local data, providing a secure and efficient alternative to sending private data through external APIs. It allows companies to extract knowledge from their own data and develop chatbots or other applications that comprehend complex domain-specific information. By combining user input with prompts and interacting with LLMs, LangChain enables seamless integration and enhances the capabilities of applications. It utilizes vector databases as [[memory]], allowing for efficient access to relevant information during the application's execution.
 +
 +
The benefits of using LangChain for [[fine-tuning]] language models include:
 +
 +
* Seamless switching between different LLM providers: LangChain offers the flexibility to easily switch between different large language model providers. This allows developers to leverage the unique capabilities and strengths of various language models, tailoring their applications to specific needs.
 +
* Dynamic and immersive user experiences: LangChain enables applications to create dynamic and immersive user experiences by allowing language models to intelligently interact and respond to their surroundings. This feature enhances the user experience and makes applications more engaging.
 +
* Prompt management and optimization: LangChain provides capabilities for prompt management and optimization. Developers can efficiently manage prompts and optimize their performance to achieve better results from the language models.
 +
* Memory integration: LangChain allows for the integration of [[memory]] into user interactions with the language models. This feature enables applications to easily access relevant information from previous interactions, enhancing the capabilities of the application
 +
* Secure and efficient data handling: LangChain provides a secure and efficient alternative to sending private data through external APIs. It allows developers to fine-tune language models on local data, ensuring data privacy and reducing reliance on external services.
 +
* Simplified application development: LangChain simplifies the process of building applications powered by large language models. It provides a framework that empowers developers, including non-NLP specialists, to create applications that were previously difficult and required extensive expertise.
 
   
 
   
  
Line 61: Line 83:
  
 
= Documents =
 
= Documents =
* [[Excel]] ... [[LangChain#Documents|Documents]] ... [[Database]] ... [[Graph]] ... [[Algorithm Administration#AIOps/MLOps|AIOps/MLOps]] ... [[Platforms: AI/Machine Learning as a Service (AIaaS/MLaaS)|AIaaS/MLaaS]]
+
* [[Excel]] ... [[LangChain#Documents|Documents]] ... [[Database|Database; Vector & Relational]] ... [[Graph]] ... [[LlamaIndex]]
 +
* [[Data Science]] ... [[Data Governance|Governance]] ... [[Data Preprocessing|Preprocessing]] ... [[Feature Exploration/Learning|Exploration]] ... [[Data Interoperability|Interoperability]] ... [[Algorithm Administration#Master Data Management (MDM)|Master Data Management (MDM)]] ... [[Bias and Variances]] ... [[Benchmarks]] ... [[Datasets]]  
 
* [https://colab.research.google.com/drive/1-BI7NBxO6O-fYwjDoTUKrC3lmrWu0I0R?usp=sharing LangChain QA over docs | Colab]
 
* [https://colab.research.google.com/drive/1-BI7NBxO6O-fYwjDoTUKrC3lmrWu0I0R?usp=sharing LangChain QA over docs | Colab]
 
* [https://hwchase17.github.io/langchainjs/docs/modules/indexes/vector_stores/ Vectorstores | LangChain]
 
* [https://hwchase17.github.io/langchainjs/docs/modules/indexes/vector_stores/ Vectorstores | LangChain]
 
* [https://chatwithdata.teachable.com/p/aichatbotdata A step-by-step beginners program on how to build a ChatGPT chatbot for your data]
 
* [https://chatwithdata.teachable.com/p/aichatbotdata A step-by-step beginners program on how to build a ChatGPT chatbot for your data]
 
* [https://github.com/mayooear/gpt4-pdf-chatbot-langchain GPT-4 & LangChain - Create a ChatGPT Chatbot for Your PDF Files]
 
* [https://github.com/mayooear/gpt4-pdf-chatbot-langchain GPT-4 & LangChain - Create a ChatGPT Chatbot for Your PDF Files]
 +
* [https://medium.com/how-ai-built-this/zero-to-one-a-guide-to-building-a-first-pdf-chatbot-with-langchain-llamaindex-part-1-7d0e9c0d62f Zero to One: A Guide to Building a First PDF Chatbot with LangChain & LlamaIndex — Part 1 | Ryan Nguyen - Medium]
  
  
Line 86: Line 110:
 
<youtube>U_eV8wfMkXU</youtube>
 
<youtube>U_eV8wfMkXU</youtube>
 
<youtube>y1pyAQM-3Bo</youtube>
 
<youtube>y1pyAQM-3Bo</youtube>
 
= Zapier =
 
* [https://zapier.com/ Zapier]
 
* [[ChatGPT#Zapier ChatGPT Plugin| Zapier ChatGPT Plugin]]
 
 
<youtube>p9v2-xEa9A0</youtube>
 
<youtube>7tNm0yiDigU</youtube>
 
  
 
= <span id="Tabular Data"></span>Tabular Data =
 
= <span id="Tabular Data"></span>Tabular Data =
Line 111: Line 128:
 
<youtube>a75O_7tsbBo</youtube>
 
<youtube>a75O_7tsbBo</youtube>
  
= Pinecone =
+
= Vector =
* [[AI-Powered Search#Pinecone|Pinecone]]
+
* [[Database#Pinecone|Pinecone]]
* [https://www.pinecone.io/learn/langchain-intro/ Pinecone LangChain Intro]
 
* [https://colab.research.google.com/github/pinecone-io/examples/blob/master/generation/langchain/handbook/00-langchain-intro.ipynb Demo using Colab]
 
* [[Generative Pre-trained Transformer (GPT)#Autonomous GPT-4|Autonomous GPT-4]]
 
 
 
<youtube>7tNm0yiDigU</youtube>
 
<youtube>2xNzB7xq8nk</youtube>
 
<youtube>-dZrNj2mVHo</youtube>
 
<youtube>nMniwlGyX-c</youtube>
 
<youtube>15TDwVSpwKc</youtube>
 
<youtube>tp0bQNDtLPc</youtube>
 
  
 
= Supabase =
 
= Supabase =
Line 168: Line 175:
 
<youtube>uoVqNFDwpX4</youtube>
 
<youtube>uoVqNFDwpX4</youtube>
  
= Taxes =
+
= <span id="Weights & Biases (W&B)"></span>Weights & Biases (W&B) =
<youtube>68WXvOl1bpk</youtube>
+
* [https://wandb.ai Weights & Biases]
 +
* [https://docs.wandb.ai/guides/prompts/quickstart Prompts Quickstart | WandB]
 +
 
 +
W&B Sweeps and LangChain integration is a feature that allows you to fine-tune LLMs with your own data using W&B Sweeps and LangChain visualization and debugging. W&B Sweeps is a hyperparameter optimization tool that helps you find the best combination of hyperparameters for your model. W&B Sweeps and LangChain integration can:
 +
 
 +
* Create a LangChain model, chain, or agent that uses an LLM as a backend.
 +
* Import WandbTracer from wandb.integration.langchain and use it to continuously log calls to your LangChain object.
 +
* Use W&B dashboard to visualize and debug your LangChain object, such as viewing the prompts, responses, metrics, and errors.
 +
* Use W&B Sweeps to optimize the hyperparameters of your LangChain object, such as the prompt template, the context length, the temperature, and the top-k.
 +
 
 +
Weights & Biases Logging/LLMops is a feature of the Weights & Biases platform, which is a developer-first MLOps platform that provides enterprise-grade, end-to-end MLOps workflow to accelerate ML activities. Weights & Biases Logging/LLMops enables you to optimize LLM operations and prompt engineering with W&B.
 +
 
 +
 
 +
<youtube>Sy-Xp-sdlh0</youtube>
 +
<youtube>gU6Ew-Rscw8</youtube>
  
 
= More =
 
= More =

Latest revision as of 13:25, 6 November 2024

YouTube ... Quora ...Google search ...Google News ...Bing News


LangChain is a Python framework built around Large Language Model (LLM) that can be used for chatbots, Generative Question-Answering (GQA), summarization, and more. The core idea of the library is that we can “chain” together different components to create more advanced use cases around LLMs. LLMs are emerging as a transformative technology, enabling developers to build applications that they previously could not. But using these LLMs in isolation is often not enough to create a truly powerful app - the real power comes when you are able to combine them with other sources of computation or knowledge.

LangChain offers a way to interact with and fine-tuning LLMs on local data, providing a secure and efficient alternative to sending private data through external APIs. It allows companies to extract knowledge from their own data and develop chatbots or other applications that comprehend complex domain-specific information. By combining user input with prompts and interacting with LLMs, LangChain enables seamless integration and enhances the capabilities of applications. It utilizes vector databases as memory, allowing for efficient access to relevant information during the application's execution.

The benefits of using LangChain for fine-tuning language models include:

  • Seamless switching between different LLM providers: LangChain offers the flexibility to easily switch between different large language model providers. This allows developers to leverage the unique capabilities and strengths of various language models, tailoring their applications to specific needs.
  • Dynamic and immersive user experiences: LangChain enables applications to create dynamic and immersive user experiences by allowing language models to intelligently interact and respond to their surroundings. This feature enhances the user experience and makes applications more engaging.
  • Prompt management and optimization: LangChain provides capabilities for prompt management and optimization. Developers can efficiently manage prompts and optimize their performance to achieve better results from the language models.
  • Memory integration: LangChain allows for the integration of memory into user interactions with the language models. This feature enables applications to easily access relevant information from previous interactions, enhancing the capabilities of the application
  • Secure and efficient data handling: LangChain provides a secure and efficient alternative to sending private data through external APIs. It allows developers to fine-tune language models on local data, ensuring data privacy and reducing reliance on external services.
  • Simplified application development: LangChain simplifies the process of building applications powered by large language models. It provides a framework that empowers developers, including non-NLP specialists, to create applications that were previously difficult and required extensive expertise.


Getting Started

Data Independent - tutorial videos

reference videos throughout page



Documents


Long documents

Memory

Emails

Tabular Data

Javascript

Vector

Supabase

Water

Visual ChatGPT

Summarization

Hugging Face

LLama

GPT-Index

Comparing Large Language Models (LLM)

Gradio

Filtering LLM

Weights & Biases (W&B)

W&B Sweeps and LangChain integration is a feature that allows you to fine-tune LLMs with your own data using W&B Sweeps and LangChain visualization and debugging. W&B Sweeps is a hyperparameter optimization tool that helps you find the best combination of hyperparameters for your model. W&B Sweeps and LangChain integration can:

  • Create a LangChain model, chain, or agent that uses an LLM as a backend.
  • Import WandbTracer from wandb.integration.langchain and use it to continuously log calls to your LangChain object.
  • Use W&B dashboard to visualize and debug your LangChain object, such as viewing the prompts, responses, metrics, and errors.
  • Use W&B Sweeps to optimize the hyperparameters of your LangChain object, such as the prompt template, the context length, the temperature, and the top-k.

Weights & Biases Logging/LLMops is a feature of the Weights & Biases platform, which is a developer-first MLOps platform that provides enterprise-grade, end-to-end MLOps workflow to accelerate ML activities. Weights & Biases Logging/LLMops enables you to optimize LLM operations and prompt engineering with W&B.


More