Embedding - Revision history

BPeat at 03:03, 6 March 2024

2024-03-06T03:03:12Z

BPeat at 02:43, 3 March 2024

2024-03-03T02:43:40Z

BPeat at 15:53, 9 October 2023

2023-10-09T15:53:15Z

BPeat at 15:51, 9 October 2023

2023-10-09T15:51:06Z

BPeat: /* Traditional Approach */

2023-10-08T22:31:48Z

‎Traditional Approach

BPeat: /* Traditional Approach */

2023-10-08T22:28:25Z

‎Traditional Approach

BPeat: /* Semantic Encoding Approach */

2023-10-08T22:21:30Z

‎Semantic Encoding Approach

BPeat: /* Text Embeddings & Text Similarity Measures */

2023-10-08T22:19:35Z

‎Text Embeddings & Text Similarity Measures

BPeat: /* Traditional Approach */

2023-10-08T22:19:19Z

‎Traditional Approach

BPeat at 22:18, 8 October 2023

2023-10-08T22:18:44Z

@@ Line 22: / Line 22: @@
 * [[Embedding]] ... [[Fine-tuning]] ... [[Retrieval-Augmented Generation (RAG)|RAG]] ... [[Agents#AI-Powered Search|Search]] ... [[Clustering]] ... [[Recommendation]] ... [[Anomaly Detection]] ... [[Classification]] ... [[Dimensional Reduction]].  [[...find outliers]]
 * [[Prompting vs AI Model Fine-Tuning vs AI Embeddings]]
-* [[AI Solver]] ... [[Algorithms]] ... [[Algorithm Administration|Administration]] ... [[Model Search]] ... [[Discriminative vs. Generative]] ... [[Optimizer]] ... [[Train, Validate, and Test]]
+* [[AI Solver]] ... [[Algorithms]] ... [[Algorithm Administration|Administration]] ... [[Model Search]] ... [[Discriminative vs. Generative]] ... [[Train, Validate, and Test]]
 * [[Math for Intelligence]] ... [[Finding Paul Revere]] ... [[Social Network Analysis (SNA)]] ... [[Dot Product]] ... [[Kernel Trick]]
 * [[Capabilities]]:

@@ Line 98: / Line 98: @@
 ** Blue: 3 -> 001
-* <b>Count-based</b>: Count-based embedding is a method of representing words or phrases as vectors based on the frequency of their co-occurrence with other words or phrases in a given context. This approach is often used in [[Natural Language Processing (NLP)]] tasks, such as [[Natural Language Classification (NLC)|text classification]] and word similarity analysis. The main idea behind count-based embedding is to capture the relationship between words by counting how often they appear together in a corpus of text. This information is then used to construct a vector representation for each word, where the elements of the vector correspond to the frequency of co-occurrence with other words. One common way to create count-based embeddings is by using a co-occurrence matrix, which is a square matrix where each row and column represents a unique word in the vocabulary, and the value at each position (i, j) represents the frequency of co-occurrence between the words i and j. This matrix can be large and memory-intensive, so [[Dimensional Reduction|dimensionality reduction techniques]] are often applied to make the model more efficient and robust. Count-based embeddings have some advantages and disadvantages compared to other methods, such as predictive models like [[Word2Vec]]. On the one hand, count-based methods can capture more significant information about word relationships, as they consider the entire co-occurrence matrix. On the other hand, they tend to consume more memory and may include some noise or less significant information
+* <b>Count-based</b>: Count-based embedding is a method of representing words or phrases as vectors based on the frequency of their co-occurrence with other words or phrases in a given context. This approach is often used in [[Natural Language Processing (NLP)]] tasks, such as [[Natural Language Classification (NLC)|text classification]] and word similarity analysis. The main idea behind count-based embedding is to capture the relationship between words by counting how often they appear together in a corpus of text. This information is then used to construct a vector representation for each word, where the elements of the vector correspond to the frequency of co-occurrence with other words. One common way to create count-based embeddings is by using a co-occurrence matrix, which is a square matrix where each row and column represents a unique word in the vocabulary, and the value at each position (i, j) represents the frequency of co-occurrence between the words i and j. This matrix can be large and [[memory]]-intensive, so [[Dimensional Reduction|dimensionality reduction techniques]] are often applied to make the model more efficient and robust. Count-based embeddings have some advantages and disadvantages compared to other methods, such as predictive models like [[Word2Vec]]. On the one hand, count-based methods can capture more significant information about word relationships, as they consider the entire co-occurrence matrix. On the other hand, they tend to consume more [[memory]] and may include some noise or less significant information
 * <b>TF-IDF techniques for vectorization</b>: [[Term Frequency–Inverse Document Frequency (TF-IDF)]] has several advantages, including its simplicity, computational efficiency, and effectiveness in capturing the importance of words in a document. However, it also has limitations, such as not capturing the semantic meaning of words.

@@ Line 21: / Line 21: @@
 * [[Embedding]] ... [[Fine-tuning]] ... [[Retrieval-Augmented Generation (RAG)|RAG]] ... [[Agents#AI-Powered Search|Search]] ... [[Clustering]] ... [[Recommendation]] ... [[Anomaly Detection]] ... [[Classification]] ... [[Dimensional Reduction]].  [[...find outliers]]
 * [[Prompting vs AI Model Fine-Tuning vs AI Embeddings]]
 * [[AI Solver]] ... [[Algorithms]] ... [[Algorithm Administration|Administration]] ... [[Model Search]] ... [[Discriminative vs. Generative]] ... [[Optimizer]] ... [[Train, Validate, and Test]]
 * [[Math for Intelligence]] ... [[Finding Paul Revere]] ... [[Social Network Analysis (SNA)]] ... [[Dot Product]] ... [[Kernel Trick]]
 * [[Capabilities]]:
-** [[AI-Powered Search|Search]] (where results are ranked by relevance to a query string)
+** [[Semantic Search]] (where results are ranked by relevance to a query string)
 ** [[Clustering]] (where text strings are grouped by similarity)
 ** [[Recommendation]]s (where items with related text strings are recommended)

@@ Line 21: / Line 21: @@
 * [[Embedding]] ... [[Fine-tuning]] ... [[Retrieval-Augmented Generation (RAG)|RAG]] ... [[Agents#AI-Powered Search|Search]] ... [[Clustering]] ... [[Recommendation]] ... [[Anomaly Detection]] ... [[Classification]] ... [[Dimensional Reduction]].  [[...find outliers]]
 * [[Prompting vs AI Model Fine-Tuning vs AI Embeddings]]
 * [[AI Solver]] ... [[Algorithms]] ... [[Algorithm Administration|Administration]] ... [[Model Search]] ... [[Discriminative vs. Generative]] ... [[Optimizer]] ... [[Train, Validate, and Test]]

@@ Line 83: / Line 83: @@
 == Traditional Approach ==
 * [https://www.educative.io/blog/one-hot-encoding Data Science in 5 Minutes: What is One Hot Encoding? | Amanda Fawcett - Educative]
 * [https://www.linkedin.com/pulse/comprehensive-guide-feature-engineering-n-grams-david-adamson-mbcs/ A Comprehensive Guide To Feature Engineering with N-Grams For Natural Language Processing | David Adamson]

← Older revision		Revision as of 22:21, 8 October 2023
Line 100:		Line 100:

	== Semantic Encoding Approach ==		== Semantic Encoding Approach ==
−		+	* [https://medium.com/nerd-for-tech/nlp-zero-to-one-dense-representations-word2vec-part-5-30-9b38c5ccfbfc NLP Zero to One: Dense Representations, Word2Vec (Part 5/30) \| Kowshik chilamkurthy - Nerd For Tech - Medium] ... Word Embeddings and Semantic Representations

@@ Line 109: / Line 109: @@
 * [https://abdulkaderhelwan.medium.com/introduction-to-word-and-sentence-embedding-991c735a2b0b Introduction to Word and Sentence Embedding | Abdulkader Helwan - Medium]
 * [https://medium.com/dair-ai/making-monolingual-sentence-embeddings-multilingual-using-knowledge-distillation-59d8a7713672 Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation | Viktor Karlsson - DAIR.AI - Medium]
 * <b>Dot product</b>: a technique for representing data as a vector of real numbers in a way that preserves semantic relationships. The dot product embedding of a vector is calculated by multiplying the vector with a learned embedding matrix. The embedding matrix is a matrix of real numbers that has been learned to capture the semantic relationships between words, sentences, or documents. Once the dot product embedding of a vector has been calculated, it can be used to compare the vector to other vectors by calculating the dot product between the embeddings. The dot product between two embeddings is a measure of their similarity. A higher dot product indicates more similarity.

@@ Line 99: / Line 99: @@
 == Semantic Encoding Approach ==
 * <b>Word2Vec and dense word embeddings</b>: [[Word2Vec]] is a semantic encoding approach that learns dense word embeddings. Dense word embeddings are vectors of real numbers that represent words in a way that captures their semantic relationships. For example, the embeddings for the words "king" and "queen" would be close together, while the embeddings for the words "king" and "apple" would be far apart. [[Word2Vec]] learns word embeddings by training a neural network to predict the context of a given word. The context of a word is the set of words that often appear around it in text. For example, some of the context words for the word "king" might be "queen", "crown", and "throne".
 == Text Embeddings & Text Similarity Measures ==
 * <b>Dot product</b>: a technique for representing data as a vector of real numbers in a way that preserves semantic relationships. The dot product embedding of a vector is calculated by multiplying the vector with a learned embedding matrix. The embedding matrix is a matrix of real numbers that has been learned to capture the semantic relationships between words, sentences, or documents. Once the dot product embedding of a vector has been calculated, it can be used to compare the vector to other vectors by calculating the dot product between the embeddings. The dot product between two embeddings is a measure of their similarity. A higher dot product indicates more similarity.