Dimensional Reduction - Revision history

BPeat at 03:54, 2 March 2024

2024-03-02T03:54:48Z

BPeat at 02:58, 6 December 2023

2023-12-06T02:58:20Z

BPeat: /* Product Quantization (PQ) */

2023-10-09T00:55:58Z

‎Product Quantization (PQ)

BPeat: /* Product Quantization (PQ) */

2023-10-09T00:54:17Z

‎Product Quantization (PQ)

BPeat: /* Product Quantization (PQ) */

2023-10-09T00:51:52Z

‎Product Quantization (PQ)

BPeat: /* Product Quantization (PQ) */

2023-10-09T00:49:59Z

‎Product Quantization (PQ)

BPeat at 09:36, 13 September 2023

2023-09-13T09:36:39Z

BPeat at 17:04, 17 August 2023

2023-08-17T17:04:24Z

BPeat: /* Product Quantization (PQ) */

2023-08-17T17:00:51Z

‎Product Quantization (PQ)

BPeat: /* Product quantization (PQ) */

2023-08-17T15:34:29Z

‎Product quantization (PQ)

@@ Line 102: / Line 102: @@
 * PQ divides and splits vectors into segments and quantizes each segment of the vectors separately.
-* Each vector in the database is converted to a short code, known as a PQ code, which is a representation that is extremely memory-efficient for the approximate nearest neighbor search.
+* Each vector in the database is converted to a short code, known as a PQ code, which is a representation that is extremely [[memory]]-efficient for the approximate nearest neighbor search.
 * PQ methods decompose the embedding manifold into a Cartesian product of M disjoint partitions and quantize each partition into K clusters.
 * PQ is highly scalable and can be used for large-scale searches.

@@ Line 87: / Line 87: @@
 ** [https://arxiv.org/pdf/1802.03426.pdf Uniform Manifold Approximation and Projection (UMAP) | L. McInnes, J. Healy, and J. Melville] ... a dimension reduction technique that can be used for visualisation similarly to [[T-Distributed Stochastic Neighbor Embedding (t-SNE) | t-SNE]], but also for general non-linear dimension reduction.
 *** [https://github.com/lmcinnes/umap UMAP]...[[Python]] version
-*** [https://github.com/pair-code/umap-js UMAP-JS] ...[[Javascript]] version
+*** [https://github.com/pair-code/umap-js UMAP-JS] ...[[JavaScript]] version
 * [https://www.sciencedirect.com/science/article/pii/S2215016120303137 Uncovering High-dimensional Structures of Projections from Dimensionality Reduction Methods | Michael Thrun & Alfred Ultsch - ScienceDirect]

@@ Line 95: / Line 95: @@
 * [[Inverted File Indexes (IVF)]]
 * [[Approximate Nearest Neighbor (ANN)]]
 Product quantization (PQ) is a technique used for vector compression and is very effective in compressing high-dimensional vectors for nearest neighbor search. The idea behind PQ is to decompose the space into a Cartesian product of low-dimensional subspaces and to quantize each subspace separately. This technique is used in [[Approximate Nearest Neighbor (ANN)|Approximate Nearest Neighbor Search (ANNs)]] and is a vital part of many vector quantization techniques.

@@ Line 106: / Line 106: @@
 * PQ is used in many vector search libraries, including Faiss, which contains different index types, including one with product quantization (IVF-PQ).
-PQ is a very effective way to compress vectors because it can achieve high compression rates while still preserving the semantic relationships between the vectors. This is because PQ preserves the pairwise distances between the vectors, which is what is important for many NLP and ML tasks. PQ is often used in conjunction with other vector indexing techniques, such as inverted files and k-nearest neighbors (kNN). This is because PQ can be used to compress the vectors, which makes them smaller and faster to search, while the other techniques can be used to further improve the search performance. Some of the disadvantages of using product quantization for vector indexing are it can be sensitive to the choice of codebook size and the number of subvectors and it can be difficult to tune the hyperparameters for optimal performance. Here are some of the advantages of using product quantization for vector indexing:
+PQ is a very effective way to compress vectors because it can achieve high compression rates while still preserving the semantic relationships between the vectors. This is because PQ preserves the pairwise distances between the vectors, which is what is important for many NLP and ML tasks. PQ is often used in conjunction with other vector indexing techniques, such as [[Inverted File Indexes (IVF)]] and [[K-Nearest Neighbors (KNN)]]. This is because PQ can be used to compress the vectors, which makes them smaller and faster to search, while the other techniques can be used to further improve the search performance. Some of the disadvantages of using product quantization for vector indexing are it can be sensitive to the choice of codebook size and the number of subvectors and it can be difficult to tune the hyperparameters for optimal performance. Here are some of the advantages of using product quantization for vector indexing:
 * It is very effective at compressing vectors while preserving their semantic relationships.
 * It is fast and scalable, making it suitable for indexing large datasets of vectors.

@@ Line 107: / Line 107: @@
 PQ is a very effective way to compress vectors because it can achieve high compression rates while still preserving the semantic relationships between the vectors. This is because PQ preserves the pairwise distances between the vectors, which is what is important for many NLP and ML tasks. PQ is often used in conjunction with other vector indexing techniques, such as inverted files and k-nearest neighbors (kNN). This is because PQ can be used to compress the vectors, which makes them smaller and faster to search, while the other techniques can be used to further improve the search performance. Some of the disadvantages of using product quantization for vector indexing are it can be sensitive to the choice of codebook size and the number of subvectors and it can be difficult to tune the hyperparameters for optimal performance. Here are some of the advantages of using product quantization for vector indexing:
-*** It is very effective at compressing vectors while preserving their semantic relationships.
+* It is very effective at compressing vectors while preserving their semantic relationships.
-*** It is fast and scalable, making it suitable for indexing large datasets of vectors.
+* It is fast and scalable, making it suitable for indexing large datasets of vectors.
-*** It is often used in conjunction with other vector indexing techniques to further improve the search performance.
+* It is often used in conjunction with other vector indexing techniques to further improve the search performance.
 <youtube>t9mRf2S5vDI</youtube>
 <youtube>PNVJvZEkuXo</youtube>

@@ Line 20: / Line 20: @@
 [https://www.bing.com/news/search?q=~Dimensional+~Reduction+AI&qft=interval%3d%228%22 ...Bing News]
-* [[Embedding]] ... [[Fine-tuning]] ... [[Agents#AI-Powered Search|Search]] ... [[Clustering]] ... [[Recommendation]] ... [[Anomaly Detection]] ... [[Classification]] ... [[Dimensional Reduction]].  [[...find outliers]]
+* [[Embedding]] ... [[Fine-tuning]] ... [[Retrieval-Augmented Generation (RAG)|RAG]] ... [[Agents#AI-Powered Search|Search]] ... [[Clustering]] ... [[Recommendation]] ... [[Anomaly Detection]] ... [[Classification]] ... [[Dimensional Reduction]].  [[...find outliers]]
 * [[Math for Intelligence]] ... [[Finding Paul Revere]] ... [[Social Network Analysis (SNA)]] ... [[Dot Product]] ... [[Kernel Trick]]
 * [[Hyperdimensional Computing (HDC)]]

← Older revision		Revision as of 17:04, 17 August 2023
Line 105:		Line 105:
	* PQ is highly scalable and can be used for large-scale searches.		* PQ is highly scalable and can be used for large-scale searches.
	* PQ is used in many vector search libraries, including Faiss, which contains different index types, including one with product quantization (IVF-PQ).		* PQ is used in many vector search libraries, including Faiss, which contains different index types, including one with product quantization (IVF-PQ).
		+
		+	<youtube>t9mRf2S5vDI</youtube>
		+	<youtube>PNVJvZEkuXo</youtube>

@@ Line 93: / Line 93: @@
 = <span id="Product Quantization (PQ)"></span>Product Quantization (PQ) =
-* [https://www.pinecone.io/learn/series/faiss/product-quantization/ a]
+* [[Inverted File Indexes (IVF)]]
-* [https://arxiv.org/pdf/1711.10775.pdf a]
+* [[Approximate Nearest Neighbor (ANN)]]
 Product quantization (PQ) is a technique used for vector compression and is very effective in compressing high-dimensional vectors for nearest neighbor search. The idea behind PQ is to decompose the space into a Cartesian product of low-dimensional subspaces and to quantize each subspace separately. This technique is used in [[Approximate Nearest Neighbor (ANN)|Approximate Nearest Neighbor Search (ANNs)]] and is a vital part of many vector quantization techniques.