Math for Intelligence - Revision history

BPeat: /* Tensors */

2026-02-04T15:11:53Z

‎Tensors

BPeat: /* Tensors */

2026-02-04T15:11:09Z

‎Tensors

BPeat: /* Scalar, Vector, Matrix & Tensor */

2026-02-04T15:10:08Z

‎Scalar, Vector, Matrix & Tensor

BPeat: /* Scalar, Vector, Matrix & Tensor */

2026-02-04T15:09:50Z

‎Scalar, Vector, Matrix & Tensor

BPeat at 15:07, 4 February 2026

2026-02-04T15:07:10Z

BPeat: /* Matrix Tensor-reduction Algorithm */

2024-06-14T16:08:19Z

‎Matrix Tensor-reduction Algorithm

BPeat: /* Matrix Tensor-reduction Algorithm */

2024-06-14T16:07:53Z

‎Matrix Tensor-reduction Algorithm

BPeat: /* Alternative Matrix Algorithms */

2024-06-14T16:04:07Z

‎Alternative Matrix Algorithms

BPeat: /* Alternative Matrix Algorithms */

2024-06-14T16:00:44Z

‎Alternative Matrix Algorithms

BPeat: /* Alternative Matrix Algorithms */

2024-06-14T15:53:31Z

‎Alternative Matrix Algorithms

@@ Line 296: / Line 296: @@
 === Tensors ===
 In mathematics, a tensor is an algebraic object that describes a (multilinear) relationship between sets of algebraic objects related to a vector space. Objects that tensors may map between include vectors and scalars, and, recursively, even other tensors. Tensors can take several different forms – for example: scalars and vectors (which are the simplest tensors), dual vectors, multi-linear maps between vector spaces, and even some operations such as the dot product. Tensors are defined independent of any basis, although they are often referred to by their components in a basis related to a particular coordinate system. [http://en.wikipedia.org/wiki/Tensor Wikipedia]
 {|<!-- T -->
@@ Line 316: / Line 314: @@
 |}
 |}<!-- B -->
 == 3blue1brown ==

@@ Line 296: / Line 296: @@
 === Tensors ===
 In mathematics, a tensor is an algebraic object that describes a (multilinear) relationship between sets of algebraic objects related to a vector space. Objects that tensors may map between include vectors and scalars, and, recursively, even other tensors. Tensors can take several different forms – for example: scalars and vectors (which are the simplest tensors), dual vectors, multi-linear maps between vector spaces, and even some operations such as the dot product. Tensors are defined independent of any basis, although they are often referred to by their components in a basis related to a particular coordinate system. [http://en.wikipedia.org/wiki/Tensor Wikipedia]
 {|<!-- T -->

@@ Line 143: / Line 143: @@
 == Scalar, Vector, Matrix & Tensor ==
 * [https://dev.to/mmithrakumar/scalars-vectors-matrices-and-tensors-with-tensorflow-2-0-1f66 Scalars, Vectors, Matrices and Tensors with Tensorflow 2.0 | Mukesh Mithrakumar - Dev]
 {|<!-- T -->
@@ Line 294: / Line 296: @@
 === Tensors ===
 In mathematics, a tensor is an algebraic object that describes a (multilinear) relationship between sets of algebraic objects related to a vector space. Objects that tensors may map between include vectors and scalars, and, recursively, even other tensors. Tensors can take several different forms – for example: scalars and vectors (which are the simplest tensors), dual vectors, multi-linear maps between vector spaces, and even some operations such as the dot product. Tensors are defined independent of any basis, although they are often referred to by their components in a basis related to a particular coordinate system. [http://en.wikipedia.org/wiki/Tensor Wikipedia]
 {|<!-- T -->

@@ Line 143: / Line 143: @@
 == Scalar, Vector, Matrix & Tensor ==
 * [https://dev.to/mmithrakumar/scalars-vectors-matrices-and-tensors-with-tensorflow-2-0-1f66 Scalars, Vectors, Matrices and Tensors with Tensorflow 2.0 | Mukesh Mithrakumar - Dev]
 {|<!-- T -->
@@ Line 296: / Line 294: @@
 === Tensors ===
 In mathematics, a tensor is an algebraic object that describes a (multilinear) relationship between sets of algebraic objects related to a vector space. Objects that tensors may map between include vectors and scalars, and, recursively, even other tensors. Tensors can take several different forms – for example: scalars and vectors (which are the simplest tensors), dual vectors, multi-linear maps between vector spaces, and even some operations such as the dot product. Tensors are defined independent of any basis, although they are often referred to by their components in a basis related to a particular coordinate system. [http://en.wikipedia.org/wiki/Tensor Wikipedia]
 {|<!-- T -->

@@ Line 19: / Line 19: @@
 [https://news.google.com/search?q=ai+math+mathematics ...Google News]
 [https://www.bing.com/news/search?q=ai+math+mathematics&qft=interval%3d%228%22 ...Bing News]
 * [[Math for Intelligence]] ... [[Finding Paul Revere]] ... [[Social Network Analysis (SNA)]] ... [[Dot Product]] ... [[Kernel Trick]]

← Older revision		Revision as of 16:08, 14 June 2024
Line 274:		Line 274:

	One key aspect of tensor reduction is the identification and elimination of redundant or unnecessary computations. By intelligently selecting and processing only the essential elements of the tensors involved, tensor reduction techniques minimize the computational workload without compromising the accuracy or quality of the model's output. This selective approach significantly reduces the time and resources required for matrix multiplication operations, making LLMs more efficient and scalable.		One key aspect of tensor reduction is the identification and elimination of redundant or unnecessary computations. By intelligently selecting and processing only the essential elements of the tensors involved, tensor reduction techniques minimize the computational workload without compromising the accuracy or quality of the model's output. This selective approach significantly reduces the time and resources required for matrix multiplication operations, making LLMs more efficient and scalable.
		+
	<youtube>fDAPJ7rvcUw</youtube>		<youtube>fDAPJ7rvcUw</youtube>

@@ Line 271: / Line 271: @@
 ===== Matrix Tensor-reduction Algorithm =====
 <youtube>fDAPJ7rvcUw</youtube>
 ===== Matrix Addition Algorithm =====

@@ Line 260: / Line 260: @@
 * <b>Tensor Reduction:</b> Google's tensor reduction technique aims to optimize the efficiency of matrix multiplication operations in LLMs by reducing the computational overhead.
-* <b>MatMul-free Gated Recurrent Unit (MLGRU):</b> architecture eliminates matrix multiplications from language models while maintaining strong performance at large scales. It combines the MLGRU token mixer and the Gated Linear Unit (GLU) channel mixer with ternary weights.
+* <b>Matrix Addition Algorithm:</b> architecture eliminates matrix multiplications from language models while maintaining strong performance at large scales. It combines the MatMul-free Gated Recurrent Unit (MLGRU) token mixer and the Gated Linear Unit (GLU) channel mixer with ternary weights.
 ===== Matrix Tensor-reduction Algorithm =====

← Older revision		Revision as of 16:00, 14 June 2024
Line 256:		Line 256:

	==== Alternative Matrix Algorithms ====		==== Alternative Matrix Algorithms ====
		+	Researchers are investigating several alternative matrix algorithms as substitutes for traditional matrix multiplication in Large Language Models (LLMs). Some notable approaches include:

		+	* <b>Tensor Reduction:</b> Google's tensor reduction technique aims to optimize the efficiency of matrix multiplication operations in LLMs by reducing the computational overhead.
		+
		+	* <b>MatMul-free Gated Recurrent Unit (MLGRU):</b> architecture eliminates matrix multiplications from language models while maintaining strong performance at large scales. It combines the MLGRU token mixer and the Gated Linear Unit (GLU) channel mixer with ternary weights.

	===== Matrix Tensor-reduction Algorithm =====		===== Matrix Tensor-reduction Algorithm =====

@@ Line 256: / Line 256: @@
 ==== Alternative Matrix Algorithms ====
 * [[Gated Recurrent Unit (GRU)]]
 * [https://arxiv.org/abs/2406.02528 Scalable MatMul-free Language Modeling | R. Zhu, Y. Zhang, E. Sifferman, T. Sheaves, Y. Wang, D. Richmond, P. Zhou, J. Eshraghian - Cornell University]
@@ Line 270: / Line 277: @@
 <youtube>cAzWnlaSX90</youtube>
 === Tensors ===