Sparse Autoencoder (SAE) - Revision history

BPeat at 22:55, 22 May 2019

2019-05-22T22:55:24Z

BPeat at 04:33, 3 February 2019

2019-02-03T04:33:12Z

BPeat at 03:04, 13 May 2018

2018-05-13T03:04:18Z

BPeat: BPeat moved page Sparse Autoencoder to Sparse Autoencoder (SAE) without leaving a redirect

2018-05-12T01:47:55Z

BPeat moved page Sparse Autoencoder to Sparse Autoencoder (SAE) without leaving a redirect

BPeat at 00:41, 12 May 2018

2018-05-12T00:41:56Z

BPeat: Created page with "[http://www.youtube.com/results?search_query=sparse+autoencoder YouTube search...] vfnxKO2rMq4 EmmLvsljmrk"

2018-05-11T03:26:12Z

Created page with "[http://www.youtube.com/results?search_query=sparse+autoencoder YouTube search...] <youtube>vfnxKO2rMq4</youtube> <youtube>EmmLvsljmrk</youtube>"

New page

[http://www.youtube.com/results?search_query=sparse+autoencoder YouTube search...]

<youtube>vfnxKO2rMq4</youtube>
<youtube>EmmLvsljmrk</youtube>

← Older revision		Revision as of 22:55, 22 May 2019
Line 9:		Line 9:

	* [http://www.asimovinstitute.org/author/fjodorvanveen/ Neural Network Zoo \| Fjodor Van Veen]		* [http://www.asimovinstitute.org/author/fjodorvanveen/ Neural Network Zoo \| Fjodor Van Veen]
		+	* [[Feature Exploration/Learning]]

	Sparse autoencoders (SAE) are in a way the opposite of AEs. Instead of teaching a network to represent a bunch of information in less “space” or nodes, we try to encode information in more space. So instead of the network converging in the middle and then expanding back to the input size, we blow up the middle. These types of networks can be used to extract many small features from a dataset. If one were to train a SAE the same way as an AE, you would in almost all cases end up with a pretty useless identity network (as in what comes in is what comes out, without any transformation or decomposition). To prevent this, instead of feeding back the input, we feed back the input plus a sparsity driver. This sparsity driver can take the form of a threshold filter, where only a certain error is passed back and trained, the other error will be “irrelevant” for that pass and set to zero. In a way this resembles spiking neural networks, where not all neurons fire all the time (and points are scored for biological plausibility). Marc’Aurelio Ranzato, Christopher Poultney, Sumit Chopra, and Yann LeCun. “Efficient learning of sparse representations with an energy-based model.” Proceedings of NIPS. 2007.		Sparse autoencoders (SAE) are in a way the opposite of AEs. Instead of teaching a network to represent a bunch of information in less “space” or nodes, we try to encode information in more space. So instead of the network converging in the middle and then expanding back to the input size, we blow up the middle. These types of networks can be used to extract many small features from a dataset. If one were to train a SAE the same way as an AE, you would in almost all cases end up with a pretty useless identity network (as in what comes in is what comes out, without any transformation or decomposition). To prevent this, instead of feeding back the input, we feed back the input plus a sparsity driver. This sparsity driver can take the form of a threshold filter, where only a certain error is passed back and trained, the other error will be “irrelevant” for that pass and set to zero. In a way this resembles spiking neural networks, where not all neurons fire all the time (and points are scored for biological plausibility). Marc’Aurelio Ranzato, Christopher Poultney, Sumit Chopra, and Yann LeCun. “Efficient learning of sparse representations with an energy-based model.” Proceedings of NIPS. 2007.

← Older revision		Revision as of 04:33, 3 February 2019
Line 1:		Line 1:
		+	{{#seo:
		+	\|title=PRIMO.ai
		+	\|titlemode=append
		+	\|keywords=artificial, intelligence, machine, learning, models, algorithms, data, singularity, moonshot, Tensorflow, Google, Nvidia, Microsoft, Azure, Amazon, AWS
		+	\|description=Helpful resources for your journey with artificial intelligence; videos, articles, techniques, courses, profiles, and tools
		+	}}
	[http://www.youtube.com/results?search_query=sparse+autoencoder YouTube search...]		[http://www.youtube.com/results?search_query=sparse+autoencoder YouTube search...]
		+	[http://www.google.com/search?q=sparse+autoencoder+machine+learning+ML+artificial+intelligence ...Google search]

	* [http://www.asimovinstitute.org/author/fjodorvanveen/ Neural Network Zoo \| Fjodor Van Veen]		* [http://www.asimovinstitute.org/author/fjodorvanveen/ Neural Network Zoo \| Fjodor Van Veen]

@@ Line 4: / Line 4: @@
 Sparse autoencoders (SAE) are in a way the opposite of AEs. Instead of teaching a network to represent a bunch of information in less “space” or nodes, we try to encode information in more space. So instead of the network converging in the middle and then expanding back to the input size, we blow up the middle. These types of networks can be used to extract many small features from a dataset. If one were to train a SAE the same way as an AE, you would in almost all cases end up with a pretty useless identity network (as in what comes in is what comes out, without any transformation or decomposition). To prevent this, instead of feeding back the input, we feed back the input plus a sparsity driver. This sparsity driver can take the form of a threshold filter, where only a certain error is passed back and trained, the other error will be “irrelevant” for that pass and set to zero. In a way this resembles spiking neural networks, where not all neurons fire all the time (and points are scored for biological plausibility).  Marc’Aurelio Ranzato, Christopher Poultney, Sumit Chopra, and Yann LeCun. “Efficient learning of sparse representations with an energy-based model.” Proceedings of NIPS. 2007.
 <youtube>vfnxKO2rMq4</youtube>
 <youtube>EmmLvsljmrk</youtube>

← Older revision	Revision as of 01:47, 12 May 2018
(No difference)