Difference between revisions of "Neural Network Pruning"
| Line 9: | Line 9: | ||
* [http://openreview.net/pdf?id=rJl-b3RcF7 The Lottery Ticket Hypothesis: Finfing Spare, Trainable Neural Networks | Jonathan Frankle & Michael Carbin - MIT CSAIL] | * [http://openreview.net/pdf?id=rJl-b3RcF7 The Lottery Ticket Hypothesis: Finfing Spare, Trainable Neural Networks | Jonathan Frankle & Michael Carbin - MIT CSAIL] | ||
| + | * [http://www.degruyter.com/downloadpdf/j/comp.2013.3.issue-3/s13537-013-0109-x/s13537-013-0109-x.pdf Pruning algorithms of neural networks - a comparative study | M. Gethsiyal Augasta | ||
| + | , T. Kathirvalavakumar] | ||
Lottery Ticket Hypothesis: The discovery hinges on the reality that the random connection strengths assigned during initialization aren’t, in fact, random in their consequences: they predispose different parts of the network to fail or succeed before training even happens. Put another way, the initial configuration influences which final configuration the network will arrive at. By focusing on this idea, the researchers found that if you prune an oversize network after training, you can actually reuse the resultant smaller network to train on new data and preserve high performance—as long as you reset each connection within this downsized network back to its initial strength. | Lottery Ticket Hypothesis: The discovery hinges on the reality that the random connection strengths assigned during initialization aren’t, in fact, random in their consequences: they predispose different parts of the network to fail or succeed before training even happens. Put another way, the initial configuration influences which final configuration the network will arrive at. By focusing on this idea, the researchers found that if you prune an oversize network after training, you can actually reuse the resultant smaller network to train on new data and preserve high performance—as long as you reset each connection within this downsized network back to its initial strength. | ||
Revision as of 14:06, 12 May 2019
Youtube search... ...Google search
- The Lottery Ticket Hypothesis: Finfing Spare, Trainable Neural Networks | Jonathan Frankle & Michael Carbin - MIT CSAIL
- [http://www.degruyter.com/downloadpdf/j/comp.2013.3.issue-3/s13537-013-0109-x/s13537-013-0109-x.pdf Pruning algorithms of neural networks - a comparative study | M. Gethsiyal Augasta
, T. Kathirvalavakumar]
Lottery Ticket Hypothesis: The discovery hinges on the reality that the random connection strengths assigned during initialization aren’t, in fact, random in their consequences: they predispose different parts of the network to fail or succeed before training even happens. Put another way, the initial configuration influences which final configuration the network will arrive at. By focusing on this idea, the researchers found that if you prune an oversize network after training, you can actually reuse the resultant smaller network to train on new data and preserve high performance—as long as you reset each connection within this downsized network back to its initial strength. From this finding, Frankle and his coauthor Michael Carbin, an assistant professor at MIT, propose what they call the “lottery ticket hypothesis.” When you randomly initialize a neural network’s connection strengths, it’s almost like buying a bag of lottery tickets. Within your bag, you hope, is a winning ticket—i.e., an initial configuration that will be easy to train and result in a successful model. A new way to build tiny neural networks could create powerful AI on your phone | Karen Hao - MIT Technology Review
