Difference between revisions of "Other Challenges"
m (BPeat moved page Challenges to Other Challenges without leaving a redirect) |
(→The Wall - Deep Learning) |
||
| Line 18: | Line 18: | ||
== The Wall - Deep Learning == | == The Wall - Deep Learning == | ||
Deep neural nets are huge and bulky inefficient creatures that allow you to effectively solve a learning problem by getting huge amounts of data and a super computer. They currently trade efficiency for brute force almost every time. | Deep neural nets are huge and bulky inefficient creatures that allow you to effectively solve a learning problem by getting huge amounts of data and a super computer. They currently trade efficiency for brute force almost every time. | ||
| + | |||
| + | |||
| + | [https://towardsdatascience.com/recent-advances-for-a-better-understanding-of-deep-learning-part-i-5ce34d1cc914 Towards Theoretical Understanding of Deep Learning | Sanjeev Arora] | ||
| + | |||
| + | * Non Convex Optimization: How can we understand the highly non-convex loss function associated with deep neural networks? Why does stochastic gradient descent even converge? | ||
| + | * Overparametrization and Generalization: In classical statistical theory, generalization depends on the number of parameters but not in deep learning. Why? Can we find another good measure of generalization? | ||
| + | * Role of Depth: How does depth help a neural network to converge? What is the link between depth and generalization? | ||
| + | * Generative Models: Why do Generative Adversarial Networks (GANs) work so well? What theoretical properties could we use to stabilize them or avoid mode collapse? | ||
| + | |||
| + | |||
| + | |||
| + | |||
<youtube>WTnxE0wjZaM</youtube> | <youtube>WTnxE0wjZaM</youtube> | ||
Revision as of 21:02, 16 February 2019
YouTube search... ...Google search
- Privacy in Data Science
- Backpropagation
- Gradient Descent Optimization & Challenges
- AI Verification and Validation
- Digital Twins
- Occlusions
- Bio-inspired Computing
The Wall - Deep Learning
Deep neural nets are huge and bulky inefficient creatures that allow you to effectively solve a learning problem by getting huge amounts of data and a super computer. They currently trade efficiency for brute force almost every time.
Towards Theoretical Understanding of Deep Learning | Sanjeev Arora
- Non Convex Optimization: How can we understand the highly non-convex loss function associated with deep neural networks? Why does stochastic gradient descent even converge?
- Overparametrization and Generalization: In classical statistical theory, generalization depends on the number of parameters but not in deep learning. Why? Can we find another good measure of generalization?
- Role of Depth: How does depth help a neural network to converge? What is the link between depth and generalization?
- Generative Models: Why do Generative Adversarial Networks (GANs) work so well? What theoretical properties could we use to stabilize them or avoid mode collapse?
The Expert
