Decentralized: Federated & Distributed - Revision history

BPeat at 20:56, 28 April 2024

2024-04-28T20:56:57Z

BPeat at 01:52, 27 April 2024

2024-04-27T01:52:50Z

BPeat at 03:12, 2 March 2024

2024-03-02T03:12:40Z

BPeat at 18:50, 18 September 2023

2023-09-18T18:50:02Z

BPeat at 18:45, 16 September 2023

2023-09-16T18:45:25Z

BPeat at 19:30, 22 August 2023

2023-08-22T19:30:08Z

BPeat at 01:44, 20 August 2023

2023-08-20T01:44:51Z

BPeat at 01:42, 20 August 2023

2023-08-20T01:42:05Z

BPeat at 01:41, 20 August 2023

2023-08-20T01:41:28Z

BPeat at 00:35, 13 July 2023

2023-07-13T00:35:57Z

@@ Line 133: / Line 133: @@
 <youtube>xtxxLWZznBI</youtube>
 <b>Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis
-</b><br>In this video from 2018 Swiss HPC Conference, Torsten Hoefler from (ETH) Zürich presents: Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis. "[[Neural Network#Deep Neural Network (DNN)|Deep Neural Networks (DNN)]] are becoming an important tool in modern computing applications. Accelerating their training is a major challenge and techniques range from distributed algorithms to low-level circuit design. In this talk, we describe the problem from a theoretical perspective, followed by approaches for its parallelization. Specifically, we present trends in DNN architectures and the resulting implications on parallelization strategies. We discuss the different types of concurrency in DNNs; synchronous and asynchronous stochastic gradient descent; distributed system architectures; [[Agents#Communication | communication]] schemes; and performance modeling. Based on these approaches, we extrapolate potential directions for parallelism in deep learning." Learn more: https://hpcadvisorycouncil.com  Sign up for our insideHPC Newsletter: https://insidehpc.com/newsletter
+</b><br>In this video from 2018 Swiss HPC Conference, Torsten Hoefler from (ETH) Zürich presents: Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis. "[[Neural Network#Deep Neural Network (DNN)|Deep Neural Networks (DNN)]] are becoming an important tool in modern computing applications. Accelerating their training is a major challenge and techniques range from distributed algorithms to low-level circuit design. In this talk, we describe the problem from a theoretical [[perspective]], followed by approaches for its parallelization. Specifically, we present trends in DNN architectures and the resulting implications on parallelization strategies. We discuss the different types of concurrency in DNNs; synchronous and asynchronous stochastic gradient descent; distributed system architectures; [[Agents#Communication | communication]] schemes; and performance modeling. Based on these approaches, we extrapolate potential directions for parallelism in deep learning." Learn more: https://hpcadvisorycouncil.com  Sign up for our insideHPC Newsletter: https://insidehpc.com/newsletter
 |}
 |<!-- M -->

@@ Line 27: / Line 27: @@
 * [[Sakana]] ... inspired by the way that fish and other animals work together in groups
 * [[Distributed Deep Reinforcement Learning (DDRL)]]
-* [[Development]] ... [[Notebooks]] ... [[Development#AI Pair Programming Tools|AI Pair Programming]] ... [[Codeless Options, Code Generators, Drag n' Drop|Codeless, Generators, Drag n' Drop]] ... [[Algorithm Administration#AIOps/MLOps|AIOps/MLOps]] ... [[Platforms: AI/Machine Learning as a Service (AIaaS/MLaaS)|AIaaS/MLaaS]]
+* [[Development]] ... [[Notebooks]] ... [[Development#AI Pair Programming Tools|AI Pair Programming]] ... [[Codeless Options, Code Generators, Drag n' Drop|Codeless]] ... [[Hugging Face]] ... [[Algorithm Administration#AIOps/MLOps|AIOps/MLOps]] ... [[Platforms: AI/Machine Learning as a Service (AIaaS/MLaaS)|AIaaS/MLaaS]]
 * [[Telecommunications]] ... [[Computer Networks]] ... [[Telecommunications#5G|5G]] ... [[Satellite#Satellite Communications|Satellite Communications]] ... [[Quantum Communications]] ... [[Agents#Communication | Agents]] ... [[AI Generated Broadcast Content|AI Broadcast; Radio, Stream, TV]]

@@ Line 23: / Line 23: @@
 * [[Risk, Compliance and Regulation]] ... [[Ethics]] ... [[Privacy]] ... [[Law]] ... [[AI Governance]] ... [[AI Verification and Validation]]
 * [[Blockchain]]
 * [[OpenMined]]
 * [[Sakana]] ... inspired by the way that fish and other animals work together in groups

@@ Line 20: / Line 20: @@
 [https://www.bing.com/news/search?q=ai+Federated+Learning+deep+machine+learning+ML&qft=interval%3d%228%22 ...Bing News]
 * [[Risk, Compliance and Regulation]] ... [[Ethics]] ... [[Privacy]] ... [[Law]] ... [[AI Governance]] ... [[AI Verification and Validation]]
 * [[Blockchain]]

@@ Line 92: / Line 92: @@
 <youtube>jkkmBpJ-Eeo</youtube>
 <b>Distinguished Lecturer : Eric Xing - Strategies & Principles for Distributed Machine Learning
-</b><br>The rise of Big Data has led to new demands for Machine Learning (ML) systems to learn complex models with millions to billions of parameters that promise adequate capacity to digest massive datasets and offer powerful [[Predictive Analytics]] (such as high-dimensional latent features, intermediate representations, and decision functions) thereupon. In order to run ML algorithms at such scales, on a distributed cluster with 10s to 1000s of machines, it is often the case that significant engineering efforts are required --- and one might fairly ask if such engineering truly falls within the domain of ML research or not. Taking the view that Big ML systems can indeed benefit greatly from ML-rooted statistical and algorithmic insights --- and that ML researchers should therefore not shy away from such systems design --- we discuss a series of principles and strategies distilled from our resent effort on industrial-scale ML solutions that involve a continuum from application, to engineering, and to theoretical research and [[development]] of Big ML system and architecture, on how to make them efficient, general, and with convergence and scaling guarantees. These principles concern four key questions which traditionally receive little attention in ML research: How to distribute an ML program over a cluster? How to bridge ML computation with inter-machine [[Agents#Communication | communication]]? How to perform such [[Agents#Communication | communication]]? What should be communicated between machines? By exposing underlying statistical and algorithmic characteristics unique to ML programs but not typical in traditional computer programs, and by dissecting successful cases of how we harness these principles to design both high-performance distributed ML software and general-purpose ML framework, we present opportunities for ML researchers and practitioners to further shape and grow the area that lies between ML and systems. This is joint work with the CMU Petuum Team.
+</b><br>The rise of Big Data has led to new demands for Machine Learning (ML) systems to learn complex models with millions to billions of parameters that promise adequate capacity to digest massive datasets and offer powerful [[Predictive Analytics]] (such as high-dimensional [[latent]] features, intermediate representations, and decision functions) thereupon. In order to run ML algorithms at such scales, on a distributed cluster with 10s to 1000s of machines, it is often the case that significant engineering efforts are required --- and one might fairly ask if such engineering truly falls within the domain of ML research or not. Taking the view that Big ML systems can indeed benefit greatly from ML-rooted statistical and algorithmic insights --- and that ML researchers should therefore not shy away from such systems design --- we discuss a series of principles and strategies distilled from our resent effort on industrial-scale ML solutions that involve a continuum from application, to engineering, and to theoretical research and [[development]] of Big ML system and architecture, on how to make them efficient, general, and with convergence and scaling guarantees. These principles concern four key questions which traditionally receive little attention in ML research: How to distribute an ML program over a cluster? How to bridge ML computation with inter-machine [[Agents#Communication | communication]]? How to perform such [[Agents#Communication | communication]]? What should be communicated between machines? By exposing underlying statistical and algorithmic characteristics unique to ML programs but not typical in traditional computer programs, and by dissecting successful cases of how we harness these principles to design both high-performance distributed ML software and general-purpose ML framework, we present opportunities for ML researchers and practitioners to further shape and grow the area that lies between ML and systems. This is joint work with the CMU Petuum Team.
 |}
 |<!-- M -->

@@ Line 14: / Line 14: @@
 </script>
 }}
-[https://www.youtube.com/results?search_query=Federated+Learning+deep+machine+learning+ML+AI Youtube search...]
+[https://www.youtube.com/results?search_query=ai+Federated+Learning+deep+machine+learning+ML YouTube]
-[https://www.google.com/search?q=Federated+Learning+deep+machine+learning+ML+AI ...Google search]
+[https://www.quora.com/search?q=ai%20Federated%20Learning%20deep%20machine%20learning%20ML ... Quora]
 * [[Risk, Compliance and Regulation]] ... [[Ethics]] ... [[Privacy]] ... [[Law]] ... [[AI Governance]] ... [[AI Verification and Validation]]

@@ Line 21: / Line 21: @@
 * [[OpenMined]]
 * [[Development]] ... [[Notebooks]] ... [[Development#AI Pair Programming Tools|AI Pair Programming]] ... [[Codeless Options, Code Generators, Drag n' Drop|Codeless, Generators, Drag n' Drop]] ... [[Algorithm Administration#AIOps/MLOps|AIOps/MLOps]] ... [[Platforms: AI/Machine Learning as a Service (AIaaS/MLaaS)|AIaaS/MLaaS]]
-* [[Computer Networks]]
+* [[Telecommunications]] ... [[Computer Networks]] ... [[Telecommunications#5G|5G]] ... [[Satellite#Satellite Communications|Satellite Communications]] ... [[Quantum#Quantum Internet / Communications|Quantum Internet]] ... [[Agents#Communication | Agents]] ... [[AI Generated Broadcast Content|AI Broadcast; Radio, Stream, TV]]