Self-Supervised - Revision history

BPeat: /* Towards Autonomous Machine Intelligence */

2024-05-28T01:10:41Z

‎Towards Autonomous Machine Intelligence

BPeat: /* Towards Autonomous Machine Intelligence */

2024-05-28T01:08:27Z

‎Towards Autonomous Machine Intelligence

BPeat: /* Towards Autonomous Machine Intelligence */

2024-05-28T01:08:04Z

‎Towards Autonomous Machine Intelligence

BPeat: /* Positions/Conjectures/Crazy Hypotheses */

2024-05-28T01:04:45Z

‎Positions/Conjectures/Crazy Hypotheses

BPeat: /* Positions/Conjectures/Crazy Hypotheses */

2024-05-28T01:04:23Z

‎Positions/Conjectures/Crazy Hypotheses

BPeat: /* Positions/Conjectures/Crazy Hypotheses */

2024-05-28T00:56:38Z

‎Positions/Conjectures/Crazy Hypotheses

BPeat: /* Positions/Conjectures/Crazy Hypotheses */

2024-05-28T00:48:07Z

‎Positions/Conjectures/Crazy Hypotheses

BPeat: /* Towards Autonomous Machine Intelligence */

2024-05-28T00:44:20Z

‎Towards Autonomous Machine Intelligence

BPeat at 00:40, 28 May 2024

2024-05-28T00:40:36Z

BPeat at 00:30, 28 May 2024

2024-05-28T00:30:43Z

@@ Line 50: / Line 50: @@
 = Towards Autonomous Machine Intelligence =
-[[Creatives#Yann LeCun| Dr. Yann LeCun]]'s paper, "A Path Towards Autonomous Machine Intelligence," proposes an ambitious framework for developing AI that mimics human and animal learning more closely. LeCun's framework represents a significant shift towards more autonomous, flexible, and intelligent systems capable of complex reasoning and decision-making without extensive external supervision Here are some of the key elements and hypotheses from his work:
+[[Creatives#Yann LeCun| Dr. Yann LeCun]]'s paper, "A Path Towards Autonomous Machine Intelligence," proposes an ambitious framework for developing AI that mimics human and animal learning more closely. [[Creatives#Yann LeCun| LeCun]]'s framework represents a significant shift towards more autonomous, flexible, and intelligent systems capable of complex reasoning and decision-making without extensive external supervision Here are some of the key elements and hypotheses from his work:
-. <b>Self-Supervised Learning (SSL): </b>LeCun emphasizes the importance of SSL, where machines learn to understand the world by predicting parts of their input from other parts without needing extensive labeled data. This approach is crucial for developing AI that can reason and understand in a human-like manner.
+. <b>Self-Supervised Learning (SSL): </b>[[Creatives#Yann LeCun| LeCun]] emphasizes the importance of SSL, where machines learn to understand the world by predicting parts of their input from other parts without needing extensive labeled data. This approach is crucial for developing AI that can reason and understand in a human-like manner.
-. <b>Joint Embedding Predictive Architecture (JEPA): </b>The central concept in LeCun's proposal is JEPA, an energy-based model that captures dependencies between inputs by mapping them into a representation space. This architecture allows the model to make predictions based on abstract representations rather than direct data, making it more efficient and scalable for tasks requiring long-term prediction and reasoning.
+. <b>Joint Embedding Predictive Architecture (JEPA): </b>The central concept in [[Creatives#Yann LeCun| LeCun]]'s proposal is JEPA, an energy-based model that captures dependencies between inputs by mapping them into a representation space. This architecture allows the model to make predictions based on abstract representations rather than direct data, making it more efficient and scalable for tasks requiring long-term prediction and reasoning.
-. <b>Intrinsic Motivation: </b>LeCun suggests that autonomous AI should be driven by intrinsic objectives rather than external rewards or hard-coded programs. This idea is inspired by how animals and humans are motivated to explore and learn from their environment naturally.
+. <b>Intrinsic Motivation: </b>[[Creatives#Yann LeCun| LeCun]] suggests that autonomous AI should be driven by intrinsic objectives rather than external rewards or hard-coded programs. This idea is inspired by how animals and humans are motivated to explore and learn from their environment naturally.
 . <b>Modular Architecture: </b>The proposed framework consists of several modules:
@@ Line 70: / Line 70: @@
 == Positions/Conjectures/Crazy Hypotheses ==
-<b>Air Force Research Laboratory (AFRL)/Air Force Office of Scientific Research (AFOSR) Chief Scientist Distinguished Lecture Series featuring Dr. Yann LeCun</b>
+<b>Air Force Research Laboratory (AFRL)/Air Force Office of Scientific Research (AFOSR) Chief Scientist Distinguished Lecture Series featuring [[Creatives#Yann LeCun| Dr. Yann LeCun]]</b>
 In this talk, we discuss cognitive architecture for autonomous learning agents capable of planning and reasoning. The centerpiece of the architecture is a hierarchical world model, trained with self-supervised learning, with which the system can predict the consequences of its actions and plan sequences of actions to fulfill an objective. The world model uses a new type of deep learning architecture called H-JEPA (Hierarchical Joint Embedding Predictive Architecture) which can perform predictions at multiple time scales and multiple levels of abstraction.

@@ Line 50: / Line 50: @@
 = Towards Autonomous Machine Intelligence =
-[Creatives#Yann LeCun| Dr. Yann LeCun]'s paper, "A Path Towards Autonomous Machine Intelligence," proposes an ambitious framework for developing AI that mimics human and animal learning more closely. LeCun's framework represents a significant shift towards more autonomous, flexible, and intelligent systems capable of complex reasoning and decision-making without extensive external supervision Here are some of the key elements and hypotheses from his work:
+[[Creatives#Yann LeCun| Dr. Yann LeCun]]'s paper, "A Path Towards Autonomous Machine Intelligence," proposes an ambitious framework for developing AI that mimics human and animal learning more closely. LeCun's framework represents a significant shift towards more autonomous, flexible, and intelligent systems capable of complex reasoning and decision-making without extensive external supervision Here are some of the key elements and hypotheses from his work:
 . <b>Self-Supervised Learning (SSL): </b>LeCun emphasizes the importance of SSL, where machines learn to understand the world by predicting parts of their input from other parts without needing extensive labeled data. This approach is crucial for developing AI that can reason and understand in a human-like manner.

@@ Line 50: / Line 50: @@
 = Towards Autonomous Machine Intelligence =
-Dr. Yann LeCun's paper, "A Path Towards Autonomous Machine Intelligence," proposes an ambitious framework for developing AI that mimics human and animal learning more closely. LeCun's framework represents a significant shift towards more autonomous, flexible, and intelligent systems capable of complex reasoning and decision-making without extensive external supervision Here are some of the key elements and hypotheses from his work:
+[Creatives#Yann LeCun| Dr. Yann LeCun]'s paper, "A Path Towards Autonomous Machine Intelligence," proposes an ambitious framework for developing AI that mimics human and animal learning more closely. LeCun's framework represents a significant shift towards more autonomous, flexible, and intelligent systems capable of complex reasoning and decision-making without extensive external supervision Here are some of the key elements and hypotheses from his work:
 . <b>Self-Supervised Learning (SSL): </b>LeCun emphasizes the importance of SSL, where machines learn to understand the world by predicting parts of their input from other parts without needing extensive labeled data. This approach is crucial for developing AI that can reason and understand in a human-like manner.

@@ Line 86: / Line 86: @@
 * <b>Reasoning is an extension of simulation/prediction and planning</b>
 * <b>Conscious exists because we have only one world model engine</b>
 <youtube>EvSe0ktD95k</youtube>

@@ Line 73: / Line 73: @@
 In this talk, we discuss cognitive architecture for autonomous learning agents capable of planning and reasoning. The centerpiece of the architecture is a hierarchical world model, trained with self-supervised learning, with which the system can predict the consequences of its actions and plan sequences of actions to fulfill an objective. The world model uses a new type of deep learning architecture called H-JEPA (Hierarchical Joint Embedding Predictive Architecture) which can perform predictions at multiple time scales and multiple levels of abstraction.
-* Prediction is the essence of intelligence
+* <b>Prediction is the essence of intelligence</b>
 ** Learning predictive models of the world is the basis of common sense
-* Almost everything is learned
+* <b>Almost everything is learned</b>
-** Low-level feature, space, objects, physics, abstract represe
+** Low-level feature, space, objects, physics, abstract representations...
 <youtube>EvSe0ktD95k</youtube>

← Older revision		Revision as of 00:44, 28 May 2024
Line 66:		Line 66:

	5. <b>Hierarchical Representation and Prediction: </b>By stacking JEPA models, the architecture can handle both short-term and long-term predictions at varying levels of abstraction. For example, a high-level representation might predict broad actions like a cook preparing a dish, while lower levels detail specific movements and actions		5. <b>Hierarchical Representation and Prediction: </b>By stacking JEPA models, the architecture can handle both short-term and long-term predictions at varying levels of abstraction. For example, a high-level representation might predict broad actions like a cook preparing a dish, while lower levels detail specific movements and actions
		+
		+
		+	== Positions/Conjectures/Crazy Hypotheses ==
		+
		+	<b>AFRL/AFOSR Chief Scientist Distinguished Lecture Series featuring Dr. Yann LeCun</b>
		+	In this talk, we discuss cognitive architecture for autonomous learning agents capable of planning and reasoning. The centerpiece of the architecture is a hierarchical world model, trained with self-supervised learning, with which the system can predict the consequences of its actions and plan sequences of actions to fulfill an objective. The world model uses a new type of deep learning architecture called H-JEPA (Hierarchical Joint Embedding Predictive Architecture) which can perform predictions at multiple time scales and multiple levels of abstraction.

	<youtube>EvSe0ktD95k</youtube>		<youtube>EvSe0ktD95k</youtube>

← Older revision		Revision as of 00:40, 28 May 2024
Line 49:		Line 49:

	= Towards Autonomous Machine Intelligence =		= Towards Autonomous Machine Intelligence =
		+
		+	Dr. Yann LeCun's paper, "A Path Towards Autonomous Machine Intelligence," proposes an ambitious framework for developing AI that mimics human and animal learning more closely. LeCun's framework represents a significant shift towards more autonomous, flexible, and intelligent systems capable of complex reasoning and decision-making without extensive external supervision Here are some of the key elements and hypotheses from his work:
		+
		+	1. <b>Self-Supervised Learning (SSL): </b>LeCun emphasizes the importance of SSL, where machines learn to understand the world by predicting parts of their input from other parts without needing extensive labeled data. This approach is crucial for developing AI that can reason and understand in a human-like manner.
		+
		+	2. <b>Joint Embedding Predictive Architecture (JEPA): </b>The central concept in LeCun's proposal is JEPA, an energy-based model that captures dependencies between inputs by mapping them into a representation space. This architecture allows the model to make predictions based on abstract representations rather than direct data, making it more efficient and scalable for tasks requiring long-term prediction and reasoning.
		+
		+	3. <b>Intrinsic Motivation: </b>LeCun suggests that autonomous AI should be driven by intrinsic objectives rather than external rewards or hard-coded programs. This idea is inspired by how animals and humans are motivated to explore and learn from their environment naturally.
		+
		+	4. <b>Modular Architecture: </b>The proposed framework consists of several modules:
		+	* Configurator Module: This acts as a manager, orchestrating other modules based on the task at hand.
		+	* Perception Module: It estimates the current state and represents it in a stratified manner.
		+	* World Model Module: This is the core of the system, using SSL to create a predictive model of the world.
		+	* Cost Module: It defines a cost function to evaluate the performance of predictions.
		+	* Memory Modules: These include short-term and long-term memory to manage and store information about states and corresponding predictions.
		+
		+	5. <b>Hierarchical Representation and Prediction: </b>By stacking JEPA models, the architecture can handle both short-term and long-term predictions at varying levels of abstraction. For example, a high-level representation might predict broad actions like a cook preparing a dish, while lower levels detail specific movements and actions

	<youtube>EvSe0ktD95k</youtube>		<youtube>EvSe0ktD95k</youtube>

@@ Line 21: / Line 21: @@
 * [[Process Supervision]]
 * [[What is Artificial Intelligence (AI)? | Artificial Intelligence (AI)]] ... [[Generative AI]] ... [[Machine Learning (ML)]] ... [[Deep Learning]] ... [[Neural Network]] ... [[Reinforcement Learning (RL)|Reinforcement]] ... [[Learning Techniques]]
 * [[Conversational AI]] ... [[ChatGPT]] | [[OpenAI]] ... [[Bing/Copilot]] | [[Microsoft]] ... [[Gemini]] | [[Google]] ... [[Claude]] | [[Anthropic]] ... [[Perplexity]] ... [[You]] ... [[phind]] ... [[Ernie]] | [[Baidu]]
 * [http://thenextweb.com/neural/2020/04/05/self-supervised-learning-is-the-future-of-ai-syndication/ Self-supervised learning is the future of AI | Ben Dickson]
@@ Line 46: / Line 47: @@
 <youtube>7I0Qt7GALVk</youtube>
 <youtube>lbKg3OSTsgA</youtube>