Difference between revisions of "Decision Jungle"
(Created page with "[http://www.youtube.com/results?search_query=+Linear+Regression+artificial+intelligence YouTube search...] * AI Solver ** ...predict categories * Capabilities *...") |
|||
| Line 1: | Line 1: | ||
| − | [http://www.youtube.com/results?search_query=+ | + | [http://www.youtube.com/results?search_query=Decision+Jungle+artificial+intelligence YouTube search...] |
* [[AI Solver]] | * [[AI Solver]] | ||
** [[...predict categories]] | ** [[...predict categories]] | ||
* [[Capabilities]] | * [[Capabilities]] | ||
| − | |||
| − | + | Randomized decision trees and forests have a rich history in machine learning and have seen considerable success in application, perhaps particularly so for computer vision. However, they face a fundamental limitation: given enough data, the number of nodes in decision trees will grow exponentially with depth. For certain applications, for example on mobile or embedded processors, memory is a limited resource, and so the exponential growth of trees limits their depth, and thus their potential accuracy. Decision jungles, revisiting the idea of ensembles of rooted decision directed acyclic graphs (DAGs), and shows these to be compact and powerful discriminative models for classification. Unlike conventional decision trees that only allow one path to every node, a DAG in a decision jungle allows multiple paths from the root to each leaf. We present and compare two new node merging algorithms that jointly optimize both the features and the structure of the DAGs efficiently. During training, node splitting and node merging are driven by the minimization of exactly the same objective function, here the weighted sum of entropies at the leaves. Results on varied datasets show that, compared to decision forests and several other baselines, decision jungles require dramatically less memory while considerably improving generalization. [http://www.microsoft.com/en-us/research/publication/decision-jungles-compact-and-rich-models-for-classification/ Decision Jungles: Compact and Rich Models for Classification | Microsoft] | |
| + | |||
| + | <youtube>q7BwufUxteE</youtube> | ||
| + | |||
| + | == Two-Class Decision Jungle == | ||
| − | + | * [http://docs.microsoft.com/en-us/azure/machine-learning/studio-module-reference/linear-regression Linear Regression | Microsoft] | |
| − | <youtube> | + | <youtube>ErVC1Tuj4IQ</youtube> |
| − | |||
Revision as of 19:52, 3 June 2018
Randomized decision trees and forests have a rich history in machine learning and have seen considerable success in application, perhaps particularly so for computer vision. However, they face a fundamental limitation: given enough data, the number of nodes in decision trees will grow exponentially with depth. For certain applications, for example on mobile or embedded processors, memory is a limited resource, and so the exponential growth of trees limits their depth, and thus their potential accuracy. Decision jungles, revisiting the idea of ensembles of rooted decision directed acyclic graphs (DAGs), and shows these to be compact and powerful discriminative models for classification. Unlike conventional decision trees that only allow one path to every node, a DAG in a decision jungle allows multiple paths from the root to each leaf. We present and compare two new node merging algorithms that jointly optimize both the features and the structure of the DAGs efficiently. During training, node splitting and node merging are driven by the minimization of exactly the same objective function, here the weighted sum of entropies at the leaves. Results on varied datasets show that, compared to decision forests and several other baselines, decision jungles require dramatically less memory while considerably improving generalization. Decision Jungles: Compact and Rich Models for Classification | Microsoft
Two-Class Decision Jungle
