Discriminative vs. Generative

YouTube search... ...Google search

Model Contrasts...

• Discriminative
  • learn the (hard or soft) boundary between classes
  • providing classification splits (probabilistic or non-probabilistic manner)
  • allow you to classify points, without providing a model of how the points are actually generated
  • don't have generative properties
  • make few assumptions of the model structure
  • less tied to a particular structure
  • better performance with lots of example data
  • can outperform generative if assumptions are not satisfied (real world is messy and assumptions are rarely perfectly satisfied)
  • not designed to use unlabeled data
  • do not generally function for outlier detection
  • do not offer such clear representations of relations between features and classes in the dataset
  • yields representations of boundaries (more than generative)

• Generative
  • model the distribution of individual classes
  • provides a model of how the data is actually generated
  • learn the underlying structure of the data
  • have discriminative properties
  • make some kind of structure assumptions on your model
  • decision boundary: where one model becomes more likely
  • often outperform discriminative models on smaller datasets because their generative assumptions place some structure on your model that prevent overfitting
  • natural use of unlabeled data
  • generally function for outlier detection
  • typically specified as probabilistic graphical models, which offer rich representations of the independence relations in the dataset
  • more straightforward to detect distribution changes and update a generative model