生成模型 Vs 判别模型

取自 自然语言处理百科

总是听到这两个术语，但是又一直不清楚它们最本质的区别。今天花了一小点时间来彻底的弄清楚了。得到的结论如下：

Discriminative Model是判别模型，又可以称为条件模型，或条件概率模型。

Generative Model是生成模型，又叫产生式模型。

二者的本质区别是

discriminative model 估计的是条件概率分布(conditional distribution)p(class|context)

generative model 估计的是联合概率分布（joint probability distribution）p()

常见的Generative Model主要有：

• Gaussians, Naive Bayes, Mixtures of multinomials
• Mixtures of Gaussians, Mixtures of experts, HMMs
• Sigmoidal belief networks, Bayesian networks
• Markov random fields

常见的Discriminative Model主要有：

• logistic regression
• SVMs
• traditional neural networks
• Nearest neighbor

Successes of Generative Methods：

NLP

• Traditional rule-based or Boolean logic systems
• Dialog and Lexis-Nexis are giving way to statistical approaches (Markov models and stochastic context grammars)

Medical Diagnosis

• QMR knowledge base, initially a heuristic expert systems for reasoning about diseases and symptoms

been augmented with decision theoretic formulation

• Genomics and Bioinformatics Sequences represented as generative HMMs

主要应用Discriminative Model：

• Image and document classification
• Biosequence analysis
• Time series prediction

Discriminative Model缺点：

• Lack elegance of generative
• Priors, structure, uncertainty
• Alternative notions of penalty functions, regularization, kernel functions
• Feel like black-boxes
• Relationships between variables are not explicit and visualizable

转自：http://hi.baidu.com/cat_ng/blog/item/5e59c3cea730270593457e1d.html