解释学习 - 智能科学.ppt

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * If then rules No program counter – no sequential code Deep Belief Net (DBN) Deep Neural Net (DNN) DBN: Undirected at top two layers which is an RBM; directed Bayes net (top-down) at lower layers (good for synthesis and recognition) DNN: Multi-layer perceptron (bottom up) + unsupervised pre-training w. RBM weights (good for recognition only) * n Layers Visible Layer Labels n Layers MANY Hidden Layers * 大数据挖掘 史忠植 Deep Belief Net (DBN) Deep Neural Net (DNN) * 大数据挖掘 史忠植 * First train a stack of three models each of which has one hidden layer. Each model in the stack treats the hidden variables of the previous model as data. Then compose them into a single Deep Belief Network. Then add outputs and train the DNN with backprop. Hinton, Deng, Yu, Mohamed, Dahl… etc. IEEE Sig. Proc. Mag. (Nov 2012) 语音识别 * 1989 2011 1999 * 大数据挖掘 史忠植 To estimate the word error rate (WER), the correct and the recognized sentence must be first aligned. Then the number of substitutions (S), deletions (D), and insertions (I) can be estimated. The WER is defined as ?????? = 100% × （?? + ?? +??）/ ?? 语音识别 * 1989 2011 * 大数据挖掘 史忠植 语音识别 * 大数据挖掘 史忠植 * DNN-HMM(replacing GMM only; longer MFCC/filter-back windows w. no transformation) * Model tied triphone states directly Many layers of nonlinear feature transformation + SoftMax 大数据挖掘 史忠植 * CD-DNN-HMM: Architecture * * 大数据挖掘 史忠植 图像识别 * 大数据挖掘 史忠植 * 声音与视频重构 * 大数据挖掘 史忠植 * 多任务学习 * 大数据挖掘 史忠植 * 深度学习网站 /reading-list/ (Bengio’s group) /deep-learning-research-groups-and-labs/ /wiki/index.php/UFLDL_Recommended_Readings /~hinton/ /tutorial/ /wiki/index.php/UFLDL_Tutorial (Andrew Ng’s group) Google+ Deep Learning community * * 大数据挖掘 史忠植 /deep-learning-research-groups-and-labs/ University of Toronto -?Machine Learning Gr