DeepLearningforNLP自然语言处理中的深度学习技术研究.PDFVIP

Deep Learning for NLP 自然语言处理中的深度学习技术研究 郭江 2014.03.03 Contents • Why NLP needs Deep Learning? • What is Deep Learning? • How do Deep Learning apply to NLP? – Current research • Open problems and Challenges • Future research direction WhyWhy WhatWhat HowHow •• Statistical NLPStatistical NLP •• FrameworkFramework •• ChallengesChallenges •• DataData SparsitySparsity •• Domain AdaptationDomain Adaptation •• Deep Semantic AnalysisDeep Semantic Analysis Statistical NLP – A coarse framework LearningLearning FeatureFeature RepresentationRepresentation Labeled DataLabeled Data Feature Engineering Statistical NLP - Challenges • Data Sparsity • Domain Adaptation • Deep Semantic Analysis Challenges 1: Data Sparsity Reason 1: Lack of Labeled Data – Hard to obtain large-scale labeled data, especially for complex tasks, e.g. Parsing, SRL. • Crowdsourcing? Challenges 1: Data Sparsity Reason 2: Language is Zipf-distributed – Very many words will appear only a few times, or will be left out of the training set. – For rare words, labeled examples are far from enough. Challenges 1: Data Sparsity • For the “Lack of labeled data” issue – Unlabeled data is unlimited - “Big Data” • Unsupervised (semi-supervised) learning • For the “Zipf distribution” issue – Distributional representation • Multinomial representatio