数据挖掘的原则和边界研究.pptVIP

Data Mining: Principles and Algorithms Data Mining:  Principles and Research Frontiers — Chapter 8.8 — — Mining Text and Web Data — ?Jiawei Han Department of Computer Science University of Illinois at Urbana-Champaign /~hanj Mining Text Data Introduction: Text mining, natural language processing and information extraction Text categorization Text classification methods Text cluster analysis Summary Research problems in text mining Mining Text and Web Data Text mining, natural language processing and information extraction: An Introduction Text categorization methods Mining Web linkage structures Summary Bag-of-Tokens Approaches Natural Language Processing General NLP—Too Difficult! Word-level ambiguity “design” can be a noun or a verb (Ambiguous POS) “root” has multiple meanings (Ambiguous sense) Syntactic ambiguity “natural language processing” (Modification) “A man saw a boy with a telescope.” (PP Attachment) Anaphora resolution “John persuaded Bill to buy a TV for himself.” (himself = John or Bill?) Presupposition “He has quit smoking.” implies that he smoked before. Shallow Linguistics WordNet Part-of-Speech Tagging Word Sense Disambiguation Parsing Obstacles Summary: Shallow NLP References for Introduction Mining Text and Web Data Text mining, natural language processing and information extraction: An Introduction Text categorization methods Mining Web linkage structures Summary Text Categorization Pre-given categories and labeled document examples (Categories may form hierarchy) Classify new documents A standard classification (supervised learning ) problem Applications News article classification Automatic email filtering Webpage classification Word sense disambiguation … … Categorization Methods Manual: Typically rule-based Does not scale up (labor-intensive, rule inconsistency) May be appropriate for special data on a particular domain Automatic: Typically exploiting machine learning techniques Vector space model based Prototype-based (Rocchio