一种基于词聚类的文本特征描述方法① - 计算机系统应用.pdfVIP

2011 年 第 20 卷 第 2 期 计 算 机 系 统 应 用 ① 一种基于词聚类的文本特征描述方法 1 2,3 陈 炯 ，张永奎 1( 山西职业技术学院 计算机工程系，太原 030006) 2( 山西大学 计算机与信息技术学院，太原 030006) 3( 山西大学 计算智能与中文信息处理教育部重点实验室，太原 030006) 摘 要：针对文本挖掘中存在的特征空间高维性问题，提出了一种基于词聚类的文本特征描述方法，旨在通过 机器学习的方法挖掘词汇之间的语义关联,动态构造特定领域的概念词典，借助构造的概念来描述文本的特征， 该方法不借助主题词典,先从训练语料中对词的共现情况进行分析,用词聚类(word clustering)生成由种子词(seed words)表示的代表某一主题概念的词类，然后用种子词作为文本的特征项。实验表明，该方法不仅压缩了特征空 间的维数，也克服了HowNet 中概念信息的局限性,提高了文本分类的精确度。 关键词：文本特征描述；词共现；词聚类；种子词 A Description Method of Text Feature Based on Word Clustering 1 2,3 CHEN Jiong , ZHANG Yong-Kui 1(Department of Computer Engineering, Shanxi Polytechnic College, Taiyuan 030006, China) 2(School of Computer Information Technology, Shanxi University, Taiyuan 030006, China) 3(Key Laboratory of Ministry of Education for Computation Intelligence and Chinese Information Processing, Shanxi University, Taiyuan 030006, China) Abstract: Feature space has the high-dimensional problem in text mining. This paper presented a new description method of text feature based on word clustering. The purpose is to mine semantic association between words using machine learning, then to construct the concept dictionary in specific areas dynamically, finally to describe the text feature with the concept constructed. This method analyzes the co-occurrence of words in training corpus firstly, without using theme dictionary, then generates word cluster expressed in seed words which represents a concept of theme by word clustering, finally takes the seed words as text features. The experimental results indicate that this method not only reduces dimensionality of feature space but also overcomes the li