基于主题的关键词提取方法对比研究.docxVIP

基于主题的关键词提取方法对比研究摘 要关键词提供了文档的概要信息，在信息检索、文本聚类和分类系统中受到了越来越多的应用，关键词的提取算法也受到了越来越多的重视。传统的方法主要依靠词汇的统计信息进行关键词提取，本文在回顾关键词提取的算方法的基础上，从文档主题的角度，综述了基于主题的关键词提取的三种算法--潜在语义分析(LSA)、概率潜在语义分析（PLSA）、隐含狄利克雷分布（LDA）。LSA方法将文档从稀疏的高维词汇空间映射到一个低维的向量空间，主要通过奇异值分解SVD的方式来求解。PLSA方法它用概率的方法来表示LSA，在文档和词汇之间引入一个潜在语义层（即主题层）。LDA的基本思想是: 利用概率推导等方式可以将单个文档表示为这些潜在主题的集合，对于主题而言，它又可以看成是若干词汇的概率分布。为了验证这三种方法的性能的优越与否，将理论与实践结合，本文通过实验将三种基于主题的关键词提取算法与TF-IDF方法进行对比，对实验结果进行总结归纳；实验证明，这三种方法无论是从召回率上还是从准确率上都优于TF-IDF方法，能够有效推荐关键词。 关键词：关键词提取；LSA；PLSA；LDAResearch on Algorithms of Topic Based Keyword Extraction Abstract Keywords provide semantic metadata producing an overview of the content of a document. They are widely used in information retrieval, text clustering and classification system. As a result, people pay a lot of attention to keyword extraction algorithm. Traditional methods for keyword extraction simply rank keywords according to the statistical information of words. After reviewing some methods of keyword extraction, this article summarizes three topic based methods of keyword extraction which are Latent Semantic Analysis (LSA), Probability Latent Semantic Analysis (PLSA), Latent Dirichlet Allocation (LDA).LSA maps the document from sparse high dimension space to a low dimensional vector space, mainly through the singular value decomposition (SVD). PLSA brings in a latent semantic layer which is called theme layer between documents and words. It explains LSA in a probabilistic way. The basic idea of LDA is: document can be regarded as the combination of several potential themes. A single document can be described as the collection of the underlying theme in a probabilistic way. To prove whether the topic based keyword extraction algorithms are efficient，this article combines theory with practice. After experiment, it compares these three algorithms with the traditional TF-IDF algorithm. It proves that both the recall and the precision get improved. The three topic based keyword extraction algorithm dose well in k