分布式朴素贝叶斯算法在文本分类中的应用

Application of Distributed Naive Bayes Algorithm in Text Classfication

中文关键词:  MapReduce框架  文本分类  朴素贝叶斯算法

英文关键词:MapReduce framework  text classification  naive Bayes algorithm

中文摘要:

      传统串行贝叶斯算法在对大规模数据进行分类时，性能较低下。为此，在TFIDF（词频-逆向文件频率）特征加权基础上，提出ICF（逆类别因子）类别加权因子，对传统贝叶斯分类模型进行改进。利用MapReduce并行计算框架在处理海量数据方面的优势，设计并实现了一种对 TFIDF 改进的分布式朴素贝叶斯文本分类算法。实验结果表明，与传统分布式朴素贝叶斯算法和TFIDF加权的分布式朴素贝叶斯算法相比，改进后的分类算法在查准率、查全率、F-measure等方面都有了较大提高。

英文摘要:

      The traditional serial Bayes algorithm performs badly when it comes to large-scale data.Based on the TFIDF weighted algorithm,an ICF factor is proposed to improve weighted Bayes classification model.Taking advantage of MapReduce in processing big data efficiently,an improved parallelized TFIDF naive Bayes text classification algorithm is designed and implemented.Experimental results indicate that compared with traditional distributed algorithm and TFIDF parallelized algorithm,the proposed algorithm performs better in precision,recall,F-measure,and so on.