基于BOOSTING框架的视觉语音多模态情感识别检测方法.docVIP

基于BOOSTING框架的视觉语音多模态情感识别检测方法 张芬 成都理工大学信息科学与技术学院 X 关注成功！ 加关注后您将方便地在 我的关注中得到本文献的被引频次变化的通知！ 新浪微博 腾讯微博 人人网 开心网 豆瓣网 网易微博 摘????要： 情感识别技术是智能人机交互的重要基础, 它涉及计算机科学、语言学、心理学等多个研究领域, 是模式识别和图像处理领域的研究热点。鉴于此, 基于Boosting框架提出两种有效的视觉语音多模态融合情感识别方法:第一种方法将耦合HMM (coupled HMM) 作为音频流和视频流的模型层融合技术, 使用改进的期望最大化算法对其进行训练, 着重学习难于识别的 (即含有更多信息的) 样本, 并将Ada Boost框架应用于耦合HMM的训练过程, 从而得到Ada Boost-CHMM总体分类器;第二种方法构建了多层Boosted HMM (MBHMM) 分类器, 将脸部表情、肩部运动和语音三种模态的数据流分别应用于分类器的某一层, 当前层的总体分类器在训练时会聚焦于前一层总体分类器难于识别的样本, 充分利用各模态特征数据间的互补特性。实验结果验证了两种方法的有效性。 关键词： 情感识别; 表情识别; Boosting方法; 情感数据库; 作者简介：张芬 (1975—) , 女, 硕士研究生, 副教授。从事计算机图形图像处理、数据库等应用研究。 收稿日期：2017-05-09 基金：四川省软件工程专业卓越工程师质量工程项目支持 (11100-14Z00327) Boosting framework based multi-modal emotion recognition detection methods fusing vision and speech ZHANG Fen School of Information Science and Technology, Chengdu University of Technology; Abstract： As the important basis of intelligent human-computer interaction, the emotion recognition technology relates to the computer science, linguistics, psychology and other research fields, and is a research hotspot in pattern recognition and image processing fields. Based on the Boosting framework, two effective multi-modal emotion recognition methods fusing vision and speech are proposed. In the first method, the coupled hidden Markov model (HMM) is taken as the model-layer fusion technology of audio and video streams, and the improved expectation maximization algorithm is used to train it, and pay attention to the learning of the samples which are difficult to recognize emphatically; the Ada Boost framework is applied to the training process of HMM coupling to get the Ada Boost-CHMM general classifier. In the second method, the multi-layer Boosted HMM (MBHMM) classifier is constructed, and the data streams with the modals of facial expression, shoulder movement and speech are respectively applied to the classifier of a certain layer. The current layer′s overall classifier while training will focus on the sample