| 442 | 9 | 750 |
| 下载次数 | 被引频次 | 阅读次数 |
微表情是一种反映人真实情感的自发性面部变化。由于微表情变化微弱且持续时间短暂，传统的神经网络难以提取到类间差异极小的微表情特征。针对上述问题，提出了一种改进的ECANet34-DA网络，在残差网络的主干部分加入DA模块和不降维局部跨通道交互策略的高效通道注意力(Efficient Channel Attention, ECA),能够关注到更细微的表情变化。使用峰值帧附近序列组成中间帧序列作为输入图像，有效解决数据量有限问题。将宏表情数据集Fer2013的先验知识通过迁移学习应用到微表情识别。将ECANet34-DA网络模型在主流数据集CASME II,SMIC和SAMM上进行实验，使用留一人交叉验证方法表明此方法有效提高了识别精度，在CASME II数据集5类微表情识别中取得了85.44%的准确率和81.96%的未加权F1指数(UF1)。
Abstract：Micro-expressions are spontaneous facial changes that reflect a person's true emotions.Due to the slightness and short duration of micro-expression changes, it is difficult for traditional neural networks to extract micro-expression features with minimal differences between classes.In response to the above problems, an improved ECANet34-DA network is proposed.A DA module and an Efficient Channel Attention(ECA) module adopting local cross-channel interaction strategy without dimensionality reduction are added to the backbone of the residual network, which can pay attention to more subtle expression changes.Secondly, an intermediate frame sequence composed of the sequence near the peak frame is used as the input image, which effectively solves the problem of limited data volume.The prior knowledge of the macro-expression dataset Fer2013 is applied to micro-expression recognition through transfer learning.Finally, the ECANet34-DA network model is tested on the mainstream datasets CASME II,SMIC and SAMM,and the leave-one-one cross-validation method is used to show that this method can effectively improve the recognition accuracy.An accuracy of 85.44% and an Unweighted F1 index(UF1) of 81.96% are achieved.
[1] EKMAN P.Emotions Revealed:Recognizing Faces and Feelings to Improve Communication and Emotional Life[M].Singapore City:Times Books,2003.
[2] MATSUMOTO D,HWANG H S.Evidence for Training the Ability to Read Microexpressions of Emotion[J].Motivation and Emotion,2011,35(2):181-191.
[3] RUSSELL T A,CHU E,PHILLIPSM L.A Pilot Study to Investigate the Effectiveness of Emotion Recognition Remediation in Schizophrenia Using the Micro-expression Training Tool[J].British Journal of Clinical Psychology,2006,45(4):579-583.
[4] WEINBERGER S.Airport Security:Intent to Deceive?[J].Nature,2010,465(7297):412-416.
[5] EKMAN P.Lie Catching and Microexpressions[J].The Philosophy of Deception,2009,1(2):118-136.
[6] EKMAN P.Telling Lies:Clues to Deceit in the Marketplace,Politics,and Marriage :Revised Edition[M].New York:WW Norton & Company,2009.
[7] EKMAN P,FRIESENW V.Facial Action Coding System:A Technique for the Measurement of Facial Movement[J].Facial Action Coding System,1978(8):1250-1261.
[8] EKMAN P.Micro Expression Training Tool (METT)[M].San Francisco:University of California,2002.
[9] NIU X S,HAN H,YANG S F,et al.Local Relationship Learning with Person-specific Shape Regularization for Facial Action Unit Detection[C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition.Long Beach:IEEE,2019:11909-11918.
[10] FRANK M,HERBASZ M,SINUK K,et al.I See How You Feel:Training Laypeople and Professionals to Recognize Fleeting Emotions[C]//Annual Meeting of the International Communication Association.New York City:[s.n.],2009:1-35.
[11] OJALA T,PIETIK?INEN M,HARWOOD D.A Comparative Study of Texture Measures with Classification Based on Featured Distributions[J].Pattern Recognition,1996,29(1):51-59.
[12] PFISTER T,LI X B,ZHAO G Y,et al.Recognising Spontaneous Facial Micro-expressions[C]//2011 International Conference on Computer Vision.Barcelona:IEEE,2011:1449-1456.
[13] HUANG X H,ZHAO G Y,HONG X P,et al.Spontaneous Facial Micro-expression Analysis Using Spatiotemporal Completed Local Quantized Patterns[J].Neurocomputing,2016,175:564-578.
[14] LIU Y J,ZHANG J K,YAN W J,et al.A Main Directional Mean Optical Flow Feature for Spontaneous Micro-expression Recognition[J].IEEE Transactions on Affective Computing,2016,7(4):299-310.
[15] PATEL D,ZHAO G,PIETIK?INEN M.Spatiotemporal Integration of Optical Flow Vectors for Micro-expression Detection[C]//International Conference on Advanced Concepts for Intelligent Vision Systems.Catania:Springer,2015:369-380.
[16] HAPPY S L,ROUTRAY A.Fuzzy Histogram of Optical Flow Orientations for Micro-expression Recognition[J].IEEE Transactions on Affective Computing,2019,10(3):394-406.
[17] PATEL D,HONG X P,ZHAO G Y.Selective Deep Features for Micro-expression Recognition[C]//2016 23rd International Conference on Pattern Recognition (ICPR).Cancun:IEEE,2016:2258-2263.
[18] PENG M,WANG C Y,BI T,et al.A Novel Apex-time Network for Cross-dataset Micro-expression Recognition[C]//2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII).Cambridge:IEEE,2019:1-6.
[19] LIONG S T,GAN Y S,YAU W C,et al.Off-apexnet on Micro-expression Recognition System[J/OL].(2018-05-10)[2022-08-10].https://arxiv.org/abs/1805.08699.
[20] LIONG S T,GAN Y S,SEE J,et al.Shallow Triple Stream Three-dimensional CNN (STSTNet) for Micro-expression Recognition[C]//2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019).Lille:IEEE,2019:1-5.
[21] XIA Z Q,HONG X P,GAO X Y,et al.Spatiotemporal Recurrent Convolutional Networks for Recognizing Spontaneous Micro-expressions[J].IEEE Transactions on Multimedia,2020,22(3):626-640.
[22] DEKA B,HUANG Z F,FRANZEN C,et al.Rico:A Mobile App Dataset for Building Data-driven Design Applications[C]//Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology.Quebec City:ACM,2017:845-854.
[23] LIU H Y,MENG W T,LIU Z.Key Frame Extraction of Online Video Based on Optimized Frame Difference[C]//2012 9th International Conference on Fuzzy Systems and Knowledge Discovery.Chongqing:IEEE,2012:1238-1242.
[24] WANG Q L,WU B G,ZHU P F,et al.ECA-Net:Efficient Channel Attention for Deep Convolutional Neural Networks [C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Seattle:IEEE,2020:11531-11539.
[25] HU J,SHEN L,SUN G.Squeeze-and-excitation Networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,42(8):2011-2023.
[26] WOO S,PARK J,LEE J Y,et al.CBAM:Convolutional Block Attention Module[J/OL].(2018-07-17)[2022-07-15].https://arxiv.org/abs/1807.06521.
[27] HE K,ZHANG X,REN S,et al.Deep Residual Learning for Image Recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas:IEEE,2016:770-778.
[28] SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-scale Image Recognition[J/OL].(2014-09-04)[2022-07-15].https://arxiv.org/abs/1409.1556.
[29] LI X B,PFISTER T,HUANG X H,et al.A Spontaneous Micro-expression Database:Inducement,Collection and Baseline[C]//10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).Shanghai:IEEE,2013:1-6.
[30] YAN W J,LI X,WANG S J,et al.CASMEⅡ:An Improved Spontaneous Micro-expression Database and the Baseline Evaluation[J].PloS One,2014,9(1):86041.
[31] DAVISON A K,LANSLEY C,COSTEN N,et al.SAMM:A Spontaneous Micro-facial Movement Dataset[J].IEEE Transactions on Affective Computing,2018,9(1):116-129.
[32] KHOR H Q,SEE J,LIONG S T,et al.Dual-stream Shallow Networks for Facial Micro-expression Recognition[C]// 2019 IEEE International Conference on Image Processing (ICIP).Taipei:IEEE,2019:36-40.
[33] SONG B,LI K,ZONG Y,et al.Recognizing Spontaneous Micro-expression Using a Three-stream Convolutional Neural Network[J].IEEE Access,2019,7:184537-184551.
[34] VAN QUANG N,CHUN J,TOKUYAMA T.CapsuleNet for Micro-expression Recognition[C]//2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019).Lille:IEEE,2019:1-7.
[35] XIA Z,PENG W,KHORH Q,et al.Revealing the Invisible with Model and Data Shrinking for Composite-database Micro-expression Recognition[J].IEEE Transactions on Image Processing,2020,29:8590-8605.
基本信息:
中图分类号:TP391.41
引用信息:
[1]李思诚,周顺勇,朱豪,等.结合多注意力机制和中间帧序列的微表情识别[J].无线电工程,2023,53(03):636-643.
基金信息:
四川省科技厅省院省校科技合作项目(2020YFSY0027,2020YFG0178)~~
2023-01-06
2023-01-06
2023-01-06