基于防疫常态化的驾驶员疲劳状态检测方法

黄玲; 洪佩鑫; 吴泽荣; 刘建荣; 黄子虚; 崔躜

doi:10.3963/j.jssn.1674-4861.2021.04.004

基于防疫常态化的驾驶员疲劳状态检测方法

doi: 10.3963/j.jssn.1674-4861.2021.04.004

华南理工大学土木与交通学院广州 510640

基金项目:

国家自然科学基金项目 51775565

广州市重点区域研发计划项目 202007050004

广东省教育厅项目 2020KQNCX205

详细信息

作者简介:
黄玲(1979—), 博士, 副教授.研究方向: 交通仿真、驾驶行为分析和智能交通.E-mail: hling@scut.edu.cn

通讯作者:
刘建荣(1984—), 博士, 讲师.研究方向: 出行行为选择研究.E-mail: ctjrliu@scut.edu.cn

中图分类号: U491.6
计量
- 文章访问数: 562
- HTML全文浏览量: 416
- PDF下载量: 29
- 被引次数: 0
出版历程
- 收稿日期: 2021-06-02

A Detection Method for Drivers' Fatigue States Based on Normalization of Epidemic Prevention

School of Civil Engineering and Transportation, South China University of Technology, Guangzhou 510640, China

摘要

摘要: 疲劳驾驶检测是交通安全领域的研究分支, 而新冠疫情形势下口罩的佩戴又提出了新的挑战。为此通过基于ResNet-10的SSD模型检测驾驶员人脸, 并使用MobileNet-V2轻量级模型判断是否佩戴口罩, 测试集验证该分类器可以达到98.50%的判断精度。在未佩戴口罩的情况下采用传统图像HOG特征结合SVM分类器检测驾驶员人脸。在后续处理中利用级联回归器定位特征点和提取时间窗口内的疲劳指标, 采用二次判定对疲劳状态采取文字和声音预警, 而在清醒状态下会调整各项判断阈值。对算法在预采集的视频样本和NTHU-DDD测试集下进行测试, 验证了该框架能以18.42帧/s的总体速度实现92.65%和86.09%的检测精度。实验结果表明, 该框架应对佩戴眼镜、脸部姿态变化和光照条件差异具有强鲁棒性, 而且能够兼顾疲劳检测的口罩干扰和实时性。
- 智能交通 /
- 疲劳检测 /
- 自适应判断 /
- 口罩分类
Abstract: The detection of fatigue driving is a research branch of traffic safety, and wearing masks in the COVID-19 situation poses a new challenge. Therefore, the driver's face is detected by the single-shot multi-box detector(SSD)model based on ResNet-10, and the MobileNet-V2 model is used to classify masks. The test set verifies that the classifier can reach an accuracy of 98.50%. The histogram of the oriented gradient(HOG)feature combined with the support vector machine(SVM)classifier is used to detect the driver's face without wearing a mask. In the subsequent processing, the cascade regress is used to locate the feature points and extract the fatigue indices in the time window. The second judgment is used to perform the text and sound warnings for the fatigue state, and the judgment thresholds are adjusted in the awaken state. The algorithm experimented on pre-collected videos and NTHU-DDD can achieve the accuracy of 92.65 and 86.09% at the overall speed of 18.42 fps, respectively. The proposed framework shows strong robustness against the variation of wearing glasses, facial posture, and illumination, considering the interference of mask and real-time performance.
- intelligent transportation /
- fatigue detection /
- adaptive judgment /
- classification of masks

HTML全文

图 1 自适应疲劳检测框架

Figure 1. Framework of adaptive fatigue detection

下载: 全尺寸图片幻灯片

图 2 HOG特征提取示例

Figure 2. Examples of HOG feature extraction

下载: 全尺寸图片幻灯片

图 3 口罩分类网络结构

Figure 3. Structure of the mask-classifying network

下载: 全尺寸图片幻灯片

图 4 数据集部分图片

Figure 4. Partial image of the dataset

下载: 全尺寸图片幻灯片

图 5 不同层数的回归器预测的特征点

Figure 5. Feature points predicted by the regressor at different levels

下载: 全尺寸图片幻灯片

图 6 眼部区域特征点

Figure 6. Feature points in the eye area

下载: 全尺寸图片幻灯片

图 7 眨眼数据包抓取示例

Figure 7. Examples of fetching the data packet of blink

下载: 全尺寸图片幻灯片

图 8 设置基准水平值应对数值变化

Figure 8. Set baseline-level values responding to numerical changes

下载: 全尺寸图片幻灯片

图 9 1次打哈欠的张开率变化曲线

Figure 9. Variable curves of the opening rates of a yawn

下载: 全尺寸图片幻灯片

图 10 疲劳分级模型判定流程

Figure 10. Determining process of the hierarchy model of fatigue

下载: 全尺寸图片幻灯片

图 11 预训练口罩模型分类效果

Figure 11. Classification effect of the pre-training mask model

下载: 全尺寸图片幻灯片

图 12 外部条件变化下的定位效果

Figure 12. Positioning effects of different external conditions

下载: 全尺寸图片幻灯片

图 13 检测界面示例

Figure 13. The detecting interface

下载: 全尺寸图片幻灯片

表 1 测试集效果评估

Table 1. Evaluation of test sets

佩戴口罩	Precision	Recall	F1-score
是	0.98	0.99	0.99
否	0.99	0.98	0.99

下载: 导出CSV

表 2 不同算法下的人脸检测器效果

Table 2. Effects of the face detector under different algorithms

佩戴口罩	Viola-Jones	HOG+SVM	SSD+esNet-10
否	97.68	93.27	99.98
是	70.81	35.45	99.47

下载: 导出CSV

表 3 图像处理的效果对比

Table 3. Effects of different image processing

处理类型	总时间/s
串行	20.11
并行	11.05

下载: 导出CSV

表 4 不同样本在连续窗口内的指标值

Table 4. Index values of different samples in contiguous windows

窗口次序	T_b/ms	l_b/%	T_c/ms	T_r/ms	是否打哈欠
清醒-1	216.7	9.1	86.4	130.3	否
清醒-2	208.9	6.7	76.7	132.2	否
疲劳-1	212.4	8.6	80.9	131.4	是
疲劳-2	257.8	33.3	95.6	162.2	否

下载: 导出CSV

表 5 不同样本集的检测精度

Table 5. Detection accuracy of different sample sets %

样本集	清醒	疲劳
佩戴口罩样本集	88.50	93.62
NTHU-DDD测试集	81.56	90.61

下载: 导出CSV

参考文献(17)

[1]	SIKANDER G, ANWAR S. Driver fatigue detection systems: a review[J]. IEEE Transactions on Intelligent Transportation Systems, 2019, 20(6): 2339-2352. doi: 10.1109/TITS.2018.2868499
[2]	柴萌. 长途客车驾驶员疲劳状态辨识与预警[D]. 长春: 吉林大学, 2019. CAI Meng. Identification and early-warning of fatigue state of intercity coach drivers[D]. Changchun: Jilin University, 2019. (in Chinese).
[3]	国家发展和改革委员会. 关于2020年国民经济和社会发展计划执行情况与2021年国民经济和社会发展计划草案的报告[N]. 人民日报, 2021-03-14(002). National Development and Reform Commission. Report on the implementation of the national economic and social development plan in 2020 and the draft national economic and social development plan in 2021[N]. People's Daily, 2021-03-14(002). (in Chinese).
[4]	李科勇. 考虑驾驶风格的驾驶人疲劳驾驶辨识方法研究[D]. 长春: 吉林大学, 2017. LI Keyong. Research on drowsiness driving detection considering driving style[D]. Changchun: Jilin University, 2017. (in Chinese).
[5]	裴玉龙, 金英群, 陈贺飞. 基于脑电信号分析的不同年龄驾驶人疲劳特性[J]. 中国公路学报, 2018, 31(4): 59-65+77 doi: 10.3969/j.issn.1001-7372.2018.04.008 PEI Yulong, JIN Yingqun, CHEN Hefei. Fatigue characteristics in drivers of different ages based on analysis of EEG[J]. China Journal of Highway and Transport, 2018, 31(4): 59-65+77. doi: 10.3969/j.issn.1001-7372.2018.04.008
[6]	DENG W, WU R. Real-time driver-drowsiness detection system using facial features[J]. IEEE Access, 2019(7): 118727-118738. http://ieeexplore.ieee.org/document/8808931/
[7]	SELVAKUMAR K, JOVITHA J, KUMAR R et al. Real-time vision based driver drowsiness detection using partial least squares analysis[J]. Journal of Signal Processing Systems for Signal Image and Video Technology, 2016, 85(2): 263-274. doi: 10.1007/s11265-015-1075-4
[8]	MONA O, SHERVI S, SHABNAM A, et al. Yawning detection using embedded smart cameras[J]. IEEE Transactions on Instrumentation and Measurement, 2016, 65(3): 570-582. doi: 10.1109/TIM.2015.2507378
[9]	田璐萍, 嵇启春. 基于眼部信息融合的疲劳驾驶检测的研究[J]. 国外电子测量技术, 2019, 38(10): 26-29. https://www.cnki.com.cn/Article/CJFDTOTAL-GWCL201910006.htm TIAN Luping, JI Qichun. Study on fatigue driving test based on eye information fusion[J]. Foreign Electronic Measurement Technology, 2019, 38(10): 26-29. (in Chinese). https://www.cnki.com.cn/Article/CJFDTOTAL-GWCL201910006.htm
[10]	李文学, 谢凯. 基于深度学习的疲劳驾驶检测方法研究[J]. 电子世界, 2019(17): 51-52. https://www.cnki.com.cn/Article/CJFDTOTAL-ELEW201917029.htm LI Wenxue, XIE Kai. Research on fatigue driving detection method based on deep learning[J]. Electronics World, 2019(17): 51-52. (in Chinese). https://www.cnki.com.cn/Article/CJFDTOTAL-ELEW201917029.htm
[11]	曾心远, 张正华, 韩雪等. 基于机器学习的疲劳检测及预警系统设计[J]. 物联网技术, 2019, 9(7): 27-29. https://www.cnki.com.cn/Article/CJFDTOTAL-WLWJ201907010.htm ZENG Xinyuan, ZHANG Zhenghua, HANG Xue, et al. Design of fatigue detection and early warning system based on machine learning[J]. Internet of Things Technologies, 2019, 9(7): 27-29. (in Chinese). https://www.cnki.com.cn/Article/CJFDTOTAL-WLWJ201907010.htm
[12]	BERGASA L M, NUEVO J, SOTELO M A, et al. Real-time system for monitoring driver vigilance[J]. IEEE Transactions on Intelligent Transportation Systems, 2006, 7(1): 63-77. doi: 10.1109/TITS.2006.869598
[13]	ANDREA NVERONIKA S. Multimodal features for detection of driver stress and fatigue: Review[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 22(6): 3214-3233. doi: 10.1109/TITS.2020.2977762
[14]	DALAL N, TRIGGS B. Histograms of oriented gradients for human detection[C]. IEEE Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA: IEEE, 2005.
[15]	KAZEMI, VAHID, SULLIVAN, et al. One millisecond face alignment with an ensemble of regression trees[C]. IEEE Conference on Computer Vision and Pattern Recognition, Columbus, Ohio: IEEE, 2014.
[16]	CAFFIER P, ULLSPERGER P, ERDMANN U. Experimental evaluation of eye-blink parameters as a drowsiness measure[J]. European Journal of Applied Physiology, 2003, 89(3/4): 319-325. http://www.onacademic.com/detail/journal_1000034466173110_b0fa.html
[17]	HUANG Rui, WANG Yan, LI Zijian, et al. RF-DCM: Multi-granularity deep convolutional model based on feature recalibration and fusion for driver fatigue detection[J/OL]. (2020-07)[2021-08-09]. https://ieeexplore.ieee.org/document/9185000.