Research progress of food packaging defect detection based on machine vision

Authors

GE Minghui, Information Engineering, China Jiliang University, Hangzhou, Zhejiang 310018, China;Zhejiang Academy of Agricultural Sciences, Institute of Food Research, Hangzhou, Zhejiang 310022, China
ZHANG Jun, Zhejiang Academy of Agricultural Sciences, Institute of Food Research, Hangzhou, Zhejiang 310022, ChinaFollow
LU Huijuan, Information Engineering, China Jiliang University, Hangzhou, Zhejiang 310018, China

Corresponding Author(s)

张俊（1977—），男，浙江省农业科学院副研究员，硕士生导师，博士。E-mail：hunterzju@163.com

Abstract

Food packaging can develop defects during the production process due to various factors. The types of packaging defects are numerous with complex background. Machine vision detection, which uses visual imaging and computer information processing to complete tasks such as identification, detection, and measurement of packaging, has faster execution speed and higher accuracy compared to traditional manual inspection. This can significantly improve the degree of production automation. This article analyzes the common defects in food packaging and their causes, introduces traditional machine vision detection algorithms, and explores the research application of deep learning algorithms in food packaging defect detection. It also analyzes the prospects and challenges of applying detection algorithms in food packaging defect detection.

Publication Date

10-30-2023

First Page

95

Last Page

102,116

DOI

10.13652/j.spjx.1003.5788.2022.80943

Recommended Citation

Minghui, GE; Jun, ZHANG; and Huijuan, LU (2023) "Research progress of food packaging defect detection based on machine vision," Food and Machinery: Vol. 39: Iss. 9, Article 15.
DOI: 10.13652/j.spjx.1003.5788.2022.80943
Available at: https://www.ifoodmm.cn/journal/vol39/iss9/15

References

[1] 华经产业研究院. 2021—2026年中国食品包装市场竞争格局及投资战略规划报告[R/OL]. （2021-08-26） [2023-05-19]. https://www.huaon.com/channel/package/742891.html. Huajing Industrial Research Institute. Report on competitive landscape and investment strategic planning of China's food packaging market from 2021 to 2026[R/OL]. （2021-08-26） [2023-05-19]. https://www.huaon.com/channel/package/742891.html.
[2] YU Q, JIANG T, ZHOU A Y, et al. Computer-aided diagnosis of malignant or benign thyroid nodes based on ultrasound images[J]. European Archives of Oto-Rhino-Laryngology, 2017, 274: 2 891-2 897.
[3] 王洪杰, 于霞, 张恩东. 人工智能辅助超声对甲状腺结节检出的研究进展[J]. 中国中西医结合影像学杂志, 2020, 18（4）: 424-426. WANG H J, YU X, ZHANG E D. Research progress of ultrasound thyroid nodule detection assisted with artificial intelligence[J]. Chinese Imaging Journal of Integrated Traditional and Western Medicine, 2020, 18（4）: 424-426.
[4] JORGE B. Automatic inspection systems vs human visual inspection\. （2020-09-24） [2023-05-21]. https://www.eines.com/automatic-inspection-systems-human-visual-inspection.
[5] COGNEX. Deep learning vs machine vision and human inspection\. [2023-05-21]. https://www.cognex.com/en-in/what-is/deep-learning/deep-learning-vs-machine-vision-and-human-inspection.
[6] 刘鹏, 刘凤义. 基于机器视觉的五金行业自动化组装生产线系统研究[J]. 工业技术创新, 2022, 9（5）: 11-18. LIU P, LIU F Y. Study of automatic assembly line system for hardware industry based on machine vision[J]. Industrial Technology Innovation, 2022, 9（5）: 11-18.
[7] 王耀南, 刘学兵, 张辉, 等. 机器视觉技术在包装行业研究进展与应用综述[J]. 包装学报, 2022, 14（2）: 1-14. WANG Y N, LIU X B, ZHANG H, et al. Research progress and application review of machine vision technology in packaging industry[J]. Packaging Journal, 2022, 14（2）: 1-14.
[8] 张红岩, 王永志, 刘庆红. 图像识别技术在食品包装缺陷检测中的应用[J]. 食品与机械, 2020, 36（8）: 225-228. ZHANG H Y, WANG Y Z, LIU Q H. Application of image recognition technology in food packaging defect monitoring[J]. Food & Machinery, 2020, 36（8）: 225-228.
[9] 陈亮, 张浩舟, 燕浩. 基于深度学习算法的尿素泵体用铝型材表面瑕疵检测[J]. 流体机械, 2020, 48（8）: 71-77. CHEN L, ZHANG H Z, YAN H. Surface flaw detection of aluminum profile for urea pump body based on deep learning algorithm[J]. Fluid Machinery, 2020, 48（8）: 71-77.
[10] 王爱艾. 基于机器视觉技术检测印刷品质量的应用[J]. 今日印刷, 2019（10）: 74-76. WANG A A. Application of machine vision technology to detect print quality[J]. Intelligent Printing, 2019（10）: 74-76.
[11] 李萌, 孙铁波. 基于机器视觉的食品包装缺陷检测研究[J]. 食品研究与开发, 2016, 37（24）: 125-127. LI M, SUN T B. Research of food packaging defects detection based on machine vision[J]. Food Research and Development, 2016, 37（24）: 125-127.
[12] 张楠, 张丹, 赵萌, 等. 基于仿人机器人的名片识别及邮件互发[J]. 计算机与网络, 2017, 43（11）: 60-62. ZHANG N, ZHANG D, ZHAO M, et al. Business card recognition and e-mail interaction based on humanoid robot[J]. Computer & Network, 2017, 43（11）: 60-62.
[13] 李丹, 白国君, 金媛媛, 等. 基于机器视觉的包装袋缺陷检测算法研究与应用[J]. 激光与光电子学进展, 2019, 56（9）: 188-194. LI D, BAI G J, JIN Y Y, et al. Machine-vision based defect detection algorithm for packaging bags[J]. Laser & Optoelectronics Progress, 2019, 56（9）: 188-194.
[14] 郑叶欣, 王耀南, 周显恩, 等. 基于POBT的空瓶规则纹路区域缺陷检测[J]. 电子测量与仪器学报, 2017, 31（4）: 549-558. ZHENG Y X, WANG Y N, ZHOU X E, et al. Empty bottle texture area defect detection based on POBT[J]. Journal of Electronic Measurement and Instrument, 2017, 31（4）: 549-558.
[15] 王宣银, 梁冬泰. 基于多元图像分析的包装罐内壁缺陷检测[J]. 农业机械学报, 2009, 40（6）: 222-226. WANG X Y, LIANG D T. Packaging cans inner surface inspection system based on multivariate image analysis[J]. Transactions of the Chinese Society for Agricultural Machinery, 2009, 40（6）: 222-226.
[16] 贾真真, 张涛, 曹兴强, 等. 基于机器视觉的食品内包装缺陷检测装置设计与实现[J]. 食品与机械, 2018, 34（7）: 111-114. JIA Z Z, ZHANG T, CAO X Q, et al. Design and realization of the food inner packaging detection device based on machine vision[J]. Food & Machinery, 2018, 34（7）: 111-114.
[17] CANNY J. A computational approach to edge detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986, 8（6）: 679-698.
[18] 张飞, 庄源昌, 郑湃. 基于机器视觉的纸盒包装系统的设计[J]. 信息系统工程, 2020（7）: 24-25. ZHANG F, ZHUANG Y C, ZHENG P. Design of cardboard packaging system based on machine vision[J]. China CIO News, 2020（7）: 24-25.
[19] JIN L H, LI D H. An adaptive spatial distance-weighted vector median filter[J]. J Image Graph, 2007, 12（6）: 970-976.
[20] BO Z, ZHENG L X, PAN X L, et al. New approach of fabric defects detection based on saliency region feature[J]. Journal of Computer Applications, 2013, 32（6）: 1 574-1 577.
[21] JIANG Y Q, SHUANG M, GAO H, et al. Research on defect detection technology of tablets in aluminum plastic package[J]. Open Automation & Control Systems Journal, 2014, 6（1）: 940-951.
[22] 高海华. 基于多特征融合的警告标志的识别[D]. 沈阳: 东北大学, 2011: 15-16. GAO H H. Warning traffic sings recognition based on multi-feature fusion[D]. Shenyang: Northeastern University, 2011: 15-16.
[23] 高奇. 基于机器视觉的网络隔离与视频传输系统的设计与实现[D]. 贵阳: 贵州大学, 2018: 33-34. GAO Q. Design and implementation of network isolation and video transmission system based on machine vision[D]. Guiyang: Guizhou University, 2018: 33-34.
[24] 郝恒恒. 基于海思平台的人体异常行为识别研究及实现[D]. 西安: 西安科技大学, 2018: 3-4. HAO H H. Research and implementation of human abnormal behavior recognition based on Hisilicon platform[D]. Xi'an: Xi'an University of Science and Technology, 2018: 3-4.
[25] XIE H W, LU F, OUYANG G, et al. A rapid inspection method for encapsulating quality of PET bottles based on machine vision[C]// 2017 3rd IEEE International Conference on Computer and Communications （ICCC）. Chengdu: IEEE, 2017: 2 025-2 028.
[26] 肖飞蛟. 基于机器视觉的啤酒包装生产线检测技术与应用研究[D]. 广州: 华南理工大学, 2010: 1-60. XIAO F J. Applied studies and inspection technology for product line of BeerPakage based on machine vision technology[D]. Guangzhou: South China University of Technology, 2010: 1-60.
[27] SHEN Y H, MO R, LEI W, et al. Bottle cap scratches detection with computer vision techniques[C]// 2013 Ninth International Conference on Natural Computation （ICNC）. Shenyang: IEEE, 2013: 1 314-1 318.
[28] FENG B, GUO S X, ZHANG F L, et al. Binary-channel can covers defects detection system based on machine vision[C]// Anti-counterfeiting, Security, and Identification. Taipei: IEEE, 2012: 1-4.
[29] 黄森林, 王耀南, 彭玉, 等. 基于迟滞阈值分割的瓶口缺陷检测方法[J]. 电子测量与仪器学报, 2017, 31（8）: 1 289-1 296. HUANG S L, WANG Y N, PENG Y, et al. Bottle mouth defect detection method based on hysteresis thresholding segmentation[J]. Journal of Electronic Measurement and Instrumentation, 2017, 31（8）: 1 289-1 296.
[30] 范涛, 朱青, 王耀南, 等. 空瓶检测机器人瓶底缺陷检测方法研究[J]. 电子测量与仪器学报, 2017, 31（9）: 1 394-1 401. FAN T, ZHU Q, WANG Y N, et al. Research on detection method of bottle bottom defects based on empty bottle detection robot system[J]. Journal of Electronic Measurement and Instrumentation, 2017, 31（9）: 1 394-1 401.
[31] 徐峥匀. 基于机器学习与深度学习的桥梁数据监测研究[D]. 贵阳: 贵州大学, 2022: 1-63. XU Z Y. Research on bridge data monitoring based on machine learning and deep learning[D]. Guiyang: Guizhou University, 2022: 1-63.
[32] 刘仁懿. 基于深度学习—机器学习融合的骨肿瘤分类模型构建与应用研究[D]. 广州: 南方医科大学, 2022: 1-51. LIU R Y. A deep learning-machine learning fusion approach for the classification of benign, malignant, and intermediate bone tumors[D]. Guangzhou: Southern Medical University, 2022: 1-51.
[33] 曾秀云. 基于机器学习的产品包装缺陷检测方法研究[D]. 芜湖: 安徽工程大学, 2022: 1-61. ZENG X Y. Research on product packaging defect detection method based on machine learning[D]. Wuhu: Anhui Polytechnic University, 2022: 1-61.
[34] 周光祥, 李鹏. 基于机器学习的烟草包装封口处视觉缺陷图像检测方法[J]. 科技通报, 2022, 38（3）: 47-51. ZHOU G X, LI P. Image detection method for visual defects at the sealing of tobacco packaging based on machine learning[J]. Bulletin of Science and Technology, 2022, 38（3）: 47-51.
[35] 孙娜, 管一弘, 崔云月, 等. 基于支持向量机的条烟包装外观缺陷检测[J]. 软件, 2020, 41（1）: 205-210. SUN N, GUAN Y H, CUI Y Y, et al. Appearance defect detection of cigarette packaging based on support vector machine[J]. Software, 2020, 41（1）: 205-210.
[36] 刘加贝. 开放式长尾遥感场景分类研究[D]. 哈尔滨: 哈尔滨工程大学, 2021: 9-10. LIU J B. Research on open set and Long-tailed remote sensing scene classification[D]. Harbin: Harbin Engineering University, 2021: 9-10.
[37] 裴利沈, 刘少博, 赵雪专. 人体行为识别研究综述[J]. 计算机科学与探索, 2022, 16（2）: 305-322. PEI L S, LIU S B, ZHAO X Z. Review of human behavior recognition research[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16（2）: 305-322.
[38] 郭克友, 廉丽冰, 李娜. 基于BP神经网络的啤酒瓶口检测方法[J]. 食品科学技术学报, 2014, 32（4）: 69-74. GUO K Y, LIAN L B, LI N. Method of beer bottle mouth inspection based on BP neural network[J]. Journal of Food Science and Technology, 2014, 32（4）: 69-74.
[39] 陈卫国, 卢锦川, 廖晋平. TRIZ物场模型和深度学习在蛋黄酥包装图案上的分类研究[J]. 装备制造技术, 2020（5）: 87-90, 97. CHEN W G, LU J C, LIAO J P. TRIZ field model and deep learning in classification of egg yolk crisp packaging patterns[J]. Equipment Manufacturing Technology, 2020（5）: 87-90, 97.
[40] 陈雪纯, 方宇伦, 杜世昌, 等. 基于深度学习的包装缺陷快速检测方法[J]. 机械设计与研究, 2021, 37（6）: 165-169, 178. CHEN X C, FANG Y L, DU S C, et al. Rapid packaging defect detection method based on deep learning[J]. Machine Design & Research, 2021, 37（6）: 165-169, 178.
[41] SA J M, LI Z H, YANG Q J, et al. Packaging defect detection system based on machine vision and deep learning[C]// 2020 5th International Conference on Computer and Communication Systems （ICCCS）. Shanghai: IEEE, 2020: 404-408.
[42] 李建明, 杨挺, 王惠栋. 基于深度学习的工业自动化包装缺陷检测方法[J]. 包装工程, 2020, 41（7）: 175-184. LI J M, YANG T, WANG H D. An industrial automation packaging defect detection method based on deep learning[J]. Packaging Engineering, 2020, 41（7）: 175-184.
[43] 彭建忠. 基于深度学习的饮品包装喷码字符缺陷检测研究[D]. 长沙: 湖南大学, 2021: 1-61. PENG J Z. Research on defect detection of code character on beverage package based on deep learning[D]. Changsha: Hunan University, 2021: 1-61.
[44] 朱为. 应用于饮料包装的复杂背景喷码质量检测方法[D]. 长沙: 湖南大学, 2020: 1-57. ZHU W. Quality inspection algorithms for inkjet codes on complex backgrounds applied to beverage package[D]. Changsha: Hunan University, 2020: 1-57.
[45] 段禄成, 谭保华, 余星雨. 基于改进YOLOv3的酒瓶盖瑕疵检测算法[J]. 电子测量技术, 2022, 45（15）: 130-137. DUAN L C, TAN B H, YU X Y. Defect detection for wine bottle caps based on improved YOLOv3[J]. Electronic Measurement Technology, 2022, 45（15）: 130-137.
[46] 吴昊然, 陈晓星, 高傲. 基于深度卷积神经网络的食品包装缺陷检测算法研究[J]. 智能计算机与应用, 2023, 13（3）: 10-15. WU H R, CHEN X X, GAO A. Research on food packaging defect detection algorithm based on deep convolutional neural network[J]. Intelligent Computer and Applications, 2023, 13（3）: 10-15.
[47] ADEM K, KZKURT C. Defect detection of seals in multilayer aseptic packages using deep learning[J]. Turkish Journal of Electrical Engineering and Computer Sciences, 2019, 27（6）: 4 220-4 230.
[48] VU T T H, PHAM D L, CHANG T W. A YOLO-based real-time packaging defect detection system[J]. Procedia Computer Science, 2023, 217: 886-894.
[49] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]// Proceedings of the IEEE conference on computer vision and pattern recognition. Boston: IEEE, 2015: 3 431-3 440.
[50] RONNEBERGER O, FISCHER P, BROX T. U-net: Convolutional networks for biomedical image segmentation[C]// International Conference on Medical Image Computing and Computer-Assisted Intervention （MICCAI）. Munich: Springer, 2015: 234-241.
[51] 田萱, 王子亚, 王建新. 基于语义分割的食品标签文本检测[J]. 农业机械学报, 2020, 51（8）: 336-343. TIAN X, WANG Z Y, WANG J X. Text detection of food labels based on semantic segmentation[J]. Transactions of the Chinese Society of Agricultural Machinery, 2020, 51（8）: 336-343.
[52] 暴泰焚, 焦慧敏, 张皓, 等. 基于语义分割的纸质包装产品表面缺陷检测[J]. 制造业自动化, 2023, 45（3）: 216-220. BAO T F, JIAO H M, ZHANG H, et al. Surface defect detection of paper packaging products based on semantic segmentation[J]. Manufacturing Automation, 2023, 45（3）: 216-220.