Knowledge Distillation Method for Surface Defect Detection.

Abstract

In this paper, we propose a multi-scale attention mechanism-guided knowledge distillation method for surface defect detection. Enables a lighter student model to mimic the complex teacher model through the use of knowledge distillation techniques, the proposed method improves the defect detection accuracy and maintains high real-time performance, simultaneously. Specifically, we first present a multi-scale fusion-based teacher network. Owing to the fusion of two resolution scales features, the teacher network can keep high compatibility with the low-resolution student network during knowledge distillation, so as to better direct the student model. Then, in the process of knowledge distillation, attentional mechanisms were introduced with the aim of enabling the student network to more effectively mimic the foreground attention map and features of the teacher network. Finally, in order to address the imbalance of foreground and background in defect detection, we introduce a class-weighted cross entropy loss. Experiments conducted on three benchmark datasets proved the validity and efficiency of the proposed method in surface defect detection.