Abstract

In contemporary industrial systems, ensuring the quality of object surfaces has become an essential and inescapable aspect of factory inspections. Cascade regional convolutional neural network (cascade R-CNN), an object detection and instance segmentation algorithm based on deep learning, has been widely applied in numerous industrial applications. Nonetheless, there is still space for improving the detection of defects on metal surfaces. The cascade R-CNN does not currently have good results in metal defect detection, but after improving it by combining some newly proposed modules, cascade R-CNN has a good performance. This article proposes an enhanced metal defect detection method based on cascade R-CNN. Specifically, the improved backbone network is employed to acquire the features of images, which enables more precise localization. Additionally, up and down sampling is combined to extract multiscale defect feature maps, and contrast histogram equalization enhancement is utilized to tackle the issue of unclear contrast in the data. Experimental results demonstrate that the proposed approach achieves a mean average precision (mAP) of 0.754 on the NEU-DET dataset and outperforms the cascade R-CNN model by 9.2%.

References

1.
Hao
,
Z.
,
Li
,
Z.
,
Ren
,
F.
,
Lv
,
S.
, and
Ni
,
H.
,
2022
, “
Strip Steel Surface Defects Classification Based on Generative Adversarial Network and Attention Mechanism
,”
Metals
,
12
(
2
), p.
311
.
2.
Xing
,
L.
,
Li
,
T.
,
Fan
,
H.
, and
Zhu
,
H.
,
2022
, “
Defect Detection and Classification of Strip Steel Based on Improved VIT Model
,”
International Federation for Information Processing
,
Catania, Italy
,
June 13–16
, Springer International Publishing, pp.
327
335
.
3.
Liu
,
Y.
,
Geng
,
J.
,
Su
,
Z.
,
Zhang
,
W.
, and
Li
,
J.
,
2018
, “
Real-Time Classification of Steel Strip Surface Defects Based on Deep CNNs
,”
2018 Chinese Intelligent Systems Conference
,
Wenzhou, China
,
Oct. 4
, Springer Singapore, pp.
257
266
.
4.
Tarafder
,
M.
,
Dey
,
S.
,
Sivaprasad
,
S.
,
Tarafder
,
S.
, and
Nasipuri
,
M.
,
2005
, “
Stretch-Zone Analysis by Image Processing for the Evaluation of Initiation Fracture Toughness of a HSLA Steel
,”
Z. Metallkunde
,
96
(
8
), pp.
924
932
.
5.
Bo
,
T.
,
Jian-yi
,
K.
,
Xing-dong
,
W.
,
Guo-zhang
,
J.
, and
Li
,
C.
,
2010
, “
Steel Strip Surface Defects Detection Based on Mathematical Morphology
,”
J. Iron Steel Res.
,
22
(
10
), pp.
56
59
.
6.
Zhang
,
Y.
,
Gong
,
W.
, and
Zhong
,
W.
,
2013
, “
Fault Automatic Detection Method of Steel Cord Conveyor Belt Based on Gabor Filter Bank
,”
2nd International Conference on Information Technology and Management Innovation (ICITMI 2013)
,
Zhuhai, China
,
July 23–24
.
7.
Medina
,
R.
,
Gayubo
,
F.
,
Gonzalez
,
L. M.
,
Olmedo
,
D.
,
Gomez
,
J.
,
Zalama
,
E.
, and
Peran
,
J. R.
,
2008
, “
Surface Defects Detection on Rolled Steel Strips by Gabor Filters
,”
3rd International Conference on Computer Vision Theory and Applications
,
Funchal, Portugal
,
Jan. 22–25
, pp.
479
485
.
8.
Wang
,
F.
,
Guohua
,
P.
, and
Xie
,
H.
,
2018
, “
Strip Steel Defect Detection Based on Morphological Enhancement and Image Fusion
,”
Laser Infrared.
,
48
(
1
), pp.
124
128
.
9.
Zuiderveld
,
K. J.
,
1994
, “Contrast Limited Adaptive Histogram Equalization,”
Graphics Gems.
, P. S. Heckbert ed.,
Academic Press Professional, Inc.
,
Cambridge, MA
, pp.
474
485
.
10.
Pizer
,
S. M.
,
Amburn
,
E. P.
,
Austin
,
J. D.
,
Cromartie
,
R.
,
Geselowitz
,
A.
,
Greer
,
T.
,
Zimmerman
,
J. B.
, and
Zuiderveld
,
K.
,
1987
, “
Adaptive Histogram Equalization and Its Variations
,”
Comput. Vis. Graph. Image Process.
,
39
(
3
), pp.
355
368
.
11.
Zhang
,
H.
,
Wu
,
C.
,
Zhang
,
Z.
,
Zhu
,
Y.
,
Lin
,
H.
,
Zhang
,
Z.
,
Sun
,
Y.
, et al.
,
2022
, “
ResNeSt: Split-Attention Networks
,”
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
,
New Orleans, LA
,
June 18–24
, pp.
2735
2745
.
12.
Liu
,
S.
,
Qi
,
L.
,
Qin
,
H.
,
Shi
,
J.
, and
Jia
,
J.
,
2018
, “
Path Aggregation Network for Instance Segmentation
,”
31st IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
,
Salt Lake City, UT
,
June 18–23
, pp.
8759
8768
.
13.
Lin
,
T.-Y.
,
Dollár
,
P.
,
Girshick
,
R.
,
He
,
K.
,
Hariharan
,
B.
, and
Belongie
,
S.
,
2017
, “
Feature Pyramid Networks for Object Detection
,”
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
,
Honolulu, HI
,
July 21–26
, pp.
936
944
.
14.
He
,
K.
,
Gkioxari
,
G.
,
Dollár
,
P.
, and
Girshick
,
R.
,
2020
, “
Mask R-CNN
,”
IEEE. Trans. Pattern. Anal. Mach. Intell.
,
42
(
2
), pp.
386
397
.
15.
Yuan
,
S.
,
Abe
,
M.
,
Taguchi
,
A.
, and
Kawamata
,
M.
,
2007
, “
High Accuracy Bicubic Interpolation Using Image Local Features
,”
IEICE Trans. Fundam. Electron. Comput. Sci.
,
E90A
(
8
), pp.
1611
1615
.
16.
SMITH
,
P.
,
1981
, “
Bilinear Interpolation of Digital Images
,”
Ultramicroscopy
,
6
(
2
), pp.
201
204
.
17.
Vu
,
T.
,
Jang
,
H.
,
Pham
,
T. X.
, and
Yoo
,
C. D.
,
2019
, “
Cascade RPN: Delving Into High-Quality Region Proposal Network With Adaptive Convolution
,”
33rd Conference on Neural Information Processing Systems (NeurIPS)
,
Vancouver, Canada
,
Dec. 8–14
.
18.
Oksuz
,
K.
,
Cam
,
B. C.
,
Akbas
,
E.
, and
Kalkan
,
S.
,
2021
, “
Rank & Sort Loss for Object Detection and Instance Segmentation
,”
18th IEEE/CVF International Conference on Computer Vision (ICCV)
,
Virtual
,
Oct. 11–17
, pp.
2989
2998
.
19.
Bhatt
,
P. M.
,
Malhan
,
R. K.
,
Rajendran
,
P.
,
Shah
,
B. C.
,
Thakar
,
S.
,
Yoon
,
Y. J.
, and
Gupta
,
S. K.
,
2021
, “
Image-Based Surface Defect Detection Using Deep Learning: A Review
,”
ASME J. Comput. Inf. Sci. Eng.
,
21
(
4
), p.
040801
.
20.
Liu
,
Y.
,
Rai
,
R.
,
Purwar
,
A.
,
He
,
B.
, and
Mani
,
M.
,
2020
, “
Special Issue: Machine Learning Applications in Manufacturing
,”
ASME J. Comput. Inf. Sci. Eng.
,
20
(
2, SI
), p.
020301
.
21.
Nand
,
G. K.
,
Noopur
, and
Neogi
,
N.
,
2014
, “
Defect Detection of Steel Surface Using Entropy Segmentation
,”
2014 Annual IEEE India Conference (INDICON)
,
Pune, India
,
Dec. 11–13
, pp.
1
6
.
22.
Di
,
S.
,
Dong-bo
,
Z.
, and
Xia
,
L.
,
2014
, “
Based on Gabor and Texture Suppression Scratch Detection for Cell Phone Accessories
,”
Comput. Eng.
,
40
, pp.
1
5
.
23.
Liu
,
M.
,
Liu
,
Y.
,
Hu
,
H.
, and
Nie
,
L.
,
2016
, “
Genetic Algorithm and Mathematical Morphology Based Binarization Method for Strip Steel Defect Image with Non-Uniform Illumination
,”
J. Vis. Commun. Image Represent.
,
37
(
May 5
), pp.
70
77
.
24.
Yun
,
J. P.
,
Kim
,
D.
,
Kim
,
K.
,
Lee
,
S. J.
,
Park
,
C. H.
, and
Kim
,
S. W.
,
2017
, “
Vision-Based Surface Defect Inspection for Thick Steel Plates
,”
Opt. Eng.
,
56
(
5
), p.
053108
.
25.
Redmon
,
J.
,
Divvala
,
S.
,
Girshick
,
R.
, and
Farhadi
,
A.
,
2016
, “
You Only Look Once: Unified, Real-Time Object Detection
,”
2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
,
Seattle, WA
,
June 27–30
, pp.
779
788
.
26.
Liu
,
Y.
,
Xu
,
K.
, and
Xu
,
J.
,
2019
, “
Periodic Surface Defect Detection in Steel Plates Based on Deep Learning
,”
Appl. Sci.
,
9
(
15
), p.
3127
.
27.
Akhil
,
V.
,
Raghav
,
G.
,
Arunachalam
,
N.
, and
Srinivas
,
D. S.
,
2020
, “
Image Data-Based Surface Texture Characterization and Prediction Using Machine Learning Approaches for Additive Manufacturing
,”
ASME J. Comput. Inf. Sci. Eng.
,
20
(
2, SI
), p.
021010
.
28.
Lv
,
X.
,
Duan
,
F.
,
Jiang
,
J.-J.
,
Fu
,
X.
, and
Gan
,
L.
,
2020
, “
Deep Metallic Surface Defect Detection: The New Benchmark and Detection Network
,”
Sensors
,
20
(
6
), p.
1562
.
29.
Liu
,
W.
,
Anguelov
,
D.
,
Erhan
,
D.
,
Szegedy
,
C.
,
Reed
,
S.
,
Fu
,
C.-Y.
, and
Berg
,
A. C.
,
2016
, “SSD: Single Shot Multibox Detector,”
Computer Vision – ECCV 2016
,
B.
Leibe
,
J.
Matas
,
N.
Sebe
, and
M.
Welling
, eds.,
Springer International Publishing
,
Cham, Switzerland
, pp.
858
864
.
30.
Girshick
,
R.
,
Donahue
,
J.
,
Darrell
,
T.
, and
Malik
,
J.
,
2016
, “
Region-Based Convolutional Networks for Accurate Object Detection and Segmentation
,”
IEEE. Trans. Pattern. Anal. Mach. Intell.
,
38
(
1
), pp.
142
158
.
31.
Wong
,
V. W. H.
,
Ferguson
,
M.
,
Law
,
K. H.
,
Lee
,
Y.-T. T.
, and
Witherell
,
P.
,
2022
, “
Segmentation of Additive Manufacturing Defects Using U-Net
,”
ASME J. Comput. Inf. Sci. Eng.
,
22
(
3
), p.
031005
.
32.
Wei
,
R.
,
Song
,
Y.
, and
Zhang
,
Y.
,
2020
, “
Enhanced Faster Region Convolutional Neural Networks for Steel Surface Defect Detection
,”
ISIJ. Int.
,
60
(
3
), pp.
539
545
.
33.
Konovalenko
,
I.
,
Maruschak
,
P.
, and
Brevus
,
V.
,
2022
, “
Steel Surface Defect Detection Using an Ensemble of Deep Residual Neural Networks
,”
ASME J. Comput. Inf. Sci. Eng.
,
22
(
1
), p.
014501
.
34.
Cai
,
Z.
, and
Vasconcelos
,
N.
,
2018
, “
Cascade R-CNN: Delving Into High Quality Object Detection
,”
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
,
Salt Lake City, UT
,
June 18–23
, pp.
6154
6162
.
35.
Girshick
,
R.
,
2015
, “
Fast R-CNN
,”
IEEE International Conference on Computer Vision
,
Santiago, Chile
,
Dec. 7–13
, pp.
1440
1448
.
36.
Ren
,
S.
,
He
,
K.
,
Girshick
,
R.
, and
Sun
,
J.
,
2017
, “
Faster R-CNN: Towards Real-Time Object Detection With Region Proposal Networks
,”
IEEE. Trans. Pattern. Anal. Mach. Intell.
,
39
(
6
), pp.
1137
1149
.
37.
Simonyan
,
K.
, and
Zisserman
,
A.
,
2015
, “
Very Deep Convolutional Networks for Large-Scale Image Recognition
,”
International Conference on Learning Representations
,
San Diego, CA
,
May 7–9
.
38.
He
,
K.
,
Zhang
,
X.
,
Ren
,
S.
, and
Sun
,
J.
,
2016
, “
Deep Residual Learning for Image Recognition
,”
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
,
Las Vegas, NV
,
June 27–30
.
39.
Zhang
,
Y.
,
Xie
,
F.
,
Huang
,
L.
,
Shi
,
J.
,
Yang
,
J.
, and
Li
,
Z.
,
2021
, “
A Lightweight One-Stage Defect Detection Network for Small Object Based on Dual Attention Mechanism and PAFPN
,”
Front. Phys.
,
9
, p.
708097
.
40.
Liu
,
X.
,
Pan
,
H.
, and
Li
,
X.
,
2020
, “Object Detection for Rotated and Densely Arranged Objects in Aerial Images Using Path Aggregated Feature Pyramid Networks,”
MIPPR 2019: Pattern Recognition and Computer Vision, Vol. 11430 of Proceedings of SPIE
,
N.
,
Sang
,
J.
,
Udupa
,
Y.
,
Wang
, and
Z.
,
Liu
, eds.,
Huazhong University of Science and Technology, National Key Lab Science and Technology Multi Spectral Informat Proc; Wuhan Institute of Technology, Automat Assoc Hubei
,
Wuhan
, 11th International Symposium on Multispectral Image Processing and Pattern Recognition (MIPPR) – Pattern Recognition and Computer Vision, Wuhan, China, Nov. 2–3, 2019.
41.
Xie
,
J.
,
Pang
,
Y.
,
Nie
,
J.
,
Cao
,
J.
, and
Han
,
J.
,
2023
, “
Latent Feature Pyramid Network for Object Detection
,”
IEEE Trans. Multimed.
,
25
, pp.
2153
2163
.
42.
Gao
,
S.
, and
Gruev
,
V.
,
2011
, “
Bilinear and Bicubic Interpolation Methods for Division of Focal Plane Polarimeters
,”
Opt. Express
,
19
(
27
), pp.
26161
26173
.
43.
Rajarapollu
,
P.
, and
Mankar
,
V.
,
2017
, “
Bicubic Interpolation Algorithm Implementation for Image Appearance Enhancement
,”
Ijcst
,
8
(
4
), pp.
23
26
. www.ijcst.com.
44.
Jaiswal
,
A.
,
Wu
,
Y.
,
Natarajan
,
P.
, and
Natarajan
,
P.
,
2021
, “
Class-Agnostic Object Detection
,”
IEEE Winter Conference on Applications of Computer Vision (WACV)
,
Virtual
,
Jan. 5–9
, pp.
918
927
.
45.
Song
,
K.
, and
Yan
,
Y.
,
2013
, “
A Noise Robust Method Based on Completed Local Binary Patterns for Hot-Rolled Steel Strip Surface Defects
,”
Appl. Surf. Sci.
,
285
(
Part B
), pp.
858
864
.
46.
Buslaev
,
A.
,
Iglovikov
,
V. I.
,
Khvedchenya
,
E.
,
Parinov
,
A.
,
Druzhinin
,
M.
, and
Kalinin
,
A. A.
,
2020
, “
Albumentations: Fast and Flexible Image Augmentations
,”
Information
,
11
(
2
), p.
125
.
47.
Redmon
,
J.
, and
Farhadi
,
A.
,
2018
, “
Yolov3: An Incremental Improvement
,”
arXiv, abs/1804.02767.
48.
Lin
,
T.-Y.
,
Goyal
,
P.
,
Girshick
,
R.
,
He
,
K.
, and
Dollár
,
P.
,
2020
, “
Focal Loss for Dense Object Detection
,”
IEEE. Trans. Pattern. Anal. Mach. Intell.
,
42
(
2
), pp.
318
327
.
49.
Law
,
H.
, and
Deng
,
J.
,
2018
, “
Cornernet: detecting objects as paired keypoints
,”
Computer Vision. 15th European Conference (ECCV 2018)
,
Munich, Germany
,
Sept. 8–14
, pp.
765
781
.
50.
Duan
,
K.
,
Bai
,
S.
,
Xie
,
L.
,
Qi
,
H.
,
Huang
,
Q.
, and
Tian
,
Q.
,
2019
, “
Centernet: Keypoint Triplets for Object Detection
,”
2019 IEEE/CVF International Conference on Computer Vision (ICCV)
,
Seoul, South Korea
,
Oct. 27–Nov. 2
, pp.
6568
6577
.
51.
Duan
,
H.
,
Huang
,
J.
,
Liu
,
W.
, and
Shu
,
F.
,
2022
, “
Defective Surface Detection Based on Improved Faster R-cnn
,”
2022 IEEE International Conference on Industrial Technology (ICIT)
,
Shanghai, China
,
Aug. 22–25
, pp.
1
6
.
52.
Chen
,
K.
,
Wang
,
J.
,
Pang
,
J.
,
Cao
,
Y.
,
Xiong
,
Y.
,
Li
,
X.
,
Sun
,
S.
, et al
2019
, “
MMDetection: Open MMLAB Detection Toolbox and Benchmark
,”
arXiv preprint arXiv:1906.07155.
You do not currently have access to this content.