Integration of Category-quantity Adaptive Deep Data Augmentation and Transfer Learning for Reef-Building Coral Recognition

Wang Lan; Wei Hao; Che Yachen; Zhang Cuicui

doi:10.12284/hyxb2024-02

Article Contents

Article Navigation > Haiyang Xuebao > 2024 > Accepted Manuscript

Wang Lan，Wei Hao，Che Yachen, et al. Integration of Category-quantity Adaptive Deep Data Augmentation and Transfer Learning for Reef-Building Coral Recognition[J]. Haiyang Xuebao，2024, 46(x)：1–11 doi: 10.12284/hyxb2024-02

Citation:

PDF( 0 KB)

Integration of Category-quantity Adaptive Deep Data Augmentation and Transfer Learning for Reef-Building Coral Recognition

doi: 10.12284/hyxb2024-02

Wang Lan^1
,,
Wei Hao¹,
Che Yachen²,
Zhang Cuicui^{1, 3
,
,}

1.
School of Marine Science and Technology, Tianjin University, Tianjin 300072
2.
National Ocean Technology Center, Key Laboratory of Ocean Observation Technology, MNR, Tianjin 300112
3.
Institute of Computing Technology, Chinese Academy of Sciences, Beijing, 100190

Available Online: 2024-08-15

Abstract

Abstract

Recognition of reef-building corals is important for protecting and monitoring coral reef ecosystems. Deep learning, as an advanced technology in image recognition, has been increasingly applied in coral recognition. However, its performance is still challenged by several issues, such as the imbalance of samples among different coral categories within a dataset and the limitation of data diversity. The former makes the deep learning model more likely to extract features from classes with a large number of samples and, therefore, decreases its ability to recognize small-sample-size corals, which often refer to endangered ones needing to be protected. The latter further reduces the performance of deep learning in recognizing corals with different appearances and are captured in variant environments. To solve these two problems, this study develops a reef-building coral recognition method by integrating a category-quantity adaptive deep data augmentation algorithm and transfer learning. To address the first problem, a category-quantity adaptive deep data augmentation algorithm named DeepSMOTE-F₁ is proposed. This algorithm improves the existing DeepSMOTE by introducing a sample-size determination stagey using an F1-score based evaluation metric. It can adaptively augment the number of samples of each category of corals according to its recognition performance so that the deep learning model can fully learn features from each class of corals. For the second problem, transfer learning is used to further enhance the model's ability to extract features. The experimental results on three widely used public coral recognition datasets, RSMAS, EILAT, and EILAT2 show that the recognition accuracy of the proposed DeepSMOTE-F₁ is improved by 2.88%, 0.39%, and 1.54%, respectively, compared with the traditional DeepSMOTE; and the accuracy of the integrated method is improved by 0.76%, 1.40% and 1.30% compared with the existing deep learning methods for coral recognition.
- Coral recognition,
- Deep learning,
- Imbalanced dataset,
- Data augmentation,
- Transfer learning.

FullText(HTML)

References(28)

References

[1]	黄晖, 陈竹, 黄林韬. 中国珊瑚礁状况报告(2010-2019)[M]. 北京: 海洋出版社, 2021. Huang Hui, Chen Zhu, Huang Lintao. Status of Coral Reefs in China (2010-2019)[M]. Beijing: China Ocean Press, 2021.
[2]	Elliff C I, Silva I R. Coral reefs as the first line of defense: shoreline protection in face of climate change[J]. Marine Environmental Research, 2017, 127: 148−154. doi: 10.1016/j.marenvres.2017.03.007
[3]	黄林韬, 黄晖, 江雷. 中国造礁石珊瑚分类厘定[J]. 生物多样性, 2020, 28(4): 515−523. doi: 10.17520/biods.2019384 Huang Lintao, Huang Hui, Jiang Lei. A revised taxonomy for Chinese hermatypic corals[J]. Biodiversity Science, 2020, 28(4): 515−523. doi: 10.17520/biods.2019384
[4]	夏荣林, 宁志铭, 余克服, 等. 长棘海星暴发对珊瑚礁区沉积物营养盐动力学的影响研究[J]. 海洋学报, 2022, 44(8): 23−30. doi: 10.12284/j.issn.0253-4193.2022.8.hyxb202208003 Xia Ronglin, Ning Zhiming, Yu Kefu, et al. Study on the impacts of crown-of-thorns starfish on nutrient dynamics in the coral reef sediments[J]. Haiyang Xuebao, 2022, 44(8): 23−30. doi: 10.12284/j.issn.0253-4193.2022.8.hyxb202208003
[5]	International Union for Conservation of Nature. The IUCN red list of threatened species, version 2022‐2[R/OL]. [2023-07-06]. https://www.iucnredlist.org.
[6]	Mahmood A, Bennamoun M, An S, et al. Deep learning for coral classification[M]//Samui P, Sekhar Roy S, Balas V E. Handbook of Neural Computation. London: Academic Press, 2017: 383-401.
[7]	Marcos M S A C, Soriano M N, Saloma C A. Classification of coral reef images from underwater video using neural networks[J]. Optics Express, 2005, 13(22): 8766−8771. doi: 10.1364/OPEX.13.008766
[8]	Pizarro O, Rigby P, Johnson-Roberson M, et al. Towards image-based marine habitat classification[C]//Proceedings of the OCEANS 2008. Quebec City, Canada: IEEE, 2008: 1−7.
[9]	Elawady M. Sparse coral classification using deep convolutional neural networks[J]. arXiv: 1511.09067, 2015. (查阅网上资料, 不确定本条文献类型和格式信息, 请确认 Elawady M. Sparse coral classification using deep convolutional neural networks[J]. arXiv: 1511.09067, 2015. (查阅网上资料, 不确定本条文献类型和格式信息, 请确认)
[10]	Mahmood A, Bennamoun M, An S, et al. Resfeats: residual network based features for image classification[C]//Proceedings of 2017 IEEE International Conference on Image Processing. Beijing, China: IEEE, 2017: 1597−1601.
[11]	Modasshir M, Li A Q, Rekleitis I. MDNet: multi-patch dense network for coral classification[C]//Proceedings of the OCEANS 2018 MTS/IEEE Charleston. Charleston, USA: IEEE, 2018: 1−6.
[12]	Gómez-Ríos A, Tabik S, Luengo J, et al. Towards highly accurate coral texture images classification using deep convolutional neural networks and data augmentation[J]. Expert Systems with Applications, 2019, 118: 315−328. doi: 10.1016/j.eswa.2018.10.010
[13]	Lumini A, Nanni L, Maguolo G. Deep learning for plankton and coral classification[J]. Applied Computing and Informatics, 2023, 19(3/4): 265−283. doi: 10.1016/j.aci.2019.11.004
[14]	Khan S H, Hayat M, Bennamoun M, et al. Cost-sensitive learning of deep feature representations from imbalanced data[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(8): 3573−3587. doi: 10.1109/TNNLS.2017.2732482
[15]	Lee H, Park M, Kim J. Plankton classification on imbalanced large scale database via convolutional neural networks with transfer learning[C]//Proceedings of 2016 IEEE International Conference on Image Processing. Phoenix, USA: IEEE, 2016: 3713−3717.
[16]	Dablain D, Krawczyk B, Chawla N V. DeepSMOTE: fusing deep learning and SMOTE for imbalanced data[J]. IEEE Transactions on Neural Networks and Learning Systems, 2023, 34(9): 6390−6404. doi: 10.1109/TNNLS.2021.3136503
[17]	Chinchor N. MUC-4 evaluation metrics[C]//Proceedings of the 4th Conference on Message Understanding. McLean, USA: Association for Computational Linguistics, 1992: 22−29.
[18]	Dablain D, Krawczyk B, Chawla N V. DeepSMOTE: fusing deep learning and SMOTE for imbalanced data[J]. IEEE Transactions on Neural Networks and Learning Systems, 2023, 34(9): 6390−6404. (查阅网上资料, 本条文献与第16条文献重复, 请确认 Dablain D, Krawczyk B, Chawla N V. DeepSMOTE: fusing deep learning and SMOTE for imbalanced data[J]. IEEE Transactions on Neural Networks and Learning Systems, 2023, 34(9): 6390−6404. (查阅网上资料, 本条文献与第16条文献重复, 请确认)
[19]	Pan S J, Yang Qiang. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345−1359. doi: 10.1109/TKDE.2009.191
[20]	Shihavuddin A S M. Coral reef dataset[EB/OL]. [2024-01-22]. https://data.mendeley.com/datasets/86y667257h/2.
[21]	WoRMS. An authoritative classification and catalogue of marine names[EB/OL]. [2024-04-05]. https://www.marinespecies.org.
[22]	Nanglu K, Lerosey-Aubril R, Weaver J C, et al. A mid-Cambrian tunicate and the deep origin of the ascidiacean body plan[J]. Nature Communications, 2023, 14(1): 3832. doi: 10.1038/s41467-023-39012-4
[23]	Rodríguez L, López C, Casado-Amezua P, et al. Genetic relationships of the hydrocoral Millepora alcicornis and its symbionts within and between locations across the Atlantic[J]. Coral Reefs, 2019, 38(2): 255−268. doi: 10.1007/s00338-019-01772-1
[24]	Shihavuddin A S M, Gracias N, Garcia R, et al. Image-based coral reef classification and thematic mapping[J]. Remote Sensing, 2013, 5(4): 1809−1841. doi: 10.3390/rs5041809
[25]	Buda M, Maki A, Mazurowski M A. A systematic study of the class imbalance problem in convolutional neural networks[J]. Neural Networks, 2018, 106: 249−259. doi: 10.1016/j.neunet.2018.07.011
[26]	Nair V, Hinton G E. Rectified linear units improve restricted boltzmann machines[C]//Proceedings of the 27th International Conference on International Conference on Machine Learning (ICML-10). Haifa, Israel: Omnipress, 2010: 807−814.
[27]	Chawla N V, Bowyer K W, Hall L O, et al. SMOTE: synthetic minority over-sampling technique[J]. Journal of Artificial Intelligence Research, 2002, 16(1): 321−357.
[28]	Deng Jia, Dong Wei, Socher R, et al. ImageNet: a large-scale hierarchical image database[C]//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, USA: IEEE, 2009: 248−255.