Original Articles Open access

Residual attention based multi-label learning for apple leaf disease identification

Authors

  • Changjian Zhou – articles in this journal College of Life Science, and High-Performance Computing and Artificial Intelligence Laboratory, Northeast Agricultural University, Harbin, China.
  • Zhenyuan Zhao – articles in this journal High-Performance Computing and Artificial Intelligence Laboratory, and School of Electrical and Information, Northeast Agricultural University, Harbin, China.
  • Wenzhuo Chen – articles in this journal High-Performance Computing and Artificial Intelligence Laboratory, and School of Electrical and Information, Northeast Agricultural University, Harbin, China.
  • Yuquan Feng – articles in this journal High-Performance Computing and Artificial Intelligence Laboratory, and School of Electrical and Information, Northeast Agricultural University, Harbin, China.
  • Jia Song – articles in this journal College of Life Science, Northeast Agricultural University, Harbin, China.
  • Wensheng Xiang – articles in this journal Corresponding author Email College of Life Science, and High-Performance Computing and Artificial Intelligence Laboratory, Northeast Agricultural University, Harbin; State Key Laboratory for Biology of Plant Diseases and Insect Pests, Institute of Plant Protection, Chinese Academy of Agricultural Sciences, Beijing, China.
Published
20 August 2024
Section
Original Articles
DOI
10.4081/jae.2024.1595

Abstract

Recent studies suggest that plant disease identification via machine learning approach is vital for preventing the spread of diseases. Identifying multiple diseases simultaneous on a single leaf is one of the most irritating issues in agricultural production. However, the existing approaches are difficult to meet the requirements of production practice in accuracy or interpretability. Here, we present residual attention based multi-label learning framework (RAMDI), a method for predicting apple leaf diseases in natural environment. Built upon an attention based multi-label learning framework, the channel and spatial attention mechanisms are investigated and embedded in residual network for multi-label disease prediction, which takes advantage of channel-wise and spatial-wise attention weights. Experimental results indicate that the RAMDI achieves 0.976 accuracy, 0.986 F-score, and 0.979 mAPs, outperforms the existing state-of-the-art apple leaf disease identification models. RAMDI not only predicts multi-disease on a single leaf simultaneously, but also reveals the interpretability among positive predictions that contribute most to identify the key features that are significant for the leaf diseases. This method achieves the following two achievements. Firstly, it provides a solution for detecting multiple diseases on a single leaf. Secondly, this approach gains an interpretable understanding for apple leaf disease identification.

Keywords

References

  1. Agarwal, M., Kaliyar, R., Singal, G., Gupta, S., 2019. FCNN-LDA: A Faster Convolution Neural Network model for Leaf Disease identification on Apple's leaf dataset. 12th International Conference on Information & Communication Technology and System (ICTS), 246-251, https://doi.org/10.1109/ICTS.2019.8850964. DOI: https://doi.org/10.1109/ICTS.2019.8850964
  2. Alice W., 2021. World apple, grape, and pear production forecast to rise in 2021/22. https://www.mintecglobal.com/top-stories/world-apple-grape-and-pear-production-forecast-to-rise-in-2021/22.
  3. Ayyub, S., Manjramkar, A., 2019. Fruit Disease Classification and Identification using Image Processing. 2019 3rd International Conference on Computing Methodologies and Communication (ICCMC), Erode, India, 2019, 754-758, https://doi.org/10.1109/ICCMC.2019.8819789. DOI: https://doi.org/10.1109/ICCMC.2019.8819789
  4. Chakraborty, S., Paul, S., Rahat-uz-Zaman, M., 2021. Prediction of Apple Leaf Diseases Using Multiclass Support Vector Machine. 2021 2nd International Conference on Robotics, Electrical and Signal Processing Techniques (ICREST), 147-151, https://10.1109/ICREST51555.2021.9331132. DOI: https://doi.org/10.1109/ICREST51555.2021.9331132
  5. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N., 2021. An image is worth 16x16 words: Transformers for image recognition at scale. The Ninth International Conference on Learning Representations (LCLR).
  6. Gao, Z., Xie J., Wang Q., Li, P., 2019. Global Second-Order Pooling Convolutional Networks. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 3019-3028. https://doi.org/10.1109/CVPR.2019.00314. DOI: https://doi.org/10.1109/CVPR.2019.00314
  7. Hu, J., Shen, L., Albanie, S., Sun, G., Wu, E., 2020. Squeeze-and-Excitation Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42, 8, 2011-2023. https://doi.org/10.1109/TPAMI.2019.2913372. DOI: https://doi.org/10.1109/TPAMI.2019.2913372
  8. Huang, R., Wu, Z., 2021.Multi-label feature selection via manifold regularization and dependence maximization, Pattern Recognition,120. https://doi.org/10.1016/j.patcog.2021.108149. DOI: https://doi.org/10.1016/j.patcog.2021.108149
  9. James, G., Sujatha,S., 2021.Categorising Apple Fruit Diseases Employing Hybrid Neural Clustering Classifier. Materials Today: Proceedings. https://doi.org/10.1016/j.matpr.2020.12.139. DOI: https://doi.org/10.1016/j.matpr.2020.12.139
  10. Lee, C., Fang, W., Yeh, C., Wang, Y., 2018. Multi-label Zero-Shot Learning with Structured Knowledge Graphs, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1576-1585, https://doi.org/10.1109/CVPR.2018.00170. DOI: https://doi.org/10.1109/CVPR.2018.00170
  11. Lee, H., Kim, H., Nam, H., 2019. Srm: A style-based recalibration module for convolutional neural networks. Preprint at: https://arxiv.org/abs/ 1903.10829. DOI: https://doi.org/10.1109/ICCV.2019.00194
  12. Li, C., Peng, J., Zhang, S., 2016. Apple leaf disease identification method based on feature fusion and local discriminant mapping. Guangdong Agric. Sci. 43 (10), 134–139. https://doi.org/10.16768/j.issn.1004-874X.2016.10.024.
  13. Li, L., Zhang, S., Wang, B., 2022. Apple Leaf Disease Identification with a Small and Imbalanced Dataset Based on Lightweight Convolutional Networks. SENSORS, 22(1), https://doi.org/10.3390/s22010173. DOI: https://doi.org/10.3390/s22010173
  14. Liu, Z., Lin, Y., Cao, Y., Hu, H., Zhang, Z., Lin, S., Guo, B., Swin Transformer: Hierarchical Vision Transformer using Shifted Windows, 2021. [Online], https://arxiv.org/abs/2103.14030. DOI: https://doi.org/10.1109/ICCV48922.2021.00986
  15. Pandiyan, S., Ashwin, M., Manikandan, R., Karthick, R, Anantah, R., 2020. Heterogeneous Internet of Things organization predictive analysis platform for apple leaf diseases recognition. Computer Communications, 154(12), 99-110, https://doi.org/10.1016/j.comcom.2020.02.054. DOI: https://doi.org/10.1016/j.comcom.2020.02.054
  16. Qin, Z., Zhang, P., Wu, F., Li, X., 2021. Fcanet: Frequency channel attention networks. Preprint at: https://arxiv.org/abs/ 2012.11879. DOI: https://doi.org/10.1109/ICCV48922.2021.00082
  17. Ramachandran, P., Parmar, N., Vaswani, A., Bello, I., Levskaya, A., Shlens, J., 2019. Stand-alone self-attention in vision models. Advances in Neural Information Processing Systems, 32. https://proceedings.neurips.cc/paper/2019/hash/3416a75f4cea9109507cacd8e2f2aefc-Abstract.html.
  18. Shi Y., Huang, W., Zhang, S., 2017. Apple disease recognition based on two-dimensionality subspace learning. Comput. Eng. Appl. 53 (22), 180–184. ISSN 1002-8331, https://doi.org/10.3778/j.issn.1002-8331.1605-0073.
  19. Sun, H., Xu, H., Liu, B., He, D., He, J., Zhang, H., Geng, N., 2021. MEAN-SSD: A novel real-time detector for apple leaf diseases using improved light-weight convolutional neural networks, Computers and Electronics in Agriculture. 189, https://doi.org/10.1016/j.compag.2021.106379. DOI: https://doi.org/10.1016/j.compag.2021.106379
  20. Thapa, R., Zhang, K., Snavely, N., Belongie, S., Belongie, S., Khan, A., 2020. The Plant Pathology Challenge 2020 data set to classify foliar disease of apples, Applications in Plant Sciences, 8,9. https://doi.org/10.1002/aps3.11390. DOI: https://doi.org/10.1002/aps3.11390
  21. Wang, P., Niu, T., Mao, YR., Zhang, Z., Liu, B., He, DJ., 2021. Identification of Apple Leaf Diseases by Improved Deep Convolutional Neural Networks with an Attention Mechanism. Frontiers in Plant Science, 12, https://doi.org/10.3389/fpls.2021.723294. DOI: https://doi.org/10.3389/fpls.2021.723294
  22. Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W., Hu, Q., 2020. ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 11531-11539. https://doi.org/10.1109/CVPR42600.2020.01155. DOI: https://doi.org/10.1109/CVPR42600.2020.01155
  23. Wang, X., Girshick, R., Gupta A., He, K., 2018. Non-local Neural Networks. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7794-7803, https://doi.org/10.1109/CVPR.2018.00813. DOI: https://doi.org/10.1109/CVPR.2018.00813
  24. Woo, S., Park, J., Lee, Y., Kweon, I., CBAM: Convolutional Block Attention Module, 2018, [online], http://arxiv.org/abs/1807.06521. DOI: https://doi.org/10.1007/978-3-030-01234-2_1
  25. Yang, B., Bender, G., Le, Q. V., & Ngiam, J.,2019. Condconv: Conditionally parameterized convolutions for efficient inference. Advances in Neural Information Processing Systems, 32.
  26. Yang, Z., Zhu, L., Wu, Y., Yang, Y., 2020. Gated channel transformation for visual recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 11 794–11 803. https://doi.org/10.1109/CVPR42600.2020.01181. DOI: https://doi.org/10.1109/CVPR42600.2020.01181
  27. Yu, H., Son, C., 2020. Leaf Spot Attention Network for Apple Leaf Disease Identification. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 229-237, https://doi.org/10.1109/CVPRW50498.2020.00034. DOI: https://doi.org/10.1109/CVPRW50498.2020.00034
  28. Zhong, Y., Zhao, M., 2020. Research on deep learning in apple leaf disease recognition, Computers and Electronics in Agriculture,168, https://doi.org/10.1016/j.compag.2019.105146. DOI: https://doi.org/10.1016/j.compag.2019.105146
  29. Zhou, C., Zhou, S., Xing, J., Song, J., 2021. Tomato Leaf Disease Identification by Restructured Deep Residual Dense Network. IEEE Access, vol. 9, pp. 28822-28831, https://doi.org/10.1109/ACCESS.2021.3058947. DOI: https://doi.org/10.1109/ACCESS.2021.3058947

Metrics

Citations and online attention from Dimensions, Altmetric and PlumX.

PlumX Metrics

License

Copyright (c) 2024 the Author(s)

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

PAGEPress has chosen to apply the Creative Commons Attribution NonCommercial 4.0 International License (CC BY-NC 4.0) to all manuscripts to be published.