Original Articles Open access

Leveraging deep semantic segmentation for assisted weed detection

Authors

Published
18 February 2025
Section
Original Articles
DOI
10.4081/jae.2025.1741

Abstract

In agriculture, it is crucial to identify and control weeds as these plant species pose a significant threat to the growth and development of crops by competing for vital resources such as nutrients, water, and light. A promising solution to this problem is adopting smart weed control systems (SWCS) that significantly reduce the use of harmful chemical products. Furthermore, SWCS leads to reduced production costs and a more sustainable and eco-friendly approach to farming. However, implementing SWCS in natural fields can be challenging, mainly due to difficulties in accurately localizing plants. To address this issue, a visual identification system can be employed to label plants from images using a process known as semantic segmentation. In this work, we have implemented, validated, and compared three deep learning approaches, including Mask Region-based Convolutional Neural Network (Mask R-CNN), Mask R-CNN enhanced with an Atrous Spatial Pyramid Pooling module (Mask R-CNN-ASPP), and a proposed model named Residual U-Net architecture, for the semantic pixel segmentation of high densities of both crops (Zea mays) and weeds (including narrow-leaf weeds and broad-leaf weeds). Data augmentation and transfer learning have also been implemented. The performance of the models was evaluated with the well-known metrics Precision, Recall, Dice similarity coefficient (DSC), and mean Intersection-Over-Union (mIoU). As a result of the analysis, the DSC and mIoU of Mask R-CNN-ASPP based models were up to 10.63% and 10.54% superior to that of the Mask R-CNN based models. Nonetheless, the proposed Residual U-Net architecture outperformed Mask R-CNN-ASPP based networks in all the metrics, reaching a DSC of 92.98% and mIoU of 87.12%. Thus, we have concluded that the proposed Residual U-Net-like architecture is the best alternative for the semantic segmentation task in images with high plant density. Our research addresses the challenge of weed identification and control in agriculture, helping farmers produce crops more efficiently while minimizing environmental impact.

Keywords

References

  1. Arai, K., Barakbah, A.R. 2007. Hierarchical k-means: an algorithm for centroids initialization for k-means. Rep. Fac. Sci. Eng. 36:22-31.
  2. Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L. 2017. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE T. Pattern Anal. 40:834-848. DOI: https://doi.org/10.1109/TPAMI.2017.2699184
  3. Dentika, P., Ozier-Lafontaine, H., Penet, L. 2021. Weeds as pathogens hosts and disease risk for crops in the wake of a reduced use of herbicides: Evidence from yam (Dioscorea alata) fields and Colletotrichum pathogens in the tropics. J. Fungi 7:283. DOI: https://doi.org/10.3390/jof7040283
  4. Dutta, A., Zisserman, A. 2019. The VIA annotation software for images, audio and video. MM ’19, Proc. 27th ACM Int. Conf. on Multimedia, New York, pp. 2276-2279. DOI: https://doi.org/10.1145/3343031.3350535
  5. Dyrmann, M., Mortensen, A.K., Midtiby, H.S., Jørgensen, R.N. 2016. Pixel-wise classification of weeds and crops in images by using a fully convolutional neural network. In: CIGR-AgEng conference, Aarhus. Available from: https://conferences.au.dk/uploads/tx_powermail/cigr2016paper_semanticsegmentation.pdf
  6. Fawakherji, M., Youssef, A., Bloisi, D.D., Pretto, A., Nardi, D. 2020. Crop and weed classification using pixel-wise segmentation on ground and aerial images. Int. J. Robot. Comput. 2:39-57. DOI: https://doi.org/10.35708/RC1869-126258
  7. Garibaldi-Márquez, F., Flores, G., Mercado-Ravell, D.A., Ramírez-Pedraza, A., Valentín-Coronado, L.M. 2022. Weed classification from natural corn field-multi-plant images based on shallow and deep learning. Sensors (Basel) 22:3021. DOI: https://doi.org/10.3390/s22083021
  8. He, K., Gkioxari, G., Dollár, P., Girshick, R. 2017. Mask R-CNN. Proc. 2017 IEEE Int. Conf. on Computer Vision (ICCV), Venice; pp. 2980-2988. DOI: https://doi.org/10.1109/ICCV.2017.322
  9. He, K., Zhang, X., Ren, S., Sun, J. 2016. Deep residual learning for image recognition. Proc. 2016 IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Las Vegas; pp. 770-778. DOI: https://doi.org/10.1109/CVPR.2016.90
  10. Jadhav, S.B., Udupi, V.R., Patil, S. 2021. Identification of plant diseases using convolutional neural networks. Int. J. Inf. Tecnol. 13:2461-2470. DOI: https://doi.org/10.1007/s41870-020-00437-5
  11. Kamath, R., Balachandra, M., Vardhan, A., Maheshwari, U. 2022. Classification of paddy crop and weeds using semantic segmentation. Cogent Engin.9:2018791. DOI: https://doi.org/10.1080/23311916.2021.2018791
  12. Khan, A., Talha, I., Umraiz, M., Ibna Mannan, Z., Kim, H. 2020. Ced-net: Crops and weeds segmentation for smart farming using a small cascaded encoder-decoder architecture. Electronics (Basel) 9:1602. DOI: https://doi.org/10.3390/electronics9101602
  13. Kolhar, S., Jayant, J. 2021. Convolutional neural network based encoder-decoder architectures for semantic segmentation of plants. Ecol Inform, 64:101373. DOI: https://doi.org/10.1016/j.ecoinf.2021.101373
  14. Krizhevsky, A., Sutskever, I., Hinton, G. 2012. ImageNet classification with deep convolutional neural networks. In: F. Pereira, C.J.C. Burges, L. Bottou and K. Weinberger (eds.), Adv. in Neural Inform. Process. Syst. 25. Curran Associates, Inc. Lake Tahoe.
  15. Long, J., Shelhamer, E., Darrell, T. 2015. Fully convolutional networks for semantic segmentation. Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Boston; pp. 3431-3440. DOI: https://doi.org/10.1109/CVPR.2015.7298965
  16. Ma, X., Deng, X., Qi, L., Jiang, Y., Li, H., Wang, Y., Xing, X. 2019. Fully convolutional network for rice seedling and weed image segmentation at the seedling stage in paddy fields. PLoS One 14:e0215676. DOI: https://doi.org/10.1371/journal.pone.0215676
  17. Monteiro, A,. Santos, S. 2022. Sustainable approach to weed management: The role of precision weed management. Agronomy (Basel) 12:118. DOI: https://doi.org/10.3390/agronomy12010118
  18. Montes de Oca, A., Arreola, L., Flores, A., Sánchez, J., Flores, G. 2018. Low-cost multispectral imaging system for crop monitoring. Proc. Int. Conf. on Unmanned Aircraft Systems (ICUAS), Dallas; pp. 443-451. DOI: https://doi.org/10.1109/ICUAS.2018.8453426
  19. Montes de Oca, A., Flores, G. 2021a. The AgriQ: A low-cost unmanned aerial system for precision agriculture. Expert Syst. Appl. 182:115–163. DOI: https://doi.org/10.1016/j.eswa.2021.115163
  20. Montes de Oca, A., Flores, G. 2021b. A UAS equipped with a thermal imaging system with temperature calibration for crop water stress index computation. Proc. Int. Conf. on Unmanned Aircraft Systems (ICUAS), Athens; pp. 714-720. DOI: https://doi.org/10.1109/ICUAS51884.2021.9476863
  21. Nedeljković, D., Knežević, S., Božić, D., Vrbnićanin, S. 2021. Critical time for weed removal in corn as influenced by planting pattern and pre-herbicides. Agriculture (Basel) 11:587. DOI: https://doi.org/10.3390/agriculture11070587
  22. Nikolić, N., Rizzo, D., Marraccini, E., Ayerdi Gotor, A., Mattivi, P., Saulet, P., Persichetti, A., Masin, R. 2021. Site- and time-specific early weed control is able to reduce herbicide use in maize- a case study. Ital. J. Agron. 16:1780. DOI: https://doi.org/10.4081/ija.2021.1780
  23. Peng, H., Li, Z., Zhou, Z., Shao, Y. 2022. Weed detection in paddy field using an improved RetinaNet network. Comput. Electron. Agr. 199:107179. DOI: https://doi.org/10.1016/j.compag.2022.107179
  24. Picon, A., San-Emeterio, M., Bereciartua-Perez, A., Klukas, C., Eggers, T., ad Navarra- Mestre, R. 2022. Deep learning-based segmentation of multiple species of weeds and corn crop using synthetic and real image datasets. Comput. Electron. Agr. 194:106719. DOI: https://doi.org/10.1016/j.compag.2022.106719
  25. Quan, L., Wu, B., Mao, S., Yang, C., Li, H. 2021. An instance segmentation-based method to obtain the leaf age and plant centre of weeds in complex field environments. Sensors (basel) 21:3389. DOI: https://doi.org/10.3390/s21103389
  26. Ren, X. Malik, J. 2003. Learning a classification model for segmentation. Proc. 9th IEEE Int. Conf. on Computer Vision, Nice; pp. 10-17. DOI: https://doi.org/10.1109/ICCV.2003.1238308
  27. Ronneberger, O., Fischer, P., Brox, T. 2015. U-net: Convolutional networks for biomedical image segmentation. In N. Navab, J. Hornegger, W. Wells, and A. Frangi (eds.), Medical image computing and computer-assisted intervention Vol. 9351. Lecture Notes in Computer Science. Cham, Springer. pp 234-241. DOI: https://doi.org/10.1007/978-3-319-24574-4_28
  28. Shi, J., Malik, J. 2000. Normalized cuts and image segmentation. IEEE T. Pattern Anal. 22:888-905. DOI: https://doi.org/10.1109/34.868688
  29. Simonyan, K., Zisserman, A. 2015. Very deep convolutional networks for large-scale image recognition. Proc. 3rd Int. Conf. on Learning Representations, San Diego.
  30. Taha, M.F., Abdalla, A., ElMasry, G., Gouda, M., Zhou, L., Zhao, N., et al. 2022. Using deep convolutional neural network for image-based diagnosis of nutrient deficiencies in plants grown in aquaponics. Chemosensors (Basel) 10:45. DOI: https://doi.org/10.3390/chemosensors10020045
  31. Tang, J.L., Chen, X.Q., Miao, R.H., Wang, D. 2016. Weed detection using image precision under different illumination for site-specific areas spraying. Comput. Electron. Agr. 122:103-111. DOI: https://doi.org/10.1016/j.compag.2015.12.016
  32. Zenkl, R., Timofte, R., Kirchgessner, N., Roth, L., Hund, A., Van Gool, L., et al. 2022. Outdoor plant segmentation with deep learning for high-throughput field phenotyping on a diverse wheat dataset. Front. Plant Sci. 12:774068. DOI: https://doi.org/10.3389/fpls.2021.774068
  33. Zhang, H., Peng, Q. 2022. Pso and k-means-based semantic segmentation toward agricultural products. Future Gener. Comp. Sy. 126:82–87. DOI: https://doi.org/10.1016/j.future.2021.06.059

Metrics

Citations and online attention from Dimensions, Altmetric and PlumX.

PlumX Metrics

License

Copyright (c) 2025 the Author(s)

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

PAGEPress has chosen to apply the Creative Commons Attribution NonCommercial 4.0 International License (CC BY-NC 4.0) to all manuscripts to be published.