Original Articles Open access

Pitaya detection using an improved lightweight Faster R-CNN based on MobileNetV3 in densely planted pitaya orchards

Authors

Published
17 July 2025
Section
Original Articles
DOI
10.4081/jae.2025.1886

Abstract

Accurate and rapid fruit detection was very important for robot picking precisely, so the large model size and slow detection speed of the detection algorithm are problems that need to be solved urgently. An improved lightweight Faster R-CNN based on MobileNetV3 was proposed in this paper, which was used to detect fruits on Ori_RGB and Rb_RGB image datasets that collected by RGB-D camera in densely planted commercial pitaya orchards. On the Rb_RGB image datasets, the detection AP of 0.929 and 0.898 were obtained using MobileNetv3_large_FRCNN and MobileNetv3_small_FRCNN, which decreased 1.38% and 4.67% than that using VGG16_FRCNN respectively, and the detection time was 35.4 and 18.8 ms per image, which decreased 46.5% and 71.6% than that using VGG16_FRCNN respectively. On the Ori_RGB image datasets, the detection AP of 0.911 and 0.856 were obtained using MobileNetv3_large_FRCNN and MobileNetv3_small_FRCNN, which decreased 2.15% and 8.06% than that using VGG16_FRCNN respectively, and the detection time was 35.2 and 19.5 ms per image, which decreased 47.2% and 70.8% than that using VGG16_FRCNN respectively. Weight sizes of MobileNetv3_large_FRCNN and MobileNetv3_small_FRCNN were 3.19%, 1.15% of that of VGG16_FRCNN respectively. The detection AP values on the Rb_RGB image test set using three networks than that on Ori_RGB image test set increased 1.98%, 4.91%, and 1.18%, but image type had no significant effect on AP. The improved lightweight Faster R-CNN based on MobilenetV3 is expected to deploy to the embedded system of the fruit picking robot to detect pitaya, which would promote the development of robot picking technology.

Keywords

Supporting Agencies

National Natural Science Foundation of China , National Key Research and Development Program of China , Jiangsu Province Agricultural Science and Technology Independent Innovation Funds Project

References

  1. Abdalla, A., Cen, H., Wan, L., Rashid, R., Weng, H., Zhou, W., et al., 2019. Fine-tuning convolutional neural network with transfer learning for semantic segmentation of ground-level oilseed rape images in a field with high weed pressure. Comput. Electron. Agr. 167:105091. DOI: https://doi.org/10.1016/j.compag.2019.105091
  2. Bargoti, S., Underwood, J.P., 2017. Image segmentation for fruit detection and yield estimation in apple orchards. J. Field Robot. 34:1039-1060. DOI: https://doi.org/10.1002/rob.21699
  3. Chiu, Y., Chen, S., Lin, J. 2013. Study of an autonomous fruit picking robot system in greenhouses. Eng. Agric. Environ. Food 6:92-98. DOI: https://doi.org/10.1016/S1881-8366(13)80017-1
  4. Fu, L., Gao, F., Wu, J., Li, R., Karkee, M., Zhang, Q., 2020a. Application of consumer RGB-D cameras for fruit detection and localization in field: A critical review. Comput. Electron. Agr. 177:105687. DOI: https://doi.org/10.1016/j.compag.2020.105687
  5. Fu, L., Majeed, Y., Zhang, X., Karkee, M., Zhang, Q., 2020b. Faster R–CNN–based apple detection in dense-foliage fruiting-wall trees using RGB and depth features for robotic harvesting. Biosyst. Eng. 197:245-256. DOI: https://doi.org/10.1016/j.biosystemseng.2020.07.007
  6. Gao, F., Fu, L., Zhang, X., Majeed, Y., Li, R., Karkee, M., et al. 2020. Multi-class fruit-on-plant detection for apple in SNAP system using Faster R-CNN. Comput. Electron. Agr. 176:105634. DOI: https://doi.org/10.1016/j.compag.2020.105634
  7. Gené-Mola, J., Sanz-Cortiella, R., Rosell-Polo, J. R., Morros, J., Ruiz-Hidalgo, J., Vilaplana, V., et al., 2020. Fruit detection and 3D location using instance segmentation neural networks and structure-from-motion photogrammetry. Comput. Electron. Agr. 169:105165. DOI: https://doi.org/10.1016/j.compag.2019.105165
  8. Gené-Mola, J., Vilaplana, V., Rosell-Polo, J. R., Morros, J., Ruiz-Hidalgo, J., Gregorio, E., 2019. Multi-modal deep learning for Fuji apple detection using RGB-D cameras and their radiometric capabilities. Comput. Electron. Agr. 162:689-698. DOI: https://doi.org/10.1016/j.compag.2019.05.016
  9. Hou, Z., Li, Z., Fadiji, T., Fu, J., 2021. Soft grasping mechanism of human fingers for tomato-picking bionic robots. Comput. Electron. Agr. 182:106010. DOI: https://doi.org/10.1016/j.compag.2021.106010
  10. Jiang, H., Zhang, W., Li, X., Shu, C., Jiang, W., Cao, J., 2021. Nutrition, phytochemical profile, bioactivities and applications in food industry of pitaya (Hylocereus spp.) peels: A comprehensive review. Trends Food Sci. Technol. 116:199-217. DOI: https://doi.org/10.1016/j.tifs.2021.06.040
  11. Li, C. E., Tang, Y., Zou, X., Zhang, P., Lin, J., Lian, G., et al., 2022. A novel agricultural machinery intelligent design system based on integrating image processing and knowledge reasoning. Appl. Sci. 12:7900. DOI: https://doi.org/10.3390/app12157900
  12. Li, Z., Miao, F., Yang, Z., Chai, P., Yang, S., 2019. Factors affecting human hand grasp type in tomato fruit-picking: A statistical investigation for ergonomic development of harvesting robot. Comput. Electron. Agr. 157:90-97. DOI: https://doi.org/10.1016/j.compag.2018.12.047
  13. Malik, M. H., Zhang, T., Li, H., Zhang, M., Shabbir, S., Saeed, A., 2018. Mature tomato fruit detection algorithm based on improved HSV and watershed algorithm. IFAC PapersOnLine 51:431-436. DOI: https://doi.org/10.1016/j.ifacol.2018.08.183
  14. Miao, Y., Zheng, J., 2020. Optimization design of compliant constant-force mechanism for apple picking actuator. Comput. Electron. Agr. 170:105232. DOI: https://doi.org/10.1016/j.compag.2020.105232
  15. Mu, L., Cui, G., Liu, Y., Cui, Y., Fu, L., Gejima, Y., 2020. Design and simulation of an integrated end-effector for picking kiwifruit by robot. Inform. Process. Agr. 7:58-71. DOI: https://doi.org/10.1016/j.inpa.2019.05.004
  16. Nan, Y., Zhang, H., Zheng, J., Yang, K., Ge, Y., 2023. Low-volume precision spray for plant pest control using profile variable rate spraying and ultrasonic detection. Front. Plant Sci. 13:1042769. DOI: https://doi.org/10.3389/fpls.2022.1042769
  17. Ni, X., Wang, X., Wang, S., Wang, S., Yao, Z., Ma, Y., 2018. Structure design and image recognition research of a picking device on the apple picking robot. IFAC PapersOnLine 51:489-494. DOI: https://doi.org/10.1016/j.ifacol.2018.08.162
  18. Ren, S., He, K., Girshick, R., Sun, J., 2017. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE T. Pattern Anal. 39:1137-1149. DOI: https://doi.org/10.1109/TPAMI.2016.2577031
  19. Sengupta, S., Lee, W.S., 2014. Identification and determination of the number of immature green citrus fruit in a canopy under different ambient light conditions. Biosyst. Eng. 117:51-61. DOI: https://doi.org/10.1016/j.biosystemseng.2013.07.007
  20. Suo, R., Gao, F., Zhou, Z., Fu, L., Song, Z., Dhupia, J., et al., 2021. Improved multi-classes kiwifruit detection in orchard to avoid collisions during robotic picking. Comput. Electron. Agr. 182:106052. DOI: https://doi.org/10.1016/j.compag.2021.106052
  21. Tang, Y. C., Chen, M.Y., Wang, C.L., Luo, L.F., Li, J.H., Lian, G.P., et al. 2020. Recognition and localization methods for vision-based fruit picking robots: a review. Front. Plant Sci. 11:510. DOI: https://doi.org/10.3389/fpls.2020.00510
  22. Tang, Y., Zhou, H., Wang, H., Zhang, Y., 2023. Fruit detection and positioning technology for a Camellia oleifera C. Abel orchard based on improved YOLOv4-tiny model and binocular stereo vision. Expert Syst. Appl. 211:118573. DOI: https://doi.org/10.1016/j.eswa.2022.118573
  23. Tu, S., Pang, J., Liu, H., Zhuang, N., Chen, Y., Zheng, C., et al. 2020. Passion fruit detection and counting based on multiple scale faster R-CNN using RGB-D images. Precis. Agric. 21:1072-1091. DOI: https://doi.org/10.1007/s11119-020-09709-3
  24. Vasconez, J. P., Delpiano, J., Vougioukas, S., Auat Cheein, F., 2020. Comparison of convolutional neural networks in fruit detection and counting: A comprehensive evaluation. Comput. Electron. Agr. 173:105348. DOI: https://doi.org/10.1016/j.compag.2020.105348
  25. Wan, S., Goudos, S., 2020. Faster R-CNN for multi-class fruit detection using a robotic vision system. Comput. Netw. 168:107036. DOI: https://doi.org/10.1016/j.comnet.2019.107036
  26. Wang, H., Zhao, Q., Li, H., Zhao, R., 2022. Polynomial-based smooth trajectory planning for fruit-picking robot manipulator. Inform. Process. Agric. 9:112-122. DOI: https://doi.org/10.1016/j.inpa.2021.08.001
  27. Wang, J., Gao, K., Jiang, H., Zhou, H., 2020. Method for detecting dragon fruit based on improved lightweight convolutional neural network. T. CSAE 36:218-225.
  28. Yang, C. H., Xiong, L. Y., Wang, Z., Wang, Y., Shi, G., Kuremot, T., et al., 2020. Integrated detection of citrus fruits and branches using a convolutional neural network. Comput. Electron. Agr. 174:105469. DOI: https://doi.org/10.1016/j.compag.2020.105469

Metrics

Citations and online attention from Dimensions, Altmetric and PlumX.

PlumX Metrics

License

Copyright (c) 2025 the Author(s)

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

PAGEPress has chosen to apply the Creative Commons Attribution NonCommercial 4.0 International License (CC BY-NC 4.0) to all manuscripts to be published.