https://doi.org/10.4081/jae.2025.1886
Pitaya detection using an improved lightweight Faster R-CNN based on MobileNetV3 in densely planted pitaya orchards
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.
Published: 17 July 2025
Accurate and rapid fruit detection was very important for robot picking precisely, so the large model size and slow detection speed of the detection algorithm are problems that need to be solved urgently. An improved lightweight Faster R-CNN based on MobileNetV3 was proposed in this paper, which was used to detect fruits on Ori_RGB and Rb_RGB image datasets that collected by RGB-D camera in densely planted commercial pitaya orchards. On the Rb_RGB image datasets, the detection AP of 0.929 and 0.898 were obtained using MobileNetv3_large_FRCNN and MobileNetv3_small_FRCNN, which decreased 1.38% and 4.67% than that using VGG16_FRCNN respectively, and the detection time was 35.4 and 18.8 ms per image, which decreased 46.5% and 71.6% than that using VGG16_FRCNN respectively. On the Ori_RGB image datasets, the detection AP of 0.911 and 0.856 were obtained using MobileNetv3_large_FRCNN and MobileNetv3_small_FRCNN, which decreased 2.15% and 8.06% than that using VGG16_FRCNN respectively, and the detection time was 35.2 and 19.5 ms per image, which decreased 47.2% and 70.8% than that using VGG16_FRCNN respectively. Weight sizes of MobileNetv3_large_FRCNN and MobileNetv3_small_FRCNN were 3.19%, 1.15% of that of VGG16_FRCNN respectively. The detection AP values on the Rb_RGB image test set using three networks than that on Ori_RGB image test set increased 1.98%, 4.91%, and 1.18%, but image type had no significant effect on AP. The improved lightweight Faster R-CNN based on MobilenetV3 is expected to deploy to the embedded system of the fruit picking robot to detect pitaya, which would promote the development of robot picking technology.
Downloads
Abdalla, A., Cen, H., Wan, L., Rashid, R., Weng, H., Zhou, W., et al., 2019. Fine-tuning convolutional neural network with transfer learning for semantic segmentation of ground-level oilseed rape images in a field with high weed pressure. Comput. Electron. Agr. 167:105091. DOI: https://doi.org/10.1016/j.compag.2019.105091
Bargoti, S., Underwood, J.P., 2017. Image segmentation for fruit detection and yield estimation in apple orchards. J. Field Robot. 34:1039-1060. DOI: https://doi.org/10.1002/rob.21699
Chiu, Y., Chen, S., Lin, J. 2013. Study of an autonomous fruit picking robot system in greenhouses. Eng. Agric. Environ. Food 6:92-98. DOI: https://doi.org/10.1016/S1881-8366(13)80017-1
Fu, L., Gao, F., Wu, J., Li, R., Karkee, M., Zhang, Q., 2020a. Application of consumer RGB-D cameras for fruit detection and localization in field: A critical review. Comput. Electron. Agr. 177:105687. DOI: https://doi.org/10.1016/j.compag.2020.105687
Fu, L., Majeed, Y., Zhang, X., Karkee, M., Zhang, Q., 2020b. Faster R–CNN–based apple detection in dense-foliage fruiting-wall trees using RGB and depth features for robotic harvesting. Biosyst. Eng. 197:245-256. DOI: https://doi.org/10.1016/j.biosystemseng.2020.07.007
Gao, F., Fu, L., Zhang, X., Majeed, Y., Li, R., Karkee, M., et al. 2020. Multi-class fruit-on-plant detection for apple in SNAP system using Faster R-CNN. Comput. Electron. Agr. 176:105634. DOI: https://doi.org/10.1016/j.compag.2020.105634
Gené-Mola, J., Sanz-Cortiella, R., Rosell-Polo, J. R., Morros, J., Ruiz-Hidalgo, J., Vilaplana, V., et al., 2020. Fruit detection and 3D location using instance segmentation neural networks and structure-from-motion photogrammetry. Comput. Electron. Agr. 169:105165. DOI: https://doi.org/10.1016/j.compag.2019.105165
Gené-Mola, J., Vilaplana, V., Rosell-Polo, J. R., Morros, J., Ruiz-Hidalgo, J., Gregorio, E., 2019. Multi-modal deep learning for Fuji apple detection using RGB-D cameras and their radiometric capabilities. Comput. Electron. Agr. 162:689-698. DOI: https://doi.org/10.1016/j.compag.2019.05.016
Hou, Z., Li, Z., Fadiji, T., Fu, J., 2021. Soft grasping mechanism of human fingers for tomato-picking bionic robots. Comput. Electron. Agr. 182:106010. DOI: https://doi.org/10.1016/j.compag.2021.106010
Jiang, H., Zhang, W., Li, X., Shu, C., Jiang, W., Cao, J., 2021. Nutrition, phytochemical profile, bioactivities and applications in food industry of pitaya (Hylocereus spp.) peels: A comprehensive review. Trends Food Sci. Technol. 116:199-217. DOI: https://doi.org/10.1016/j.tifs.2021.06.040
Li, C. E., Tang, Y., Zou, X., Zhang, P., Lin, J., Lian, G., et al., 2022. A novel agricultural machinery intelligent design system based on integrating image processing and knowledge reasoning. Appl. Sci. 12:7900. DOI: https://doi.org/10.3390/app12157900
Li, Z., Miao, F., Yang, Z., Chai, P., Yang, S., 2019. Factors affecting human hand grasp type in tomato fruit-picking: A statistical investigation for ergonomic development of harvesting robot. Comput. Electron. Agr. 157:90-97. DOI: https://doi.org/10.1016/j.compag.2018.12.047
Malik, M. H., Zhang, T., Li, H., Zhang, M., Shabbir, S., Saeed, A., 2018. Mature tomato fruit detection algorithm based on improved HSV and watershed algorithm. IFAC PapersOnLine 51:431-436. DOI: https://doi.org/10.1016/j.ifacol.2018.08.183
Miao, Y., Zheng, J., 2020. Optimization design of compliant constant-force mechanism for apple picking actuator. Comput. Electron. Agr. 170:105232. DOI: https://doi.org/10.1016/j.compag.2020.105232
Mu, L., Cui, G., Liu, Y., Cui, Y., Fu, L., Gejima, Y., 2020. Design and simulation of an integrated end-effector for picking kiwifruit by robot. Inform. Process. Agr. 7:58-71. DOI: https://doi.org/10.1016/j.inpa.2019.05.004
Nan, Y., Zhang, H., Zheng, J., Yang, K., Ge, Y., 2023. Low-volume precision spray for plant pest control using profile variable rate spraying and ultrasonic detection. Front. Plant Sci. 13:1042769. DOI: https://doi.org/10.3389/fpls.2022.1042769
Ni, X., Wang, X., Wang, S., Wang, S., Yao, Z., Ma, Y., 2018. Structure design and image recognition research of a picking device on the apple picking robot. IFAC PapersOnLine 51:489-494. DOI: https://doi.org/10.1016/j.ifacol.2018.08.162
Ren, S., He, K., Girshick, R., Sun, J., 2017. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE T. Pattern Anal. 39:1137-1149. DOI: https://doi.org/10.1109/TPAMI.2016.2577031
Sengupta, S., Lee, W.S., 2014. Identification and determination of the number of immature green citrus fruit in a canopy under different ambient light conditions. Biosyst. Eng. 117:51-61. DOI: https://doi.org/10.1016/j.biosystemseng.2013.07.007
Suo, R., Gao, F., Zhou, Z., Fu, L., Song, Z., Dhupia, J., et al., 2021. Improved multi-classes kiwifruit detection in orchard to avoid collisions during robotic picking. Comput. Electron. Agr. 182:106052. DOI: https://doi.org/10.1016/j.compag.2021.106052
Tang, Y. C., Chen, M.Y., Wang, C.L., Luo, L.F., Li, J.H., Lian, G.P., et al. 2020. Recognition and localization methods for vision-based fruit picking robots: a review. Front. Plant Sci. 11:510. DOI: https://doi.org/10.3389/fpls.2020.00510
Tang, Y., Zhou, H., Wang, H., Zhang, Y., 2023. Fruit detection and positioning technology for a Camellia oleifera C. Abel orchard based on improved YOLOv4-tiny model and binocular stereo vision. Expert Syst. Appl. 211:118573. DOI: https://doi.org/10.1016/j.eswa.2022.118573
Tu, S., Pang, J., Liu, H., Zhuang, N., Chen, Y., Zheng, C., et al. 2020. Passion fruit detection and counting based on multiple scale faster R-CNN using RGB-D images. Precis. Agric. 21:1072-1091. DOI: https://doi.org/10.1007/s11119-020-09709-3
Vasconez, J. P., Delpiano, J., Vougioukas, S., Auat Cheein, F., 2020. Comparison of convolutional neural networks in fruit detection and counting: A comprehensive evaluation. Comput. Electron. Agr. 173:105348. DOI: https://doi.org/10.1016/j.compag.2020.105348
Wan, S., Goudos, S., 2020. Faster R-CNN for multi-class fruit detection using a robotic vision system. Comput. Netw. 168:107036. DOI: https://doi.org/10.1016/j.comnet.2019.107036
Wang, H., Zhao, Q., Li, H., Zhao, R., 2022. Polynomial-based smooth trajectory planning for fruit-picking robot manipulator. Inform. Process. Agric. 9:112-122. DOI: https://doi.org/10.1016/j.inpa.2021.08.001
Wang, J., Gao, K., Jiang, H., Zhou, H., 2020. Method for detecting dragon fruit based on improved lightweight convolutional neural network. T. CSAE 36:218-225.
Yang, C. H., Xiong, L. Y., Wang, Z., Wang, Y., Shi, G., Kuremot, T., et al., 2020. Integrated detection of citrus fruits and branches using a convolutional neural network. Comput. Electron. Agr. 174:105469. DOI: https://doi.org/10.1016/j.compag.2020.105469
Supporting Agencies
How to Cite
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
PAGEPress has chosen to apply the Creative Commons Attribution NonCommercial 4.0 International License (CC BY-NC 4.0) to all manuscripts to be published.