https://doi.org/10.4081/jae.2023.1301
Cassava stalk detection for a cassava harvesting robot based on YOLO v4 and Mask R-CNN
Published: 1 August 2023
Abstract Views: 1011
PDF: 480
HTML: 97
HTML: 97
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.
Authors
The quality of fresh cassava roots can be increased through the use of precision equipment. As a first step towards developing an automatic cassava root cutting system, this study demonstrates the use of a computer vision system with deep learning for cassava stalk detection. An RGB image of a cassava tree mounted on a cassava-pulling machine was captured, and the YOLO v4 model and two Mask R-CNN models with ResNet 101 and ResNet 50 base architectures were employed to train the weights to predict the position of the cassava stalk. One hundred test images of stalks of various shapes and sizes were used to determine the grasping point and inclination, and the results from manual annotation were compared with the predicted results. Regarding localisation, Mask R-CNN with ResNet 101 gave a significantly higher performance than the other models, with an F1 score and a mean IoU of 0.81 and 0.70, respectively. YOLO v4 showed the highest correlation for the x- and y-coordinates for the prediction of the grasping point, with values for R2 of 0.89 and 0.53, respectively. For inclination prediction, Mask R-CNN with ResNet 101 and Mask R-CNN with ResNet 50 gave the same level of correlation, with values for R2 of 0.50 and 0.61, respectively. These results were acceptable for use as design criteria for developing a cassava rootcutting robot.
Abdulla W. 2016. Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow. GitHub Repository. Available from: https://github.com/matterport/Mask_RCNN
Arsawang S., Chansrakoo W., Chamsing A., Sangphanta P., Chawkongchak S. 2015. Design and development of Cassava root plucking out machine. Agric. Sci. J. 47:463-6.
Bapat K. 2018. Find the center of a blob (Centroid) using OpenCV (C++/Python). LearnOpenCV; July 19, 2018. Available from: https://www.learnopencv.com/find-center-of-blob-centroidusing-opencv-cpp-python/
Blok P.M., Barth R., van den Berg W. 2016. Machine vision for a selective broccoli harvesting robot. IFAC-PapersOnLine 49:66-71. Available from: https://doi.org/10.1016/j.ifacol.2016.10.013
DOI: https://doi.org/10.1016/j.ifacol.2016.10.013
Boonsang, S. 2020a. CiRA CORE : community. KMITL. Available from: https://www.facebook.com/groups/cira.core.comm/
Boonsang S. 2020b. KSL cira core. KLS. Available from: https://sites.google.com/site/klsrobotcenter/kls-cira-core
Brownlee J. 2021. A gentle introduction to object recognition with deep learning. Available from: https://machinelearningmastery.com/object-recognition-with-deep-learning/
Chansiri, C., & Wongpichet, S. 2011. Research and development of the digging and gathering machine for cassava harvesting. Khon Kaen University, Khon Kaen, Thailand.
Dutta, A., & Zisserman, A. 2019. The VIA annotation software for images, audio and video. MM 2019 - Proceedings of the 27th ACM International Conference on Multimedia, 2276-2279. Available from: https://doi.org/10.1145/3343031.335053.
DOI: https://doi.org/10.1145/3343031.3350535
FAO. 2013. Cassava, a 21st century crop. In: Save and grow: cassava, a guide to sustainable production intensification. Available from: http://www.fao.org/ag/save-and-grow/cassava/en/1/index.html
Font D., Pallejà T., Tresanchez M., Runcan D., Moreno J., Martínez D., Teixidó M., Palacín J. 2014. A proposal for automatic fruit harvesting by combining a low cost stereovision camera and a robotic arm. Sensors (Switzerland) 14:11557-79.
DOI: https://doi.org/10.3390/s140711557
Fu L., Majeed Y., Zhang X., Karkee M., Zhang Q. 2020. Faster RCNN- based apple detection in dense-foliage fruiting-wall trees using RGB and depth features for robotic harvesting. Biosyst. Engine. 197:245-56.
DOI: https://doi.org/10.1016/j.biosystemseng.2020.07.007
Ganesh P., Volle K., Burks T.F., Mehta S.S. 2019. Deep orange: Mask R-CNN based orange detection and segmentation. IFAC-PapersOnLine 52:70-5.
DOI: https://doi.org/10.1016/j.ifacol.2019.12.499
Ge Y., Xiong Y., From P.J. 2019. Instance segmentation and localization of strawberries in farm conditions for automatic fruit harvesting. IFAC-PapersOnLine 52:294-9.
DOI: https://doi.org/10.1016/j.ifacol.2019.12.537
Girshick R. 2015. Fast R-CNN. pp 1440-1448 in Proceedings of the IEEE International Conference on Computer Vision 2015 Inter, 1. Available from: https://doi.org/10.1109/ICCV.2015.169
DOI: https://doi.org/10.1109/ICCV.2015.169
He K., Gkioxari G., Dollár P., Girshick R. 2020. Mask R-CNN. IEEE Trans. Pattern Anal. Machine Intell. 42:386-97.
DOI: https://doi.org/10.1109/TPAMI.2018.2844175
Jia W., Tian Y., Luo R., Zhang Z., Lian J., Zheng Y. 2020. Detection and segmentation of overlapped fruits based on optimized mask R-CNN application in apple harvesting robot. Comput. Electron. Agric. 172:105380.
DOI: https://doi.org/10.1016/j.compag.2020.105380
Jordan J. 2018. Evaluating image segmentation. Available from: https://www.jeremyjordan.me/evaluating-image-segmentation-models/
Koirala A., Walsh K.B., Wang Z., McCarthy C. 2019. Deep learning for real-time fruit detection and orchard fruit load estimation: benchmarking of ‘MangoYOLO.’ Precis. Agric. 20:1107-35.
DOI: https://doi.org/10.1007/s11119-019-09642-0
KURDI. 2020. index @ www3.rdi.ku.ac.th. KU. Available from: http://www3.rdi.ku.ac.th/?p=58386
Langkapin J., Kalsirisilp R., Tantrabandit M. 2012. Design and fabrication of cassava root picking machine. Agric. Sci. J. 30:1-11.
Ling X., Zhao Y., Gong L., Liu C., Wang T. 2019. Dual-arm cooperation and implementing for robotic harvesting tomato using binocular vision. Robot. Auton. Syst. 114:134-43.
DOI: https://doi.org/10.1016/j.robot.2019.01.019
Majeed Y., Karkee M., Zhang Q., Fu L., Whiting M.D. 2019. A study on the detection of visible parts of cordons using deep learning networks for automated green shoot thinning in vineyards. IFAC-PapersOnLine 52:82-6.
DOI: https://doi.org/10.1016/j.ifacol.2019.12.501
Manthamkan, V., Rattanasrimetha, S., & Suriwong, M. 2011. Development of cassava root lifted up by pulling stump harvester type. Karsetsart Extension Journal, 56(2), 52–60. Available from: https://kukrdb.lib.ku.ac.th/journal/ETO/search_detail/result/43687.
Mao S., Li Y., Ma Y., Zhang B., Zhou J., Kai W. 2020. Automatic cucumber recognition algorithm for harvesting robots in the natural environment using deep learning and multi-feature fusion. Comput. Electron. Agric. 170:105254.
DOI: https://doi.org/10.1016/j.compag.2020.105254
Maraphum K., Saengprachatanarug K., Wongpichet S., Phuphaphud A., Posom J. 2020. In-field measurement of starch content of cassava tubers using handheld vis-near infrared spectroscopy implemented for breeding programmes. Comput. Electron. Agric. 175:105607.
DOI: https://doi.org/10.1016/j.compag.2020.105607
Mauntumkarn W. 2010. Cassava harvesting machine. Available from: http://www.rdi.ku.ac.th/kasetresearch53/group06/wichar/index_04html
Mooney J.G., Johnson E.N. 2014. Performance evaluation of a harvesting robot for sweet pepper. J. Field Robot. 33:1-17.
Mordvintsev A., Abid K. 2013. opencv python. Available from: https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_contours/py_contours_begin/py_con
tours_begin.html
OAE. 2021. Agricultural economic information by commodity. Office Agric. Econ. Available from: http://www.oae.go.th/view/1/ปัจจัยการผลิต/TH-TH
Redmon J., Divvala S., Girshick R., Farhadi A. 2016. You only look once: unified, real-time object detection. pp. 779-788 in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Available from: https://doi.org/10.1109/CVPR.2016.91
DOI: https://doi.org/10.1109/CVPR.2016.91
Redmon J., Farhadi A. 2018. YOLO v.3. Tech Report, 1–6. Available from: https://pjreddie.com/media/files/papers/YOLOv3.pdf
Ren S., He K., Girshick R., Sun J. 2017. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Machine Intell. 39:1137-49.
DOI: https://doi.org/10.1109/TPAMI.2016.2577031
Sangphanta P., Chansrakoo W., Arsawang S., Chamsing A. 2011. Research and development of cassava harvesting machine. p. 12 in Proceedings of the 12th Thai Society of Agricultural Engineering Annual Academic Meeting.
Saxena S. 2021. Image augmentation techniques for training deep learning models. Analytics Vidhya. Available from: https://www.analyticsvidhya.com/blog/2021/03/image-augmentation-techniques-for-training-deep-learning-models/
Singhpoo T. 2019. Factors affecting cassava root cutting by using cylinder saw mechanism. Khon Kaen Univesity, Khon Kaen, Thailand.
Singhpoo T., Wongpichet S., Saengprachatanarug K., Posom J., Watyotha C., Yangyuen S. 2019. A study of stalk shape for designing the operational mechanism of gripping equipment for cassava tuber cut preparation process. IOP Conf. Ser. Earth Environ. Sci. 301:1-6.
DOI: https://doi.org/10.1088/1755-1315/301/1/012014
Suvanapa K., Wongpichet S. 2014. Feasibility study of cassava rhizome cutting by using square tube blade. pp. 335-342 in The 16th Thai Society of Agricultural Engineering Annual Academic Meeting.
Tian Y., Yang G., Wang Z., Wang H., Li E., Liang Z. 2019. Apple detection during different growth stages in orchards using the improved YOLO-V3 model. Comput. Electron. Agricult. 157:417-26.
DOI: https://doi.org/10.1016/j.compag.2019.01.012
Vatakit K., Somphong C., Junyusen P., Arjharn W. 2014. Development of cassava harvester for cutting cassava tuber from rhizome. Agric. Sci. J. 45:353-6.
Williams H.A.M., Jones M.H., Nejati M., Seabright M.J., Bell J., Penhall N.D., Barnett J.J., Duke M.D., Scarfe A.J., Ahn H.S., Lim J.Y., MacDonald B.A. 2019. Robotic kiwifruit harvesting using machine vision, convolutional neural networks, and robotic arms. Biosyst. Engine. 181:140-56.
DOI: https://doi.org/10.1016/j.biosystemseng.2019.03.007
Yu Y., Zhang K., Yang L., Zhang D. 2019. Fruit detection for strawberry harvesting robot in non-structural environment based on Mask-RCNN. Comput. Electron. Agric. 163:104846.
DOI: https://doi.org/10.1016/j.compag.2019.06.001
Supporting Agencies
Khon Kaen UniversityHow to Cite
Singhpoo, T. (2023) “Cassava stalk detection for a cassava harvesting robot based on YOLO v4 and Mask R-CNN”, Journal of Agricultural Engineering, 54(2). doi: 10.4081/jae.2023.1301.
Copyright (c) 2023 the Author(s)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
PAGEPress has chosen to apply the Creative Commons Attribution NonCommercial 4.0 International License (CC BY-NC 4.0) to all manuscripts to be published.
Similar Articles
- Yun Zhu, Shuwen Liu, Xiaojun Wu, Lianfeng Gao, Youyun Xu, Multi-class segmentation of navel orange surface defects based on improved DeepLabv3+ , Journal of Agricultural Engineering: Vol. 55 No. 2 (2024)
- Wei Wang, Yuanjuan Gong, Xuewei Bai, Rui Tan, Real-time straw moisture content detection system for mobile straw granulator , Journal of Agricultural Engineering: Vol. 55 No. 2 (2024)
- Zhongwei Hua, Min Guan, Lightweight sandy vegetation object detection algorithm based on attention mechanism , Journal of Agricultural Engineering: Vol. 54 No. 1 (2023)
- Luhua Han, Haorui Ma, Menghan Mo, Francis Kumi, Jianping Hu, Hanping Mao, Design and test of an efficient seedling pick-up device with a combination of air jet ejection and mechanical action , Journal of Agricultural Engineering: Vol. 55 No. 3 (2024)
- Diwei Wu, Shaohua Zeng, Shuai Wang, Yanan Chen, Yidan Xu, Zanthoxylum infructescence detection based on adaptive density clustering , Journal of Agricultural Engineering: Vol. 55 No. 2 (2024)
- Jingtao Li, Hao Chen, Guisong Li, Yueqi Liu, Yanli Yang, Xia Liu, Chang Yi Wang, Detection method of potato leaf disease based on YOLOv5s , Journal of Agricultural Engineering: Vol. 55 No. 3 (2024)
- Xu Xiao, Yaonan Wang, Yiming Jiang, Haotian Wu, Zhe Zhang, Rujing Wang, AC-YOLO: citrus detection in the natural environment of orchards , Journal of Agricultural Engineering: Vol. 55 No. 4 (2024)
- Cesar Augusto Terán-Chaves, Sonia Mercedes Polo-Murcia, Cropping pattern simulation-optimization model for water use efficiency and economic return , Journal of Agricultural Engineering: Vol. 52 No. 4 (2021)
- Zhi He, Xinting Ding, Wei Hao, Kai Li, Weixin Gong, Zixu Li, Yongjie Cui, Calibration and experiments of discrete element flexible model parameters for kiwifruit stalk , Journal of Agricultural Engineering: Vol. 55 No. 4 (2024)
- Yanmei Meng, Tingting Zhang, Jin Wei, Jinlai Zhang, Xulei Zhai, Design and experiments of an automatic pipe winding machine , Journal of Agricultural Engineering: Vol. 54 No. 2 (2023)
<< < 1 2 3 4 5 6 7 8 9 10 > >>
You may also start an advanced similarity search for this article.
