Automated tea shoot picking using the YOLO network and Mamba images segmentation for top-view detection with a monocular camera

Detection and localization of tea shoots (one bud with two leaves) are critical steps in the automation of tea harvesting. Using red, green, blue-depth (RGB-D) camera to detect and locate tea shoots from side angles results in significant occlusion of tea shoots, as well as loss of depth information. To achieve automated, intelligent, and precise tea harvesting, this paper proposes a method for detecting and locating tea shoots from the top using a monocular camera. Firstly, the “You Only Look Once” (YOLO) network is employed to detect tea shoots regions in images collected by the monocular camera and to crop individual tea shoot top images. For these cropped images, a U-shaped images segmentation model based on Mamba is proposed. This model achieves a mean intersection over union (MIoU) of 87.80% and an accuracy (ACC) of 95.63%, precisely locating the specific tea shoots top regions. The center of the circumscribed circle of this region is used as the position for the next step in the picking process, accurately guiding the picking effector to the top of the tea shoot. Finally, the picking effector, controlled by feedback signals from infrared sensors, performs up-and-down reciprocation and cutting actions to complete the picking process. This method effectively avoids the problem of depth information loss during localization with RGB-D camera. To verify the effectiveness of the proposed approach, picking experiments were conducted on HouKui tea within a simulated tea garden environment, achieving a tea shoot picking success rate of 75.54%. The results indicate that this method offers significant application value and provides a new perspective for the development of automated tea shoots picking.