Grapevine Pruner Bumblebee
Bumblebee is a rugged field robot to autonomously prune dormant season grapevines.
Dormant season grapevine pruning involves removal of plant tissues in the form of spurs and excess one-year-old canes from the previous year’s growth. This is a primary tool used by growers to manipulate vine size and regulate crop load, maintaining balance between the size of the vine and the amount of fruit it should produce. It is a labor-intensive task that requires skilled seasonal workers during the winter season which are becoming less available. Inconsistent pruning results in vines that are over-cropped (trained to produce more fruit than the vine can sustain, resulting in low quality) or under-cropped (trained to produce less fruit than the vine can sustain, resulting in low yield), either of which requires several years of careful mitigation to recover from.
From the robotics perspective, pruning for vine balance poses multiple interesting challenges that require research advances in the fields of perception, control, robotic manipulation, and artificial intelligence (AI), in general. Dormant season vines contain dense, criss-crossing branches that effectively fill a 3D volume while also leaving many small unoccupied spaces. The resulting highly occluded complex geometry is difficult to model, and existing modeling methods such as SLAM are not capable of generating complete maps. Deciding where to make a pruning cut requires intelligence to understand the canopy at multiple levels, including its geometry, its topology (what is connected to what), and its semantic meaning (what parts are canes, buds, etc.). The ability to automatically generate this level of understanding does not currently exist. From a manipulation standpoint, the vine is cluttered and full of flexible objects that must be avoided or pushed away in order to reach into the canopy. Standard manipulation planning approaches are not equipped to handle this case. The balanced pruning application will drive our investigation into these topics, but they are fundamental research advances that will have broader impact, ranging from automation in more general tree canopies to a range of everyday tasks that require intelligent interaction with flexible materials in cluttered spaces. This work build on top of our previous vine pruning research.
Automated Grapevine Skeletonization for Balanced Pruning
Summary:
The end goal of this effort is to process images of grapevines and create high-quality skeletal models of complex, natural vines. In the end, these skeletons are intended to help enable robotic pruning of grapevines. Winter grapevine pruning is an annual operation to maximize high-quality grape yields, but the current most cost-effective methods for pruning are one-size-fits-all mechanized approaches, which cause some vines to be unhealthy. If robotic pruning could operate on individual plants, it could improve plant health and overall yield. The basic method for skeletonization is to create a unified 3d point cloud for a vine by combining multiple stereo views, then fit cylindrical segments to the point cloud. There are two goals that we hope can be satisfied when a skeleton is created. First, plant vigor can be assessed by calculating the length of the vines. Second, the pruning robot can use the skeleton to plan a cutting path. Skeletons are a simple and effective model for that use case.
Status:
[10/17/2022]: So far we have turned the original stereo images of grapevines into a single combined point cloud per vine. In addition we have trained a neural network on the grapevine images to segment pixels into the classes of background, vine, and trunk. The public dataset for these labeled images is found here. Combining these two results we can select the points in the point cloud that we believe represent vines. The next step is to combine with the work that John Kim has been doing around gap-filling in tree models, and try that gap-filling approach on vines. The basis of that approach is to project elliptical probabilities out from each seen line segment, and overlay those from all line segments in order to find the most likely gap filling paths.
Sponsor/Partners:
CLEREL / Cornell Experimental Vineyards
Related Presentations:
Skeletonization literature overview
Repository: