Tomato Gantry T- Rex Disease Detection
VaTech
The Gantry was the platform for multiple projects: T-REX, Disease Detection and ...
T-REX: Robot for Extracting DNA Samples from Tomato plants
Gantry Robot Platform
Research Effort:
My research effort was in the design and development of the 3DoF end effector attached to the Gantry robot. Once a working prototype of the grab-and-cut end effector was made, the objective shifted to 3D reconstruction of plants (on a Gantry robot by taking multi-view images using a stereo camera) and grasping control. The stereo images were processed from the Rosbag files to create disparity images using a Deep learning-based approach called RAFT-Stereo. These disparity images were in turn used to create point cloud models of each disparity. These models were fused to provide a more detailed point cloud, capturing all 6 different views for each plant. Leaf segmentation was done to identify the top N leaves of each plant sorted by size. These were filtered to get individual leaf point clouds. Ransac best-fit plane method was employed to obtain the plane of the leaf and estimate its pose. This pose was used to compute desired end effector pose.
Current Status:
Currently, the pipeline created is able to process rosbag files containing the stereo images, produce the disparity images using RAFT-Stereo and produce the raw point clouds from the disparity. These point clouds undergo pre-processing and filtering. Segmenting the leaves and obtaining the end effector pose is also done. The next stage is to control the robot grasping and refine the process.
Sponsors:
Partners: VirginiaTech, who are concerned with disease detection of the sampled leaves
Project Objective: The goal of the project is early-stage disease detection in young tomato plants. The plants are grown on a “Gantry” table, where the Gantry robot is a Cartesian robot capable of traversing in X, Y and Z axes. In addition, a 3DoF end-effector is attached to the Z-axis. This end effector consists of a stereo camera and a “gripper” that can grip and cut tomato leaves for sampling purpose. These leaves are then dropped to a specific location on the table, where (genome sequencing?) is carried out to analyze the leaf for (leaf spot?) disease.