Tomato Gantry T- Rex Disease Detection
The Gantry was the platform for multiple projects: T-REX, Disease Detection and ...
T-REX: Robot for Extracting DNA Samples from Tomato plants
Gantry Robot Platform
My research effort was in the design and development of the 3DoF end effector attached to the Gantry robot. Once a working prototype of the grab-and-cut end effector was made, the objective shifted to 3D reconstruction of plants (on a Gantry robot by taking multi-view images using a stereo camera) and grasping control. The stereo images were processed from the Rosbag files to create disparity images using a Deep learning-based approach called RAFT-Stereo. These disparity images were in turn used to create point cloud models of each disparity. These models were fused to provide a more detailed point cloud, capturing all 6 different views for each plant. Leaf segmentation was done to identify the top N leaves of each plant sorted by size. These were filtered to get individual leaf point clouds. Ransac best-fit plane method was employed to obtain the plane of the leaf and estimate its pose. This pose was used to compute desired end effector pose.
Currently, the pipeline created is able to process rosbag files containing the stereo images, produce the disparity images using RAFT-Stereo and produce the raw point clouds from the disparity. These point clouds undergo pre-processing and filtering. Segmenting the leaves and obtaining the end effector pose is also done. The next stage is to control the robot grasping and refine the process.
Partners: VirginiaTech, who are concerned with disease detection of the sampled leaves
Project Objective: The goal of the project is early-stage disease detection in young tomato plants. The plants are grown on a “Gantry” table, where the Gantry robot is a Cartesian robot capable of traversing in X, Y and Z axes. In addition, a 3DoF end-effector is attached to the Z-axis. This end effector consists of a stereo camera and a “gripper” that can grip and cut tomato leaves for sampling purpose. These leaves are then dropped to a specific location on the table, where (genome sequencing?) is carried out to analyze the leaf for (leaf spot?) disease.