Pepper Harvesting

Project Overview

This project explores the use of robotics for precision agriculture, specifically in automating pepper harvesting. Combining human demonstrations and advanced machine learning techniques, the goal is to deploy an autonomous harvesting system that can efficiently navigate real-world challenges in crop fields.

Our Approach

At the core of this project is imitation learning, where robots learn by observing human actions. Through hundreds of carefully curated human demonstrations, the system is trained using a diffusion policy, enabling it to recognize key features of the harvesting task, such as identifying the pepper, estimating its position, and executing the necessary movements to harvest it autonomously.

 Field Testing

The system was deployed in real-world field conditions, undergoing rigorous testing across various environmental challenges. These trials involved harvesting peppers in different lighting conditions, accounting for occlusion by leaves, and dealing with a range of pepper orientations and shapes. The tests demonstrated the system’s ability to generalize to new, unseen environments, handling natural variations in crop layouts.

Challenges We Face

Pepper Occlusion by Leaves: One of the most difficult obstacles is when leaves or stems block the view of the pepper, which can mislead the robot’s policy.

Peduncle Orientation: The orientation and shape of the pepper’s stem (peduncle) greatly influence how effectively the robot can cut and retrieve the pepper. Complex shapes and angles often pose significant difficulties.

Looking Forward

To further enhance the system, we are exploring ways to improve the robot’s ability to handle occlusion, optimize cutting tools, and extend its capabilities to other crops. These advancements will make the technology more adaptable to diverse agricultural environments, paving the way for smarter and more efficient farming practices.