Solutions > Agriculture > Human Detection and Tracking in Agriculture

Human Detection and Tracking in Agriculture

NREC developed advanced machine vision techniques for safety around agricultural vehicles. Robotics offers the opportunity to improve efficiency on the farm, but these systems must reliably detect other workers to ensure their safety.

Enabling the full promise of robotics in agriculture requires reliable detection and tracking of human coworkers so that people and machines can effectively and safely perform required tasks. Many agricultural machines are powerful and potentially dangerous, and certain tasks require humans to work closely to these machines. Other applications may need to enforce a safety buffer, and agricultural fields generally have minimal access controls. Even for smaller agricultural robots, it is often important for them to understand where the people in their environment are to effectively complete their tasks.

Our previous work resulted in a spatially distributed multi-vehicle system of autonomous tractors that shared task responsibilities with multiple human co-workers to accomplish agricultural operations in a citrus orchard. This system has demonstrated over 2400km of autonomous operation and performed significant useful work at a higher productivity level than current methods. The system includes a sophisticated obstacle detection system, but a key limiting factor was the reliable detection of people when partially occluded by tree branches and weeds or when lying on the ground or in other non-standard poses. [1] [2]

[1] Carnegie Mellon University. “Integrated Automation for Sustainable Specialty Crops.” http://www.rec.ri.cmu.edu/usda/index.html

[2] S. J. Moorehead, C. K. Wellington, B. J. Gilmore, and C. Vallespi, “Automating orchards: A system of autonomous tractors for orchard maintenance,” in IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS) Workshop on Agricultural Robotics, 2012.

The aim of this work was to advance the state of the art in detection and tracking of people in agricultural environments. We benchmarked many current methods in pedestrian detection and developed new ones, and released the dataset below [3,4]. We hope that this common benchmark allows the field to move forward with a spirit of both competition and cooperation.

[3] T. Tabor, Z. Pezzementi, C. Vallespi and C. Wellington, ‘People in the Weeds: Pedestrian Detection Goes Off-road’, in 2015 IEEE International Symposium on Safety, Security, and Rescue Robotics, Purdue University, West Lafayette, IN, 2015.

[4] Z. Pezzementi, T. Tabor, P. Hu, J. Chang, D. Ramanan, C. Wellington, B. Babu, and H. Herman. Comparing apples and oranges: Off-road pedestrian detection on the National Robotics Engineering Center agricultural person-detection dataset. J Field Robotics. 2017;00:1–19. https://doi.org/10.1002/rob.21760. arXiv preprint arXiv:1707.07169.

The NREC Person Detection Dataset is a collection of off-road videos taken in an apple orchard and orange grove. The videos are collected with a set of visible people in a variety of outfits, locations, and times. We encourage you to train a detector on our dataset and submit your curves for display on this webpage.

Labels are provided in Pascal VOC format and images are provided as rectified pngs. A training set has been partitioned for algorithm training. A full validation set has been partitioned for algorithm tuning and development results. Finally, a test set is provided for final evaluation and publication. We ask that the test set be used only after completion of development, in order to preserve the integrity of the dataset.

Details, analysis, and initial results on the data set can be found in our paper. Please cite this paper for any work making use of the data set:

Z. Pezzementi, T. Tabor, P. Hu, J. Chang, D. Ramanan, C. Wellington, B. Babu, and H. Herman. Comparing apples and oranges: Off-road pedestrian detection on the National Robotics Engineering Center agricultural person-detection dataset. J Field Robotics. 2017;00:1–19. https://doi.org/10.1002/rob.21760. arXiv preprint arXiv:1707.07169.

Scripts for working with the dataset are available at: https://github.com/zpezz/nrecAgPersonEval

This work was supported by the USDA National Institute of Food and Agriculture as part of the National Robotics Initiative under award number 2014-67021-22171.