AI INNOVATIONS FOR BUILDING A DIGITAL TWIN
Acquiring Manipulation Skills: Oliver Kroemer and George Kantor
Addressing the challenge of reasoning about manipulation tasks in the presence of forceful contacts and deformable objects. The framework will include interactive system identification, wherein the robot uses actions and sensory feedback to estimate the environment’s model parameters, i.e., the mechanical parameters of the digital twin. While previous work focused on individual interactions, we will model continuous deformable structures and the connections between them. We will also incorporate multiple sensor modalities, such as vision, joint torque, tactile, and acoustic, to increase the information gained from each interaction. We will explore methods for incorporating prior domain knowledge in manipulation processes. Bimanual task executions have two or more manipulators, allowing the robot to introduce additional constraints with one arm to facilitate the manipulation task of another arm. This approach will treat the other arms as part of the environment.
Soft Manipulation: Nancy Pollard
Approaching soft robot hand design as an inverse optimization problem, where human dexterous manipulation of delicate objects is assumed to be optimal. Our objective function relates to ease of manipulation; the best gripper will be the one for which the demonstrated manipulations come most “naturally.” Our experience shows that the robotic hand design may make certain tasks either
difficult or “easy.” Soft grippers can make dexterous manipulation tasks easier than previously believed.
We propose three approaches to solve the inverse optimization problem: (a) A lazy volume-filling approach allows us to create hand-body designs with material to support contacts; (b) Evolutionary algorithms for tendon routing allow global exploration of the space of body deformations; and (c) User-guided designs allow interaction with the optimizer and lead to greater simplicity and symmetry in the results
Fieldable Intelligent Systems: George Kantor and David Wettergreen
Using an application-oriented approach to systems development, we will leverage prior projects in terms of hardware platforms and software architectures. This will allow us to rapidly port AI developments from the lab to the field, resulting in new fielded capabilities that grow in sophistication and robustness. We will use the Robotic Operating System (ROS) as our primary software integration platform. We will also use existing wheeled ground platforms of various sizes, ranging from the custom-built Robotanist to the much larger Clearpath Warthog, and we will use COTS aerial platforms, such as DJI Matrice 600 Pro hexocopters. We will use COTS manipulators (e.g., UR5 and Franka arms) to demonstrate the proposed intelligent manipulation capabilities. Custom soft grippers will be lab tested and then integrated with mobility platforms for in-field testing and validation.
Closed-loop AI with Formal Guarantees: Arti Singh with other AIIRA investigators
Combining methods for high-dimensional search with adaptive sampling methods that leverage structure of variables (i.e., domain knowledge in terms of algebraic, differential, or network constraints) to enable searching for target multi-trait ideotypes in a high-dimensional space while considering stochastic effects. This approach will utilize the digital twin(s) and historical data to provide a robust search method for (pareto-)optimal multiscale traits (for instance, canopy architecture vs. resource use efficiency) to allow systematic reasoning of trade-offs between yield, inputs, and profitability. The result will develop formal guarantees for these methods, capturing the accuracy-sample complexity trade-off and closed-loop AI gains.
AI Enabled Sensing Innovations: Srinivasa Narasimham
We have developed an affordable and programmable imaging system that can image both surface and subsurface (leaves, stem, crop, flower). An improved version of the Episcan color sensor to capture direct and indirect light transport to image the surface and subsurface structures has been designed. Using structured lighting, the sensor can also be used to reconstruct the 3D shapes of the plants. We designed and developed an approach to backlight the leaves to enhance vein structures during water stress experiments and have found that backlighting provides a clearer view of the structures of thin leaves.
Distributed adaptive sampling: Katia Sycara
Using a two-pronged investigation into online multiagent adaptive sampling for opportunistic data collection as feedback when adapting the digital twin to local conditions. In one research direction, we consider the problem of a team of robots spread out over the field in order to optimize the sensing performance over physiological traits whose distribution is a density function. We propose a distributed mixture of Gaussian processes framework that enables robots to collaboratively learn the global density function by exchanging only model-related parameters, thus increasing communication and computation efficiency. The second direction will utilize recently developed tensor-based methods to perform sampling over high-dimensional data. An example research question using this approach is, How should a (swarm of) UAS plan routes and perform sampling over both space and frequency domains, so that one can use a small number of data samples to represent the global information?