Tactile Optical Simulation
We built the fully general optical tactile simulation system for GelSight using physics based rendering techniques. We propose physically accurate light models and show in-depth analysis of individual components of our simulation pipeline. The technique used is fully general and can be extended to simulated sensors of arbitrary geometry, surface material and light sources. Our system outperforms previous simulation techniques qualitatively and quantitative on image similarity metrics like SSIM.
Overview
We develop an optical simulation system using physics-based rendering (PBR) techniques. PBR focuses on accurately modeling the physics of light scattering. PBR allows modifying the physical location of optical elements like cameras and lights; changing the deformable surface geometry; changing the optical properties of sensor surfaces (shiny vs diffuse). Therefore, our system can be used as a design tool and simulation software for producing accurate tactile images. Our system is able to successfully capture the variation in color, light intensity and shape at different locations of the elastomer surface.
We propose specific models of light, translucent “Gelcore” and deformable elastomer surface generation process which allows us to simulate GelSight.
Simulated Sensor with physically-accurate components (from left to right bottom row) a) Gelcore b) Light models c) Lights mounted on the Gelcore d) Outer supporting shell
Surface heightfield generation
Experiments
We collected images using our sensor prototype and compare the images generated by our simulation when the object is pressed against the sensor at multiple spatial locations and for arbitrarily complex objects. Please refer to our paper for complete description of the experiments and refer to the supplementary material for additional ablation studies.
Spatial variation: Shapes are pressed on different locations on the deformable sensor surface and multiple surface geometry. The visual highlights the need to model the illumination variation across the sensor surface.
Simulating Complex shapes : Our approach could be used to model arbitrarily complex geometry and apply image processing operations to match the real world surface deformations.
Resources
3D Printable STL files : objects used in our experiments
Image Dataset 1 : real sensor images used for comparing with prior work
Simulated Images 1 : images generated by our method for dataset 1
SSIM maps 1 : ssim maps representing the pixelwise error for dataset 1
Image Dataset 2 : real sensor images used for comparing the performance for various sensor location
Simulated Images 2 : images generated by our method for dataset 2
SSIM maps 2 : ssim maps representing the pixelwise error for dataset 2