Inferring 3D Shapes of Unknown Rigid Objects in Clutter Through Inverse Physics Reasoning

Changkyu Song, Abdeslam Boularias

Research output: Contribution to journalArticle

Abstract

We present a probabilistic approach for building, on the fly, three dimensional (3D) models of unknown objects while being manipulated by a robot. We specifically consider manipulation tasks in piles of clutter that contain previously unseen objects. Most manipulation algorithms for performing such tasks require known geometric models of the objects in order to grasp or rearrange them robustly. One of the novel aspects of this work is the utilization of a physics engine for verifying hypothesized geometries in simulation. The evidence provided by physics simulations is used in a probabilistic framework that accounts for the fact that mechanical properties of the objects are uncertain. We present an efficient algorithm for inferring occluded parts of objects based on their observed motions and mutual interactions. Experiments using a robot show that this approach is efficient for constructing physically realistic 3D models, which can be useful for manipulation planning. Experiments also show that the proposed approach significantly outperforms alternative approaches in terms of shape accuracy.

Original languageEnglish (US)
Article number8567926
Pages (from-to)201-208
Number of pages8
JournalIEEE Robotics and Automation Letters
Volume4
Issue number2
DOIs
StatePublished - Apr 1 2019

Fingerprint

3D shape
Clutter
Physics
Reasoning
Unknown
Manipulation
Robots
3D Model
Robot
Piles
Experiments
Engines
Geometric Model
Probabilistic Approach
Planning
Mechanical properties
Geometry
Mechanical Properties
Experiment
Simulation

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Human-Computer Interaction
  • Biomedical Engineering
  • Mechanical Engineering
  • Control and Optimization
  • Artificial Intelligence
  • Computer Science Applications
  • Computer Vision and Pattern Recognition

Cite this

@article{c480601a4b5d4c1c941af93c32517962,
title = "Inferring 3D Shapes of Unknown Rigid Objects in Clutter Through Inverse Physics Reasoning",
abstract = "We present a probabilistic approach for building, on the fly, three dimensional (3D) models of unknown objects while being manipulated by a robot. We specifically consider manipulation tasks in piles of clutter that contain previously unseen objects. Most manipulation algorithms for performing such tasks require known geometric models of the objects in order to grasp or rearrange them robustly. One of the novel aspects of this work is the utilization of a physics engine for verifying hypothesized geometries in simulation. The evidence provided by physics simulations is used in a probabilistic framework that accounts for the fact that mechanical properties of the objects are uncertain. We present an efficient algorithm for inferring occluded parts of objects based on their observed motions and mutual interactions. Experiments using a robot show that this approach is efficient for constructing physically realistic 3D models, which can be useful for manipulation planning. Experiments also show that the proposed approach significantly outperforms alternative approaches in terms of shape accuracy.",
author = "Changkyu Song and Abdeslam Boularias",
year = "2019",
month = "4",
day = "1",
doi = "10.1109/LRA.2018.2885579",
language = "English (US)",
volume = "4",
pages = "201--208",
journal = "IEEE Robotics and Automation Letters",
issn = "2377-3766",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "2",

}

Inferring 3D Shapes of Unknown Rigid Objects in Clutter Through Inverse Physics Reasoning. / Song, Changkyu; Boularias, Abdeslam.

In: IEEE Robotics and Automation Letters, Vol. 4, No. 2, 8567926, 01.04.2019, p. 201-208.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Inferring 3D Shapes of Unknown Rigid Objects in Clutter Through Inverse Physics Reasoning

AU - Song, Changkyu

AU - Boularias, Abdeslam

PY - 2019/4/1

Y1 - 2019/4/1

N2 - We present a probabilistic approach for building, on the fly, three dimensional (3D) models of unknown objects while being manipulated by a robot. We specifically consider manipulation tasks in piles of clutter that contain previously unseen objects. Most manipulation algorithms for performing such tasks require known geometric models of the objects in order to grasp or rearrange them robustly. One of the novel aspects of this work is the utilization of a physics engine for verifying hypothesized geometries in simulation. The evidence provided by physics simulations is used in a probabilistic framework that accounts for the fact that mechanical properties of the objects are uncertain. We present an efficient algorithm for inferring occluded parts of objects based on their observed motions and mutual interactions. Experiments using a robot show that this approach is efficient for constructing physically realistic 3D models, which can be useful for manipulation planning. Experiments also show that the proposed approach significantly outperforms alternative approaches in terms of shape accuracy.

AB - We present a probabilistic approach for building, on the fly, three dimensional (3D) models of unknown objects while being manipulated by a robot. We specifically consider manipulation tasks in piles of clutter that contain previously unseen objects. Most manipulation algorithms for performing such tasks require known geometric models of the objects in order to grasp or rearrange them robustly. One of the novel aspects of this work is the utilization of a physics engine for verifying hypothesized geometries in simulation. The evidence provided by physics simulations is used in a probabilistic framework that accounts for the fact that mechanical properties of the objects are uncertain. We present an efficient algorithm for inferring occluded parts of objects based on their observed motions and mutual interactions. Experiments using a robot show that this approach is efficient for constructing physically realistic 3D models, which can be useful for manipulation planning. Experiments also show that the proposed approach significantly outperforms alternative approaches in terms of shape accuracy.

UR - http://www.scopus.com/inward/record.url?scp=85063304865&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85063304865&partnerID=8YFLogxK

U2 - 10.1109/LRA.2018.2885579

DO - 10.1109/LRA.2018.2885579

M3 - Article

AN - SCOPUS:85063304865

VL - 4

SP - 201

EP - 208

JO - IEEE Robotics and Automation Letters

JF - IEEE Robotics and Automation Letters

SN - 2377-3766

IS - 2

M1 - 8567926

ER -