RSS 2021 Workshop on

Integrating Planning and Learning

July 12, 2021

Overview

Planning and learning are two fundamental capabilities of robots. While planning seeks to solve a task by synthesizing policies on the fly, learning seeks to obtain solutions from experiences provided by data. The two domains have independently advanced in the past decades and both achieved remarkable success on their most suitable tasks. There is a noticeable trend to combine these two capabilities. Particularly, there are the following complementary directions:

  • Learning to plan: Learning heuristics, sampling distributions, action abstractions, etc. for planning.
  • Planning to learn: Learning from planners; active learning; uncertainty-aware learning, etc.
  • Differentiable planners: Embedding the structure of planners into neural networks to learn both the model and planner parameters end-to-end.
  • Model learning: Learning dynamics and observation models for complex domains and use them for planning.
  • Learning representations: Learning state, observation or belief representations to enable efficient planning with high-dimensional observation spaces.
  • Model augmented RL: Leveraging models to augment learning and achieve sample-efficiency.

However, what is missing is a systematic discussion on where and how these strategies shall be best applied and how they can be seamlessly combined. For example:

  • How can we quantify that a learned model is useful beyond its prediction accuracy, i.e. it enables efficient planning and reasoning?
  • How are the principles of planning to learn and learning to plan different from standard planning and learning principles?
  • How to use learning subcomponents “properly” in planning, desirablly with theoretical guarantees?
  • What meta-problems in learning can be formulated as planning problem?
  • Are differentiable planners merely structural priors for deep RL, or beyond that?

In this workshop, we aim to bring together researchers in the planning and learning sub-domains, to discuss new opportunities to integrate them and to incubate the next level of intelligence.


Thank you for your participation. The workshop has come to an end.


You may now find the recorded videos of each panel in the "Schedule" section.

Speakers

Discussions

Panel I. Learning to plan, differentiable planner, and planning to learn.

  • How are the principles of planning to learn and learning to plan different from standard planning and learning principles?
  • How to use learning subcomponents “properly”: What are the meaningful theoretical properties for the integrated systems?
  • Are differentiable planners merely structural priors for deep RL, or beyond that? How general are differentiable planners? How do they scale up?
  • Meta-problems in learning that can be solved using planning: which data to collect, hyper-parameter search, etc.
  • Using concepts in planning to improve and innovate learning algorithms, and vice versa.

Panel II. Model learning, learning representations, and model-augmented learning.

  • How to learn models that enable efficient reasoning for planning? How to quantify that? Is accuracy/dimensionality a good metric for judging the usefulness of a model for planning?
  • Should models predict in some (abstract) state or perceptual space?
  • Controversial: A differentiable simulator is all you need?
  • What are good object models? Meshes, implicit surfaces, shape primitives?
  • What parts of the world have to be modeled? Object interactions, dynamics, rewards, state reconstruction, discrete vs continuous aspects?

Panel III. New opportunities for integrating planning and learning.

  • What is a model? Do we want classical elementary models or end-to-end models? Or something in between?
  • When do we want to integrate planning and learning? Where to use planning as the main tool, and where to use learning as the main tool?
  • How do we determine what to plan for and what to learn for?
  • What have we achieved till now by integrating planning and learning? What is still lacking?
  • With recent developments of planning and learning sub-domains, what are the new opportunities for planning to learn, learning representations, learning to plan?

Schedule

Time Participants Description
Aleksandra Faust Aleksandra Faust
George Konidaris George Konidaris
Alberto Rodriguez Alberto Rodriguez
Aviv Tamar Aviv Tamar
Moderator:
Panpan Cai Panpan Cai
Moderated Panel I: Learning to plan, differentiable planner, and planning to learn.

Video recording available here.
Igor Mordatch Igor Mordatch
Michael Posa Michael Posa
Nicholas Roy Nicholas Roy
Amy Zhang Amy Zhang
Moderator:
Danny Driess Danny Driess
Moderated Panel II: Model learning, learning representations, and model-augmented learning.

Video recording available here.
Accepted abstracts Spotlight Talks (Group 1)
All participants Poster Session
All participants Social Hang-out
Chelsea Finn Chelsea Finn
Dieter Fox Dieter Fox
David Hsu David Hsu
Leslie Kaelbling Leslie Kaelbling
Moderator:
Russ Tedrake Russ Tedrake
Moderated Panel III: New opportunities for integrating planning and learning.

Video recording available here.
Accepted abstracts Spotlight Talks (Group 2)
All participants Poster Session
All participants Social Hang-out

Accepted Contributions

  • Interaction Prediction and Monte-Carlo Tree Search for Robot Manipulation in Clutter [paper] [video]
    Baichuan Huang, Shuai D. Han, Abdeslam Boularias, Jingjin Yu

  • Information-Theoretic Reward Shaping for Multimodal Object Attribute Learning [paper] [video]
    Xiaohan Zhang, Jivko Sinapov, Shiqi Zhang

  • Safe Exploration for Reinforcement Learning Using Unsupervised Action Planning [paper] [video]
    Hao-Lun Hsu, Qiuhua Huang, Sehoon Ha

  • Imitation Learning-Based Path Generation for the Complex Assembly of Deformable Objects [paper] [video]
    Yitaek Kim, Christoffer Sloth

  • Integrated Commonsense Reasoning and Interactive Learning in Robotics [paper] [video]
    Mohan Sridharan

  • Adversarially Regularized Policy Learning Guided by Trajectory Optimization [paper] [video]
    Zhigen Zhao, Simiao Zuo, Tuo Zhao, Ye Zhao

  • Gaussian Process Constraint Learning for Scalable Safe Motion Planning from Demonstrations [paper] [video]
    Hao Wang, Glen Chou, Dmitry Berenson

  • Policy-Guided Exploration for Efficient Sampling-Based Motion Planning in High Dimensions [video]
    Liam Schramm, Abdeslam Boularias

  • Towards Safe, Abstraction-based Online Learning and Synthesis for Unknown Systems [paper] [video]
    John Jackson, Luca Laurenti, Eric W Frew, Morteza Lahijanian

  • Entropy Regularized Motion Planning via Stein Variational Inference [paper] [video]
    Alexander Lambert, Byron Boots

  • Receding horizon inverse reinforcement learning [paper] [video]
    Wei Gao, Yiqing Xu, David Hsu

Organizers


* Main organizers

Contact

Enquiries: Panpan Cai (dcscaip@nus.edu.sg), Danny Driess (danny.driess@ipvs.uni-stuttgart.de)