Matt Strong

Matt Strong

CS PhD at Stanford University

← Paper Digest

Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents

Wenlong Huang, Pieter Abbeel, Deepak Pathak, Igor Mordatch

Seminal January 2022

The VirtualHome agent carries out the task "Browse internet".

TL;DR Frozen GPT-3 and Codex models write household task plans one step at a time, and a sentence-embedding model maps each step to the closest action VirtualHome can execute. No model is trained, and executability rises from 18% to 79% for Codex 12B, at some cost in human-judged correctness.

What they do

The paper asks whether large pre-trained language models already know how to break a high-level household task (“make breakfast”) into executable steps (“open fridge”), without any training. In VirtualHome, large GPT-3 and Codex models prompted with one example write plans that human raters find plausible. Most of these plans cannot be executed, because their steps do not match the environment’s action vocabulary or they break its preconditions. The authors add an inference-time procedure that grounds each step in an admissible action, with every model kept frozen.

How it works

  • Planning LM. A causal LLM (GPT-2, GPT-Neo, Codex or GPT-3) is prompted with an example task and its plan written in natural language.
  • Action translation. A Sentence-RoBERTa model embeds each generated step and all 47,522 admissible VirtualHome actions, and the step is replaced by the action with the highest cosine similarity.
  • Autoregressive trajectory correction. The plan is generated one step at a time, and each step is translated before it is appended to the prompt. Candidates are ranked by similarity plus a weighted LM log-probability. Generation stops early when the best score falls below a threshold.
  • Dynamic example selection. The prompt example is the demonstration task, out of 204, whose name is closest to the query task.

Results

  • Tested on 88 held-out tasks in 7 scenes, with correctness rated by 10 MTurk annotators. Vanilla GPT-3 175B plans are judged correct 77.86% of the time, more than human-written plans (70.05%), but only 7.79% of them are executable.
  • Executability rises to 78.57% with Translated Codex 12B (18.07% vanilla) and to 73.05% with Translated GPT-3 175B. Correctness falls to 54.88% and 66.13%.
  • Removing action translation hurts most: Translated Codex 12B drops to 31.49% executability.
  • Plans that are both executable and correct: 35.23% for Translated GPT-3 175B, at most 6.82% for vanilla models, and 65.91% for human plans.

Limitations

Translation lowers correctness. Compound steps map poorly to a single action, and plans sometimes end too early, partly because VirtualHome lacks some of the actions needed. The method grounds plans only at the mid level and assumes a low-level controller exists. Plans ignore observations and feedback from the environment and refer to only one instance of each object. Correctness depends on human judgment; the automatic LCS metric is only a proxy.

Videos

The VirtualHome agent carries out the task "Turn off TV".
The VirtualHome agent carries out the task "Wash face".
Full video (YouTube)

Figures

From Figure 1: Executability vs. semantic correctness of generated plans (left), sample plans from different models (...
From Figure 1: Executability vs. semantic correctness of generated plans (left), sample plans from different models (right) and an example execution in the environment (bottom). Large models write plans close to human ones but often not executable; the proposed techniques raise executability at some cost in correctness.
From Figure 2: A frozen causal LLM breaks a high-level task into mid-level steps (left), a frozen masked LLM translat...
From Figure 2: A frozen causal LLM breaks a high-level task into mid-level steps (left), a frozen masked LLM translates each step into an admissible action (middle), and the translated action is appended to the prompt before the next step is generated (right).
From Figure 4: Percentage of generated plans that are both executable and correct, for each language model.
From Figure 4: Percentage of generated plans that are both executable and correct, for each language model.
Abstract

Can world knowledge learned by large language models (LLMs) be used to act in interactive environments? In this paper, we investigate the possibility of grounding high-level tasks, expressed in natural language (e.g. "make breakfast"), to a chosen set of actionable steps (e.g. "open fridge"). While prior work focused on learning from explicit step-by-step examples of how to act, we surprisingly find that if pre-trained LMs are large enough and prompted appropriately, they can effectively decompose high-level tasks into mid-level plans without any further training. However, the plans produced naively by LLMs often cannot map precisely to admissible actions. We propose a procedure that conditions on existing demonstrations and semantically translates the plans to admissible actions. Our evaluation in the recent VirtualHome environment shows that the resulting method substantially improves executability over the LLM baseline. The conducted human evaluation reveals a trade-off between executability and correctness but shows a promising sign towards extracting actionable knowledge from language models. Website at https://huangwl18.github.io/language-planner

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.