Revisiting Active Perception
TL;DR A history and position paper that defines an active perceiver as an agent that knows why it wants to sense, then chooses what, how, when and where to sense, and organizes 45+ years of work around those five questions. It runs no new experiments and argues that a complete artificial agent must perceive actively.
What they do
A historical review and position paper on active perception. It focuses on vision but is meant to apply to other senses. The authors define an active perceiver as an agent that “knows why it wishes to sense, and then chooses what to perceive, and determines how, when and where to achieve that perception”. They then organize work since SHAKEY (1966-1972) and Tenenbaum’s 1970 thesis around that definition.
How it works
- The active pentuple. Why is an expectation-action tuple derived from the agent’s current state. What is scene selection. How covers mechanical alignment, sensor alignment and priming. When is temporal selection, and Where is viewpoint selection (agent and sensor pose). Table 2 lists the earliest work on each component.
- Why and what. Twelve reasons to move a camera, from fixation and occlusion to disambiguating degenerate views. Attention does the selecting: Kelly’s 1971 face outlines, Garvey’s 1976 indirect search, the saliency-driven Harvard Head and Burt’s foveated tracker.
- How. Robot heads from the Stanford Hand-Eye project through the U Penn, Rochester and KTH heads and TRISH. Active touch uses exploratory procedures: press for hardness, slide for texture, lift for weight.
- When and where. Gaze control, aspect graphs for choosing viewpoints, and sensor planning for visual search.
Results
The paper reports no new experiments. It collects results from earlier work:
- In general, visual matching is NP-complete (Tsotsos 1989), but task knowledge can make it linear. Choosing robot actions that maximize the chance of finding a target is NP-hard (Ye & Tsotsos).
- Under certain conditions, an active hypothesis sieve over an image sequence costs far less than interpreting a single image (Tsotsos 1992). Active vision is not always cheaper, though.
- In 1979, Moravec’s Stanford Cart crossed a chair-filled room without human help in about five hours. It moved in one-meter spurts with 10-15 minute pauses between them.
- A knife detector with 90% accuracy works for image retrieval, but a robot using it would act on the wrong object 10% of the time. The authors suggest using such a detector only to guide attention.
Limitations
The authors call their history selective and say they may have missed important work. Active systems cannot be benchmarked on fixed datasets: they need live environments or a scene generator. Learning is also hard, because the agent’s actions change the statistics of its own input. Table 2 notes that many problems remain open for each component.
Figures
Abstract
Despite the recent successes in robotics, artificial intelligence and computer vision, a complete artificial agent necessarily must include active perception. A multitude of ideas and methods for how to accomplish this have already appeared in the past, their broader utility perhaps impeded by insufficient computational power or costly hardware. The history of these ideas, perhaps selective due to our perspectives, is presented with the goal of organizing the past literature and highlighting the seminal contributions. We argue that those contributions are as relevant today as they were decades ago and, with the state of modern computational tools, are poised to find new life in the robotic perception systems of the next decade.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.