| Volume 24 Issue 1 - Publication Date: 1 January 2005 |
| |
| Interactive Multimodal Robot
Programming |
| |
| S. Iba, P.K. Khosla
The Robotics Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania
15213-3890, USA and C.J.J. Paredis Systems Realization Laboratory,
G. W. Woodruff School of Mechanical Engineering, Georgia Institute of
Technology, Atlanta, Georgia 30332-0405, USA |
| |
| As robots enter the human
environment and come into contact with inexperienced users, they need
to be able to interact with users in a multimodal fashion - keyboard and
mouse are no longer acceptable as the only input modalities. In this paper
we introduce a novel approach for programming robots interactively through
a multimodal interface. The key characteristic of this approach is that
the user can provide feedback interactively at any time - during both
the programming and the execution phase. The framework takes a three-step
approach to the problem: multimodal recognition, intention interpretation,
and prioritized task execution. The multimodal recognition module translates
hand gestures and spontaneous speech into a structured symbolic data stream
without abstracting away the user’s intent. The intention interpretation
module selects the appropriate primitives to generate a task based on
the user's input, the system’s current state, and robot sensor data.
Finally, the prioritized task execution module selects and executes skill
primitives based on the system’s current state, sensor inputs, and
prior tasks. The framework is demonstrated by interactively controlling
and programming a vacuum-cleaning robot. The demonstrations are used to
exemplify the interactive programming and the plan recognition aspect
of the research. |
| |
| Multimedia Key |
= Video |
 = Data |
= Code |
= Image |
|
| |
|
Extension |
Type |
Description |
1 |
|
Example
One: Demonstration of the programming phase. (13.3 MB) |
2 |
|
Example Two:
Demonstration of the execution phase. (11.6 MB) |
3 |
|
Example Three:
Data associated with the plan recognition demonstrations shown in
Figure 10 and 11. (201kb) |
|
| |
| Return
to Contents |
