In Fiscal Year 2011 (FY11), preparatory work was completed in the form of literature reviews; observations of NASA robot systems; interviews with NASA robotic operators and trainers; and a space HRI workshop. These activities resulted in the selection of three research areas that are the focus of the proposed work. The three research areas are: Video Overlays, Camera Views, and Command Modalities.

Studies proposed in this Directed Research Project in the area of Video Overlays consider two factors in the implementation of augmented reality (AR) for operator displays during teleoperation. The first of these factors is the type of navigational guidance provided by AR symbology. Participants’ performance during teleoperation of a robot arm will be compared when they are provided with command-guidance symbology (i.e., directing the operator what commands to make) or situation-guidance symbology (i.e., providing natural cues so that the operator can infer what commands to make). The second factor to be considered for AR symbology is the effect of overlays that are either superimposed or integrated into the external view of the world. A study is proposed that compares the effects of superimposed and integrated overlays on operator task performance during teleoperated driving tasks.

Studies proposed in the area of Camera Views investigate inclusion/exclusion of a robot within the video feed and camera frame of reference. One study will investigate the effects of including and excluding the robot’s chassis within the video feed presented to operators on path-following and maze traversal task performance. Another study will investigate the effects of the addition of an exocentric camera frame of reference to egocentric frames of reference on operator task performance for these same tasks.

Lastly, studies in the area of Command Modalities will systematically build and evaluate gesture and voice vocabularies for commanding a ground-based mobile robot. The first in this series of studies will have participants produce robot commands for a set of critical control functions. The characteristics of the commands will be analyzed. In a second phase of this study, the strength of association between command and voice/gesture inputs will be evaluated. The next two studies will test the learnability and memorability of the developed vocabularies in the context of a representative task.

The results of the camera view studies will be applicable to mobile robots such as rovers and search and rescue robots.

Finally, the method applied and tested for the development of a gesture and voice command vocabulary can be used for any other system to develop similar kinds of communication systems.

Follow-on studies need to investigate these types of overlays with improved designs and in more realistic environments.

In Fiscal Year 2011 (FY11), preparatory work was completed in the form of literature reviews; observations of NASA robot systems; interviews with NASA robotic operators and trainers; and a space HRI workshop. These activities resulted in the selection of three research areas that are the focus of the proposed work. The three research areas are: Video Overlays, Camera Views, and Command Modalities.

Studies proposed in this Directed Research Project in the area of Video Overlays consider two factors in the implementation of augmented reality (AR) for operator displays during teleoperation. The first of these factors is the type of navigational guidance provided by AR symbology. Participants’ performance during teleoperation of a robot arm will be compared when they are provided with command-guidance symbology (i.e., directing the operator what commands to make) or situation-guidance symbology (i.e., providing natural cues so that the operator can infer what commands to make). The second factor to be considered for AR symbology is the effect of overlays that are either superimposed or integrated into the external view of the world. A study is proposed that compares the effects of superimposed and integrated overlays on operator task performance during teleoperated driving tasks.

Studies proposed in the area of Camera Views investigate inclusion/exclusion of a robot within the video feed and camera frame of reference. One study will investigate the effects of including and excluding the robot’s chassis within the video feed presented to operators on path-following and maze traversal task performance. Another study will investigate the effects of the addition of an exocentric camera frame of reference to egocentric frames of reference on operator task performance for these same tasks.

Lastly, studies in the area of Command Modalities will systematically build and evaluate gesture and voice vocabularies for commanding a ground-based mobile robot. The first in this series of studies will have participants produce robot commands for a set of critical control functions. The characteristics of the commands will be analyzed. In a second phase of this study, the strength of association between command and voice/gesture inputs will be evaluated. The next two studies will test the learnability and memorability of the developed vocabularies in the context of a representative task.

The results of the camera view studies will be applicable to mobile robots such as rovers and search and rescue robots.

Finally, the method applied and tested for the development of a gesture and voice command vocabulary can be used for any other system to develop similar kinds of communication systems.

A second study is looking at superimposed and integrated overlays for teleoperation of a mobile robot using a hand controller. When AR is superimposed on the external world, it appears to be fixed onto the display and internal to the operators’ workstation. Unlike superimposed overlays, integrated overlays often appear as three-dimensional objects and move as if part of the external world. Studies conducted in the aviation domain show that integrated overlays can improve situation awareness and reduce the amount of deviation from the optimal path. The purpose of this ongoing study is to investigate whether these results apply to navigation with a mobile robot.

In Fiscal Year 2011 (FY11), preparatory work was completed in the form of literature reviews; observations of NASA robot systems; interviews with NASA robotic operators and trainers; and a space HRI workshop. These activities resulted in the selection of three research areas that are the focus of the proposed work. The three research areas are: Video Overlays, Camera Views, and Command Modalities.

Studies proposed in this Directed Research Project in the area of Video Overlays consider two factors in the implementation of augmented reality (AR) for operator displays during teleoperation. The first of these factors is the type of navigational guidance provided by AR symbology. Participants’ performance during teleoperation of a robot arm will be compared when they are provided with command-guidance symbology (i.e., directing the operator what commands to make) or situation-guidance symbology (i.e., providing natural cues so that the operator can infer what commands to make). The second factor to be considered for AR symbology is the effect of overlays that are either superimposed or integrated into the external view of the world. A study is proposed that compares the effects of superimposed and integrated overlays on operator task performance during teleoperated driving tasks.

Studies proposed in the area of Camera Views investigate inclusion/exclusion of a robot within the video feed and camera frame of reference. One study will investigate the effects of including and excluding the robot’s chassis within the video feed presented to operators on path-following and maze traversal task performance. Another study will investigate the effects of the addition of an exocentric camera frame of reference to egocentric frames of reference on operator task performance for these same tasks.

Lastly, studies in the area of Command Modalities will systematically build and evaluate gesture and voice vocabularies for commanding a ground-based mobile robot. The first in this series of studies will have participants produce robot commands for a set of critical control functions. The characteristics of the commands will be analyzed. In a second phase of this study, the strength of association between command and voice/gesture inputs will be evaluated. The next two studies will test the learnability and memorability of the developed vocabularies in the context of a representative task.

The first study focused on video overlays, and investigated how Augmented Reality (AR) symbology can be added to the human-robot interface to improve teleoperation performance. Three types of AR symbology were explored in this study, command guidance (CG), situation guidance (SG), and both (SCG). CG symbology gives operators explicit instructions on what commands to input, whereas SG symbology gives operators implicit cues so that operators can infer the input commands. The combination of CG and SG provided operators with explicit and implicit cues allowing the operator to choose which symbology to utilize. The objective of the study was to understand how AR symbology affects the human operator’s ability to align a robot arm to a target using a joystick and the ability to allocate attention between the symbology and external views of the world. The study evaluated the effect of type of symbology (CG and SG) on operator task performance and attention allocation during teleoperation of a robot arm.

Planned studies for the near future:

The second study will expand on the first study by evaluating the effect of type of navigational guidance (CG and SG) on operator task performance and attention allocation during teleoperation of a robot arm through uplinked, manually entered commands. Although this study complements the first study on navigational guidance with hand controllers, it is a separate investigation due to the distinction in intended operators (i.e., crewmembers versus ground-operators).

A third study will look at superimposed and integrated overlays for teleoperation of a mobile robot using a hand controller. When AR is superimposed on the external world, it appears to be fixed onto the display and internal to the operators’ workstation. Unlike superimposed overlays, integrated overlays often appear as three-dimensional objects and move as if part of the external world. Studies conducted in the aviation domain show that integrated overlays can improve situation awareness and reduce the amount of deviation from the optimal path. The purpose of the study is to investigate whether these results apply to HRI tasks, such as navigation with a mobile robot.

HRP Gaps:

This HRI research contributes to closure of HRP gaps by providing information on how display and control characteristics--those related to guidance, feedback, and command modalities--affect robot operator performance. The overarching goals are to improve interface usability, reduce operator error, and develop candidate guidelines to design effective human-robot interfaces.

In Fiscal Year 2011 (FY11), preparatory work was completed in the form of literature reviews; observations of NASA robot systems; interviews with NASA robotic operators and trainers; and a space HRI workshop. These activities resulted in the selection of three research areas that are the focus of the proposed work. The three research areas are: Video Overlays, Camera Views, and Command Modalities.

Studies proposed in this Directed Research Project in the area of Video Overlays consider two factors in the implementation of augmented reality (AR) for operator displays during teleoperation. The first of these factors is the type of navigational guidance provided by AR symbology. Participants’ performance during teleoperation of a robot arm will be compared when they are provided with command-guidance symbology (i.e., directing the operator what commands to make) or situation-guidance symbology (i.e., providing natural cues so that the operator can infer what commands to make). The second factor to be considered for AR symbology is the effect of overlays that are either superimposed or integrated into the external view of the world. A study is proposed that compares the effects of superimposed and integrated overlays on operator task performance during teleoperated driving tasks.

Studies proposed in the area of Camera Views investigate inclusion/exclusion of a robot within the video feed and camera frame of reference. One study will investigate the effects of including and excluding the robot’s chassis within the video feed presented to operators on path-following and maze traversal task performance. Another study will investigate the effects of the addition of an exocentric camera frame of reference to egocentric frames of reference on operator task performance for these same tasks.

Lastly, studies in the area of Command Modalities will systematically build and evaluate gesture and voice vocabularies for commanding a ground-based mobile robot. The first in this series of studies will have participants produce robot commands for a set of critical control functions. The characteristics of the commands will be analyzed. In a second phase of this study, the strength of association between command and voice/gesture inputs will be evaluated. The next two studies will test the learnability and memorability of the developed vocabularies in the context of a representative task.

[Ed. note 9/17/13: added to Task Book when received information from HRP]