Menu

 

The NASA Task Book
Advanced Search     

Project Title:  Advanced Displays for Efficient Training and Operation of Robotic Systems Reduce
Fiscal Year: FY 2011 
Division: Human Research 
Research Discipline/Element:
HRP SHFH:Space Human Factors & Habitability (archival in 2017)
Start Date: 09/01/2007  
End Date: 09/30/2011  
Task Last Updated: 02/14/2012 
Download report in PDF pdf
Principal Investigator/Affiliation:   Oman, Charles M. Ph.D. / Massachusetts Institute of Technology 
Address:  Department of Aeronautics and Astronautics 
77 Massachusetts Avenue 37-219 
Cambridge , MA 02139-4301 
Email: coman@mit.edu 
Phone: 617-253-7508  
Congressional District:
Web:  
Organization Type: UNIVERSITY 
Organization Name: Massachusetts Institute of Technology 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Young, Laurence  Massachusetts Institute of Technology 
Natapoff, Alan  Massachusetts Institute of Technology 
Liu, Andrew  Massachusetts Institute of Technology 
Tomlinson, Zakiya  NASA Johnson Space Center 
Project Information: Grant/Contract No. NCC 9-58-SA01301 
Responsible Center: NSBRI 
Grant Monitor:  
Center Contact:   
Solicitation / Funding Source: 2007 NSBRI-RFA-07-01 Human Health in Space 
Grant/Contract No.: NCC 9-58-SA01301 
Project Type: GROUND 
Flight Program:  
TechPort: No 
No. of Post Docs:
No. of PhD Candidates:
No. of Master's Candidates:
No. of Bachelor's Candidates:
No. of PhD Degrees:
No. of Master's Degrees:
No. of Bachelor's Degrees:
Human Research Program Elements: (1) SHFH:Space Human Factors & Habitability (archival in 2017)
Human Research Program Risks: (1) HARI:Risk of Inadequate Design of Human and Automation/Robotic Integration
(2) Sensorimotor:Risk of Altered Sensorimotor/Vestibular Function Impacting Critical Mission Tasks (Revised as of IRP Rev M)
Human Research Program Gaps: (1) SHFE-HARI-02:We need to develop design guidelines for effective human-automation-robotic systems in operational environments that may include distributed, non-colocated adaptive mixed-agent teams with variable transmission latencies (IRP Rev F) (Previously: How can performance, efficiency, and safety guidelines be developed for effective information sharing between humans and automation, such that appropriate trust and situation awareness is maintained?)
(2) SM12:Develop standards for spaceflight cockpit displays and inputs. (OBSOLETE - Merged with SM6 to create SM6.1, per IRP Rev F)
Task Description: The long term objectives of this 4 year NSBRI Sensorimotor Team project addressed three specific aims related to astronaut performance during space telerobotics training. We collaborated with the JSC Robotic Systems Training Group (DX-2). Astronaut robotics trainees vary significantly in their initial performance, ability, learning rate, and level of mastery. Because the process of training astronauts to be qualified robotics operators is so long and expensive, NASA needs tools to predict performance and customize training. Our principal scientific goal has been to understand how individual differences in spatial and bimanual control abilities impact learning and performance.

Aim 1. Astronaut applicants currently take an Aptitude for Robotics Test (ART) and those selected proceed to Generic Robotics Training (GRT). Using a logistic modeling approach we investigated how well an astronaut's ART scores and an additional set of mental rotation, perspective taking and visualization tests predicted spatial performance in subsequent training. We found ART was not a reliable predictor and proposed changes in ART metrics to improve the predictive power. These were implemented and used in the last round of astronaut testing and GRT training. Logistic regression analysis of mental rotation and visualization scores allowed us to predict who will achieve a top score in qualification evaluations, but not those who fail (partly because very few do). Model predictions were reliable enough to use in customization of regular and remedial training, but not to make career defining decisions.

Aim 2: Our second objective was to study performance and learning using a standardized pedagogy in controlled laboratory settings using a space telerobotics training simulator at MIT. The simulator recreated the BORIS training environment used in GRT, and eventually also the ISS docked Shuttle and Progress rendezvous spacecraft environment. In a series of five previous experiments, we consistently found that a trainee's early performance and learning in relatively simple GRT-like "fly-to" and pregrapple tasks correlated with their spatial abilities. We believe this is because mental rotation and visualization abilities are important for integrating the multiple video camera views used when performing robotics tasks. Robotics operators are encouraged to move the arm on more than one axis at a time, beginning in the earliest phases of training. This year we conducted an experiment to see whether multi-axis movement feedback improved learning, and whether real time visual, real time aural or post-trial feedback was best. We studied the performance of 16 subjects learning to perform a specific fly-to telerobotics task (Forman, et al. 2011). Performance (movement time, % multi-axis movement, % bimanual movement) improved with practice as expected, but improvement differed between subjects. Subjects preferred the real time visual feedback modality. However we were unable to show reliable effects of the particular display modality options we tested. Nonetheless, the development and evaluation of quantiative fly-to task performance metrics has been valuable. We believe quantitative performance metrics based on our research should be built into both the JSC Dynamic Skills Trainer (DST) and the inflight ROBoT laptop trainer used by ISS astronauts for training in orbit.

Aim 3: In space telerobotic tasks, the right hand controls rotational velocity and the left hand controls translational velocity, each with a unique three-axis control stick (interceptor). This arrangement compels the operator to decompose the desired 6 DOF arm motion into a pair of 3D rotational and translational components, and execute them with different hands, but in coordinated fashion. Unfortunately, simultaneous movement of both hands can result in undesired inter-manual neural cross-coupling, as exemplified by the difficulty of the pat your head while rubbing your stomach task. In addition to inter-manual cross coupling due to neuromotor causes, intra-manual CC within interceptors can occur due to joystick mechanical design, which can cause the operator to have difficulties when attempting to actuate a single axis. To quantify effects and separate inter- and intra-manual effects, this year we developed a Bimanual Cross-Coupling Test (BCCT). Our BCCT consists of six trials of a 3D simulated tracking task projected onto a 2D display. The subject uses a pair of interceptors to control the cursor in tracking the target, which for each trial moves in a different combination of two of the six axes, always with one axis per hand. Each hand thus tracks a pseudorandom signal with frequency content separable from that of the other hand. Through Fourier analysis on all six axes for each trial and compiling the information from the entire forty-minute test, we generate a six-by-six matrix of coupling percentages between each pair of axes. An eighteen-subject study with three sessions over two weeks per subject was conducted to validate the BCCT. Inter-manual cross coupling (CC) exhibited learning effects across sessions, and was significantly different across subjects. Principal component analysis combined with matrix-wise elimination of first-order indirect effects allowed us to characterize intermanual CC by a linear combination of yaw to lateral translation and pitch to vertical translation, which is often greater than 10% even among fully trained subjects. Intra-manual CC was characterized by high roll to yaw coupling, indicating poor ergonomics of the joystick used – this study should be repeated with NASA's rotational control interceptor, which has different ergonomics due to the handle angle and pitch axis location. This final report also includes a 4 year overview of research results.

Research Impact/Earth Benefits: Our goal is to improve the efficiency of robotic training via improvement of current pedagogies and performance metrics. Improved training methods provide a framework for designing future in-flight training procedures during long duration missions. The project will also demonstrate how individual differences in spatial and manual control skills affect performance of critical operational skills, including complex robotics tasks associated with post-Shuttle era ISS operations. The project results also inform the design of telerobotic displays and bimanual control inceptors (control sticks). The Bimanual Cross Coupling Test method for quantifying neural cross coupling between hands during movement has potential research and clinical applications.

Task Progress & Bibliography Information FY2011 
Task Progress: Spatial Ability Tests Predict Training Performance

To see whether spatial ability scores predicted an astronaut's performance in their initial robotics training course we have tested the spatial abilities of 50 current astronauts (10 astronauts added this year) who had finished at least one robotics training course. We found a significant correlation between the astronauts' spatial test scores and performance in their first robotics course using logistic regression models. We concluded that our spatial ability test score predictors are suitable for identifying trainees who may require extra or remedial sessions.

Multiaxis Feedback during Fly-To Training

Robotics operators are trained to move the arm on more than one axis. We conducted a second experiment to see whether real time visual, real time aural, or post-trial feedback of multi axis was best when 16 subjects learning to perform a specific fly-to telerobotics task. Performance improved with practice as expected, but differed between subjects. Subjects preferred the real time visual feedback, but no reliable performance differences of feedback mode were found. Nonetheless, we believe providing quantitative metrics of other aspects of performance during training will assist both trainees and trainers in improving performance.

Bimanual Cross-Coupling Test

In typical telerobotic tasks, the right hand controls rotational velocity and the left hand controls translational velocity, each with a unique three-axis control stick (interceptor). Simultaneous motion of both hands can result in undesired motions through cross-coupling (CC), as shown by the pat your head and rub your stomach task. In space telerobotics, in addition to intermanual CC between interceptors due to these neuromotor causes, intramanual CC within interceptors occurs due to ergonomic difficulties in isolating actuation to a single axis. We developed a Bimanual Cross-Coupling Test (BCCT) to quantify intra- and inter-manual CC effects. The BCCT consists of six trials of a 3D simulated tracking task projected onto a 2D display. The subject uses a pair of interceptors to control the cursor in tracking the target, which for each trial moves in a different combination of two of the six axes, always with one axis per hand. Each hand thus tracks a pseudorandom signal with frequency content separable from that of the other hand. An 18-subject study with three sessions over two weeks per subject was conducted. Intermanual CC exhibited learning effects across sessions, and was significantly different across subjects. Principal component analysis allowed us to characterize intermanual CC by a linear combination of yaw to lateral translation and pitch to vertical translation, which is often greater than 10% even among trained subjects. Intramanual CC was characterized by high roll to yaw coupling, indicating poor ergonomics of the joystick used.

Bibliography Type: Description: (Last Updated: 01/17/2020) 

Show Cumulative Bibliography Listing
 
Abstracts for Journals and Proceedings Liu AM, Wang V, Forman RE, Galvan RC, Natapoff N, Oman CM. "Advanced Displays for Efficient Training and Operation of Robotic Systems." 2012 NASA Human Research Program Investigators’ Workshop, Houston, TX, Feb 14-16, 2012.

2012 NASA Human Research Program Investigators’ Workshop, Houston, TX, Feb 14-16, 2012. , Feb-2012

Abstracts for Journals and Proceedings Pontillo TM, Liu AM, Natapoff A, Oman CM. "Spatial ability, joystick configuration, and handedness as predictors of space teleoperation performance." 18th IAA Humans in Space Symposium, Houston, TX, April 11-15, 2011.

18th IAA Humans in Space Symposium, Houston, TX, April 11-15, 2011. , Apr-2011

Abstracts for Journals and Proceedings Forman RE, Lowenthal CS, Oman CM, Liu AM, Natapoff A. "A review of space robotics metrics and proposed performance metrics for improved telerobotics training." 18th IAA Humans in Space Symposium, Houston, TX, April 11-15, 2011.

18th IAA Humans in Space Symposium, Houston, TX, April 11-15, 2011. Abstract and poster 2241. , Apr-2011

Articles in Peer-reviewed Journals Liu AM, Oman CM, Galvan R, Natapoff A. "Predicting space telerobotic operator performance from human spatial ability assessments." Updated (n=50) and submitted to Acta Astronautica, November 2011. , Nov-2011
Awards Oman CM. "International Academy of Astronautics. Elected to full membership, April 2011." Apr-2011
Dissertations and Theses Forman RE. "Objective performance metrics for improved space telerobotics training." SM dissertation, Massachusetts Institute of Technology, Department of Aeronautics and Astronautics, September 2011. , Sep-2011
Project Title:  Advanced Displays for Efficient Training and Operation of Robotic Systems Reduce
Fiscal Year: FY 2010 
Division: Human Research 
Research Discipline/Element:
HRP SHFH:Space Human Factors & Habitability (archival in 2017)
Start Date: 09/01/2007  
End Date: 08/31/2011  
Task Last Updated: 09/14/2010 
Download report in PDF pdf
Principal Investigator/Affiliation:   Oman, Charles M. Ph.D. / Massachusetts Institute of Technology 
Address:  Department of Aeronautics and Astronautics 
77 Massachusetts Avenue 37-219 
Cambridge , MA 02139-4301 
Email: coman@mit.edu 
Phone: 617-253-7508  
Congressional District:
Web:  
Organization Type: UNIVERSITY 
Organization Name: Massachusetts Institute of Technology 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Young, Laurence  Massachusetts Institute of Technology 
Natapoff, Alan  Massachusetts Institute of Technology 
Liu, Andrew  Massachusetts Institute of Technology 
Tomlinson, Zakiya  NASA Johnson Space Center 
Project Information: Grant/Contract No. NCC 9-58-SA01301 
Responsible Center: NSBRI 
Grant Monitor:  
Center Contact:   
Solicitation / Funding Source: 2007 NSBRI-RFA-07-01 Human Health in Space 
Grant/Contract No.: NCC 9-58-SA01301 
Project Type: GROUND 
Flight Program:  
TechPort: No 
No. of Post Docs:
No. of PhD Candidates:
No. of Master's Candidates:
No. of Bachelor's Candidates:
No. of PhD Degrees:
No. of Master's Degrees:
No. of Bachelor's Degrees:
Human Research Program Elements: (1) SHFH:Space Human Factors & Habitability (archival in 2017)
Human Research Program Risks: (1) HARI:Risk of Inadequate Design of Human and Automation/Robotic Integration
(2) Sensorimotor:Risk of Altered Sensorimotor/Vestibular Function Impacting Critical Mission Tasks (Revised as of IRP Rev M)
Human Research Program Gaps: (1) SHFE-HARI-02:We need to develop design guidelines for effective human-automation-robotic systems in operational environments that may include distributed, non-colocated adaptive mixed-agent teams with variable transmission latencies (IRP Rev F) (Previously: How can performance, efficiency, and safety guidelines be developed for effective information sharing between humans and automation, such that appropriate trust and situation awareness is maintained?)
(2) SM12:Develop standards for spaceflight cockpit displays and inputs. (OBSOLETE - Merged with SM6 to create SM6.1, per IRP Rev F)
Task Description: The long term objectives of this 4 year NSBRI Sensorimotor Team project address three specific aims related to astronaut performance during space telerobotics training. We are collaborating with the JSC Robotic Systems Training Group (DX-2). The project is in its third year. Astronaut robotics trainees vary significantly in their initial performance, ability, learning rate, and level of mastery. Because the process of training astronauts to be qualified robotics operators is so long and expensive, NASA needs tools to predict performance and customize training. Our scientific goal has been to understand how individual differences in spatial and manual control abilities impact learning and performance.

Aim 1. Astronaut candidates currently take an "Aptitude for Robotics Test" (ART) and those selected proceed to Generic Robotics Training (GRT). Using a logistic modeling approach we investigated how well an astronaut's ART scores and an additional set of mental rotation, perspective taking and visualization tests predicted spatial performance in subsequent training. We found ART was not a reliable predictor and proposed changes in ART metrics to improve the predictive power. These were implemented and used in the current round of astronaut testing and GRT training. During the coming year we plan to re-evaluate ART using this new data. Logistic regression analysis of mental rotation and visualization scores allows us to predict who will achieve a top score in qualification evaluations, but not those who fail (partly because very few do). Model predictions are reliable enough to use in customization of regular and remedial training, but not to make career defining decisions. Additional GRT and spatial ability data is also being obtained for analysis this year, in collaboration with the JSC Robotics Training Branch.

Aim 2: Our second objective has been to study performance and learning in a controlled laboratory setting using a space telerobotics training simulator at MIT. The simulator recreates the BORIS training environment used in GRT, and also the ISS environment. In a series of three previous experiments, we consistently found that a trainee's early performance and learning in relatively simple GRT-like "fly-to" and pregrapple tasks correlate with their spatial abilities. We believe this is because mental rotation and visualization abilities are important for integrating the multiple video camera views used when performing robotics tasks. This year we completed two more experiments. In our prior research, camera configurations were controlled by the experimenter. In reality, cameras are selected by the primary or secondary operator. In the first experiment (n=21) we found that while acting as a secondary operator, camera selection performance and ability to identify arm clearance issues was also correlated with the individual subject's (Vandenberg) mental rotation, (Purdue Spatial) visualization, and (Kozhevnikov 2D) perspective taking ability. A second experiment (n=20) studied the effects of spatial ability, handedness, and joystick configuration on "fly-to" performance requiring multi-axis movements. Like spacecraft, the Shuttle and ISS robotic arms are always controlled using a 3 DOF rotational controller in the right hand, and a 3DOF translational controller on the left. Current hand preference theories (e.g. Guiard) suggest that right handed astronauts should be at a particular advantage with this physical hand controller arrangement. As in our prior experiments, we found that spatial ability scores predicted task performance. However to our surprise we found no large or consistent effect of handedness (Edinburgh questionnaire), or whether the RHC was in the dominant or non-dominant hand, even in the early stage of training. As when using an "Etch-A-Sketch", robotics operators must learn to must parse different spatial axes to different hands. (Happily for left handed astronauts), right handed operators apparently have no significant advantage in this regard.

Aim 3: So far our MIT experiments have studied performance only during the first 1-2 days of training while operators perform relatively simple fly-to, camera selection, and clearance monitoring tasks. Are spatial abilities as important during the later phases of GRT and more advanced training? Based on discussions with the Robotics Training Branch, this year we developed a new series of training protocols and performance metrics in order to study performance during more advanced tasks. New experimental scenarios developed include grapple, loaded-fly-to, autosequencing and free-floating payload track-and-capture. (With the impending retirement of ISS, astronauts must be able to capture drifting HTV logistic supply vehicle.) We have also developed a side task to assess workload. Trainees are able to perform complete sequences of robotic tasks, and the more realistic protocols will allow us to study the process of complex skill acquisition under multi-day advanced training scenarios, and quantify spatial ability effects. Also, in a companion study being initiated this year (NSBRI project NBPF02001) we plan to assess the effects of fatigue and sleep deprivation.

Research Impact/Earth Benefits: Our goal is to improve the efficiency of robotic training via improvement of current pedagogies and development of new teaching tools. Improved training methods provide a framework for designing future in-flight training procedures during long duration missions. The project will also demonstrate how individual differences in spatial and manual control skills affect performance of critical operational skills, including complex robotics tasks associated with post-Shuttle era ISS operations

Task Progress & Bibliography Information FY2010 
Task Progress: Overall our project is running ahead of original schedule. All members of our MIT scientific team have taken part or substantially all of the JSC GRT course. One (Tomlinson) completed her graduate work at MIT, and joined the JSC DX2 group as a robotics trainer.

Aim 1: Completed logistic model analysis of spatial ability and JSC-ART data on 40 NASA astronauts to predict GRT scores. Results presented at NASA HRP-BIW and ASMA, and manuscript submitted. ART scoring changes implemented by JSC. Over the past 10 months we have received a partial data set for the 14 astronauts in the 2009 class : ART data is complete, 10 with spatial ability data, 3 with GRT data (most have completed GRT - data coming).

Aim 2: Developed MIT robotic skills trainer, developed several new BORIS and ISS tasks, and completed 5 experiments (n=121). This year completed two new experiments: a) effects of spatial skills on secondary operator camera selection performance (n=20) and b) effects of spatial ability, joystick configuration and subject handedness during fly-to tasks. Both experiments confirmed strong spatial ability effects. Results are described in a Master's Thesis (June 10) and in articles in preparation. To our surprise, we found no reliable effects of handedness and joystick arrangement (left vs right hand). Camera selection results were presented at ASMA (May 10 poster and abstract). Our MIT Robotic Skills Trainer was demonstrated at the NSBRI booth at ASMA. We have advised two other NASA-HRP projects (PIs Steven Moore and Angelia Sebok) on metrics and robotics simulation platforms. At the request of the Boston Museum of Science we recently developed a stand-alone version of our robotic skills training software for an upcoming (late 2010) museum exhibit.

Aim 3: So far our MIT experiments have addressed performance only during the early stages of training, and while trainees perform relatively simple fly-to, camera selection, and clearance monitoring tasks. The relationship between early performance and individual spatial ability metrics has been consistently demonstrated. Based on discussions with the Robotics Training Branch this year we developed training protocols and new performance metrics to study complex skill development during more advanced phases of training, including grapple, loaded-fly-to, autosequencing and free-floating payload track-and-capture. We are also incorporating a side task workload metric. These protocols will allow us to quantify spatial ability effects on ultimate performance on advanced tasks. We also plan to leverage these developments by incorporating them into a new collaborative study - initiated this year - on the effects of fatigue on robotics performance. We may also be able to incorporate our performance metrics into JSC training and even onboard ISS robotic software as a training aid.

Bibliography Type: Description: (Last Updated: 01/17/2020) 

Show Cumulative Bibliography Listing
 
Abstracts for Journals and Proceedings Oman CM, Liu AM, Natapoff A, Tomlinson ZA, Pontillo TM. "Influence of spatial abilities and fatigue on space telerobotics operator performance." Aerospace Medical Association 81st Annual Meeting, Phoenix Ariz., May 10, 2010.

Aviat Space Environ Med 2010 Mar;81(3):214. , Mar-2010

Abstracts for Journals and Proceedings Pontillo TM, Oman CM, Liu AM, Natapoff A, Tomlinson ZA. "Role of spatial ability in camera selection for space teleoperation tasks." Aerospace Medical Association 81st Annual Meeting Phoenix, Ariz., May 13, 2010.

Aviat Space Environ Med 2010 Mar;81(3):255. , Mar-2010

Articles in Peer-reviewed Journals Liu AM, Oman CM, Natapoff A. "Predicting space telerobotic operator performance from human spatial ability assessments." Human Factors, Submitted, 2009. (not yet published as of September 2010--ed.) , Sep-2010
Dissertations and Theses Pontillo TM. "Spatial ability and handedness as potential predictors of space teleoperation performance." Dissertation, Massachusetts Institute of Technology, June 2010. , Jun-2010
Project Title:  Advanced Displays for Efficient Training and Operation of Robotic Systems Reduce
Fiscal Year: FY 2009 
Division: Human Research 
Research Discipline/Element:
HRP SHFH:Space Human Factors & Habitability (archival in 2017)
Start Date: 09/01/2007  
End Date: 08/31/2011  
Task Last Updated: 09/15/2009 
Download report in PDF pdf
Principal Investigator/Affiliation:   Oman, Charles M. Ph.D. / Massachusetts Institute of Technology 
Address:  Department of Aeronautics and Astronautics 
77 Massachusetts Avenue 37-219 
Cambridge , MA 02139-4301 
Email: coman@mit.edu 
Phone: 617-253-7508  
Congressional District:
Web:  
Organization Type: UNIVERSITY 
Organization Name: Massachusetts Institute of Technology 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Young, Laurence  Massachusetts Institute of Technology 
Natapoff, Alan  Massachusetts Institute of Technology 
Liu, Andrew  Massachusetts Institute of Technology 
Project Information: Grant/Contract No. NCC 9-58-SA01301 
Responsible Center: NSBRI 
Grant Monitor:  
Center Contact:   
Solicitation / Funding Source: 2007 NSBRI-RFA-07-01 Human Health in Space 
Grant/Contract No.: NCC 9-58-SA01301 
Project Type: GROUND 
Flight Program:  
TechPort: No 
No. of Post Docs:
No. of PhD Candidates:
No. of Master's Candidates:
No. of Bachelor's Candidates:
No. of PhD Degrees:
No. of Master's Degrees:
No. of Bachelor's Degrees:
Human Research Program Elements: (1) SHFH:Space Human Factors & Habitability (archival in 2017)
Human Research Program Risks: (1) HARI:Risk of Inadequate Design of Human and Automation/Robotic Integration
(2) Sensorimotor:Risk of Altered Sensorimotor/Vestibular Function Impacting Critical Mission Tasks (Revised as of IRP Rev M)
Human Research Program Gaps: (1) SHFE-HARI-02:We need to develop design guidelines for effective human-automation-robotic systems in operational environments that may include distributed, non-colocated adaptive mixed-agent teams with variable transmission latencies (IRP Rev F) (Previously: How can performance, efficiency, and safety guidelines be developed for effective information sharing between humans and automation, such that appropriate trust and situation awareness is maintained?)
(2) SM12:Develop standards for spaceflight cockpit displays and inputs. (OBSOLETE - Merged with SM6 to create SM6.1, per IRP Rev F)
Task Description: The long term objectives of this 4 year NSBRI Sensorimotor Team project address three specific aims related to human performance during space telerobotics training. We are collaborating with the JSC Robotic Systems Training Group (DX-2). The project is in its second year.

Aim 1. Our goal is to improve NASA teleoperation training efficiency by scientifically customizing remedial training based on the measured spatial abilities of individual astronauts. Astronaut robotics trainees vary significantly in their initial performance, ability, learning rate, and level of mastery. Because the process of training astronauts to be qualified robotics operators is so long and expensive, NASA needs tools to predict robotics performance prior to training. NASA’s existing “Aptitude for Robotics Test” (ART) had not been statistically validated. Using a logistic modeling approach we investigated how well an astronaut’s ART scores predicted their spatial performance in subsequent evaluation testing (either Shuttle PDRS or Generic Robotics Training). ART was not found to be a reliable predictor. Based on our data analysis, we proposed changes in ART performance metrics to improve the predictive power. These have now been implemented and are being used in the current round of astronaut candidate testing. We expect to re-evaluate the modified ART dataset in year 4. Also, as described in a paper recently submitted to J. Human Factors, we tested the mental rotation and perspective taking spatial abilities of 40 active astronauts who had completed at least one robotics training course. We found logistic regression models that predicted who would achieve the top score in qualification evaluations. The model predictions appear reliable enough to be used to customize regular and remedial training, but not to make career defining decisions. The models could not reliably predict who would completely fail, because so few did. We have proposed improvements in GRT scoring methodology that should improve prediction reliability. At JSC’s request, we are also evaluating whether GRT scores could be used to predict performance in later training (e.g. SSRMS), or under operational conditions on ISS.

Aim 2: Our second major objective is to perform experiments using a space telerobotics training simulator at MIT to quantify how a trainee’s individual spatial and manual control abilities, use of camera views, and hand controller reference frame impacts learning and final level of performance as both primary and secondary robotics operator. This aim was the main focus of our activity this year. We hypothesized that individual ability to manipulate the arm and integrate camera views correlates with 3 subcomponents of spatial intelligence: spatial visualization (SV), mental rotation (MR) and perspective taking (PT). In particular, PT (the ability to imagine an object from another viewpoint) was thought to be important for integrating camera views. This year we completed three different experiments using MIT’s Dynamic Skills Trainer, a virtual space telerobotic training system similar to that used at NASA JSC:

In the first experiment, 19 subjects were trained to manipulate a robotic arm using a pair of hand controllers in a virtual environment almost identical to that in NASA’s Basic Operational Robotic Instruction System (BORIS), used in NASA’s introductory “Generic Robotics Training” course. Over 18 “fly to” trials, the disparity between the arm's control frame and the cameras was varied between low (< 90 degrees) and high (> 90 degrees) conditions. We used the Cube Comparisons (CC) test to assess SV, the Vandenberg Mental Rotations Test (MRT) to assess MR, and the Purdue Spatial Visualization of Views Test (PSVT) and a Perspective Taking Ability (PTA) test to assess PT. We showed that subjects with high PSVT scores moved the arm more directly to the target and were better at maintaining the required clearance between the arm and obstacles, even without a direct camera view. The subjects' performance degraded under the high disparity condition.

Our second experiment addressed trainee performance as both primary and then secondary (monitoring) operator. Twenty subjects were trained to manipulate the arm during 6 trials in a BORIS environment and then acted as a secondary operator observing an additional 32 trials in an ISS-like environment. We recorded which of three display monitors the trainee was looking at. The MRT, PSVT, and PTA were used to assess spatial abilities. Though the primary operator task was slightly different than that used in Experiment 1, we prospectively confirmed many results of the first experiment. Subjects with high PTA scores took less time, moved the arm more directly to the target, and moved the arm more fluidly, especially under the high disparity condition. High scorers on the PSVT and PTA were better at maintaining required clearance. Low PTA scorers looked from monitor to map more often. Prior experience with the arm didn't significantly improve task performance, and performance as primary operator didn’t reliably predict performance as a secondary operator. However, subjects with high PSVT scores had better overall secondary operator performance and high PTA scorers were better at detecting problems before they occurred. These two experiments are the Master’s thesis of Ms. Z. Tomlinson, and have so far been presented in two conference abstracts and posters.

Aim 3: Our third major goal is to identify and develop new interfaces and tools to support future in-space and lunar surface teleoperation and teleoperation training. Our original 2007 plan was to develop an adjunct spatial situation display and a scheme for switching camera views using operator gestures. We plan to focus on bimanual control skill assessment this year, while acquiring the necessary tracking hardware and address gesture control or spatial situation displays during Year 4.

Research Impact/Earth Benefits: Our goal is to improve the efficiency of robotic training via improvement of current pedagogies and development of new teaching tools. Improved training methods provide a framework for designing future in-flight training procedures during long duration missions. The project will also demonstrate how individual differences in spatial and bimanual control skills affect performance of a critical operational skill and provide initial designs of controls, displays and procedures that better match the operator’s cognitive skills with task demands.

Task Progress & Bibliography Information FY2009 
Task Progress: Overall the project is running ahead of the original schedule.

Aim 1: Completed logistic model analysis of spatial test and JSC-ART data on 40 NASA astronauts to predict GRT scores. Manuscript submitted to Journal of Human Factors. Results presented at NASA HRP-BIW and ASMA. Changes in ART scoring proposed and have been implemented by JSC for next round. Reevaluation planned for year 4. GRT scoring methodology changes suggested.

Aim 2: Modified MIT dynamic skills trainer, developed new BORIS and ISS SSRMS tasks, developed IRB protocols and completed 3 series of experiments (n= total 60) on effects of spatial skills on primary and secondary operator performance and gaze patterns during simulated teleoperation training. Preliminary results of first two experiments presented at NASA HRP-BIW (Feb 09). Complete results in ASMA (May 09) poster and abstract and Master’s Thesis (February 09). Articles in preparation.

Aim 3: Discussions with the Robotics Training Branch suggested that we should also focus our research attention on the control and display issues associated with their newest challenge: training crews to use the ISS arms to successfully grab and dock ISS logistics supply vehicles such as ATV and HTV, which may be slowly drifting. Although the task does require spatial skills, it particularly demands that the operator develop a high degree of skill in bimanual control, so the end of the robotic arm can follow a three dimensional arc, all the while maintaining proper angular alignment with the docking pin. Arm translations are controlled with the left hand, and rotations with the right, so the operator must be able to instinctively decompose a 6 DOF movement into the corresponding 3DOF tasks for each hand. (The task feels a bit like to learning to draw smooth lines using a children’s Etch-a-Sketch tablet, except using rate rather than position control, and with six degrees of freedom, not just two.) Some individuals acquire much more proficiency than others. High performance normally requires extensive training and sustained practice. Relatively little is known about acquisition and retention of asymmetric coordinated bimanual control skills, or the origins of individual differences. The bimanual control literature largely addresses human computer interface tasks, where the left and right hands work cooperatively but separately on different (usually 2 DOF) tasks. The USAF employs a Two Handed Coordination Test to screen pilot candidates. However in flying the right hand controls attitude (3 DOF), while the left hand controls throttle (1DOF). There are no fully validated tests of bimanual telerobotic skill. We plan to develop such a test this year. So far our project has largely addressed spatial skills, so investigating the motor control aspects will usefully broaden our scope. Assessments of ATV/HTV docking task, bimanual control literature, and bimanual control test methodologies are underway.

Bibliography Type: Description: (Last Updated: 01/17/2020) 

Show Cumulative Bibliography Listing
 
Abstracts for Journals and Proceedings Collins A, Tomlinson Z, Oman C, Liu A, Natapoff A. "Investigating the effects of frame disparity on the performance of telerobotic tasks." 59th International Astronautical Congress, Glasgow, Scotland, Sept 29 - Oct 3, 2008.

59th International Astronautics Congress, Abstract Book, October 2008. , Oct-2008

Abstracts for Journals and Proceedings Liu AM, Oman CM, Natapoff A, Coleman C. "Spatial ability as a predictor of space robotics training performance." 79th Annual Scientific Meeting of the Aerospace Medical Association, Boston, MA, May 11-15, 2008.

Aviat Space Environ Med. 2008 Mar;79(3):288-9. , Mar-2008

Abstracts for Journals and Proceedings Oman CM, Liu AM, Tomlinson ZA, Natapoff A, Collins A, Pontillo TM, Silverman JB. "Advanced displays for efficient training and operation of robotic systems." Neurophysiology session poster, 2009 NASA Human Research Program Bioastronautics Investigators' Workshop, League City, TX, February 2- 4, 2009.

NASA Human Research Program Bioastronautics Investigators' Workshop, Abstract Book, February 2009. , Feb-2009

Abstracts for Journals and Proceedings Tomlinson ZA, Oman CM, Liu AM, Natapoff A, Silverman J. "Influence of spatial ability on primary and secondary space telerobotic operator performance." 80th Annual Meeting of the Aerospace Medical Association Los Angeles, CA, May 3-7, 2009.

Aviat Space Environ Med. 2009 Mar;80(3):221. , Mar-2009

Articles in Peer-reviewed Journals Liu AM, Oman CM, Natapoff A. "Predicting space telerobotic operator performance from human spatial ability assessments." Human Factors, Submitted, 2009. , Jul-2009
Awards Oman CM. "2009 HRP Bioastronautics Investigators' Workshop L.R. Young Bioastronautics Achievement Award, February 2009." Feb-2009
Dissertations and Theses Tomlinson ZA. "Influence of spatial abilities on primary and secondary space telerobotics operator performance." Dissertation, Massachusetts Institute of Technology, Cambridge, MA, February 2009. , Feb-2009
Papers from Meeting Proceedings Collins A, Tomlinson Z, Oman C, Liu A, Natapoff A. "Investigating the effects of frame disparity on the performance of telerobotic tasks." 59th International Astronautical Congress, Edinburgh, Scotland, Sept 29- Oct 3, 2008.

International Astronautical Congress, Paper IAC-08-B3.6, September 2008. , Sep-2008

Project Title:  Advanced Displays for Efficient Training and Operation of Robotic Systems Reduce
Fiscal Year: FY 2008 
Division: Human Research 
Research Discipline/Element:
HRP SHFH:Space Human Factors & Habitability (archival in 2017)
Start Date: 09/01/2007  
End Date: 08/31/2011  
Task Last Updated: 10/08/2008 
Download report in PDF pdf
Principal Investigator/Affiliation:   Oman, Charles M. Ph.D. / Massachusetts Institute of Technology 
Address:  Department of Aeronautics and Astronautics 
77 Massachusetts Avenue 37-219 
Cambridge , MA 02139-4301 
Email: coman@mit.edu 
Phone: 617-253-7508  
Congressional District:
Web:  
Organization Type: UNIVERSITY 
Organization Name: Massachusetts Institute of Technology 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Young, Laurence  Massachusetts Institute of Technology 
Natapoff, Alan  Massachusetts Institute of Technology 
Liu, Andrew  Massachusetts Institute of Technology 
Project Information: Grant/Contract No. NCC 9-58-SA01301 
Responsible Center: NSBRI 
Grant Monitor:  
Center Contact:   
Solicitation / Funding Source: 2007 NSBRI-RFA-07-01 Human Health in Space 
Grant/Contract No.: NCC 9-58-SA01301 
Project Type: GROUND 
Flight Program:  
TechPort: No 
No. of Post Docs:
No. of PhD Candidates:
No. of Master's Candidates:
No. of Bachelor's Candidates:
No. of PhD Degrees:
No. of Master's Degrees:
No. of Bachelor's Degrees:
Human Research Program Elements: (1) SHFH:Space Human Factors & Habitability (archival in 2017)
Human Research Program Risks: (1) HARI:Risk of Inadequate Design of Human and Automation/Robotic Integration
(2) Sensorimotor:Risk of Altered Sensorimotor/Vestibular Function Impacting Critical Mission Tasks (Revised as of IRP Rev M)
Human Research Program Gaps: (1) SHFE-HARI-02:We need to develop design guidelines for effective human-automation-robotic systems in operational environments that may include distributed, non-colocated adaptive mixed-agent teams with variable transmission latencies (IRP Rev F) (Previously: How can performance, efficiency, and safety guidelines be developed for effective information sharing between humans and automation, such that appropriate trust and situation awareness is maintained?)
(2) SM12:Develop standards for spaceflight cockpit displays and inputs. (OBSOLETE - Merged with SM6 to create SM6.1, per IRP Rev F)
Task Description: The long term objectives of this 4 year MIT/NASA/JSC project are: 1) to develop tests of astronaut spatiomotor abilities that predict the need for remedial training or performance in final telerobotic qualification tests, and 2) to improve teleoperation training techniques and develop new teleoperator interfaces that improve the efficiency of teleoperation training and flight operations.

Our work towards these objectives has been separated into three specific aims:

Aim 1. To improve NASA teleoperation training efficiency by scientifically customizing remedial training based on the measured spatial abilities of individual astronauts. We have examined whether NASA-JSC’s current Aptitude for Robotics Test (ART) predicts the need for remedial work in Generic Robotic Training (GRT) and Shuttle manipulator training (PDRS) or whether additional psychometric tests will sharpen performance predictions.

Based on data from 40 astronauts, we have found that a logistic model statistically described the relationship between a standardized Mental Rotation Test score and General Situation Awareness and Clearance category scores during Generic Robotics Training (GRT) and Payload Deployment and Retrieval System (PDRS) qualification evaluations. Because the current training approach minimizes the total number of poorly scoring performers by training everyone to competence, our model estimates whether an astronaut will show excellent performance in training versus average or worse performance. We have evaluated the logistic model as a predictor of training performance (using the current data set) using an ROC methodology and found prediction performance is significantly better than chance. Before NASA could use such a model in practice, it will have to attach a cost to making erroneous predictions and a value to accurate predictions.

We found that the ART, as currently implemented, is not a very reliable predictor of performance during GRT. Of the five ART tasks, only two tasks are strongly correlated with astronauts’ evaluation scores. However we were not able to establish that these task scores are reliable predictors of GRT training performance using a logistic model. Additional measures of performance (e.g., quantitative temporal, spatial, or smoothness scores such as task-completion time, RMS error, and time-derivative of control inputs, respectively) both during training and during the final evaluation arguable could improve our ability to predict both outstanding performance and the need for remedial training. The three ART tasks that did not clearly correlate with training performance could be modified by imposing task-time constraints on them.

Aim 2. To perform a series of experiments using the MIT Remote Manipulation System Simulator to quantify how a trainee’s individual spatial and manual control abilities, use of camera views and choice of hand controller reference frame impacts learning and final level of performance as primary operator. Secondary operator performance in a clearance detection and estimation task is assessed using a signal detection/situation awareness probe paradigm.

For these experiments at MIT we have re-created the BORIS training virtual environment used in GRT at JSC. Subjects performed as series of arm positioning tasks as the primary robotic operator under various visual conditions to test the role of spatial ability. We found that subjects with higher perspective-taking test scores tended to perform better in certain metrics of performance, such as showing smaller deviations from the ideal trajectory when positioning the robot arm and having fewer clearance violations during the trials, especially when the camera views do not provide direct estimates of the distance between robot arm and environment structure. These results suggest that training could be customized to emphasize different aspects of the task according an assessment of individual spatial abilities.

As expected, subjects’ performance was degraded, in terms of larger path deviations from the ideal trajectory, when the disparity between the camera and control frames of reference was greater than 90 degrees. However, there was no significant difference in task performance between subjects with low and high spatial test scores. There were no differences between the subjects in terms of their improvement in task performance. While the higher scoring subjects may grasp the spatial aspects of the task more quickly, all subjects may have similar difficulties learning the appropriate motor mappings to control the arm. This suggests that tests of individual ability in manual control or dexterity could also be useful predictors of robotic task performance.

Aim 3. To develop and evaluate two interactive interfaces for future in-space and lunar surface operations. Work on this specific Aim is scheduled to begin by Year 3.

Proposed work for Year 2

We will continue data collection at NASA-JSC for Aim 1 to reach the desired sample size. With a sufficiently large sample size, we may be able to test the reliability of our predictions using a split-half technique. We will also carry out a new analysis suggested by the NASA Astronaut Office of predicting "real-arm" training performance from the Generic Robotics Training performance.

Work on Aim 2 will continue with the completion of the second experiment investigating the role of spatial ability in secondary operator performance of monitoring telerobotic operations. Further experiments will be developed that will investigate control mode awareness, and monitoring performance when an end effector camera is in use. We will also refine the use of gaze tracking in these experiments to understand how visual information is utilized during operations.

Research Impact/Earth Benefits: Our goal is to improve the efficiency of robotic training through the modification of current procedures and development of new teaching tools. Improved training methods provide a framework for designing future in-flight training procedures during long duration missions. The project will also demonstrate how individual differences affect performance of a critical operational skill and provide initial designs of controls, displays and procedures that better match the operator’s cognitive skills with task demands.

Task Progress & Bibliography Information FY2008 
Task Progress: Progress on our specific aims include:

Aim 1 - To improve NASA teleoperation training efficiency by scientifically customizing remedial training based on the measured spatial abilities of individual astronauts.

PROGRESS – Four MIT investigators took Generic Robotics Training at JSC. We collected additional data at JSC to bring our subject population to 40. Analyzed the correlations between spatial ability tests and ART, Generic Robotics Training and Shuttle arm training using a logistic regression technique. Demonstrated a technique to predict GRT performance from spatial tests scores. Made recommendations to Scott Hobaugh, Astronaut Office Robotics Branch Chief, and James Tinch, Robotic Branch Chief Engineer, on the efficacy of ART. Beginning analysis on suitability of GRT performance as a predictor of Shuttle or Space Station arm performance. Presented results at the 2008 AsMA scientific meeting.

Aim 2 - To perform a series of experiments using the MIT Remote Manipulation System Simulator to quantify how a trainee's individual spatial and manual control abilities, use of camera views and choice of hand controller reference frame impacts learning and final level of performance as primary or secondary operator.

PROGRESS – Developed a simulation of the BORIS virtual reality teaching environment used in GRT. Completed the experiments for Experiment 2.1 studying performance of a primary operator, including test scenarios and instructional materials. Completed data analysis and submitted an abstract for the 59th International Astronomical Congress, Glasgow, Scotland (Oct, 2008). Modified the task scenarios and instructions for Experiment 2.2 studying performance of a secondary operator. Developed a simple video data recording system to collect eye gaze information during the tasks. Data collection for this experiment started July 2008. Developed a demonstration version of the experiments for educational and outreach purposes.

Aim 3 - To develop and evaluate an improved spatial situation display and new camera control interaction techniques for future in-space and lunar surface operations. PROGRESS – No work scheduled during this year.

Bibliography Type: Description: (Last Updated: 01/17/2020) 

Show Cumulative Bibliography Listing
 
Abstracts for Journals and Proceedings Collins A, Tomlinson Z, Oman C, Liu A, Natapoff A. "Investigating the effects of frame disparity on the performance of telerobotic tasks." 59th International Astronautical Congress, Glasgow, Scotland, 29 Sept - 3 Oct, 2008.

59th International Astronautics Congress, Abstract Book, Oct 2008. , Oct-2008

Abstracts for Journals and Proceedings Liu AM, Oman CM, Natapoff A, Coleman C. "Spatial ability as a predictor of space robotics training performance." 79th Annual Scientific Meeting of the Aerospace Medical Association, Boston, MA, May 11-15, 2008.

Aviat Space Environ Med. 2008 Mar;79(3):288-9. , Mar-2008

Project Title:  Advanced Displays for Efficient Training and Operation of Robotic Systems Reduce
Fiscal Year: FY 2007 
Division: Human Research 
Research Discipline/Element:
HRP SHFH:Space Human Factors & Habitability (archival in 2017)
Start Date: 09/01/2007  
End Date: 08/31/2011  
Task Last Updated: 11/29/2007 
Download report in PDF pdf
Principal Investigator/Affiliation:   Oman, Charles M. Ph.D. / Massachusetts Institute of Technology 
Address:  Department of Aeronautics and Astronautics 
77 Massachusetts Avenue 37-219 
Cambridge , MA 02139-4301 
Email: coman@mit.edu 
Phone: 617-253-7508  
Congressional District:
Web:  
Organization Type: UNIVERSITY 
Organization Name: Massachusetts Institute of Technology 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Liu, Andrew  Massachusetts Institute of Technology 
Natapoff , Alan  Massachusetts Institute of Technology 
Young, Laurence  Massachusetts Institute of Technology 
Project Information: Grant/Contract No. NCC 9-58-SA01301 
Responsible Center: NSBRI 
Grant Monitor:  
Center Contact:   
Solicitation / Funding Source: 2007 NSBRI-RFA-07-01 Human Health in Space 
Grant/Contract No.: NCC 9-58-SA01301 
Project Type: GROUND 
Flight Program:  
TechPort: No 
No. of Post Docs:  
No. of PhD Candidates:  
No. of Master's Candidates:  
No. of Bachelor's Candidates:  
No. of PhD Degrees:  
No. of Master's Degrees:  
No. of Bachelor's Degrees:  
Human Research Program Elements: (1) SHFH:Space Human Factors & Habitability (archival in 2017)
Human Research Program Risks: (1) HARI:Risk of Inadequate Design of Human and Automation/Robotic Integration
(2) Sensorimotor:Risk of Altered Sensorimotor/Vestibular Function Impacting Critical Mission Tasks (Revised as of IRP Rev M)
Human Research Program Gaps: (1) SHFE-HARI-02:We need to develop design guidelines for effective human-automation-robotic systems in operational environments that may include distributed, non-colocated adaptive mixed-agent teams with variable transmission latencies (IRP Rev F) (Previously: How can performance, efficiency, and safety guidelines be developed for effective information sharing between humans and automation, such that appropriate trust and situation awareness is maintained?)
(2) SM12:Develop standards for spaceflight cockpit displays and inputs. (OBSOLETE - Merged with SM6 to create SM6.1, per IRP Rev F)
Task Description: The long term objectives of this project are:

1. To develop tests of astronaut spatiomotor abilities that predict the need for remedial training or performance in final telerobotic qualification tests; and

2. To improve teleoperation training techniques and develop new teleoperator interfaces that improve the efficiency of teleoperation training and flight operations.

Specific aims are:

1. To improve NASA teleoperation training efficiency by scientifically customizing remedial training based on the measured spatial abilities of individual astronauts. We propose an experiment to determine whether NASA Johnson Space Centers (JSC) current Robotic Aptitude Assessment test predicts the need for remedial work in Generic Robotic Training and Shuttle PDRS manipulator training or whether, as we expect, additional psychometric testing will sharpen performance predictions.

2. To perform a series of experiments using the Massachusetts Institute of Technology (MIT) Remote Manipulation System Simulator to quantify how a trainees individual spatial and manual control abilities, use of camera views and choice of hand controller reference frame impacts learning and final level of performance as primary operator. Secondary operator performance in a clearance detection and estimation task is assessed using a signal detection/situation awareness probe paradigm.

3. To develop and evaluate two interactive interfaces for future in-space and lunar surface operations:

* An improved, user-controllable work area spatial situation display; and

* A new head-gesture controlled method for switching between camera views, thereby reducing or eliminating the requirement for multiple monitors in telerobotic workstations.

The short psychometric spatial ability test subjects we employ are sensitive to cognitive impairments and are candidates for Flight Medicine fitness-for-duty tests for astronaut telerobotic system operators. Our approach builds on evidence from our prior research that specific spatial abilities are correlated with teleoperation performance metrics.

Research Impact/Earth Benefits: 0

Task Progress & Bibliography Information FY2007 
Task Progress: New project for FY2007.

Bibliography Type: Description: (Last Updated: 01/17/2020) 

Show Cumulative Bibliography Listing
 
 None in FY 2007