The long term objectives of this 4 year MIT/NASA/JSC project are: 1) to develop tests of astronaut spatiomotor abilities that predict the need for remedial training or performance in final telerobotic qualification tests, and 2) to improve teleoperation training techniques and develop new teleoperator interfaces that improve the efficiency of teleoperation training and flight operations.
Our work towards these objectives has been separated into three specific aims:
Aim 1. To improve NASA teleoperation training efficiency by scientifically customizing remedial training based on the measured spatial abilities of individual astronauts. We have examined whether NASA-JSC’s current Aptitude for Robotics Test (ART) predicts the need for remedial work in Generic Robotic Training (GRT) and Shuttle manipulator training (PDRS) or whether additional psychometric tests will sharpen performance predictions.
Based on data from 40 astronauts, we have found that a logistic model statistically described the relationship between a standardized Mental Rotation Test score and General Situation Awareness and Clearance category scores during Generic Robotics Training (GRT) and Payload Deployment and Retrieval System (PDRS) qualification evaluations. Because the current training approach minimizes the total number of poorly scoring performers by training everyone to competence, our model estimates whether an astronaut will show excellent performance in training versus average or worse performance. We have evaluated the logistic model as a predictor of training performance (using the current data set) using an ROC methodology and found prediction performance is significantly better than chance. Before NASA could use such a model in practice, it will have to attach a cost to making erroneous predictions and a value to accurate predictions.
We found that the ART, as currently implemented, is not a very reliable predictor of performance during GRT. Of the five ART tasks, only two tasks are strongly correlated with astronauts’ evaluation scores. However we were not able to establish that these task scores are reliable predictors of GRT training performance using a logistic model. Additional measures of performance (e.g., quantitative temporal, spatial, or smoothness scores such as task-completion time, RMS error, and time-derivative of control inputs, respectively) both during training and during the final evaluation arguable could improve our ability to predict both outstanding performance and the need for remedial training. The three ART tasks that did not clearly correlate with training performance could be modified by imposing task-time constraints on them.
Aim 2. To perform a series of experiments using the MIT Remote Manipulation System Simulator to quantify how a trainee’s individual spatial and manual control abilities, use of camera views and choice of hand controller reference frame impacts learning and final level of performance as primary operator. Secondary operator performance in a clearance detection and estimation task is assessed using a signal detection/situation awareness probe paradigm.
For these experiments at MIT we have re-created the BORIS training virtual environment used in GRT at JSC. Subjects performed as series of arm positioning tasks as the primary robotic operator under various visual conditions to test the role of spatial ability. We found that subjects with higher perspective-taking test scores tended to perform better in certain metrics of performance, such as showing smaller deviations from the ideal trajectory when positioning the robot arm and having fewer clearance violations during the trials, especially when the camera views do not provide direct estimates of the distance between robot arm and environment structure. These results suggest that training could be customized to emphasize different aspects of the task according an assessment of individual spatial abilities.
As expected, subjects’ performance was degraded, in terms of larger path deviations from the ideal trajectory, when the disparity between the camera and control frames of reference was greater than 90 degrees. However, there was no significant difference in task performance between subjects with low and high spatial test scores. There were no differences between the subjects in terms of their improvement in task performance. While the higher scoring subjects may grasp the spatial aspects of the task more quickly, all subjects may have similar difficulties learning the appropriate motor mappings to control the arm. This suggests that tests of individual ability in manual control or dexterity could also be useful predictors of robotic task performance.
Aim 3. To develop and evaluate two interactive interfaces for future in-space and lunar surface operations. Work on this specific Aim is scheduled to begin by Year 3.
Proposed work for Year 2
We will continue data collection at NASA-JSC for Aim 1 to reach the desired sample size. With a sufficiently large sample size, we may be able to test the reliability of our predictions using a split-half technique. We will also carry out a new analysis suggested by the NASA Astronaut Office of predicting "real-arm" training performance from the Generic Robotics Training performance.
Work on Aim 2 will continue with the completion of the second experiment investigating the role of spatial ability in secondary operator performance of monitoring telerobotic operations. Further experiments will be developed that will investigate control mode awareness, and monitoring performance when an end effector camera is in use. We will also refine the use of gaze tracking in these experiments to understand how visual information is utilized during operations. |