NOTE: End date changed to 4/14/2024 per A. Beitman/JSC (Ed., 2/20/23)

NOTE: Start date changed to 4/15/2019 per NSSC information (Ed., 5/18/21)

NOTE: End date changed to 4/14/2023 per NSSC information (Ed., 1/22/2020)

Effective space exploration will require proper task coordination between humans and robotic systems. These systems can be characterized in a variety of ways, from level of autonomy to the number of functions provided. At the most basic level a robotic system can be considered a hand tool while something more complex could be a humanoid companion. To ensure the robotic system is effective, the crew must trust that the system performs its intended function(s), or retain enough Situation Awareness (SA) and capability to find another way to execute the required task.

Currently, there are no comprehensive standards for measuring, monitoring, and evaluating task performance with regard to crewmember capabilities, the design of the task, and the dynamic spacecraft environment. This work seeks to address this missing performance infrastructure by providing a conceptual framework for measuring task design quality and developing a path for validation using a task performance metric through experimentation both in university labs and using NASA's analog missions.

The focus for this grant year was to continue collecting data for our Block 2 testing, which investigates whether biomeasures can be used to classify different task types. For this grant year, we were able to run 22 more subjects under this Block 2 testing regime. Initial data analysis for classification of the tasks indicates that the best accuracies that we could reasonably achieve ranges from 20-40% using the Random Forest Classifier, Support Vector Machine, Linear Discriminant Analysis, and K-Nearest Neighbor classifiers. This classification accuracy is only just above guessing (25% for four categories of workload). There are a number of possible interpretations as to why the accuracy performs poorly, including potential ordering effects of the protocol, fatigue induced from the test session, learning or habituation effects, or environmental or demographic impacts. These will continue to be investigated using other analysis techniques, including Linear Mixed Models.

And continued analysis on the missions of the HERA Campaign #6 has indicated that the fNIRS measures collected over a 20-minute task may provide a different perspective of task performance than our in-lab five-minute cognitive tasks, in that there may be indicators of mental fatigue present. Additionally, the functional near-infrared spectroscopy (fNIRS) data from HERA indicates that it can clearly identify when the subject is switching from one cognitive task to the next. More thorough analysis will be done with advanced modeling techniques to again capture mixed effects modeling.

NOTE: Start date changed to 4/15/2019 per NSSC information (Ed., 5/18/21)

NOTE: End date changed to 4/14/2023 per NSSC information (Ed., 1/22/2020)

Effective space exploration will require proper task coordination between humans and robotic systems. These systems can be characterized in a variety of ways, from level of autonomy to the number of functions provided. At the most basic level a robotic system can be considered a hand tool while something more complex could be a humanoid companion. To ensure the robotic system is effective, the crew must trust that the system performs its intended function(s), or retain enough Situation Awareness (SA) and capability to find another way to execute the required task.

Currently, there are no comprehensive standards for measuring, monitoring, and evaluating task performance with regard to crewmember capabilities, the design of the task, and the dynamic spacecraft environment. This work seeks to address this missing performance infrastructure by providing a conceptual framework for measuring task design quality and developing a path for validation using a task performance metric through experimentation both in university labs and using NASA's analog missions.

Additionally, the previous year we had begun a variety of engineering tests with the Biosignalsplux fNIRS systems to compare the expected results regarding different cognitive tests. A protocol for investigating the quality of the fNIRS sensor from Biosignalsplux was developed. The study was designed to determine if we can discern differences in data quality collected during cognitive activity between two functional near-infrared spectroscopy (fNIRS) systems: the NIRx NIRSport2 and the biosignalsplux fNIRS Pioneer. The NIRx NIRSport2 is a widely used but invasive and operationally infeasible sensor package while the biosignalsplux Pioneer is non-invasive and operationally feasible but has a limited presence in research studies. This protocol was submitted and approved by the Institutional Review Board (IRB) with potential data collection to start in the spring and summer of 2023.

And lastly, initial analysis from the first three missions of the HERA Campaign #6 has begun and indicated that the fNIRS measures collected over a 20-minute task may provide a different perspective of task performance than our in-lab five-minute cognitive tasks, in that there may be indicators of mental fatigue present. Additionally, the fNIRS data from HERA indicates that it can clearly identify when the subject is switching from one cognitive task to the next. More thorough analysis will be done as the final Human Exploration Research Analog (HERA) mission concludes.

NOTE: End date changed to 4/14/2023 per NSSC information (Ed., 1/22/2020)

Effective space exploration will require proper task coordination between humans and robotic systems. These systems can be characterized in a variety of ways, from level of autonomy to the number of functions provided. At the most basic level a robotic system can be considered a hand tool while something more complex could be a humanoid companion. To ensure the robotic system is effective, the crew must trust that the system performs its intended function(s), or retain enough Situation Awareness (SA) and capability to find another way to execute the required task.

Currently, there are no comprehensive standards for measuring, monitoring, and evaluating task performance with regard to crewmember capabilities, the design of the task, and the dynamic spacecraft environment. This work seeks to address this missing performance infrastructure by providing a conceptual framework for measuring task design quality and developing a path for validation using a task performance metric through experimentation both in university labs and using NASA's analog missions.

In response to the ongoing coronavirus national crisis, steps were taken to ensure that testing could proceed without introducing any added risk of COVID-19 contraction among test participants or the test operator(s). In order to implement this mitigation strategy, several updates to the original proposal research approach were required. The primary change was moving the testing to an At-Home protocol consisting of step-by-step instructions for the test participant and the development of a mobile/portable testing system that includes a stationary bike set-up with a tablet with pre-loaded instructional software with specific testing instructions.

The majority of this year’s accomplishments revolved around the deployment of the At-Home test protocol and data collection and analysis. Thirteen test participants (age 23.77 ± 4.11 years; six female), conducted between eight and ten days of testing each, depending on availability. The participants were mostly undergraduate or graduate aerospace engineering students, and all had backgrounds in science/technology/engineering/mathematics (STEM).

Our test protocol had the test subject perform two main tasks interspersed with questionnaires and rests in between. The first task is slow biking on a stationary bike while performing a set of 50 arithmetic multiple choice problems. Then, after a five minute rest, they perform a two minute warmup on the bicycle to get up to an 85-90 RPM pace, and then begin the second task, which is to maintain the fast biking pace while simultaneously answering the same set of 50 randomized arithmetic problems.

While there are many approaches to analyzing this data to identify predictors for performance on the two main tasks, for now we are starting with the most basic analysis using characteristic statistics for each biomeasure signal, where we represent the signal as means, medians, slope, and standard deviation values for each subject over each test day. We use these aggregated values to compare against the performance outcomes and determine if there are any correlations.

We found that there was no single biomeasure statistic that could reliably predict performance for either of the main tasks, but this could be for a number of reasons. Potentially our hardware has limited resolution and cannot adequately capture small effect sizes for some of the more delicate measures, such as blood oxygenation in the prefrontal cortex. Or it is possible that the tasks we selected do not adequately challenge the subjects, so we lack a range of performance outcomes to compare against. Or we may not have the appropriate biomeasure characteristics; for example, using the average electrodermal activity (EDA) over a specific timeframe may not be as relevant as the measure of latency of the EDA response to stressors.

From this testing we are working towards defining a second test protocol to address these confounding factors more clearly, as well as revisiting our framework and restructuring it based on our findings. Additional data analysis is still underway with this first test block.

We structured our testing into three blocks that coincide with increasing operational fidelity. We finished Block 1 testing, and are now moving into our Block 2 testing and using our findings from Block 1 to address many of the confounding factors we discovered. Additionally, our results from Block 1 suggests it will be important to have a separate validation test of the biosignalsplux functional near-infrared spectroscopy (fNIRS) hardware with a higher fidelity sensor, as it is one of our more intricate hardware systems. Our Block 3 test would be done in a space habitat mock-up developed at the University of Colorado and be more representative of spaceflight. Meanwhile, we are participating in NASA’s HERA Campaign 6 and are just starting to get back data from the first mission for analysis.

NOTE: End date changed to 4/14/2023 per NSSC information (Ed., 1/22/2020)

Effective space exploration will require proper task coordination between humans and robotic systems. These systems can be characterized in a variety of ways, from level of autonomy to the number of functions provided. At the most basic level a robotic system can be considered a hand tool while something more complex could be a humanoid companion. To ensure the robotic system is effective, the crew must trust that the system performs its intended function(s), or retain enough Situation Awareness (SA) and capability to find another way to execute the required task.

Currently, there are no comprehensive standards for measuring, monitoring, and evaluating task performance with regard to crewmember capabilities, the design of the task, and the dynamic spacecraft environment. This work seeks to address this missing performance infrastructure by providing a conceptual framework for measuring task design quality and developing a path for validation using a task performance metric through experimentation both in university labs and using NASA's analog missions.

The work performed this year was adjusted to fit the extenuating circumstances of a global pandemic. Our team’s assumption is that the pandemic will still be ongoing through the following year or two and therefore we made accommodations to our human subject testing to ensure safe protocols for both the subject and the test operator(s). Additionally, all of the work described in this report was done remotely with no direct contact between people.

During the first part of this grant year, the Human Exploration Research Analog (HERA) campaign test sets were finalized and submitted for inclusion in the HERA #6 campaign. These test sets utilized both arithmetic questions and a modified trail mapping test to stress HERA participants’ cognition for quantification of their cognitive states. Cognitive load measures will be monitored with a portable fNIRS system, which was also packaged and shipped to Johnson Space Center (JSC). To track the impacts of environmental factors, a CO2 meter, sound meter, light meter, and temperature/humidity sensors were also delivered to JSC. A simple Food Log spreadsheet was developed and sent to HERA for the subjects to log their daily water and food intake.

In response to the ongoing coronavirus national crisis, steps were taken to ensure that testing could proceed without introducing any added risk of COVID-19 contraction among test participants or the test operator(s). In order to implement this mitigation strategy several updates to the original proposal research approach were required. The primary change was moving the testing to an At-Home protocol consisting of step-by-step instructions for the test participant and the development of a mobile/portable testing system that includes a stationary bike set-up with a tablet with pre-loaded instructional software with specific testing instructions.

Prior to deploying our At-Home Test Protocol, our team tested the protocol extensively to ensure we could extract the appropriate data and that the instructions were easy to follow for a subject with limited guidance from our team. Through this testing phase, our team characterized the suite of biometric sensor behavior across four task types represented as low cognitive, no physical; low cognitive, high physical; high cognitive, no physical; and high cognitive, high physical tasks. This testing helped guide modifications to the protocol including changing the duration of testing phases, modifying the cognitive load task to increase difficulty and streamlining the software instructions.

An engineering test subject was recruited to specifically test the full deployment of our At-Home Test Protocol. The test subject did not have prior familiarization of the hardware or instructions and provided feedback throughout the process. The data gathered from the engineering test subject has provided an wealth of data to analyze and is being used to develop our preliminary post-processing algorithms for interpreting the data and integrating it into the Crew Performance Framework.

Effective space exploration will require proper task coordination between humans and robotic systems. These systems can be characterized in a variety of ways, from level of autonomy to the number of functions provided. At the most basic level a robotic system can be considered a hand tool while something more complex could be a humanoid companion. To ensure the robotic system is effective, the crew must trust that the system performs its intended function(s), or retain enough Situation Awareness (SA) and capability to find another way to execute the required task.

Currently, there are no comprehensive standards for measuring, monitoring, and evaluating task performance with regard to crewmember capabilities, the design of the task, and the dynamic spacecraft environment. This work seeks to address this missing performance infrastructure by providing a conceptual framework for measuring task design quality and developing a path for validation using a task performance metric through experimentation both in university labs and using NASA's analog missions.

1) Evaluate Quantitative Framework for Measuring Crew Capabilities and Task Interaction. As part of this objective, a detailed review of the Task Design Framework was done to map the task types to the specific biometric outputs of interest. This mapping required two stages: 1) mapping specific task types to specific human capabilities needed to perform the task, and 2) mapping of the human capabilities to possible biometric outputs such as heart rate acceleration/deceleration, increased electrodermal activity, etc. Leveraging the Task Design Framework updates, the research team focused on identifying appropriate non-invasive wearable sensors and monitors that would be best used for the HERA operational setting. Several factors were taken into consideration when down selecting the sensor suite to ensure the sensors were suitable for 45-day mission including aspects of usability, subject comfort, onboard memory capacity, ease of use by the crew, etc.

2) Establish Protocols for Measuring Human Capabilities. The outcome of this objective was the submission and full approval of the IRB (Institutional Review Board) protocol required to run the human test subjects in NASA’s HERA Campaign #6. The research team approached this objective with a systematic process and accomplished the tasks in a somewhat chronological order:

a) Candidate Task Identification: Identified over 50 representative tasks done by astronauts and provided detailed characteristics and protocols for each task.

b) Sensor Operation: Developed a basic checklist on how to use the sensors and what other components are needed (i.e., laptop, charging device, stopwatch, paper, pencil, etc.) for data acquisition.

c) Calibrate (Verify) Sensor Data: Compared sensors to standard measure (accuracy) BioPac and normalized the output data and plots. This step ensured the data we measured could be seen across higher accuracy sensors such as the BioPac.

d) Validate Sensors: Compared data to expected task output (example: heart rate goes up during a high physical activity).

e) Task Down Selection: Chose tasks that will have the largest differences in biometric changes that can be observed in the data. Also identified tasks that are representative of astronaut activities and can be implemented in HERA.

f) IRB Protocol Submission and Approval: Protocol was written, edited, and submitted to the IRB panels at both the University of Colorado and NASA Johnson Space Center (JSC). Both required a few modifications and edits and were fully approved.

Summary of Accomplishments

• Reviewed and Updated Task Design Framework

• Identified and Down selected Appropriate Sensor Suite

• Identified Candidate Tasks for Experiment Protocol

• Operated, Calibrated, and Validated Sensor Application

• Down selected Tasks for Experiment Protocol

• Submitted and Received Approval for IRB Protocol (Approved by NASA and University of Colorado Boulder IRB)

• Submitted Data Sharing Agreement with NASA LSDA (Life Sciences Data Archive)

• Coordinated with HERA Experiment Support Scientist (ESS) for delivery of Science Requirement Document (SRD) for HERA Campaign #6.

Effective space exploration will require proper task coordination between humans and robotic systems. These systems can be characterized in a variety of ways, from level of autonomy to the number of functions provided. At the most basic level a robotic system can be considered a hand tool while something more complex could be a humanoid companion. To ensure the robotic system is effective, the crew must trust that the system performs its intended function(s), or retain enough Situation Awareness (SA) and capability to find another way to execute the required task.

Currently, there are no comprehensive standards for measuring, monitoring, and evaluating task performance with regard to crewmember capabilities, the design of the task, and the dynamic spacecraft environment. This work seeks to address this missing performance infrastructure by providing a conceptual framework for measuring task design quality and developing a path for validation using a task performance metric through experimentation both in university labs and using NASA's analog missions.