The research objective of this proposal is to investigate the impact of using Virtual Assistants (VA) to support crewmembers in the context of anomaly treatment during Long Duration Exploration Missions (LDEM), when ground support will be limited. A VA will be developed building upon the software architecture from existing VAs developed by the Principal Investigator (PI) for similar purposes. The VA will provide support for various aspects of anomaly treatment, including detecting and diagnosing the anomaly, as well as recommending a course of action. It will also have the ability to take initiative in the dialog with the user (mixed-initiative mode), and the ability to provide explanations for its actions. The impact of the VA on performance, cognitive workload, situational awareness, and trust, will be assessed through a set of three experiments with human subjects in a laboratory environment. The first experiment will establish the baseline impact (master-slave, no explanations), and subsequent experiments will study the effect of switching to the mixed-initiative mode and adding explanations. The system will also be deployed and tested in the Human Exploration Research Analog (HERA) analog environment.

Concerning Specific Aim 2, we concluded Lab Experiment 2 on the effect of explanations and have submitted the corresponding two journal papers. This experiment addresses the following research question: How do explanations affect human performance, trust, cognitive workload (CW), situational awareness (SA), user satisfaction, and self-confidence for different levels of agent accuracy and uncertainty in human-AI collaborative anomaly diagnosis? The first paper focuses on the effects of explanations for different levels of agent accuracy, while the second paper focuses on the effects of uncertainty.

The protocol for this experiment was as follows. Subjects start with a background survey and test and then they proceed to do two sessions, one in each explanation condition (order counterbalanced). In each session they work on 8 anomaly scenarios, 4 with high uncertainty and 4 with low uncertainty, in random order. After each anomaly, subjects fill out a confidence survey, a Jian survey, and a satisfaction survey. After each session, they also fill out a NASA Task Load Index (TLX), a Situational Awareness Rating Technique (SART) questionnaire, another Jian survey, and another satisfaction survey.

30 subjects were recruited and performed the experiment. The major findings from the experiment are as follows:

Effect of explanations (within-subjects comparison using surveys done after each condition) 1. Explanations improve trust (p<0.001) 2. Explanations improve #anomalies correctly diagnosed (p=0.0039), but do not significantly change the time to diagnosis (p=0.37) 3. Explanations improve SA (p=0.009) 4. Explanations slightly increase CW, but effect is not significant (p=0.108) 5. Explanations improve user satisfaction (p<0.001) 6. Explanations improve user confidence (p<0.001)

Effect of uncertainty (within-subjects comparison using surveys done after each anomaly) 1. (High) Uncertainty decreases trust (p<0.001) 2. (High) Uncertainty decreases #anomalies correctly diagnosed (p<0.001) and time to diagnosis (p<0.001) 3. (High) Uncertainty decreases user satisfaction (p<0.001) 4. (High) Uncertainty decreases user confidence (p<0.001)

The effect of uncertainty on CW and SA could not be measured due to the experimental design – they were only measured once with explanations and once without, averaging over the effects of uncertainty in both cases.

The effects of accuracy were not significant, presumably due to individual differences leading to large variability between subjects. The only exception is that accuracy significantly improved the number of anomalies correctly diagnosed (p<0.001). The interaction effect between explanations and accuracy was also not significant.

Concerning Specific Aim 3, we completed the C6 campaign and prepared for the C7 campaign.

Results from the HERA C6 campaign: The major findings from the HERA C6 campaign (N=16) are as follows: • All subjects correctly resolved all scenarios. • Primary results did now show significant effects of the VA on any metrics. • The only exception is that attentional demand (a component of SA) was significantly higher with VA than without (n=16, V=21, p=0.03). • For group scenarios only, CW was higher with Daphne (t (15) = -6.0207, p <0.001).

Some of the insights we got from the exit interviews are as follows. The crew generally exhibited strong interest in using VAs for anomaly resolution, and enjoyed using Daphne. Many subjects showed an interest in the social aspects of VA and mentioned that they “attributed a personality to Daphne” and “talked about her as if she were another crewmember”. They all mentioned establishing trust very quickly once and for all thanks to Daphne “getting it right the first 3 times or so”. Almost none of them found the question answering capabilities essential because they were “going for speed” and didn’t feel like they needed to ask any questions. However, many subjects mentioned that question answering would be very useful in cases where Daphne recommended more than one diagnosis with the same confidence level. Moreover, crewmembers expressed an interest in the more interactive diagnosis and advanced explanations capabilities we are currently developing as something that would “significantly increase the usefulness” of the tool. Finally, they all confirmed that the scenarios generally felt very easy to diagnose and adding some more complexity would make it more interesting and fun. Note that detailed data regarding the interactions between the crewmembers and Daphne has not been analyzed yet.

Preparation of HERA C7: In C7, we will test a more advanced version of the Daphne agent with improved self-explaining abilities while also testing more complex anomaly scenarios in which there are cascading and simultaneous failures. The timeline for the campaign is shown below. • Phase 1 – Group sessions to study the effect of time delay (crew allowed to talk to MCC) – MD1-4: Group study with and without VA at 0s time delay – 2x1h – MD5-10: Group study with and without VA at 5s time delay – 2x1h – MD11-12: Group study with and without VA at 1min time delay – 2x1h • Phase 2 – Individual sessions for main study (not allowed to talk to MCC) – Week 3: Individual training, 1h per crewmember, 2 simple scenarios, 1 with VA and 1 without – Weeks 4-6: 2 sessions per crewmember per week, 1 with VA and 1 without, 3x4x2x1h=24h

As shown above, the group scenarios from C6 have been replaced with a shorter study on the effect of cislunar time delays (5sec) compared to no delays or longer delays (1 min). For these scenarios, the crew will also be allowed to communicate with MCC, to gain insight into the roles a VA can play for cislunar operators given that MCC is available albeit with a non-negligible time delay of a few seconds.

The second phase is the main phase of individual scenarios and follows the same design as in C6, with the caveat that a (re)-training session is done for each crewmember in week 3 before they start their individual tests.

Finally, in the next year, we also plan on finalizing the 3 publications under review (which should finalize work related to Specific Aims 1 and 2) and submitting a fourth one on the results of C6.

The research objective of this proposal is to investigate the impact of using Virtual Assistants (VA) to support crewmembers in the context of anomaly treatment during Long Duration Exploration Missions (LDEM), when ground support will be limited. A VA will be developed building upon the software architecture from existing VAs developed by the Principal Investigator (PI) for similar purposes. The VA will provide support for various aspects of anomaly treatment, including detecting and diagnosing the anomaly, as well as recommending a course of action. It will also have the ability to take initiative in the dialog with the user (mixed-initiative mode), and the ability to provide explanations for its actions. The impact of the VA on performance, cognitive workload, situational awareness, and trust, will be assessed through a set of three experiments with human subjects in a laboratory environment. The first experiment will establish the baseline impact (master-slave, no explanations), and subsequent experiments will study the effect of switching to the mixed-initiative mode and adding explanations. The system will also be deployed and tested in the Human Exploration Research Analog (HERA) analog environment.

Concerning Specific Aim 2, we added the explanation capabilities to the VA and started data collection and analysis for Lab Experiment 2. This experiment addresses the following research question: How do explanations affect human performance, trust, cognitive workload (CW), situational awareness (SA), user satisfaction, and self-confidence for different levels of accuracy and uncertainty in human-AI collaborative anomaly diagnosis?

The protocol is as follows. Subjects start with a background survey and test, and then they proceed to do two sessions, one in each explanation condition (order counterbalanced). In each session, they work on 8 anomaly scenarios, 4 with high uncertainty and 4 with low uncertainty, in random order. After each anomaly, subjects fill out a confidence survey, a Jian survey, and a satisfaction survey. After each session, they also fill out a NASA Task Load Index (TLX), a Situational Awareness Rating Technique (SART) questionnaire, another Jian survey, and another satisfaction survey. A Situation Awareness Global Assessment Technique (SAGAT) test is done twice per session (once per uncertainty level).

So far, 18 subjects have been recruited and performed the experiment, but data for 3 subjects had to be deleted because of a procedural mistake during those sessions. Currently, data collection is halted while we wait for approval of an IRB modification regarding the details of this experiment’s protocol and compensation. The major findings from the preliminary results (N=15) are as follows: 1. Explanations improve trust (p=0.01) 2. Explanations do not significantly improve performance; #anomalies correctly diagnosed increases (p=0.14), but time to diagnosis also increases (p=0.30) 3. Explanations improve SA (p=0.018), primarily by improving understanding (P<0.01) 4. Explanations increase CW but effect is not significant (p=0.14) 5. Explanations improve user satisfaction (p<0.01) 6. Explanations improve user confidence (p<0.01) 7. Performance is higher for low uncertainty (p<0.01) 8. Trust is higher for low uncertainty (p<0.01)

A more detailed analysis, including the accuracy levels, will be done with the full dataset (N=36).

Concerning Specific Aim 3, we completed the C6M3 mission and will complete the last mission by the end of this year’s period of performance, which should get us close to completing the nominal scope of Specific Aim 3, pending publication of the results. The major findings from the HERA C6M1-M3 missions (N=12) are as follows: 1. All subjects correctly resolved all scenarios. 2. Time to resolution is lower without Daphne but not significant (p=0.2). We attribute this to some logon issues we encountered, especially in mission 1. Our perception is that the diagnosis was very fast, both with and without Daphne, suggesting that the anomaly scenarios were “too easy.” The subjects did spend considerable time conducting the procedures to resolve the anomalies, but that was not the focus of our study. 3. CW is higher with Daphne, but this difference was not significant (p=0.3). 4. SA is lower with Daphne, but this difference was not significant (p=0.19).

Some of the insights we got from the exit interviews are as follows: The crew generally exhibited a strong interest in using VAs for anomaly resolution and enjoyed using Daphne. Most of them showed an interest in the social aspects of VA and mentioned that they “attributed a personality to Daphne” and “talked about her as if she were another crewmember.” They all mentioned establishing trust very quickly once and for all thanks to Daphne “getting it right the first 3 times or so.” Almost none of them found the question-answering capabilities essential because they were “going for speed” and didn’t feel like they needed to ask any questions. However, many subjects mentioned that question-answering would be very useful in cases where Daphne recommended more than one diagnosis with the same confidence level. Moreover, crewmembers expressed an interest in the more interactive diagnosis and advanced explanations capabilities we are currently developing as something that would “significantly increase the usefulness” of the tool. Finally, they all confirmed that the scenarios generally felt very easy to diagnose and adding some more complexity would make it more interesting and fun. Note that detailed data regarding the interactions between the crewmembers and Daphne has not been analyzed yet.

The final data analysis will be done once the C6M4 mission is complete. Of note, we have extended the scope of specific Aim 3 through an extension of the grant that will allow us to participate in the HERA C7 campaign, which we have started to prepare. In C7, we will be able to test a more advanced version of the Daphne agent while also testing more complex anomaly scenarios.

In terms of publications, we submitted a journal paper on Specific Aim 1 to the Human Factors journal, which is currently under a second round of review and presented a paper on Specific Aim 2 at the 2022 Applied Human Factors and Ergonomics (AHFE) conference [15]. We will also have one talk and one poster presentation at the 2023 Human Research Project Investigator's Workshop (HRP IWS) in February.

As for the next steps, in Year 4, we will finalize the C6 campaign and start C7 (Specific Aim 3). We plan on publishing a journal paper with the results of C6. Otherwise, most of the activity will focus on finalizing lab experiments 2 and 3. We will work on additional journal papers on lab experiment, lab experiment 3, and the C7 campaign.

The research objective of this proposal is to investigate the impact of using Virtual Assistants (VA) to support crewmembers in the context of anomaly treatment during Long Duration Exploration Missions (LDEM), when ground support will be limited. A VA will be developed building upon the software architecture from existing VAs developed by the Principal Investigator (PI) for similar purposes. The VA will provide support for various aspects of anomaly treatment, including detecting and diagnosing the anomaly, as well as recommending a course of action. It will also have the ability to take initiative in the dialog with the user (mixed-initiative mode), and the ability to provide explanations for its actions. The impact of the VA on performance, cognitive workload, situational awareness, and trust, will be assessed through a set of three experiments with human subjects in a laboratory environment. The first experiment will establish the baseline impact (master-slave, no explanations), and subsequent experiments will study the effect of switching to the mixed-initiative mode and adding explanations. The system will also be deployed and tested in the Human Exploration Research Analog (HERA) analog environment.

We measured performance metrics by evaluating the number of anomalies that participants could successfully resolve with and without the use of Daphne-AT. Additional performance metrics included the time it took for each subject to solve each anomaly, whether or not the subject detected the anomaly, diagnosed the root cause, and selected the correct anomaly resolution procedure. We also recorded the number of attempts for the latter three metrics. In addition, we measured cognitive workload by employing the NASA Task Load Index (TLX) survey, and situational awareness using the Situational Awareness Rating Technique (SART). Finally, the participants’ trust in autonomous systems was measured using Jian's trust scale.

The summary of the results is as follows. For performance, a significant difference (p=0.002) was found in the number of anomalies correctly resolved with Daphne (4.1±0.3) vs. without Daphne (2.7±0.4). This supports our hypothesis that a VA can improve performance in anomaly resolution. For cognitive workload, a significant difference (p=0.002) was found in the TLX score with Daphne (45.53±6.67) vs. without Daphne (65.97±4.81). This supports our hypothesis that a VA can reduce cognitive workload in anomaly resolution. For situational awareness, a marginally significant difference (p=0.052) was found in the SART score with Daphne (5.19±0.56) vs. without Daphne (4.41±0.71). This does not fully support our hypothesis that a VA can improve situational awareness in anomaly resolution. Finally, the responses to the Jian scale items showed generally high trust in automation. A much more detailed description of the experiment and the results is in the paper about to be submitted to the Human Factors and Ergonomics Society (HFES) journal. The draft paper will be provided to the NASA Human Research Program (HRP) as soon as it is submitted.

This year we also completed the first HERA mission Campaign 6, Mission XXII (C6M1). Campaign 6 / C6M2 will also be complete by the time the period of performance of this report is exhausted, which gets us about halfway in meeting Specific Aim 3. Our HERA study was divided into two phases. In Phase 1, each crewmember performed two experiment sessions per week. In each session, the crewmember worked on a single anomaly scenario, either with Daphne or without. Thus, at the end of Phase 1, each subject had worked individually on 3 anomalies with Daphne and 3 without. Because of a technical issue with Daphne, one of the crewmembers ended up with only 2 anomalies with Daphne and 4 without Daphne. In Phase 2, crew members worked together as a team. They conducted two sessions per week, and worked on one anomaly scenario per session. This resulted in a total of 3 anomalies with Daphne and 3 without. Preliminary results are as follows.

Concerning performance, all scenarios were successfully resolved and therefore this was not an interesting metric. We are currently looking at the time to diagnosis of the anomaly to see if there are any interesting trends there. Trends appear to indicate that in some cases the crew took more time to solve scenarios with Daphne than without, especially in Phase 2, but we attribute this to some logon issues we encountered towards the end of the mission. Our perception is that the diagnosis was very fast both with and without Daphne, suggesting that the anomaly scenarios were “too easy”. The subjects did spend considerable time conducting the procedures to resolve the anomalies, but that was not the focus of our study.

Concerning cognitive workload, we observed a small difference between the TLX scores with Daphne (23.42±5.17) and without Daphne (25.20±3.54) but this difference was not significant with only 4 subjects (p=0.42). It remains to be seen if the effect becomes significant with the full 16 subjects. It is also interesting to note that these scores are very low compared to the scores obtained in the laboratory experiments. This could be due to the shorter training and less experience of the participants of the lab experiments compared to the HERA crew. This is also consistent with the explanation that the scenarios were too “easy” for the HERA crew.

Concerning situational awareness, we observed the opposite trend compared to the lab experiments, although in both cases the effects are not significant. In this case, SART scores were slightly lower with Daphne (20.46±3.35) than without Daphne (21.21±3.50), for a p-value of 0.41. We note that the effects were not consistent among crewmembers, and the overall result is driven by one of the 4 crewmembers, who reported a significantly higher level of situational awareness without Daphne vs. with Daphne. We are looking in more detail into the components of the SART score (understanding, demand, supply) because the trends appear to be conflicting there too. While the literature has reported cases where using some kind of assistance results in a decrease of situational awareness (e.g., autopilots in commercial aircraft), this may be due to other factors (e.g., minor technical issues with the VA) as well. More research is needed to understand this – we hope that missions 2,3,4 will help us elucidate this issue. We also note that crewmembers mentioned in the exit debrief interviews that adding explanations would likely significantly increase situational awareness with Daphne, and this is something that we plan on testing in our lab experiments 2 and 3.

Concerning trust, the results were similar to the ones from the lab experiments, showing high trust in the VA across the board. In the exit interviews, crewmembers indicated that they trusted Daphne “right away”, after the first few of her recommendations were proven to be correct.

Some of the insights we got from the exit interviews are as follows. The crew generally exhibited strong interest in using VAs for anomaly resolution, and enjoyed using Daphne. Most of them showed an interest in the social aspects of VA and mentioned that they “attributed a personality to Daphne” and “talked about her as if she were another crewmember”. They all mentioned establishing trust very quickly, once and for all, thanks to Daphne “getting it right the first 3 times or so”. Almost none of them found the question answering capabilities essential because they were “going for speed” and didn’t feel like they needed to ask any questions. However, all of them mentioned that question answering would be very useful in cases where Daphne recommended more than one diagnosis with the same confidence level. Moreover, crewmembers expressed an interest in the more interactive diagnosis and advanced explanations capabilities we are currently developing as something that would “significantly increase the usefulness” of the tool. Finally, they all confirmed that the scenarios generally felt very easy to diagnose and adding some more complexity would make it more interesting and fun.

As for next steps, in Year 4 we will support the remaining missions of C6, thus finalizing Specific Aim 3. We plan on publishing a journal paper with the results. Otherwise, most of the activity will focus on achieving Specific Aim 2: we will finalize the development of the enhanced VA capabilities (explanations and mixed initiative) and conduct lab experiments 2 and 3.

The research objective of this proposal is to investigate the impact of using Virtual Assistants (VA) to support crew members in the context of anomaly treatment during Long Duration Exploration Missions (LDEM), when ground support will be limited. A VA will be developed building upon the software architecture from existing VAs developed by the Principal Investigator (PI) for similar purposes. The VA will provide support for various aspects of anomaly treatment, including detecting and diagnosing the anomaly, as well as recommending a course of action. It will also have the ability to take initiative in the dialog with the user (mixed-initiative mode), and the ability to provide explanations for its actions. The impact of the VA on performance, cognitive workload, situational awareness, and trust, will be assessed through a set of three experiments with human subjects in a laboratory environment. The first experiment will establish the baseline impact (master-slave, no explanations), and subsequent experiments will study the effect of switching to the mixed-initiative mode and adding explanations. The system will also be deployed and tested in the Human Exploration Research Analog (HERA) analog environment.

To be able to do experiments at Texas A&M, we developed an immersive 3d user interface to simulate the Human Exploration Research Analog (HERA) habitat during our experiments at TAMU. The goal was for the simulator to have reasonable fidelity in terms of the general dimensions and shape as well as the main components present in the habitat related to the ECLSS system. The simulator is basically a first-person game developed in Unreal Engine 4 (UE4). All of the major components of the HERA Environmental Control and Life Support Subsystem (ECLSS) system are present and the user can interact with them (e.g., press buttons, open/close panels) as required by the anomaly resolution procedures. More details about this simulator are provided in

Woodruff R, Beebe N, Josan PK, Wong RKW, Dunbar BJ, Selva D, Diaz-Artiles A. (2021). 3D Interactive Model of HERA to support ECLSS anomaly resolution using a Virtual Assistant. 2021 IEEE Aerospace Conference Proceedings.

After validation of the UE4 simulator, we designed and conducted a pilot study of the first “Baseline” prototype of the assistant. It included five male subjects within the age range of 22-40 years old, and with a Standard Deviation (SD) of ±7.2. Prior to the experimental sessions, subjects were provided a 2-hour virtual training session about all the experiment elements (Daphne-AT, ECLSS subsystems, 3D virtual HERA environment, and experimental protocols) and they were guided through an example anomaly scenario. After the training, subjects participated in two in-person experiment sessions: one with Daphne-AT and the other one without Daphne-AT (counterbalanced among subjects). These sessions were conducted in two consecutive days at approximately the same time of the day. In each one of the sessions, the subjects solved four anomalies. Once the experiment sessions were over, the subjects completed a set of surveys.

Dependent variables included performance, cognitive workload, situational awareness, and trust in the context of ECLSS anomaly treatment, with and without the use of a VA. Performance is described as an individual’s ability to successfully treat the anomaly. On average, subjects solved 3.2 ± 0.37 anomalies using Daphne-AT and 2.0 ± 0.without Daphne-AT. Based on these limited data, our preliminary results support the hypothesis that the use of Daphne-AT will increase performance in the context of ECLSS anomaly resolution operations.

The NASA Task Load Index (TLX) survey was used to measure the effect of using Daphne-AT on cognitive workload. NASA TLX is an evaluation technique developed by NASA to assess relative importance of six pre-determined factors in determining how much workload a subject experience during a particular task, in this case, the task of anomaly treatment. These six factors include mental demand, physical demand, temporal demand, perceived performance, perceived effort, and frustration level. On average, subjects indicated higher cognitive workload during the sessions without Daphne-AT (57.2 ± 5.78 with Daphne-AT and 66 ± 7.31 without Daphne). These preliminary results support the hypothesis that the use of Daphne-AT will decrease cognitive workload. More details about this pilot study can be found in:

Josan PK, Dutta P, Woodruff R, Beebe N, York K, Balcells-Quintana O, Kluis L, Viros-i-Martin A, Dunbar BJ, Wong RKW, Selva D, Diaz-Artiles A. (2021). Experimental Design & Pilot Testing for ECLSS Anomaly Resolution using Daphne-AT Virtual Assistant. 2021 IEEE Aerospace Conference Proceedings.

As a result of the pilot testing and HERA integration activities, we have substantially refined the VA interface and added functionality to the back end. Specifically: 1) We adapted the telemetry feed to the new version of the HERA Environmental Control and Life Support System (ECLSS) simulator (now called Habitat System Simulator or HSS). 2) We updated the anomaly database to include more information for each anomaly (e.g., a time signature). 3) We expanded the knowledge base with more information about ECLSS anomalies, their associated risks, and how to resolve them. 4) We expanded the anomaly detection functions beyond basic thresholding to be able to detect outliers in the telemetry feed. 5) We made slight changes to the knowledge graph and procedures to adapt to HERA C6 schedule constraints.

In terms of publications, we had two papers accepted at the 2021 IEEE Aerospace Conference which will be presented in March. We also gave a presentation at the 2020 American Institute of Aeronautics and Astronautics (AIAA) ASCEND conference (Ed. note 1/7/2021: see FY2020 Bibliography in Task Book report) and will have one talk presentation and one poster presentation at the 2021 Human Research Program workshop in February.

The research objective of this proposal is to investigate the impact of using Virtual Assistants (VA) to support crew members in the context of anomaly treatment during Long Duration Exploration Missions (LDEM), when ground support will be limited. A VA will be developed building upon the software architecture from existing VAs developed by the Principal Investigator (PI) for similar purposes. The VA will provide support for various aspects of anomaly treatment, including detecting and diagnosing the anomaly, as well as recommending a course of action. It will also have the ability to take initiative in the dialog with the user (mixed-initiative mode), and the ability to provide explanations for its actions. The impact of the VA on performance, cognitive workload, situational awareness, and trust, will be assessed through a set of three experiments with human subjects in a laboratory environment. The first experiment will establish the baseline impact (master-slave, no explanations), and subsequent experiments will study the effect of switching to the mixed-initiative mode and adding explanations. The system will also be deployed and tested in the Human Exploration Research Analog (HERA) analog environment.

In the first year of this project, we have completed a project definition phase in which we have more crisply defined the scientific objectives and the technical approach to design the virtual assistant. We started by selecting the Environmental Control and Life Support Subsystem (ECLSS) as the technical scope for the virtual assistant. We then proceeded to elicit high level requirements and use cases and perform the early design of the virtual assistant, which we are getting ready to test in our laboratory at Texas A&M University in the next month.

Specifically, in this first year, we have accomplished the following: 1) We have defined the high-level requirements and overall software architecture of the virtual assistant. 2) We have defined how the virtual assistant will interface with the Human Exploration Research Analog (HERA) infrastructure. 3) We have developed the first very basic prototype of the virtual assistant which includes: a) a simple anomaly database; b) a simple database containing knowledge about how the ECLSS system works in HERA and how to perform some maintenance and repair operations; c) an interface with the HERA simulated telemetry feed, which provides a data stream with all the measurements of the ECLSS subsystem; d) a web-based graphical user interface containing a main plot window displaying the telemetry feed and a chat box to chat with the virtual assistant; e) the natural language processing layer that the assistant uses to process the natural language requests; f) the machine learning layer that is used to understand the intent of the question and send it to the relevant part of the software, i.e., the one that knows how to answer it; f) the first version of the back end of the software, which is precisely the part of the software that performs the calculations needed to answer the questions and requests from the user. While this first prototype is not sophisticated in terms of the breadth and depth of questions it can answer, it still has most of the main functionality expected of the assistant. Specifically: 1) it warns the user when a measurement from the telemetry feed exceed some user-defined thresholds; 2) it also performs statistical analysis on the data stream and warns the user if the measurement is considered an outlier, even if it is within the user-defined thresholds; 3) once an anomaly is detected (e.g., increase in CO2 concentration), it provides the users with some possible root causes for it, in order of likelihood (e.g., a malfunctioning filter or a leak); 4) once the anomaly has been diagnosed (faulty filter), it provides the user with relevant procedures to reconfigure the system (how to change the CO2 filter).

In parallel of the software development, we have also started planning and designing the experiments. We are currently getting ready to start a pilot experiment at Texas A&M in order to refine both the software implementation and the experimental design. The results of this pilot experiment will help us get ready for the HERA campaign, which is currently scheduled for August 2020.

Since the beginning of the project, we have been working with the HERA team to prepare for the experiment. We obtained approval from the Institutional Review Boards of both NASA and Texas A&M. We provided our input to the experiment support specialist to create our science requirements document and are currently working on the investigator working group, getting ready for a software delivery to NASA by March.

We have also been in constant communication with the rest of the VNSCOR to ensure compatibility of our experiments and try to make the most of any potential synergies.

Finally, in terms of publications, we are about to present a peer-reviewed full paper at the 2020 American Institute of Aeronautics and Astronautics (AIAA) Scitech conference. We also have two poster presentations at the 2020 Human Research Program workshop later this month.

In the next few months, we will keep adding functionality to the assistant so that it can answer more questions better and provide more useful information to the user. We anticipate that some rework of the telemetry feed interface will be necessary to adapt to some changes on the HERA side. We will also refine the user interface with the results from the pilot. We are getting ready to conduct a preliminary design review in March before delivery to HERA. Between March and August 2020 we will work on integrating our software with HERA. We will conduct a Critical Design Review in June 2020 and an Operational Readiness Review in August 2020, right before ingress. Between August 2020 and September 2021 we will support the 4 missions of the C6 HERA campaign.

The research objective of this proposal is to investigate the impact of using Virtual Assistants (VA) to support crew members in the context of anomaly treatment during Long Duration Exploration Missions (LDEM), when ground support will be limited. A VA will be developed building upon the software architecture from an existing VA developed by the Principal Investigator. The VA will provide support for various aspects of anomaly treatment, including detecting and diagnosing the anomaly, as well as recommending a course of action. It will also have the ability to take initiative in the dialog with the user (mixed-initiative mode), and the ability to provide explanations for its actions. The impact of the VA on performance, cognitive workload (CW), situational awareness (SA), and trust, will be assessed through a set of three experiments with human subjects in a laboratory environment. The first experiment will establish the baseline impact (master-slave, no explanations), and subsequent experiments will study the effect of switching to the mixed-initiative mode and adding explanations. Finally, the system will be deployed in an analog environment.