This project will prototype and deliver an extensible (scalable) embedded training system (RIVET) based on graphical and speech interfaces for the user and procedural and non-programming interfaces for task training developers. RIVET is an instructional task visualization and embedded training tool that:

• Is based on rapid generation of procedures by non-programming methods.

• Is designed for multiple application environments (e.g., operations, maintenance, medical).

• Is sympathetic to the various manners in which instructions are provided to crew today, but which permit transformations between forms to accommodate individual preferences or work conditions.

• Is extensible (scalable) by instruction authors or other individuals, whether in flight or on the ground.

• Supports a variety of output media including 3D virtual reality (VR), navigable animations, 2D movies, text, and speech.

• Provides user interaction tools appropriate to both training and operational environments (e.g. portable systems or hands-free voice actuation). The significance of these objectives and the methodology for procedural task simulation is that crew training and instruction will be enhanced with the concomitant expectation that human errors will be reduced, infrequent tasks can be quickly refreshed, and novel problems may be tackled. If instruction and training materials can be created, visualized, and validated by individuals who need not possess deep computer programming skills, the number and skill level of people engaged in producing this material may result in cost-savings at NASA as well.

The end results of this project would be:

• Integrated procedure development, training, and aiding strategies and technologies.

• Embedded training that is scalable, adaptive to crew information requirements, and addresses different levels of task familiarity and information needs.

• Examine role of the International Procedures Viewer (IPV) and existing/developmental NASA instructional and training systems. These results will be evaluated through usability studies undertaken by the project team. Performance measures will include the amount of facilitation provided by the RIVET system that enables the performer to complete a given task in an efficient and timely manner when compared to existing instruction delivery systems.

Ultimately, a java enabled web browser was used as the base platform. This platform also enabled us to port the application to a PDA. The interface contains an area for instruction display including the current instruction, previous instructions, and future instructions dependent current instruction outcome. There are also controls for switching between performing and training modes, turning on and off speech recognition and output as well as instruction timers. A user can also restart the algorithm at anytime or undo procedures one at a time. Patient state is clearly displayed and can be altered by the user which results in jumping to the place in the algorithm that first detects the new patient state. Finally, there is a window to display images, movies, or detailed information that might accompany the current instruction.

Additionally, there is a help system that includes tool tips and a help menu. As the user of our system completes instructions, they can be checked off. This helps the user to keep there place in the algorithm and enables RIVET to keep an inventory of items used as well as time taken to perform the procedures. Some of the ACLS procedures are time critical. Most instructions in the ACLS algorithm include yes or no questions that determine the next procedure that should be performed. Our system provides users the ability to look ahead to the next possible instructions before choosing an answer to the current instruction.

Training mode includes sample patients and their symptoms. A user can practice procedures and algorithm navigation with the same interface that they would use in a real world application. This provides better familiarity with the system and better use of the tool.

This project will prototype and deliver an extensible (scalable) embedded training system (RIVET) based on graphical and speech interfaces for the user and procedural and non-programming interfaces for task training developers. RIVET is an instructional task visualization and embedded training tool that:

• Is based on rapid generation of procedures by non-programming methods.

• Is designed for multiple application environments (e.g., operations, maintenance, medical).

• Is sympathetic to the various manners in which instructions are provided to crew today, but which permit transformations between forms to accommodate individual preferences or work conditions.

• Is extensible (scalable) by instruction authors or other individuals, whether in flight or on the ground.

• Supports a variety of output media including 3D virtual reality (VR), navigable animations, 2D movies, text, and speech.

• Provides user interaction tools appropriate to both training and operational environments (e.g. portable systems or hands-free voice actuation). The significance of these objectives and the methodology for procedural task simulation is that crew training and instruction will be enhanced with the concomitant expectation that human errors will be reduced, infrequent tasks can be quickly refreshed, and novel problems may be tackled. If instruction and training materials can be created, visualized, and validated by individuals who need not possess deep computer programming skills, the number and skill level of people engaged in producing this material may result in cost-savings at NASA as well.

The end results of this project would be:

• Integrated procedure development, training, and aiding strategies and technologies.

• Embedded training that is scalable, adaptive to crew information requirements, and addresses different levels of task familiarity and information needs.

• Examine role of the International Procedures Viewer (IPV) and existing/developmental NASA instructional and training systems. These results will be evaluated through usability studies undertaken by the project team. Performance measures will include the amount of facilitation provided by the RIVET system that enables the performer to complete a given task in an efficient and timely manner when compared to existing instruction delivery systems.

We chose a web-based interface for the application so that it is accessible using a web browser. The application is written as a Java applet and can be placed somewhere on the web; clients or users can access it using any standard web browser with Java plug-in. As the applet code and input files for the ACLS algorithm reside on the server side, they can later be modified at any stage without the necessity of sending updates to the users. The applet has been implemented for laptop as well as PDA; it is mouse enabled and also has the ability to recognize voice commands so that it can be run hands-free. In addition, we have also incorporated speech synthesis in the applet, but more work is needed to correctly narrate the instructions.

The ACLS algorithm is a complicated directed cyclic graph, but the applet hides the complexity of the algorithm by creating an illusion that the algorithm is merely a linear collection of instructions. The instructions are delivered one at a time and as one instruction has been executed, the state of the patient is queried and an appropriate next instruction is delivered based on the answer provided for the query. For look-ahead purposes, the next one or two possible instructions are always displayed beneath the current instruction. The user can also browse through the algorithm if desired, without actually executing the instructions. This feature allows RIVET to be used for training as well as operations, and is an important motivation for and component of the design.

As the algorithm progresses, the applet keeps track of the state of patient’s vital signs and displays them in a separate panel; the tracked vital signs are consciousness, breath, and pulse. The (paper) algorithm provides no proper solution to a situation when the patient’s vital signs suddenly change. To take this fact into account, the applet allows the user to indicate a sudden change in patient’s vital signs and upon such indication prompts the user to confirm and then re-evaluates and rolls back the algorithm to the closest instruction in past that queried about that particular vital sign. This is important to prevent situations when the physician is, for example, about to insert a catheter in the patient’s chest and the patient suddenly regains consciousness. There is still room for other optimizations in the algorithm that could be implemented after consulting appropriate medical professionals.

In addition to delivering instructions, the applet also delivers appropriate warnings, cautions, and advisories for each instruction. The instructions can also be delivered in a detailed mode that provides illustrative examples and a more detailed description of instructions. The applet also makes a panel available for visual illustration of the procedures and instructions. This panel is intended for future use and will provide 3D visualizations of the medical procedures and 3D interactive simulations of the equipment used during the ACLS algorithm.

As the ACLS algorithm is a cyclic flowchart, it is possible to get into situations when the same question is asked multiple times even though the answer to that question could not have changed over time. For example, if the answer to the availability of a particular medicine was negative, then that medicine cannot become available over time. These queries can be optimized by keeping track of the previous answers to such queries. In fact, this kind of question can be completely eliminated by integrating RIVET with an inventory control database. In the near future, we intend to implement an interface to an inventory control system.

We noticed that by following certain paths in the algorithm tree, one could be asked to perform the same medical procedure over and over again. In some cases, it might be necessary to perform the same procedure multiple times, but in others it might be harmful. One example of the latter is a place where the physician inserts a catheter into the patient’s chest to check for a collapsed lung; the catheter is left in place, and then it is possible for the doctor to follow a certain path and come back to this instruction again where he is asked to insert another catheter in patient’s chest while the previous catheter is still in place. We think these cycles in the algorithm need to be detected and modified in the immediate future. The RIVET algorithm structure, with conditions (e.g., vital signs) that could change asynchronously, will lead to re-evaluation of on-board ACLS flowchart procedures, their potential redesign, and likely improvement.

Another very important aspect of our future work is to investigate generic instruction authoring mechanisms. We want to be able to allow the users to provide any algorithm as an input and be able to author instructions. We intend to provide either an XML- or a GUI based instruction authoring mechanism that will enable users of this system to specify instructions, illustrative pictures or animations, inventory items, state variables, and any other information necessary for the execution of the algorithm. We also intend to incorporate parameterized action representations for the textual as well as visual delivery of instructions.

SAE International Digital Human Modeling For Design and Engineering Symposium, June 2005. , Jun-2005