|
Principal Investigator:
|
Laurence R. Young, Sc.D.
|
|
Organization:
|
Massachusetts Institute of Technology
|
|
Project Ended:
|
2000
|
The efficiency of crew performance on the ISS will depend critically on training and coaching, both before launch and during a flight increment. This project investigated the ability of a real-time expert system to improve performance and reduce the time needed to diagnose errors and troubleshoot a space life science experiment. The experiment tested the efficacy of Principal Investigator-in-a-Box, or PI, a tool for assisting relatively untrained "astronaut surrogates" to detect, diagnose, and correct realistic instrumentation anomalies. It is the first evaluation of such a decision aid under controlled conditions, and will help determine the applicability of expert systems in future space flight research. Two groups of subjects, receiving identical training on sleep monitoring, were tested on two days. Half the subjects were tested with PI assistance only on day one; the other half only on day two. For all subjects, time to detect and identify randomly introduced artifacts and to complete a normal sleep monitoring calibration was measured with and without PI. The expert system rules build on the existing PI software for troubleshooting and error detection developed for the Neurolab sleep experiment.
Key Findings
Results from the study indicate that an expert system can be used for fault management in a space life science experiment. Furthermore, astronauts who used PI during missions found it to be a useful decision aid. However, its utility depends, at least in part, on training and the user's computer literacy.
The feasibility of the PI-in-a-Box concept has been shown in ground studies as well, confirming the favorable experience with it in space. With appropriate training, PI reduced time to detect and time to correctly troubleshoot faults in a sleep instrumentation setup. We found that by observing the reliability of the indicator lights, PI was helpful for subjects on day one, and was a hindrance for them on day two. There were also fewer undetected anomalies and undiagnosed faults with PI than without it.
Satisfaction of Hypotheses
As stated in the proposal, our hypothesis is that use of a computer decision aid (PI-in-a-Box) will improve experiment performance on three independent measures, compared to the same subjects' performance without the decision aid. These independent measures are:
- Average time to detect the deterioration of signal quality to beyond a pre-determined level,
- Average time to identify correctly the source of unanalyzable data in a complex situation with several alternative causes,
- Average time to complete a normal calibration and run a physiological experiment.
The pilot study and observations of Neurolab and STS-95 data were used to evaluate PI's ability to help with detection times for anomalous signals. The results of the pilot study showed that PI assistance reduced the detection time, though not by a statistically significant amount. Training, or the cross effect of PI and day, was found to be significant. The study also found that the number of undetected anomalies was significantly lower when PI was available. Gender effects were also found to be significant for the detection task.
The Neurolab and STS-95 data were comprised of signal recordings of the first few minutes of each instrumentation session. It was found that PI correctly detected 84 percent of the anomalies that were not saturated in the signals from the Neurolab data. In the STS-95 data, PI correctly detected 86 percent of all signal anomalies. Overall, the cardiorespiratory indicator lights were the most reliable, while the electroencephalogram (EEG), and electro-oculogram (EOG) signals were the most prone to false alarms from PI indicator lights.
The study completed in phase one showed that the use of PI assistance has a different impact for different types of stimulus files. It seems to neutralize differences between signal anomalies of different simulation files. Furthermore, in phase two, subjects allowed fewer faults to go undiagnosed (i.e. fewer timeouts) when PI help was available. These are positive indications that PI acts so as to make complex faults easier to detect and diagnose.
Phase two of the study demonstrated a beneficial effect of PI and training in reducing anomaly troubleshooting time. Questionnaires showed that most subjects preferred monitoring the PI indicator lights while monitoring waveforms, rather than monitoring the waveforms alone. On one hand, PI did not improve the reliability of detection, since subjects were not any more correct in their anomaly detection with PI than without it. On the other hand PI did even out performance by reducing the chance of an undiagnosed fault, and by helping subjects with different tasks based on their experience level. It was shown that PI's indicator lights only needed to be 40 percent reliable for subjects to achieve optimum performance, which shows its flexibility. PI correctly detected the anomalous signal for up to 85 percent of the time. There was no difference in fault management performance between genders.
Implications of Results
Our space experiences with computer decision aids for astronaut scientists have all been demonstrations, rather than formal experiments with testable hypotheses. The drive to develop useable new technology in feasible, cost-effective ways outweighed the scientific need to fly placebo devices as controls for experiments. (These devices would have contributed nothing to the ongoing experiments, would have consumed valuable space resources, and so were considered unessential/dispensable.) Our study performed thorough ground tests to evaluate the efficacy of our expert system for assisting astronauts in the Space Station era. The diagnostic aids, experimental scheduler, and interesting data monitor were shown to be beneficial for carrying out space experiments. Each of the tools developed throughout the history of PI - from STS-40 and STS-58 through STS-90 and STS-95 - can be applied to experiments aboard ISS. Autonomous systems are already being implemented in the ISS, and having a software with embedded knowledge such as PI will ensure the scientific and operational success of a mission. These developments could reduce the chance of error caused by human-system interface problems, a concern outlined in section 6.09 of NASA's Critical Path Roadmap.
The development of PI can be applied to earth-based domains too. Subjects could be helped by an intelligent fault management system for diagnostics and repairs. Earth-based space research includes projects such as the BIOPLEX. Autonomous fault management systems are already being used for this testbed of life support systems. The results of human behavior in a fault management situation, such as in this ground study, could lead to better designs for the interface of such systems. Other earth-based applications include home sleep monitoring. Patients or caregivers who are not familiar with sleep instrumentation can use a diagnostic engine to help them detect and repair failures, without data being lost. Therefore, the concept of embedding the knowledge in an autonomous system in the spirit of PI can benefit technology on earth.
Print