Menu

 

The NASA Task Book
Advanced Search     

Project Title:  Speech monitoring of cognitive deficits and stress Reduce
Fiscal Year: FY 2008 
Division: Human Research 
Research Discipline/Element:
HRP BHP:Behavioral Health & Performance (archival in 2017)
Start Date: 07/01/2004  
End Date: 07/31/2008  
Task Last Updated: 11/05/2009 
Download report in PDF pdf
Principal Investigator/Affiliation:   Lieberman, Philip  Ph.D. / Brown University 
Address:  Department of Cognitive and Linguistic Sciences 
190 Thayer Street Room 120 
Providence , RI 02912 
Email: philip_lieberman@brown.edu 
Phone: 401-863-1857  
Congressional District:
Web:  
Organization Type: UNIVERSITY 
Organization Name: Brown University 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Dinges, David  University of Pennsylvania 
Project Information: Grant/Contract No. NCC 9-58-NBPF00406 
Responsible Center: NSBRI 
Grant Monitor:  
Center Contact:   
Solicitation / Funding Source: 2003 Biomedical Research & Countermeasures 03-OBPR-04 
Grant/Contract No.: NCC 9-58-NBPF00406 
Project Type: GROUND 
Flight Program:  
TechPort: No 
No. of Post Docs:
No. of PhD Candidates:
No. of Master's Candidates:
No. of Bachelor's Candidates:
No. of PhD Degrees:
No. of Master's Degrees:
No. of Bachelor's Degrees:
Human Research Program Elements: (1) BHP:Behavioral Health & Performance (archival in 2017)
Human Research Program Risks: (1) Bmed:Risk of Adverse Behavioral Conditions and Psychiatric Disorders
Human Research Program Gaps: (1) BMed03:We need to identify and quantify the key threats to and promoters of mission relevant behavioral health and performance during autonomous, long duration and/or long distance exploration missions (IRP Rev F)
Flight Assignment/Project Notes: NOTE: End date change to 7/31/2008 from 6/30/2008 (info received 11/2009)

Task Description: Radiation in space or on the moon can result in brain damage that degrades an astronaut's ability to perform cognitive tasks, particularly ones involving changing the course of one's actions. Shifts in personality can also occur. Task demands, sleep deprivation and psychosocial stressors can also degrade cognitive performance. Our goal was a speech monitoring system suitable for space-flight that would detect degraded cognitive ability and stress by means of automatic, ongoing acoustic analysis of an astronaut's speech. The system would provide astronauts and ground-control with timely warnings before profound disability occurs, and provide ongoing assessment of ability to perform. No additional sensors attached to individuals or tasks would be involved; it would be impossible to evade the voice monitoring system. Moreover, our voice analysis techniques preserve confidentiality because the relevant acoustic measures reflect impaired speech motor control rather than message content.

Our procedures are based on recent insight on how brains work. Complex behaviors such as walking, talking, comprehending the meaning of a sentence, or deciding what you should do when circumstance change, involve linked activity in different parts of the brain. The subcortical basal ganglia are structures of the brain that support "circuits" (akin to electrical pathways) connecting different regions of the brain. Independent studies show that circuits involving the basal ganglia regulate motor control, cognition, emotional responses and some aspects of a person's personality. Damage to the basal ganglia, which are sensitive to both radiation and oxygen deprivation (hypoxia), thus can degrade these aspects of behavior. We have confirmed that acoustic measures quantifying slow speech motor control can be used to monitor cognitive impairment induced by hypoxic and cosmic-ray induced insult to the brain, as well as degraded cognitive performance resulting from task difficulty, other stressors and sleep deprivation. Other acoustic metrics can identify sleep deprivation and stress derived from perceived poor performance. We were moving towards an operational system and developed a prototype computer algorithm that automatically measures speech rates in low noise environments.

Our project had two complementary components. 1- Our Everest Space-Analog studies provide the foundation for an operational system that uses acoustic measures of a person's speech to detect cognitive deficits resulting from exposure to radiation in space or on the moon, as well as hypoxia in spacewalks. 2- Our cooperative laboratory study with the NSBRI project directed by Dr. David Dinges at the Medical School of the University of Pennsylvania has yielded a procedure that uses these acoustic measures and additional ones to detect impaired cognitive performance resulting from the stress of task difficulty, as well as psychosocial stressors and sleep deprivation. We developed prototype computer algorithms that automatically derive relevant acoustic measures from running speech.

Everest Space-Analog.

Independent NSBRI research confirms that the basal ganglia are sensitive to radiation; they also are susceptible to oxygen deprivation (hypoxic insult) in climbers breathing thin air at extreme altitudes. Thus we can use climbers ascending Everest as models for the some of the effects of radiation on crews in space. The research is ethical because subjects willfully expose themselves to the dangers of climbing Everest. Climbing Everest entails ascending to a series of high camps. At each camp with progressively lower oxygen, our climber-subjects perform sentence comprehension tests, the Wisconsin Card Sorting Test (WCST), and mental arithmetic tests that simulate operational tasks encountered in spaceflight. WCST performance translates to the ability to change plans when circumstances change. At higher altitudes, error rates on the WCST and arithmetic tests tend to increase and it takes longer to comprehend the meanings of sentences. We have used the BLISS interactive speech analysis computer algorithms developed at Brown University to derive acoustic speech measures that reflect slower motor control. These acoustic metrics track cognitive dysfunction. Our procedures detects lower sentence comprehension or degraded WCST performance 91% of the time. A system that used measures of speech rate to monitor these cognitive deficits would have had a 3% miss rate and 6% "false alarm" rate (decisions that do not reflect with impairment).

Task difficulty, stress and sleep deprivation.

In the Dinges laboratory setting subjects had to perform easy and difficult mental arithmetic tasks. Subjects also performed these tasks after sleep deprivation and with a psychosocial stressor (being informed that their performance was deficient). Measures of slower speech tracked higher error rates and fewer solutions as they performed the more difficult task. Acoustic measures that reflect laryngeal activity identified those subjects who were sleep deprived or subjected to the psychosocial stressor.

Earthbound applications.

Our objective acoustic analyses provide direction for focused intervention for children having verbal apraxia (speech motor sequencing difficulties). Our procedures have been used to evaluate new procedures for the treatment of Parkinson's disease, which involves basal ganglia degeneration. The effects of task difficulty, sleep deprivation and other stressors could be voice monitored in applications ranging from enhancing computer-implemented instruction to safely driving a truck.

Research Impact/Earth Benefits: The techniques we developed for unobtrusively monitoring cognitive status and stress via automated measurement of speech parameters have applications in general aviation. Systems based on these techniques could be used to monitor air crews for gradual effects of partial or slow failure of aircraft pressurization systems. Hypoxia resulting from such depressurization degrades cognitive function and crew members not only are unable to perform their tasks but fail to notice their own impairment, leading to disaster. Speech-based systems could monitor both motor and cognitive dysfunction resulting from stress and sleep deprivation in occupations ranging from air traffic controllers to truck drivers. Measures of vowel and pause durations could be used to pace computer-aided instruction, adjusting the presentation of information to an individual's cognitive ability.

Our project's techniques have already been used to assess the efficacy of new surgical procedures for the treatment of Parkinson's disease. They may also provide instruments that can detect memory loss in the early stages of Alzheimer's disease. Such early detection would permit clinicians to take maximal advantage of therapies that can delay or even arrest further decline. Our techniques may have application to the diagnosis, assessment, and treatment of other human pathologies stemming from impaired basal ganglia function in neural circuits regulating speech production, cognition and personality. These include not only neurodegenerative diseases but also the results of acute insult. For example, hypoxia during birth can lead to verbal apraxia in children - a syndrome where speech motor and orofacial motor control is degraded and which can result in cognitive and linguistic deficits. Our computer-implemented speech analysis techniques identify specific deficits that are not evident by listening to the children; therapy can then be directed towards the remediation of these problems. We also can identify the specific cognitive deficits accompanying many instances of verbal apraxia, again directing treatment. Our research can be useful in identifying genetic and environmental factors underlying the condition. We are extending this line of enquiry to autism, working with specialists in pediatric neurology.

Another potential application is in assessing the truthfulness of verbal statements. Dissimulation generally involves greater cognitive effort, activating more brain structures that would be the case for truthful statements. This yields slower speech which we can readily detect.

Task Progress & Bibliography Information FY2008 
Task Progress: We directed our attention towards deriving speech measures that could have been implemented to monitor cognitive deficits deriving from cosmic-ray induced brain dysfunction, cognitive load, sleep deprivation and other stressors. Structures of the brain that support cognition also take part in regulating motor control. Stressors impeding cognition also slow down motor activity. We confirmed that acoustic measures quantifying slow speech, can be used to monitor cognitive impairment induced by hypoxic and cosmic-ray induced insult to the brain, as well as degraded cognitive performance resulting from task difficulty, other stress and sleep deprivation. We have developed a computer algorithm that automatically measures speech rates in normal environments, up to and including a conference room. We identified signal processing techniques that would have resulted in an operational system. Unfortunately funding to deliver that system was not awarded.

Hypoxic cognitive dysfunction

As our climber-subjects ascended Mount Everest cognitive performance was impaired to varying degrees in different individuals. Set shifting error rates on the Odd-Man-Out (OMO) test, which we used this year to minimize practice effects, increase. OMO performance translates to the ability to adjust to change plans as circumstances change. It generally took longer to comprehend the meanings of sentences. The rate at which the subjects talked slowed down. In our 2007 and 2008 studies, the climber-subjects also performed mental arithmetic tests which simulate many of the cognitive tasks encountered in space-flight. Data from subjects studied between 2003 and 2007 showed that our procedures detect lower sentence comprehension or degraded set-shifting performance 86% of the time. A system that used measures of speech rate to monitor these cognitive deficits would have had a 6% miss rate and 8% "false alarm" rate (decisions that do not reflect with impairment). Data analysis from May 2008, which started in June, is in progress.

Task difficulty, stress and sleep deprivation.

In a laboratory setting subjects had to perform easy and difficult mental arithmetic tasks that simulate some of the tasks performed by astronauts. Similar speech metrics tracked degraded performance on the difficult task. Slower speech and acoustic measures that reflect laryngeal activity identify those subjects who were sleep deprived or subjected to psychosocial stress.

Algorithms for a space-capable system.

We developed a prototype computer algorithm that automatically derives acoustic measures that reflect slow speech rates in normal room environments. Our goal was a speech monitoring system suitable for space-flight that will provide astronauts and ground-control with timely warnings before profound disability occurs, and provide ongoing assessment of ability to perform.

Earthbound applications.

Studies of children having verbal apraxia (speech motor sequencing difficulties) have similar cognitive and speech deficit that appear to derive from damage to basal ganglia.

Bibliography Type: Description: (Last Updated: 02/27/2014) 

Show Cumulative Bibliography Listing
 
Articles in Peer-reviewed Journals Lieberman P, Mc Carthy R. "Tracking the evolution of human language and speech: comparing vocal tracks to identify speech capabilities." Expedition. 2007 Summer;49(2):15-20. http://www.penn.museum/documents/publications/expedition/PDFs/49-2/Lieberman.pdf , Jul-2007
Articles in Peer-reviewed Journals Kugler SL, Bali B, Lieberman P, Strug L, Gagnon B, Murphy PL, Clarke T, Greenberg DA, Pal DK. "An autosomal dominant genetically heterogeneous variant of rolandic epilepsy and speech disorder." Epilepsia. 2008 Jun;49(6):1086-90. Epub 2008 Jan 31. http://dx.doi.org/10.1111/j.1528-1167.2007.01517.x ; PubMed PMID: 18248446; PubMed Central PMCID: PMC2435390 , Jun-2008
Articles in Peer-reviewed Journals Pal DK, Li W, Clarke T, Lieberman P, Strug LJ. "Pleiotropic effects of the 11p13 locus on developmental verbal dyspraxia and EEG centrotemporal sharp waves." Genes Brain Behav. 2010 Nov;9(8):1004-12. http://dx.doi.org/10.1111/j.1601-183X.2010.00648.x ; PubMed PMID: 20825490 , Nov-2010
Project Title:  Speech monitoring of cognitive deficits and stress Reduce
Fiscal Year: FY 2007 
Division: Human Research 
Research Discipline/Element:
HRP BHP:Behavioral Health & Performance (archival in 2017)
Start Date: 07/01/2004  
End Date: 06/30/2008  
Task Last Updated: 02/07/2008 
Download report in PDF pdf
Principal Investigator/Affiliation:   Lieberman, Philip  Ph.D. / Brown University 
Address:  Department of Cognitive and Linguistic Sciences 
190 Thayer Street Room 120 
Providence , RI 02912 
Email: philip_lieberman@brown.edu 
Phone: 401-863-1857  
Congressional District:
Web:  
Organization Type: UNIVERSITY 
Organization Name: Brown University 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Dinges, David  University of Pennsylvania 
Project Information: Grant/Contract No. NCC 9-58-NBPF00406 
Responsible Center: NSBRI 
Grant Monitor:  
Center Contact:   
Solicitation / Funding Source: 2003 Biomedical Research & Countermeasures 03-OBPR-04 
Grant/Contract No.: NCC 9-58-NBPF00406 
Project Type: GROUND 
Flight Program:  
TechPort: No 
No. of Post Docs:
No. of PhD Candidates:
No. of Master's Candidates:
No. of Bachelor's Candidates:
No. of PhD Degrees:
No. of Master's Degrees:
No. of Bachelor's Degrees:
Human Research Program Elements: (1) BHP:Behavioral Health & Performance (archival in 2017)
Human Research Program Risks: (1) Bmed:Risk of Adverse Behavioral Conditions and Psychiatric Disorders
Human Research Program Gaps: (1) BMed03:We need to identify and quantify the key threats to and promoters of mission relevant behavioral health and performance during autonomous, long duration and/or long distance exploration missions (IRP Rev F)
Task Description: Radiation in space or on the moon can result in brain damage that degrades an astronaut’s ability to perform cognitive tasks, particularly ones involving changing the course of one’s actions. Shifts in personality can also occur. Task demands, sleep deprivation and psychosocial stressors can also degrade cognitive performance. Our goal is a speech monitoring system suitable for space-flight that will detect degraded cognitive ability and stress by means of automatic, ongoing acoustic analysis of an astronaut’s speech. The system would provide astronauts and ground-control with timely warnings before profound disability occurs, and provide ongoing assessment of ability to perform. No additional sensors attached to individuals or tasks would be involved; it would be impossible to evade the voice monitoring system. Moreover, our voice analysis techniques preserve confidentiality because the relevant acoustic measures reflect impaired speech motor control rather than message content.

Our procedures are based on recent insight on how brains work. Complex behaviors such as walking, talking, comprehending the meaning of a sentence, or deciding what you should do when circumstance change, involve linked activity in different parts of the brain. The subcortical basal ganglia are structures of the brain that support "circuits" (akin to electrical pathways) connecting different regions of the brain. Independent studies show that circuits involving the basal ganglia regulate motor control, cognition, emotional responses and some aspects of a person's personality. Damage to the basal ganglia, which are sensitive to both radiation and oxygen deprivation (hypoxia), thus can degrade these aspects of behavior. We have confirmed that acoustic measures quantifying slow speech motor control can be used to monitor cognitive impairment induced by hypoxic and cosmic-ray induced insult to the brain, as well as degraded cognitive performance resulting from task difficulty, other stressors and sleep deprivation. Other acoustic metrics can identify sleep deprivation and stress derived from perceived poor performance. We are moving towards an operational system and have developed a prototype computer algorithm that automatically measures speech rates in low noise environments.

Our project has two complementary components. 1- Our Everest Space-Analog studies provide the foundation for an operational system that uses acoustic measures of a person’s speech to detect cognitive deficits resulting from exposure to radiation in space or on the moon, as well as hypoxia in spacewalks. 2- Our cooperative laboratory study with the NSBRI project directed by Dr. David Dinges at the Medical School of the University of Pennsylvania has yielded a procedure that uses these acoustic measures and additional ones to detect impaired cognitive performance resulting from the stress of task difficulty, as well as psychosocial stressors and sleep deprivation. We have developed prototype computer algorithms that automatically derive relevant acoustic measures from running speech

Everest Space–Analog.

Independent NSBRI research confirms that the basal ganglia are sensitive to radiation; they also are susceptible to oxygen deprivation (hypoxic insult) in climbers breathing thin air at extreme altitudes. Thus we can use climbers ascending Everest as models for the some of the effects of radiation on crews in space. The research is ethical because subjects willfully expose themselves to the dangers of climbing Everest. Climbing Everest entails ascending to a series of high camps. At each camp with progressively lower oxygen, our climber–subjects perform sentence comprehension tests, the Wisconsin Card Sorting Test (WCST), and mental arithmetic tests that simulate operational tasks encountered in spaceflight. WCST performance translates to the ability to change plans when circumstances change. At higher altitudes, error rates on the WCST and arithmetic tests tend to increase and it takes longer to comprehend the meanings of sentences. We have used the BLISS interactive speech analysis computer algorithms developed at Brown University to derive acoustic speech measures that reflect slower motor control. These acoustic metrics track cognitive dysfunction. Our procedures detect lower sentence comprehension or degraded WCST performance 91% of the time. A system that used measures of speech rate to monitor these cognitive deficits would have had a 3% miss rate and 6% "false alarm” rate (decisions that do not reflect with impairment).

Task difficulty, stress and sleep deprivation.

In the Dinges laboratory setting subjects had to perform easy and difficult mental arithmetic tasks. Subjects also performed these tasks after sleep deprivation and with a psychosocial stressor (being informed that their performance was deficient). Measures of slower speech tracked higher error rates and fewer solutions as they performed the more difficult task. Acoustic measures that reflect laryngeal activity identified those subjects who were sleep deprived or subjected to the psychosocial stressor.

Earthbound applications

Our objective acoustic analyses provide direction for focused intervention for children having verbal apraxia (speech motor sequencing difficulties). Our procedures have been used to evaluate new procedures for the treatment of Parkinson's disease, which involves basal ganglia degeneration. The effects of task difficulty, sleep deprivation and other stressors could be voice monitored in applications ranging from enhancing computer-implemented instruction to safely driving a truck.

Research Impact/Earth Benefits: The techniques we are developing for unobtrusively monitoring cognitive status and stress via automated measurement of speech parameters have applications in general aviation. Systems based on these techniques could be used to monitor air crews for gradual effects of partial or slow failure of aircraft pressurization systems. Hypoxia resulting from such depressurization degrades cognitive function and crew members not only are unable to perform their tasks but fail to notice their own impairment, leading to disaster. Speech-based systems could monitor both motor and cognitive dysfunction resulting from stress and sleep deprivation in occupations ranging from air traffic controllers to truck drivers. Measures of vowel and pause durations could be used to pace computer-aided instruction, adjusting the presentation of information to an individual’s cognitive ability.

Our project’s techniques have already been used to assess the efficacy of new surgical procedures for the treatment of Parkinson’s disease. They may also provide instruments that can detect memory loss in the early stages of Alzheimer's disease. Such early detection would permit clinicians to take maximal advantage of therapies that can delay or even arrest further decline. Our techniques may have application to the diagnosis, assessment, and treatment of other human pathologies stemming from impaired basal ganglia function in neural circuits regulating speech production, cognition and personality. These include not only neurodegenerative diseases but also the results of acute insult. For example, hypoxia during birth can lead to verbal apraxia in children – a syndrome where speech motor and orofacial motor control is degraded and which can result in cognitive and linguistic deficits. Our computer-implemented speech analysis techniques identify specific deficits that are not evident by listening to the children; therapy can then be directed towards the remediation of these problems. We also can identify the specific cognitive deficits accompanying many instances of verbal apraxia, again directing treatment.

Our research can be useful in identifying genetic and environmental factors underlying the condition. We will be extending this line of enquiry to autism, working with specialists in pediatric neurology.

Another potential application is in assessing the truthfulness of verbal statements. Dissimulation generally involves greater cognitive effort, activating more brain structures that would be the case for truthful statements. This yields slower speech which we can readily detect.

Task Progress & Bibliography Information FY2007 
Task Progress: We have directed our attention towards deriving speech measures that can be implemented in a space-capable system monitoring cognitive deficits deriving from cosmic-ray induced brain dysfunction, cognitive load, sleep deprivation and other stressors. Structures of the brain that support cognition also take part in regulating motor control. Stressors impeding cognition also slow down motor activity. We have confirmed that acoustic measures quantifying slow speech, can be used to monitor cognitive impairment induced by hypoxic and cosmic-ray induced insult to the brain, as well as degraded cognitive performance resulting from task difficulty, other stress and sleep deprivation. We have moved towards an operational system and have developed a prototype computer algorithm that automatically measures speech rates in low noise environments.

Hypoxic cognitive dysfunction

As our climber-subjects ascended Mount Everest cognitive performance was impaired to varying degrees in different individuals. Set shifting error rates on the Wisconsin Card Sorting Test (WCST) tended to increase. WCST performance translates to the ability to adjust to change plans as circumstances change. It generally took longer to comprehend the meanings of sentences. The rate at which the subjects talked slowed down. In our 2007 study, the climber-subjects also performed mental arithmetic tests which simulate many of the cognitive tasks encountered in space-flight. Combined data for subjects studied between 2003 and 2006 showed that our procedures detects lower sentence comprehension or degraded WCST performance 91% of the time. A system that used measures of speech rate to monitor these cognitive deficits would have had a 3% miss rate and 6% "false alarm” rate (decisions that do not reflect with impairment). Data analysis from May 2007, which started in June, is in progress.

Task difficulty, stress and sleep deprivation.

In a laboratory setting subjects had to perform easy and difficult mental arithmetic tasks that simulate some of the tasks performed by astronauts. Similar speechmetrics tracked degraded performance on the difficult task. Slower speech and acoustic measures that reflect laryngeal activity identify those subjects who were sleep deprived or subjected to psychosocial stress.

Algorithms for a space-capable system

We developed a prototype computer algorithm that automatically derives acoustic measures that reflect slow speech rates from discourse in relatively noise-free environments. Our goal is a speech monitoring system suitable for space-flight that will provide astronauts and ground-control with timely warnings before profound disability occurs, and provide ongoing assessment of ability to perform.

Earthbound applications

Studies of children having verbal apraxia (speech motor sequencing difficulties) show an additional earthbound application. They have cognitive and speech deficits occur that appear to derive from damage to basal ganglia.

Bibliography Type: Description: (Last Updated: 02/27/2014) 

Show Cumulative Bibliography Listing
 
Articles in Peer-reviewed Journals Lieberman P, McCarthy R. "Tracking the evolution of language and speech: comparing vocal tracks to identify speech capabilities." Expedition. 2007 Summer;49(2):15-20. http://www.penn.museum/documents/publications/expedition/PDFs/49-2/Lieberman.pdf , Jul-2007
Articles in Peer-reviewed Journals Hochstadt J, Nakano H, Lieberman P, Friedman J. "The roles of sequencing and verbal working memory in sentence comprehension deficits in Parkinson's disease." Brain Lang. 2006 Jun;97(3):243-57. PMID: 16332387 , Jun-2006
Articles in Peer-reviewed Journals Lieberman P. "The evolution of human speech: its anatomical and neural bases." Current Anthropology 2007 Feb;48(1):39-66. http://dx.doi.org/10.1086/509092 , Feb-2007
Books/Book Chapters Lieberman P. "Toward an evolutionary biology of language." Cambridge. Mass. : Harvard University Press, 2006., May-2006
Project Title:  Speech monitoring of cognitive deficits and stress Reduce
Fiscal Year: FY 2006 
Division: Human Research 
Research Discipline/Element:
HRP BHP:Behavioral Health & Performance (archival in 2017)
Start Date: 07/01/2004  
End Date: 06/30/2008  
Task Last Updated: 01/08/2007 
Download report in PDF pdf
Principal Investigator/Affiliation:   Lieberman, Philip  Ph.D. / Brown University 
Address:  Department of Cognitive and Linguistic Sciences 
190 Thayer Street Room 120 
Providence , RI 02912 
Email: philip_lieberman@brown.edu 
Phone: 401-863-1857  
Congressional District:
Web:  
Organization Type: UNIVERSITY 
Organization Name: Brown University 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Dinges, David  University of Pennsylvania 
Project Information: Grant/Contract No. NCC 9-58-NBPF00406 
Responsible Center: NSBRI 
Grant Monitor:  
Center Contact:   
Solicitation / Funding Source: 2003 Biomedical Research & Countermeasures 03-OBPR-04 
Grant/Contract No.: NCC 9-58-NBPF00406 
Project Type: GROUND 
Flight Program:  
TechPort: No 
No. of Post Docs:
No. of PhD Candidates:
No. of Master's Candidates:
No. of Bachelor's Candidates:
No. of PhD Degrees:
No. of Master's Degrees:
No. of Bachelor's Degrees:
Human Research Program Elements: (1) BHP:Behavioral Health & Performance (archival in 2017)
Human Research Program Risks: (1) Bmed:Risk of Adverse Behavioral Conditions and Psychiatric Disorders
Human Research Program Gaps: (1) BMed03:We need to identify and quantify the key threats to and promoters of mission relevant behavioral health and performance during autonomous, long duration and/or long distance exploration missions (IRP Rev F)
Task Description: Our goal is a speech monitoring system that will automatically and unobtrusively monitor the effects of stress and neurological impairment on astronauts’ ability to perform in extended deep-space missions. The project establishes a synergy between a space-analog study of climbers ascending Mount Everest, who experience stress and neural dysfunction similar to that from exposure to cosmic rays, and laboratory studies of task-induced stress. Our proposed system will detect changes before performance is severely impaired, giving crews time to invoke countermeasures. Because the system's acoustic measures are largely outside of conscious control, crewmembers will not be able to improve with practice or “fool” the system. It will protect privacy and mission security because it does not depend on the content of speech, only its acoustics.

Task induced stress and cognitive overload. In the past year, working with David Dinges’s NSBRI-funded group at the University of Pennsylvania, we validated robust acoustic measures that track task-induced stress. We measured the duration of the utterances and pauses of the spoken responses made by Dinges’s subjects while performing mental arithmetic exercises that varied in difficulty. These acoustic parameters reflect the rate at which the subjects performed the task. We also measured the fundamental frequency of phonation (F0), which reflects the pitch of the subjects’ voices. Utterance and pause durations discriminated the high and low-stress conditions for the 24 subjects measured with total certainty. Mean and maximum F0 and F0 range were significantly greater in the high-stress condition for 23 of the 24 subjects. The subjects all performed the tests on two different days. The acoustic parameters identified subjects who appear to have been sleep deprived when they performed the arithmetic tests the second time.

Exposure to cosmic rays. We also advanced research on speech and cognition in climbers on Everest. Deep-space missions will expose crews to cosmic rays, which may damage the brain, especially the basal ganglia and hippocampus. These subcortical brain structures are also vulnerable to hypoxia (oxygen deprivation) at the extreme altitudes on Everest. Thus we use climbers on Everest as models for crews in space. Everest also resembles long-term spaceflight in that a small group in close contact must make critical decisions in life-threatening situations. The research is ethical because subjects willfully expose themselves to the dangers of climbing Everest. The basal ganglia are linked with cortical regions in circuits regulating motor control, cognition and personality. A major function of the basal ganglia is regulating sequences of motor acts and cognitive operations; previous studies show impairment of motor control as well as cognitive abilities that require processing sequences, such as sentence comprehension and set-shifting.

Basal ganglia dysfunction slows speech down and impairs sequential speech motor acts such as the production of voice-onset time (VOT). VOT is the time for word-initial stop consonants, between the noise “burst” when an oral tract closure (e.g., the lips for [b] or [p]) is opened and the onset of the vowel. VOTs for voiced stops like [b] are usually shorter than 20 msec, much less than those for voiceless stops like [p]. These acoustic speech parameters in our Everest study correlate with cognitive set shifting deficits that translate into an inability to change plans when circumstances dictate, as well as slowing down the ability to comprehend the meaning of sentences. VOT ranges converge and speech slows down. We obtain speech measurements by using radios and PDAs recording climbers reading words at different altitudes. We give the Wisconsin Card Sorting Task, which requires subjects to sort stimuli using different criteria and to shift criteria. We administer sentence-picture matching tests of language comprehension. Climbers also use the PDAs to take tests of verbal and special working memory and choice reaction time on the system, developed by Stephen Kosslyn’s NSBRI research group at Harvard. We added an implicit memory test reflecting hippocampal function to the PDA test battery. Our recent Everest results show that we can track impairment on sentence comprehension and set-shifting using acoustic measures of speech rate similar to that used for task-induced stress, achieving a “hit “ rate of 81% by noting instances of slow speech. VOT convergence adds to the certainty of the speech monitoring technique for some subjects.

These findings set the stage for an operational system. Normative data for individual astronauts can be obtained before flight, permitting adjustment to individual speaking rates and VOTs. Our techniques have already been used to evaluate new treatments for Parkinson’s disease and have potential for detecting early Alzheimer’s disease. Similar techniques could be used in general aviation, where hypoxia has led to disasters. We are applying the techniques to assess verbal apraxia, a disorder affecting language and cognition in children, with the aim of isolating genetic anomalies underlying this condition. Moreover, our techniques have more general applications in detecting sleep-deprivation, monitoring stress in occupations like air traffic control, pacing computer-implemented instruction to meet individual cognitive capabilities. There also are possible forensic applications in assessing the truthfulness of statements since dissimulation generally requires greater mental effort with concomitant slower speech.

Research Impact/Earth Benefits: The techniques we are developing for unobtrusively monitoring cognitive status and stress via automated measurement of speech parameters have applications in general aviation. Systems based on these techniques could be used to monitor air crews for gradual effects of partial or slow failure of aircraft pressurization systems. The hypoxia resulting from such depressurization – which degrades cognitive function so that crew members not only are unable to perform their tasks but fail to notice their own impairment – has led to flight disasters in the past. Speech-based systems could monitor both motor and cognitive dysfunction resulting from stress and sleep deprivation in occupations ranging from air traffic controllers to truck drivers.

Our project’s techniques have already been used to assess the efficacy of new surgical procedures for the treatment of Parkinson’s disease. They may also provide instruments that can detect memory loss in the early stages of Alzheimer's disease. Such early detection would permit clinicians to take maximal advantage of therapies that can delay or even arrest further decline. Our techniques may have application to the diagnosis, assessment, and treatment of other human pathologies stemming from impaired basal ganglia function in neural circuits regulating speech production, cognition and personality. These include not only neurodegenerative diseases but also the results of acute insult. For example, hypoxia during birth can lead to verbal apraxia in children – a syndrome where speech motor and orofacial motor control is degraded and which can result in cognitive and linguistic deficits. In a pilot study we have already applied our techniques to the diagnosis of verbal apraxia; we aim to use results of this research in isolating potential genetic anomalies underlying the condition.

Another area is stress analysis and the assessing the truthfulness of verbal statements. Dissimulation generally involves greater cognitive effort, activating more brain structures that would be the case for truthful statements. This yields slower speech which we can readily detect. Similar measures of utterance duration and pauses can be used to pace computer-aided instruction, adjusting the presentation of information to an individual’s cognitive ability.

Task Progress & Bibliography Information FY2006 
Task Progress: We have directed our attention towards a space-capable system monitoring both cognitive deficits deriving from cosmic-ray induced brain dysfunction and cognitive load. Acoustic measures of slow speech can be used to monitor cognitive impairment induced by hypoxic and cosmic-ray induced insult to the brain, as well as degraded cognitive performance resulting from task-induced stress. These finding facilitate system development since acoustic measures quantifying slow speech can be more readily automatized than other parameters.

Hypoxic cognitive dysfunction

As in previous years, as our climber-subjects ascended Mount Everest cognitive performance was impaired to varying degrees in different individuals. Set shifting error rates on the Wisconsin Card Sorting Test (WCST) tended to increase. WCST performance translates to the ability to adjust to change plans as circumstances change. It generally took longer to comprehend the meanings of sentences. The rate at which the subjects talked slowed down, quantified by longer vowel durations. Slowdowns in sentence comprehension or degraded WCST performance co-occurred with increases in vowel duration 91% of the time. A system that used increased vowel duration to monitor these cognitive deficits would have had a 91% “hit” rate and a 3% miss rate. Increased vowel duration would have yielded a 6%"false alarm” rate (increases in vowel duration that were not concomitant with cognitive impairment). Other acoustic parameters, such as voice-onset-time, the fundamental frequency of phonation (F0), and rapid F0 fluctuations, “jitter,” increase the accuracy of voice monitoring of cognitive impairment.

Task-induced stress

In a laboratory experiment in which subjects had to perform both an easy and a difficult mental arithmetic task, the duration of their spoken responses and pause durations tracked cognitive difficulties 100 % of the time. All 24 subjects spoke more slowly when they had to perform the more difficult task. Slower speech also appears to be a means of identifying those subjects who were sleep deprived.

Algorithms for a space-capable system

We are developing algorithms to automatically derive acoustic measures of slow speech. Our goal is a speech monitoring system suitable for space-flight that will provide astronauts and ground-control with timely warnings of brain dysfunction before profound disability occurs and that will assess the crews’ ability to perform.

Earthbound application

A pilot study of nine children having verbal apraxia (speech motor sequencing difficulties) shows an additional earthbound application. Our findings show that they have cognitive deficits similar in nature to hypoxic Everest climbers that appear to derive from damage to neural circuits involving basal ganglia. Clinical studies show damage to the basal ganglia in many of these children owing to difficult births in which oxygen flow was interrupted.

Bibliography Type: Description: (Last Updated: 02/27/2014) 

Show Cumulative Bibliography Listing
 
Articles in Peer-reviewed Journals Hochstadt J, Nakano H, Lieberman P, Friedman J. "The Roles of Sequencing and Verbal Working Memory in Sentence Comprehension Deficits in Parkinson's Disease." Brain and Language, in press, June 2006. , Jun-2006
Articles in Peer-reviewed Journals Lieberman P. "On the evolution of human speech: Its anatomical and neural bases." Current Anthropology. In Press, June 2006. , Jun-2006
Awards Lieberman P. "Invited speaker at the International Morris Symposium on the Evolution of Language held at the State University of New York at Stony Brook, October 2005." Oct-2005
Books/Book Chapters Lieberman P. "Toward an evolutionary biology of language." Cambridge, Mass. : Harvard University Press, 2006., Jun-2006
Books/Book Chapters Lieberman P. "The FOXP2 gene, human cognition and language." in "Integrative approaches to human health and evolution : proceedings of the International Symposium: Integrative Approaches to Human and Evolution held in Madrid, Spain, between 18 and 20, April 2005." Ed. T. C. Brommage, E. Aguirre and A. Perez-Ochoa. Amsterdam ; San Diego, Calif. : Elsevier, 2006. (International Congress Series, vol. 1296). http://dx.doi.org/10.1016/j.ics.2006.03.039 , Jun-2006
Project Title:  Speech monitoring of cognitive deficits and stress Reduce
Fiscal Year: FY 2005 
Division: Human Research 
Research Discipline/Element:
HRP BHP:Behavioral Health & Performance (archival in 2017)
Start Date: 07/01/2004  
End Date: 06/30/2008  
Task Last Updated: 11/02/2005 
Download report in PDF pdf
Principal Investigator/Affiliation:   Lieberman, Philip  Ph.D. / Brown University 
Address:  Department of Cognitive and Linguistic Sciences 
190 Thayer Street Room 120 
Providence , RI 02912 
Email: philip_lieberman@brown.edu 
Phone: 401-863-1857  
Congressional District:
Web:  
Organization Type: UNIVERSITY 
Organization Name: Brown University 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Dinges, David  University of Pennsylvania 
Project Information: Grant/Contract No. NCC 9-58-NBPF00406 
Responsible Center: NSBRI 
Grant Monitor:  
Center Contact:   
Solicitation / Funding Source: 2003 Biomedical Research & Countermeasures 03-OBPR-04 
Grant/Contract No.: NCC 9-58-NBPF00406 
Project Type: GROUND 
Flight Program:  
TechPort: No 
No. of Post Docs:
No. of PhD Candidates:
No. of Master's Candidates:
No. of Bachelor's Candidates:
No. of PhD Degrees:
No. of Master's Degrees:
No. of Bachelor's Degrees:
Human Research Program Elements: (1) BHP:Behavioral Health & Performance (archival in 2017)
Human Research Program Risks: (1) Bmed:Risk of Adverse Behavioral Conditions and Psychiatric Disorders
Human Research Program Gaps: (1) BMed03:We need to identify and quantify the key threats to and promoters of mission relevant behavioral health and performance during autonomous, long duration and/or long distance exploration missions (IRP Rev F)
Task Description: Our project’s goal is to derive and validate acoustic measures of speech that permit automatic, unobtrusive on-line monitoring of effects of stress and neurological impairment on astronauts’ ability to perform in extended deep-space missions. Our project integrates ongoing NSBRI research aimed at systems that monitor the ability to perform using facial markers of stress, acoustic measures of stress and impaired cognition, and tests that permit rapid psychometric assessment of cognitive status. The project establishes a synergy between a space-analog study of climbers ascending Mount Everest, who experience stress and neurologic impairment similar to that from exposure to cosmic rays, and laboratory studies of task-induced stress. Our proposed system will detect psychological changes before performance is severely impaired, giving crews time to invoke countermeasures. It will derive acoustic parameters from conversation via automated procedures and without intrusive devices. Because the acoustic measures are largely outside of conscious control, crewmembers will not be able to improve with practice or “fool” the system. It will protect privacy and mission security because it does not depend on the content of speech, only its acoustics. In the past year, working with David Dinges’s NSBRI-funded group at the University of Pennsylvania, we validated measurements of speech acoustics as stress indices. We measured fundamental frequency of phonation (F0), word duration and pause duration in spoken responses made by Dinges’s subjects while performing a Stroop task in high-stress (incongruent) and low-stress (congruent) conditions. Mean, minimum and maximum F0; F0 range; pause duration and word length were significantly greater in the high-stress condition. Moreover, a process based on three measures – pause duration, word duration and maximum F0 – discriminated with total certainty between high- and low-stress conditions for the 42 subjects measured. We also continued research on speech and cognition in climbers on Everest. Deep-space missions will expose crews to cosmic rays, which may damage the basal ganglia and hippocampus. These subcortical brain structures are also vulnerable to oxygen deprivation. Humans cannot acclimatize to the low oxygen levels at altitudes above Everest Base Camp. Thus we use climbers on Everest as models for crews in space. Everest also resembles long-term spaceflight in that a small group in close contact must make critical decisions in life-threatening situations. The research is ethical because subjects willfully expose themselves to the dangers of climbing Everest. The basal ganglia are linked with cortical regions in circuits regulating motor control, cognition and personality. A major function of the basal ganglia is regulating sequences of motor acts and cognitive operations. Parkinson’s disease (PD), which results from dysfunction of the basal ganglia, impairs not only motor control but cognitive abilities that require processing sequences, such as sentence comprehension and set-shifting. PD also impairs sequential speech motor acts such as production of voice-onset time (VOT). VOT is the time, for word-initial stop consonants, between the noise “burst” when an oral tract closure (e.g., the lips for [b] or [p]) is opened and the onset of the vowel. VOTs for voiced stops like [b] are usually at least 20 msec less than those for voiceless stops like [p]; in PD patients this difference may be reduced. This VOT convergence correlates with sentence comprehension deficits in PD. Hypoxia on Everest yields speech and cognitive deficits similar to if less extreme than those in PD. We obtain speech measurements by using radios to record climbers reading words at different altitudes. We also give the Wisconsin Card Sorting Task, which requires subjects to sort stimuli using different criteria and to shift criteria based on feedback. Climbers carry picture booklets so that we can administer sentence-picture matching tests of language comprehension. Climbers also take the MiniCog test battery, developed in Stephen Kosslyn’s NSBRI-funded research at Harvard, on Palm Pilot PDAs that they carry. We have in recent years added an implicit memory test reflecting hippocampal function. Past Everest results show that VOT convergence and increased vowel duration at high altitudes track impairment on cognitive tasks dependent on basal ganglia function, including sentence comprehension, set-shifting and working memory. Preliminary analyses indicate that hypoxia also impairs hippocampal function. In April-May 2005 we obtained speech recordings and cognitive test results for 20 more climber-subjects. Preliminary analyses appear consistent with our past results. Results on implicit memory were also obtained for 2 additional climbers. Our findings set the stage for an operational system for detecting potentially dangerous shifts in psychological state via speech acoustic measures. Our techniques may be used to monitor PD and have potential for detecting early Alzheimer’s disease. We are applying the techniques to assess verbal apraxia, a disorder affecting language and cognition in children, with the aim of isolating genetic anomalies underlying this condition. Our techniques have potential applications in general aviation, where hypoxia has led to disasters, and for monitoring stress in occupations like air traffic control. In the coming year we will complete analysis of this year’s Everest data and plan for a spring 2006 Everest research trip. We will better characterize our stress metrics by correlating them with physiological and behavioral indices obtained by the Dinges group. We plan to validate other acoustic stress measures, namely jitter (the period-by-period variability in wavelength) and spectral properties. We also intend to validate our metrics of task-induced stress, derived in lab recordings, by testing them on speech recordings obtained in stressful conditions on Everes

Research Impact/Earth Benefits: The techniques we are developing for unobtrusively monitoring cognitive status and stress via automated measurement of speech parameters may have applications in general aviation. Systems based on these techniques could be used to monitor air crews for gradual effects of partial or slow failure of aircraft pressurization systems. The hypoxia resulting from such depressurization – which degrades cognitive function so that crew members not only are unable to perform their tasks but fail to notice their own impairment – has led to flight disasters in the past. A speech-based system may also be useful in monitoring motor and cognitive dysfunction resulting from stress and sleep deprivation in, for example, air traffic controllers. Our project’s techniques have already been used to assess the efficacy of new surgical procedures for the treatment of Parkinson’s disease. They can be used to monitor disease state in Parkinson’s. They may also provide instruments that can detect memory loss in the early stages of Alzheimer's disease. Such early detection would permit clinicians to take maximal advantage of therapies, now under development, that can delay or even arrest further decline. In addition to Parkinson’s disease, our techniques may have application to the diagnosis, assessment, and treatment of other human pathologies stemming from impaired dopaminergic basal ganglia function in neural circuits regulating speech production, cognition and personality. These include not only neurodegenerative diseases but also the results of acute insult. For example, hypoxia during birth can lead to verbal apraxia in children – a syndrome where speech motor and orofacial motor control is degraded and which can result in cognitive and linguistic deficits. In an independent pilot study we have already applied our techniques to the diagnosis and possible remediation of verbal apraxia; we aim to use results of this research in isolating potential genetic anomalies underlying the condition.

Task Progress & Bibliography Information FY2005 
Task Progress: One of our project’s main goals is to derive and validate robust, reliable indices of cognitive degradation due to impairment of subcortical brain structures – the basal ganglia and hippocampus – that are vulnerable to damage from cosmic radiation during long-term spaceflight. Because these same structures are susceptible to hypoxic damage, we study climbers ascending Mount Everest as an analog to crews in deep space. Research on Parkinson’s disease (PD) indicates that voice-onset time (VOT), a speech parameter whose production involves sequencing the opening of a supralaryngeal closure and the onset of laryngeal phonation, depends on intact basal ganglia function - consistent with the structure’s role as a “sequencing engine” in motor and cognitive domains. VOT “convergence” in PD correlates with deficits in comprehending sentences with moderately complex syntax. In past NSBRI-funded research on Everest we found VOT “convergence” and lengthened vowels in climbers at high altitudes. Moreover, we found reliable relationships between these changes (which are not seen in all climbers) and impaired performance on cognitive tests reflecting basal ganglia function, including sentence comprehension, cognitive set-shifting and working memory. (Such relationships were not seen for tasks that do not depend on the basal ganglia.) Our latest Everest expedition, in April-May 2005, involved several improvements on past years’ procedures. We stressed the need for familiarization with the Mini-Cog test battery (developed in Kosslyn’s NSBRI-funded research). This battery tests a variety of cognitive capacities, including both ones that are sensitive to basal ganglia function and ones that are not. Familiarization allows subjects to reach performance plateaus before attempting to summit Everest; otherwise comparisons between lower and higher altitudes may reflect both performance decrements due to hypoxia and improvements due to practice. We also used fewer Mini-Cog tests to reduce time demands while still adequately assessing cognition. Our research team substantially expanded our data base, obtaining speech recordings and cognitive test results for 20 climbers. Preliminary analyses appear consistent with our past results. We also obtained results on an implicit memory test reflecting hippocampal function for two additional climbers, adding to data obtained in 2004. Our other main goal is to derive and validate speech acoustic indices of stress based on speech acoustic properties. This past year we obtained results showing that mean, minimum and maximum fundamental frequency of phonation (F0); F0 range; pause duration and word length were significantly greater in spoken responses during a high-stress, incongruent Stroop task than a low-stress, congruent Stroop task. Moreover, a process based on three measures – pause duration, word duration and maximum F0 – discriminated w

Bibliography Type: Description: (Last Updated: 02/27/2014) 

Show Cumulative Bibliography Listing
 
Books/Book Chapters Lieberman P. "Towards an evolutionary biology of language." Harvard University Press, in press as of January 2004., Jan-2004
Presentation Hochstadt, J., and Lieberman, P. "Eye Movements During Sentence-Picture Matching Track Linguistic Processing: Novel Findings in Normal Participants and Parkinson's Disease Patients" N/A

Apr-2005

Presentation Lieberman, P. "The FOXP2 gene and human motor, cognitive and linguistic ability" N/A

Apr-2005

Presentation Lieberman, P. "The NSBRI Everest Space-Analog Study" N/A

Jun-2005

Presentation Lieberman, P., J. Hochstadt, A. Morey, M. Larson, S. Mather, and J. Mertus "Speech Monitoring of Cognitive Deficits and Stress: A Space-Analog Study" N/A

Jan-2005

Project Title:  Speech monitoring of cognitive deficits and stress Reduce
Fiscal Year: FY 2004 
Division: Human Research 
Research Discipline/Element:
HRP BHP:Behavioral Health & Performance (archival in 2017)
Start Date: 07/01/2004  
End Date: 06/30/2008  
Task Last Updated: 03/29/2006 
Download report in PDF pdf
Principal Investigator/Affiliation:   Lieberman, Philip  Ph.D. / Brown University 
Address:  Department of Cognitive and Linguistic Sciences 
190 Thayer Street Room 120 
Providence , RI 02912 
Email: philip_lieberman@brown.edu 
Phone: 401-863-1857  
Congressional District:
Web:  
Organization Type: UNIVERSITY 
Organization Name: Brown University 
Joint Agency:  
Comments:  
Project Information: Grant/Contract No. NCC 9-58-NBPF00406 
Responsible Center: NSBRI 
Grant Monitor:  
Center Contact:   
Solicitation / Funding Source: 2003 Biomedical Research & Countermeasures 03-OBPR-04 
Grant/Contract No.: NCC 9-58-NBPF00406 
Project Type: GROUND 
Flight Program:  
TechPort: No 
No. of Post Docs:  
No. of PhD Candidates:  
No. of Master's Candidates:  
No. of Bachelor's Candidates:  
No. of PhD Degrees:  
No. of Master's Degrees:  
No. of Bachelor's Degrees:  
Human Research Program Elements: (1) BHP:Behavioral Health & Performance (archival in 2017)
Human Research Program Risks: (1) Bmed:Risk of Adverse Behavioral Conditions and Psychiatric Disorders
Human Research Program Gaps: (1) BMed03:We need to identify and quantify the key threats to and promoters of mission relevant behavioral health and performance during autonomous, long duration and/or long distance exploration missions (IRP Rev F)
Task Description: The goal of this project is to derive and validate acoustic measures of speech that permit automatic and unobtrusive on-line monitoring of the effects of stress and neurological impairment on astronauts' ability to perform in extended deep space missions. Our project will integrate and validate ongoing NSBRI research projects aimed at systems that monitor astronauts ability to perform using (a) video recognition of facial markers of stress, (b) acoustic measures of stress and impaired cognition and (c) psychometric test procedures that permit the rapid assessment of cognitive ability. The project will establish a synergy between (1) a space-analog study involving climbers ascending Mount Everest, where life-threatening stress and neurologic impairment similar to that which may result from exposure to cosmic rays in deep-space missions occurs, and (2) the findings of laboratory-based studies of task-induced stress.

Research Impact/Earth Benefits:

Task Progress & Bibliography Information FY2004 
Task Progress: New project for FY2004.

Bibliography Type: Description: (Last Updated: 02/27/2014) 

Show Cumulative Bibliography Listing
 
 None in FY 2004