Significance:

If we were able to create a retinal imaging-based quantitative tool to establish the presence of stroke, and stroke type (ischemic versus hemorrhagic), we would be able to indicate the appropriate treatment in a deep space mission stroke emergency, when prompt intervention is of utmost importance in saving astronauts' lives. This is a high-risk high-reward project aiming to create and validate software prototypes towards this goal with a ground-based study which involves a data collection and algorithm development effort.

The system proposed has the potential to enable lifesaving treatment in case of a stroke event.

Innovation:

- We will create an acute stroke database with subjects within a few hours of stroke onset and OCT-A imaging, in addition to neuroimaging, fundus retina images, and clinical assessment.

- We will drive innovation by establishing the feasibility of machine learning models to identify acute stroke events and stroke type from retina data, effectively acting as a proxy for brain imaging.

- We will be using a first-of-its-kind deep learning model using symmetry-sensitive relationships between the retinal images of each eye. This could enable the algorithm to be robust to space travel induced changes which affect both eyes at the same time, such as Spaceflight Associated Neuro-Ocular Syndrome (SANS).

- Our model will be interpretable without having to compromise on specific architecture, as we will be able to study the regions of activation using the epsilon-LRP (Layer-wise Relevance Propagation) algorithm, to understand the image areas responsible for the model decisions.

- While some initial work has been done to create machine learning models combining information from fundus images and OCT data, to our knowledge, we will be the first to experiment with a combination of fundus imaging and OCT-A imaging using machine learning approaches. Such combination will capture the optic disc/vasculature with a large field of view (fundus) and the finer information for blow flow (OCT-A).

On Earth, acute strokes can be successfully treated with anticoagulant or thrombolytic drugs if the event is rapidly diagnosed and the type of stroke (ischemic versus hemorrhagic) is rapidly identified with Computer Tomography (CT) or Magnetic Resonance Imaging (MRI) brain imaging. However, these brain imaging capabilities do not exist in space and alternative robust means of diagnosing and classifying stroke are needed. Due to the homology between retinal and cerebral vessels, and the ease with which retinal images can be acquired non-invasively, retinal images have been studied as a marker for cerebrovascular events. We propose to use a combination of color fundus photos and Optical Coherence Tomography Angiography (OCT-A) images to identify stroke events and stroke type, effectively acting as a proxy for brain imaging. These imaging modalities are non-invasive and deployable in currently existing technologies for deep space missions. We will adapt our automated interpretable image-based deep learning algorithm to identify stroke and stroke type from retinal vascular images, enabling an automated life-saving tool usable on a deep space exploration mission.

Original project aims/objectives:

Aim 1: Build a terrestrial acute stroke dataset retinal images and stroke clinical outcomes.

Aim 2: Develop and validate an interpretable deep learning model to identify the presence of stroke and stroke type from color fundus photos.

Aim 3: Development and validation of interpretable deep learning model to identify presence of stroke and stroke type from OCT-A (and fundus image combination).

Project Highlights and key findings:

• Camera acquisition setup completed, data acquisition team trained and 128 subjects recruited.

• All images have been externally graded by researchers blinded to the stroke condition of the subjects.

• In our dataset, it is possible to detect acute ischemic stroke with relatively high-performance using OCT-A images (AUC 0.87 [CI 0.78-0.99]). Age does not appear to be a confounder.

• 10 macular microvasculature density variables per retina are enough to achieve these performance.

• Fractal dimension on fundus images and OCT-A are associated with acute ischemic stroke but they are worse predictors than macular microvasculature density variables.

• Multi-modality integration (fundus+OCT-A) does not increase the predictive performance in our dataset.

• State-of-art self-supervised deep learning algorithms do not increase the predictive performance even when pre-trained on other two external OCT-A datasets (without the stroke information).

• Developed a live web application to generate the stroke predictions from the 10 macular microvasculature density variables per eye. This would allow other researchers to use and further validate our model. The web app is available at a web address, as source code and as Docker container.

• Developed a new deep learning model to extract a synthetic OCT-A image from fundus images. This has the potential of having better vessel segmentations without the need of being trained on manual segmentation.

• Released a dataset of OCT-A, fundus images, our synthetic OCT-A and vessel segmentations to allow other researchers to use and further refine our synthetic OCT-A model.

• Retina vasculature embeddings tested on non-acute stroke data. We found a significant predictive association with our retina vasculature imaging biomarker and stroke subjects. The model developed outperformed two state-of-the-art deep neural networks. All the embeddings have been submitted to UK-Biobank database for sharing with other investigators.

• Identified multiple challenges in the data acquisition that can inform future hardware development.

Impact of key findings on hypotheses, technology requirements, objectives and specific aims of the original proposal:

• Our results support our initial hypothesis that it is possible to detect acute stroke from retina images.

• We did not need to use symmetric differences between the two retinas as initially planned to train our model.

• We were not able to recruit enough hemorrhagic stroke patients to test our ability to distinguish between ischemic and hemorrhagic stroke. This is partly due to the COVID pandemic did not allow us to start the data acquisition as initially planned, partly to the fact that hemorrhagic stroke subjects have more disability which leads to issues in focusing onto the camera and/or suffer from droopy eyelids, which does not allow successful imaging.

• There is the need of better portable or robot-arm mounted OCT-A cameras to image bedridden subjects.

Significance:

If we were able to create a retinal imaging-based quantitative tool to establish the presence of stroke, and stroke type (ischemic versus hemorrhagic), we would be able to indicate the appropriate treatment in a deep space mission stroke emergency, when prompt intervention is of utmost importance in saving astronauts' lives. This is a high-risk high-reward project aiming to create and validate software prototypes towards this goal with a ground-based study which involves a data collection and algorithm development effort.

The system proposed has the potential to enable lifesaving treatment in case of a stroke event.

Innovation:

- We will create an acute stroke database with subjects within a few hours of stroke onset and OCT-A imaging, in addition to neuroimaging, fundus retina images, and clinical assessment.

- We will drive innovation by establishing the feasibility of machine learning models to identify acute stroke events and stroke type from retina data, effectively acting as a proxy for brain imaging.

- We will be using a first-of-its-kind deep learning model using symmetry-sensitive relationships between the retinal images of each eye. This could enable the algorithm to be robust to space travel induced changes which affect both eyes at the same time, such as Spaceflight Associated Neuro-Ocular Syndrome (SANS).

- Our model will be interpretable without having to compromise on specific architecture, as we will be able to study the regions of activation using the epsilon-LRP (Layer-wise Relevance Propagation) algorithm, to understand the image areas responsible for the model decisions.

- While some initial work has been done to create machine learning models combining information from fundus images and OCT data, to our knowledge, we will be the first to experiment with a combination of fundus imaging and OCT-A imaging using machine learning approaches. Such combination will capture the optic disc/vasculature with a large field of view (fundus) and the finer information for blow flow (OCT-A).

If we were able to create a retinal imaging-based quantitative tool to establish the presence of stroke and stroke type, we would be able to indicate the appropriate treatment in a deep space mission stroke emergency, when prompt intervention is of utmost importance in saving astronauts' lives. This technology would also have a significant impact on Earth when deployed in mobile stroke units and ambulance, as they will be able to reduce the time of intervention.

This is a high-risk high-reward project aiming to create and validate software prototypes towards this goal with a ground-based study which involves a data collection and algorithm development effort. In addition to the significance and impact of the long-term goal described above, the aims of this project will allow to generate new hypotheses and validate existing association about connections between retina microvasculature and acute stroke.

On Earth, acute strokes can be successfully treated with anticoagulant or thrombolytic drugs if the event is rapidly diagnosed and the type of stroke (ischemic versus hemorrhagic) is rapidly identified with Computer Tomography (CT) or Magnetic Resonance Imaging (MRI) brain imaging. However, these brain imaging capabilities do not exist in space and alternative robust means of diagnosing and classifying stroke are needed. Due to the homology between retinal and cerebral vessels, and the ease with which retinal images can be acquired non-invasively, retinal images have been studied as a marker for cerebrovascular events. We propose to use a combination of color fundus photos and Optical Coherence Tomography Angiography (OCT-A) images to identify stroke events and stroke type, effectively acting as a proxy for brain imaging. These imaging modalities are non-invasive and deployable in currently existing technologies for deep space missions. We will adapt our automated interpretable image-based deep learning algorithm to identify stroke and stroke type from retinal vascular images, enabling an automated life-saving tool usable on a deep space exploration mission.

Original project aims/objectives

Aim 1: Build a terrestrial acute stroke dataset retinal images and stroke clinical outcomes.

Aim 2: Develop and validate an interpretable deep learning model to identify the presence of stroke and stroke type from color fundus photos.

Aim 3: Development and validation of interpretable deep learning model to identify presence of stroke and stroke type from OCT-A (and fundus image combination).

Project Highlights and Key Findings

• Camera acquisition setup completed, data acquisition team trained and 84 subjects recruited.

• Preliminary analysis on acute stroke data using OCT-A/OCT with feature engineering indicates that it is possible to detect acute ischemic stroke (AUC 0.89 [CI 0.74-0.99] / AUC 0.86 [CI 0.74-0.96]); however, age is currently a confounder.

• Retina vasculature embeddings tested on non-acute stroke data. We found a significant predictive association with our retina vasculature imaging biomarker and stroke subjects (AUC 0.69 [CI 0.65-0.73]).

• Identified multiple challenges in the data acquisition that can inform future hardware development.

• On Jan 1st, a National Library of Medicine Scholarship was awarded to Ivan Coronado, PhD student working in Dr. Giancardo's Lab, for extending this project by developing algorithms using Adversarial Learning strategies.

Impact of key findings on hypotheses, technology requirements, objectives, and specific aims of the original proposal

• Our initial results, while preliminary, support our initial hypothesis that it is possible to detect acute stroke from retina images.

• The COVID pandemic did not allow us to start the data acquisition as initially planned. In addition, we found that many acute stroke subjects have issue in focusing onto the camera and/or suffer from droopy eyelids, which does not allow successful imaging. As such our dataset is growing at a slower pace than expected. We will be extending the recruitment period and if this is not sufficient, start acquiring data from non-acute stroke subjects.

On Earth, acute strokes can be successfully treated with anticoagulant or thrombolytic drugs if the event is rapidly diagnosed and the type of stroke (ischemic versus hemorrhagic) is rapidly identified with Computer Tomography (CT) or Magnetic Resonance Imaging (MRI) brain imaging. However, these brain imaging capabilities do not exist in space and alternative robust means of diagnosing and classifying stroke are needed. Due to the homology between retinal and cerebral vessels, and the ease with which retinal images can be acquired non-invasively, retinal images have been studied as a marker for cerebrovascular events. We propose to use a combination of color fundus photos and Optical Coherence Tomography Angiography (OCT-A) images to identify stroke events and stroke type, effectively acting as a proxy for brain imaging. These imaging modalities are non-invasive and deployable in currently existing technologies for deep space missions. We will adapt our automated interpretable image-based deep learning algorithm to identify stroke and stroke type from retinal vascular images, enabling an automated life-saving tool usable on a deep space exploration mission. This approach will leverage the symmetry relationships between the retinal images of each eye in order to identify subtle vasculature changes and at the same time be robust to confounders that affect both eyes at the same time.

Significance:

If we were able to create a retinal imaging-based quantitative tool to establish the presence of stroke, and stroke type (ischemic versus hemorrhagic), we would be able to indicate the appropriate treatment in a deep space mission stroke emergency, when prompt intervention is of utmost importance in saving astronauts' lives. This is a high-risk high-reward project aiming to create and validate software prototypes towards this goal with a ground-based study which involves a data collection and algorithm development effort.

The system proposed has the potential to enable lifesaving treatment in case of a stroke event.

Innovation:

- We will create an acute stroke database with subjects within a few hours of stroke onset and OCT-A imaging, in addition to neuroimaging, fundus retina images, and clinical assessment.

- We will drive innovation by establishing the feasibility of machine learning models to identify acute stroke events and stroke type from retina data, effectively acting as a proxy for brain imaging.

- We will be using a first-of-its-kind deep learning model using symmetry-sensitive relationships between the retinal images of each eye. This could enable the algorithm to be robust to space travel induced changes which affect both eyes at the same time, such as Spaceflight Associated Neuro-Ocular Syndrome (SANS).

- Our model will be interpretable without having to compromise on specific architecture, as we will be able to study the regions of activation using the epsilon-LRP (Layer-wise Relevance Propagation) algorithm, to understand the image areas responsible for the model decisions.

- While some initial work has been done to create machine learning models combining information from fundus images and OCT data, to our knowledge, we will be the first to experiment with a combination of fundus imaging and OCT-A imaging using machine learning approaches. Such combination will capture the optic disc/vasculature with a large field of view (fundus) and the finer information for blow flow (OCT-A).