Clog Loss: Advance Alzheimer’s Research with Stall Catchers

Help accelerate Alzheimer's research by automatically classifying which blood vessels are flowing and which are stalled. #health

$10,000 in prizes
aug 2020
915 joined

Stall Catchers example image


We see tremendous opportunity to incorporate these models into Stall Catchers, where they could work in concert with human-based analysis to double or triple our analytic throughput without compromising data quality. This could get us closer to an Alzheimer’s treatment at unprecedented analytic speeds.

— Pietro Michelucci, Project Lead, Stall Catchers

Why

5.8 million Americans live with Alzheimer’s dementia, including 10% of all seniors 65 and older. Scientists at Cornell have discovered links between “stalls,” or clogged blood vessels in the brain, and Alzheimer’s. The ability to prevent or remove stalls may transform how Alzheimer’s disease is treated. However, finding these stalls is extremely time intensive, especially as only around 1% of image stacks contain stalls.

The Solution

In this challenge, participants were tasked with building machine learning models that could classify blood vessels in 3D image stacks as stalled or flowing. The exceptional challenge dataset came from Stall Catchers, a citizen science project that crowdsources the identification of stalls using research data provided by Cornell University’s Department of Biomedical Engineering.

The researchers behind Stall Catchers believe that some portion of the data may be within reach of machine learning models. Model predictions would only be used in cases where models had been validated to meet the researchers’ data quality requirements.

The Results

The top-performing model was able to achieve an impressive 0.855 Matthew's correlation coefficient (MCC) score, surpassing 85% sensitivity (true positive rate, i.e. classifying stalled vessels as such) and 99% specificity (true negative rate, i.e. classifying flowing vessels as such). This level of specificity means that models like this can be used to process large volumes of data while minimizing the risk of losing the stalled cases that are so important to research.

alt-text

The greatest immediate promise from these results lies in combining machine and human predictions to accelerate research, by 1) using algorithms and humans in tandem to process data far more quickly, and 2) continuing experimentation to develop methods that combine algorithms and humans in ways that exceed the performance that either demonstrates alone. All prize-winning solutions from this competition are linked below and made available for anyone to use and learn from.


RESULTS ANNOUNCEMENT + MEET THE WINNERS

WINNING MODELS ON GITHUB