Snowcast Showdown: Evaluation Stage

SUBMIT HERE FOR 2022 PREDICTIONS AND PRIZES! Seasonal snowpack is a critical water resource throughout the Western U.S. Help the Bureau of Reclamation estimate snow water equivalent (SWE) at a high spatiotemporal resolution using near real-time data sources. #climate

$500,000 in prizes
jul 2022
333 joined

alt-text


Getting better high-resolution snow water equivalent estimates for mountain watersheds and headwater catchments will help to improve runoff and water supply forecasts. This helps water managers better operate with limited water supplies and respond to extreme weather events such as floods and droughts.

— David Raff, Chief Engineer, Bureau of Reclamation

Why

Seasonal mountain snowpack is a critical water resource throughout the Western U.S. Snow water equivalent (SWE) is the most commonly used measurement in water forecasts because it combines information on snow depth and density.

While ground-based instruments can be used to monitor snowpacks, ground stations tend to be spatially limited and are not easily installed at high elevations. Given the diverse landscape in the Western U.S. and shifting climate, new and improved methods are needed to accurately measure SWE at a high spatiotemporal resolution to inform water management decisions.

The Solution

The goal of this challenge was to estimate SWE in real-time each week for 1km x 1km grid cells across the Western U.S. Participants could use real-time satellite, ground station, and climate data, as well as static data sources on elevation, soil, water, and land cover.

During the development phase, participants tested submissions using historical data. Then, for the first time, predictions generated by participants each week were scored against new ground truth measures as they came in! In total, competitors were evaluated against over 42,000 ground truth SWE measurements spanning 20 weeks across the 2022 peak snow and melt seasons.

The Results

Over the course of the competition, participants tested over 900 solutions and were able to significantly advance existing methods of SWE estimation. During the real-time evaluation phase, the top model achieved an impressive RMSE of 3.88 and an R-squared of 0.70.

Bar chart showing RMSE scores of top 15 participants.

The winning solutions were varied in their modeling approaches, ranging from Gaussian processes and multilinear regressions to tree-based ensembles and neural networks. Participants took advantage of both real-time and static data sources, and derived creative features to best predict SWE.

Submitting teams also had the opportunity to further analyze and explain their approaches through the Model Report competition, which focused on model interpretability and robustness.

See the results announcement for more information on the winning approaches. All of the prize-winning solutions and model reports from this competition, along with the real-time evaluation dataset assembled for the challenge, are linked below and available for anyone to use and learn from.


RESULTS ANNOUNCEMENT + MEET THE WINNERS

WINNING MODELS + MODEL REPORTS ON GITHUB

CHALLENGE DATASET (MUST BE LOGGED IN)