Snowcast Showdown: Evaluation Stage Hosted By Bureau of Reclamation

3 days left
$500,000

alt-text

Overview


Seasonal mountain snowpack is a critical water resource throughout the Western U.S. Snowpack acts as a natural reservoir by storing precipitation throughout the winter months and releasing it as snowmelt when temperatures rise during the spring and summer. This meltwater becomes runoff and serves as a primary freshwater source for major streams, rivers and reservoirs. As a result, snowpack accumulation on high-elevation mountains significantly influences streamflow as well as water storage and allocation for millions of people.

Snow water equivalent (SWE) is the most commonly used measurement in water forecasts because it combines information on snow depth and density. SWE refers to the amount of liquid water contained in a snowpack, or the depth of water that would result if a column of snow was completely melted. Water resource managers use measurements and estimates of SWE to support a variety of water management decisions, including managing reservoir storage levels, setting water allocations, and planning for extreme weather events.

Over the past several decades, ground-based instruments including snow course and SNOwpack TELemetry (SNOTEL) stations have been used to monitor snowpacks. While ground measures can provide accurate SWE estimates, ground stations tend to be spatially limited and are not easily installed at high elevations. Recently, high resolution satellite imagery has strengthened snow monitoring systems by providing data in otherwise inaccessible areas at frequent time intervals.

Given the diverse landscape in the Western U.S. and shifting climate, new and improved methods are needed to accurately measure SWE at a high spatiotemporal resolution to inform water management decisions.

Task

The goal of this challenge is to estimate snow water equivalent (SWE) at a high spatiotemporal resolution over the Western U.S. using near real-time data sources. Prizes will be awarded based on the accuracy of model predictions and write-ups explaining the solutions as described below.

Getting better SWE estimates for mountain watersheds and headwater catchments will help to improve runoff and water supply forecasts, which in turn will help reservoir operators manage limited water supplies. Improved SWE information will also help water managers respond to extreme weather events such as floods and droughts.

Challenge structure


This competition will include two tracks. For more information on each track, see the Problem Description.

TRACK 1: Prediction Competition is the core machine learning competition, where participants train models to estimate SWE at 1km resolution across 11 states in the Western U.S.

  • Stage 1: Model Development (Dec 7 - Feb 15)
    Historical ground truth is made available along with input data sources for model training. This period is an opportunity to build your data pipelines and test modeling approaches. A public leaderboard will be made available to provide feedback, but prizes will not be awarded during this stage.

  • Stage 2: Model Evaluation (Jan 11 - Jul 1) - YOU ARE HERE

    • Stage 2a: Submission Testing (Jan 11 - Feb 15)
      Package everything needed to perform inference on new data each week. This is an opportunity to make any final improvements to your model and ensure it works with approved data sources to generate predictions for real-time evaluation. Submit your code and frozen model weights to be eligible for Stage 2b.

    • Stage 2b: Real-time Evaluation (Feb 15 - Jul 1)
      After the ❄ model freeze ❄, run your model on a weekly basis to generate and submit near real-time predictions throughout the winter and spring season. Predictions will be evaluated against ground truth labels as they become available and prizes will be awarded based on final private leaderboard rankings.

TRACK 2: Model Report Competition (entries due by Mar 15) is a model analysis competition. Everyone who successfully submits a model for real-time evaluation can also submit a report that discusses their solution methodology and explains its performance on historical data.

Teaming deadline: All teams must be formed by the submission testing deadline on February 15, 2022 23:59:59 UTC, prior to the beginning of Stage 2b.

Prizes


Overview

Track Prize Pool
Track 1: Prediction Competition $440,000
Track 2: Model Report Competition $60,000
Total $500,000

Breakdown

Place Prize Amount
1st $150,000
2nd $75,000
3rd $50,000
4th $25,000
5th $20,000

Overall Predictive Performance

Feb. 15 - Jul. 1, 2022

TRACK 1 - Evaluated on predicted labels across all grid cells in the real-time evaluation period. Final rankings determined by the scoring metric and displayed on the leaderboard at the end of the challenge.

Prize Amount
1st $30,000
2nd $20,000
3rd $10,000

Regional Predictions: Sierras

TRACK 1 - Evaluated on predicted labels from regional grid cells in the real-time evaluation period. Final rankings determined by the scoring metric.

Prize Amount
1st $30,000
2nd $20,000
3rd $10,000

Regional Predictions: Central Rockies

TRACK 1 - Evaluated on predicted labels from regional grid cells in the real-time evaluation period. Final rankings determined by the scoring metric.

Prize Amount
1st $30,000
2nd $20,000
3rd $10,000

Model Report

Submissions due Mar. 15, 2022

TRACK 2 - Evaluated on write-ups of modeling approaches. Everyone who successfully submits a model for real-time evaluation in the Prediction Contest is eligible to submit a report for judging. Final winners will be selected by a judging panel.


Note on prize eligibility: The term Competition Sponsor in the Competition Rules includes the Bureau of Reclamation as well as all federal employees acting within the scope of their employment and federally-funded researchers acting within the scope of their funding. These parties are not eligible to win a prize in this challenge.


How to compete (Stage 2)

  1. Click the “Compete” button in the sidebar to enroll in the competition.
  2. Get familiar with the problem through the overview and problem description. You might also want to reference some of the additional resources from the about page.
  3. Access the data from the data tab.
  4. Prepare your model for Stage 2.
  5. Use your model to generate predictions that match the submission format
  6. Click “Submit” in the sidebar, and “Make new submission”. You’re in! You'll continue to make submission for each week during the real-time evaluation period.
  7. Make sure to follow instructions on submitting your model code and frozen weights before the submission testing deadline on Feb 15.

This challenge is sponsored by the Bureau of Reclamation



With support from NASA Tournament Lab



And with collaborators from Bonneville Power Administration, USDA Natural Resources Conservation Service, U.S. Army Corps of Engineers, U.S. Geological Survey, NASA, and National Center for Atmospheric Research