Water Supply Forecast Rodeo: Final Prize Stage

[submissions closed] Water managers in the Western U.S. rely on accurate water supply forecasts to better operate facilities and mitigate drought. Help the Bureau of Reclamation improve seasonal water supply estimates in this probabilistic forecasting challenge! [Final Prize Stage] #climate

$400,000 in prizes
3 months left
34 joined

Overview

Accurate seasonal water supply forecasts are crucial for effective water resources management in the Western United States. This region faces dry conditions and high demand for water, and these forecasts are essential for making informed decisions. They guide everything from water supply management and flood control to hydropower generation and environmental objectives.

Yet, hydrological modeling is a complex task that depends on natural processes marked by inherent uncertainties, such as antecedent streamflow, snowpack accumulation, soil moisture dynamics, and rainfall patterns. To maximize the utility of these forecasts, it's essential to provide not just accurate predictions, but also comprehensive ranges of values that effectively convey the inherent uncertainties in the predictions.

Task

The goal of this challenge is to develop probabilistic forecast models that predict seasonal water supply at the 0.10, 0.50, and 0.90 quantiles. The seasonal water supply refers to the naturalized cumulative streamflow volume over specified seasonal periods. The challenge is occuring over multiple stages: the Hindcast Stage evaluated models on historical data simulating real-time forecasting, the Forecast Stage will run in real-time during the 2024 season, and a final prize stage will ask solvers to submit additional materials for the overall and bonus prizes, like model reports and cross-validation predictions, which will be judged by a panel of technical experts.

By improving the accuracy, explainability, and uncertainty characterization of seasonal streamflow forecasts, water resources managers will be better equipped to operate facilities during high flows, mitigate impacts of drought, improve hydropower generation, and meet environmental targets.

Challenge overview

Timeline Overview

The challenge is occurring over multiple stages, each with its own prizes:

  1. Hindcast Stage: Models were evaluated on historical ground truth data to simulate forecasts that would have been made in the past.
  2. Forecast Stage: Models are be run four times each month from January through July 2024 to issue real-time forecasts for the 2024 season.
  3. Final Prize Stage: In a final stage, solvers will submit updated models and additional materials to compete for Overall Prizes and for the Explainability and Communication Bonus Track prizes.

Prize Overview

Category Prize Pool
Overall $325,000
Hindcast Stage $50,000
Forecast Stage $50,000
Explainability and Communication Bonus Track $75,000
Total $500,000

Final Prize Stage Details

You are currently in the Final Prize Stage for the challenge. In the Final Prize Stage, you will submit cross-validation results on the historical data and a final model report. Your submission will be evaluated by a panel of technical experts and compete for the challenge's Overall Prizes.

The Final Prize Stage will also include the Forecast Explainability and Communication Bonus Track. The details for the Explainability Bonus Track are still being finalized and will be announced at a later time.

Final Stage Key Dates

Final Prize Stage opens February 16, 2024
Cross-validation predictions and final model report due March 28, 2024 at 11:59 pm UTC
Explainability Bonus Track submissions opens April 9, 2024
Explainability Bonus Track submissions due May 16, 2024 at 11:59 pm UTC

Prizes Breakdown

Place Prize Amount
1st $100,000
2nd $75,000
3rd $50,000
4th $30,000
5th $20,000
Bonus Prize: Cascades $10,000
Bonus Prize: Sierra Nevada $10,000
Bonus Prize: Colorado Headwaters $10,000
Bonus Prize: Challenging Basins $10,000
Bonus Prize: Early Lead Time $10,000

Overall Prizes

Overall prizes will be awarded based on an evaluation by a panel of technical experts. The evaluation will consider the overall performance across all stages of the challenge as well as a final model report detailing solutions.

Bonus prizes will also be awarded for best performance in forecast subcategories. See the problem description page for additional details.

You must participate in both Hindcast and Forecast Stages by submitting successfully executed code to be eligible for overall prizes.

Place Prize Amount
1st $25,000
2nd $15,000
3rd $10,000

Hindcast Stage Prizes

October–December 2023

Test your model against historical ground truth data! Train your models and submit code. Your code will be executed to perform inference on a held-out test set. Prizes will be awarded based on a combination of your leaderboard performance and an evaluation of your model report by a panel of judges.

Place Prize Amount
1st $25,000
2nd $15,000
3rd $10,000

Forecast Stage Prizes

December 2023–July 2024

Find out how your model performs with live forecasts for the 2024 season! DrivenData will execute your model at a regular cadence from January through July 2024. After July, forecasts will be evaluated against the true water supply measurements and top leaderboard performers will win Forecast Stage prizes.

You must participate in the Hindcast Stage by submitting successfully executed code in order to be registered for the Forecast Stage.

Place Prize Amount
1st $25,000
2nd $20,000
3rd $15,000
4th $10,000
5th $5,000

Forecast Explainability and Communication Prizes

Augment your model to produce forecast explanation outputs alongside its normal predictions. Understanding how predictors drive forecasts and changes in forecasts is important to operational decision makers. Submissions will be judged by a panel of technical experts to select the winners.

You must participate in both Hindcast and Forecast Stages and submit a final model report to be eligible.

How to compete

  1. Only teams that made successful code submissions in the Hindcast and Forecast Stages are eligible to participate in the Final Prize Stage. Eligible teams are automatically registered. If you need to make changes to your team, please contact info@drivendata.org.
  2. Review the problem description page to refresh yourself on the challenge setup.
  3. Evaluate your model using cross-validation as instructed on the cross-validation submission format page. Click "Submissions" in the sidebar, and then "Make new submission" to submit your cross-validation predictions.
  4. Write your final model report according to the instructions on the final model report format page. Click "Report submissions" in the sidebar to submit your report.

The challenge rules are in place to promote fair competition and useful solutions. If you are ever unsure whether your solution meets the rules, ask the challenge organizers in the competition forum or send an email to info@drivendata.org.

Note on prize eligibility: The term Competition Sponsor in the rules includes the Bureau of Reclamation as well as all federal employees acting within the scope of their employment and federally-funded researchers acting within the scope of their funding. These parties are not eligible to win a prize in this challenge.


Sponsors

This challenge is sponsored by the Bureau of Reclamation


With support from NASA


And with collaborators from the USDA Natural Resources Conservation Service and the U.S. Army Corps of Engineers