Differential Privacy Temporal Map Challenge: Sprint 3 (Open Arena) Hosted By NIST PSCR

competition
complete
Glory!

Woohoo! This competition has come to a close!

Many thanks to the participants for all of their hard work and commitment to using data for good!

Differential Privacy Temporal Map Challenge

Overview


Large data sets containing personally identifiable information (PII) are exceptionally valuable resources for research and policy analysis in a host of fields supporting America's First Responders such as emergency planning and epidemiology.

Temporal map data is of particular interest to the public safety community in applications such as optimizing response time and personnel placement, natural disaster response, epidemic tracking, demographic data and civic planning. Yet, the ability to track information related to a person's location over a period of time presents particularly serious privacy concerns.

In the Differential Privacy Temporal Map Challenge (DeID2) your task is to develop algorithms that preserve data utility as much as possible while guaranteeing individual privacy is protected. The challenge features a series of coding sprints to apply differential privacy methods to temporal map data, where one individual in the data may contribute to a sequence of events. The goal is to create a privacy-preserving dashboard map that shows changes across different map segments over time.

Submissions will be assessed based on

  1. their ability to prove they satisfy differential privacy; and
  2. the accuracy of output data as compared with ground truth.

This is a hard problem, and we need your help! For more details on how to get started check out the Problem Description.

alt-text

Sample illustration of the privacy-utility tradeoff.
From Liu et al. “Privacy-Preserving Monotonicity of Differential Privacy Mechanisms.” 2018.

Contest arenas


You are in the Open Arena. This is a great place to get started! Once your submission is successfully prescreened, head on over to the Prescreened Arena to make executable code submissions and qualify for the $75,000 in final scoring prizes.

This de-identification algorithm sprint features two competition arenas which provide different access levels and capabilities.

The Open Arena is the first step in the competition process. Here all participants can enter the outputs of their solutions-in-development to see how they fare against others on the open leaderboard. You will also be able to submit a write-up of the solution for pre-screening that confirms you have an essentially correct understanding of differential privacy as applied to your submission. Solutions that pass this process are designated Prescreened.

Participants that are Prescreened as satisfying differential privacy can then access the Prescreened Arena. Here participants can continue to tweak their solutions, submit their executable code, and see how they perform on the prescreened leaderboard. Submissions in the Prescreened Arena take precedence in determining Progressive Prizes and are required for final scoring.

The Open Arena

  • Is available to all registered participants
  • Provides access to the public ground truth data
  • Ingests and scores submission of .csv files with privatized data sets
  • Displays an open leaderboard with live results from the best-scoring submissions (no pre-screening required)
  • Accepts privacy write-ups for pre-screening review

The Prescreened Arena

  • Is available to participants that have been granted Prescreened status
  • Provides access to the public ground truth data
  • Ingests and scores executable privatization code to run on the cloud
  • Displays a prescreened leaderboard with live results from the best-scoring submissions (these are given precedence in determining progressive prizes)
  • Accepts Final Scoring code submissions and final privacy write-ups at the end of the sprint

Only those participating in the Prescreened Arena will be eligible to win Final Scoring prizes. For more information on staging, submissions, and scoring check out the Problem Description.

Timeline and prizes


This is the third and final sprint in a series of three sprints! Each algorithm sprint in the Differential Privacy Temporal Map Challenge invites participants to explore new methods in differential privacy applied with provided data sets and scoring constraints. Participants are welcome—and encouraged—to build on their solutions and compete in multiple sprints!

  • Sprint 1 ($29,000 prize pool): Oct - Dec 2020
  • Sprint 2 ($39,000 prize pool): Jan - Mar 2021
  • Sprint 3 ($79,000 prize pool): Apr - Jun 2021

Sprint 3 timeline

Open to participants March 29, 2021
Webinar: Differential privacy and how to compete April 5, 2021 1:00 - 2:00 pm EDT
Deadline for pre-screening submissions to be reviewed before Progressive Prizes April 20, 2021 by 10:00 pm EDT
Progressive Prize rankings determined April 26, 2021 at 10:00 pm EDT
Development phase closes May 10, 2021 by 8:00 pm EDT
Final code submissions and write-ups due May 17, 2021 by 8:00 pm EDT
Evaluation period May 17 - June 15, 2021
Winners announced June 16, 2021

Prizes

Place Prize Amount
1st $25,000
2nd $20,000
3rd $15,000
4th $10,000
5th $5,000

Final Scoring

Awarded at the end of the sprint based on evaluation of final code submissions and write-ups.

Prize Amount
Prize 1 $1,000
Prize 2 $1,000
Prize 3 $1,000
Prize 4 $1,000

Progressive Prizes

Awarded part-way through each sprint to four eligible teams with the best scores to date, with precedence given to solutions that are pre-screened as satisfying differential privacy.

Note: For this challenge, the Official Representative (individual or team lead) must be age 18 or older and a U.S. citizen or permanent resident of the United States or its territories in order to be eligible to receive a cash prize. See the Challenge Rules for full details and requirements.


This challenge is sponsored by NIST PSCR:

With support from NASA: