Differential Privacy Temporal Map Challenge: Sprint 1 (Prescreened Arena)
CALLING PRESCREENED PARTICIPANTS! Help public safety agencies share data while protecting privacy. If you haven't been prescreened yet, head on over to the Open Arena to learn more and get started. Competition hosted by NIST PSCR. #privacy
Large data sets containing personally identifiable information (PII) are exceptionally valuable resources for research and policy analysis in a host of fields supporting America's First Responders such as emergency planning and epidemiology.
Temporal map data is of particular interest to the public safety community in applications such as optimizing response time and personnel placement, natural disaster response, epidemic tracking, demographic data and civic planning. Yet, the ability to track a person's location over a period of time presents particularly serious privacy concerns.
In the Differential Privacy Temporal Map Challenge (DeID2) your task is to develop algorithms that preserve data utility as much as possible while guaranteeing individual privacy is protected. The challenge will feature a series of coding sprints to apply differential privacy methods to temporal map data, where one individual in the data may contribute to a sequence of events. The goal is to create a privacy-preserving dashboard map that shows changes across different map segments over time.
Submissions will be assessed based on
- their ability to prove they satisfy differential privacy; and
- the accuracy of output data as compared with ground truth.
This is a hard problem, and we need your help! For more details on how to get started check out the Problem Description.
Sample illustration of the privacy-utility tradeoff.
From Liu et al. “Privacy-Preserving Monotonicity of Differential Privacy Mechanisms.” 2018.
This de-identification algorithm sprint features two competition arenas which provide different access levels and capabilities.
The Open Arena is the first step in the competition process. Here all participants can enter the outputs of their solutions-in-development to see how they fare against others on the open leaderboard. You will also be able to submit a write-up of the solution for pre-screening that confirms you have an essentially correct understanding of differential privacy as applied to your submission. Solutions that pass this process are designated Prescreened.
Participants that are Prescreened as satisfying differential privacy can then access the Prescreened Arena. Here participants can continue to tweak their solutions, submit their executable code, and see how they perform on the prescreened leaderboard. Submissions in the Prescreened Arena take precedence in determining Progressive Prizes and are required for final scoring.
The Open Arena
- Is available to all registered participants
- Provides access to the public ground truth data
- Ingests and scores submission of .csv files with privatized data sets
- Displays an open leaderboard with live results from the best-scoring submissions (no pre-screening required)
- Accepts privacy write-ups for pre-screening review
The Prescreened Arena
- Is available to participants that have been granted Prescreened status
- Provides access to the public ground truth data
- Ingests and scores executable privatization code to run on the cloud
- Displays a prescreened leaderboard with live results from the best-scoring submissions (these are given precedence in determining progressive prizes)
- Accepts Final Scoring code submissions and final privacy write-ups at the end of the sprint
Only those participating in the Prescreened Arena will be eligible to win Final Scoring prizes. For more information on staging, submissions, and scoring check out the Problem Description.
Measuring the utility of privatized data
Do you have additional ideas for how privatized data outputs should be evaluated against the ground truth? We’d love to hear from you!
This contest is one part of the Differential Privacy Temporal Map Challenge (DeID2). In addition to seeking de-identification algorithms, NIST PSCR also invites solvers to develop metrics that best assess the accuracy of the data output of algorithms that de-identify temporal map data.
In particular, methods are sought that:
- measure the quality of data with respect to temporal utility and/or geographic utility
- evaluate data quality in contexts beyond this challenge
- are clearly explained and straightforward to correctly implement and use
To learn more and sign up, check out DeID2 - A Better Meter Stick for Differential Privacy.
Timeline and prizes
This is the first in a series of three sprints! Each algorithm sprint in the Differential Privacy Temporal Map Challenge invites participants to explore new methods in differential privacy applied with provided data sets and scoring constraints. Participants are welcome—and encouraged—to build on their solutions and compete in multiple sprints!
- Sprint 1 ($29,000 prize pool): Oct - Dec 2020
- Sprint 2 ($39,000 prize pool): Jan - Mar 2021
- Sprint 3 ($79,000 prize pool): Apr - Jun 2021
Sprint 1 timeline
Awarded at the end of the sprint based on evaluation of final code submissions and write-ups.
Awarded part-way through each sprint to four eligible teams with the best scores to date, with precedence given to solutions that are pre-screened as satisfying differential privacy.
Note: For this challenge, the Official Representative (individual or team lead) must be age 18 or older and a U.S. citizen or permanent resident of the United States or its territories in order to be eligible to receive a cash prize. See the Challenge Rules for full details and requirements.
This challenge is sponsored by NIST PSCR: