Differential Privacy Temporal Map Challenge: Sprint 1 (Open Arena)

START HERE! Help public safety agencies share data while protecting privacy. This is part of a series of contests to develop algorithms that preserve the usefulness of temporal map data while guaranteeing individual privacy is protected. #privacy

nov 2020
169 joined

Differential Privacy Temporal Map Challenge

Overview


Large data sets containing personally identifiable information (PII) are exceptionally valuable resources for research and policy analysis in a host of fields supporting America's First Responders such as emergency planning and epidemiology.

Temporal map data is of particular interest to the public safety community in applications such as optimizing response time and personnel placement, natural disaster response, epidemic tracking, demographic data and civic planning. Yet, the ability to track a person's location over a period of time presents particularly serious privacy concerns.

In the Differential Privacy Temporal Map Challenge (DeID2) your task is to develop algorithms that preserve data utility as much as possible while guaranteeing individual privacy is protected. The challenge will feature a series of coding sprints to apply differential privacy methods to temporal map data, where one individual in the data may contribute to a sequence of events. The goal is to create a privacy-preserving dashboard map that shows changes across different map segments over time.

Submissions will be assessed based on

  1. their ability to prove they satisfy differential privacy; and
  2. the accuracy of output data as compared with ground truth.

This is a hard problem, and we need your help! For more details on how to get started check out the Problem Description.

alt-text

Sample illustration of the privacy-utility tradeoff.
From Liu et al. “Privacy-Preserving Monotonicity of Differential Privacy Mechanisms.” 2018.

Contest arenas


You are in the Open Arena. This is a great place to get started! Once your submission is successfully prescreened, head on over to the Prescreened Arena to make executable code submissions and qualify for the $25,000 in final scoring prizes.

This de-identification algorithm sprint features two competition arenas which provide different access levels and capabilities.

The Open Arena is the first step in the competition process. Here all participants can enter the outputs of their solutions-in-development to see how they fare against others on the open leaderboard. You will also be able to submit a write-up of the solution for pre-screening that confirms you have an essentially correct understanding of differential privacy as applied to your submission. Solutions that pass this process are designated Prescreened.

Participants that are Prescreened as satisfying differential privacy can then access the Prescreened Arena. Here participants can continue to tweak their solutions, submit their executable code, and see how they perform on the prescreened leaderboard. Submissions in the Prescreened Arena take precedence in determining Progressive Prizes and are required for final scoring.

The Open Arena

  • Is available to all registered participants
  • Provides access to the public ground truth data
  • Ingests and scores submission of .csv files with privatized data sets
  • Displays an open leaderboard with live results from the best-scoring submissions (no pre-screening required)
  • Accepts privacy write-ups for pre-screening review

The Prescreened Arena

  • Is available to participants that have been granted Prescreened status
  • Provides access to the public ground truth data
  • Ingests and scores executable privatization code to run on the cloud
  • Displays a prescreened leaderboard with live results from the best-scoring submissions (these are given precedence in determining progressive prizes)
  • Accepts Final Scoring code submissions and final privacy write-ups at the end of the sprint

Only those participating in the Prescreened Arena will be eligible to win Final Scoring prizes. For more information on staging, submissions, and scoring check out the Problem Description.

Measuring the utility of privatized data


Do you have additional ideas for how privatized data outputs should be evaluated against the ground truth? We’d love to hear from you!

This contest is one part of the Differential Privacy Temporal Map Challenge (DeID2). In addition to seeking de-identification algorithms, NIST PSCR also invites solvers to develop metrics that best assess the accuracy of the data output of algorithms that de-identify temporal map data.

In particular, methods are sought that:

  • measure the quality of data with respect to temporal utility and/or geographic utility
  • evaluate data quality in contexts beyond this challenge
  • are clearly explained and straightforward to correctly implement and use

To learn more and sign up, check out DeID2 - A Better Meter Stick for Differential Privacy.

Timeline and prizes


This is the first in a series of three sprints! Each algorithm sprint in the Differential Privacy Temporal Map Challenge invites participants to explore new methods in differential privacy applied with provided data sets and scoring constraints. Participants are welcome—and encouraged—to build on their solutions and compete in multiple sprints!

  • Sprint 1 ($29,000 prize pool): Oct - Dec 2020
  • Sprint 2 ($39,000 prize pool): Jan - Mar 2021
  • Sprint 3 ($79,000 prize pool): Apr - Jun 2021

Sprint 1 timeline

Open to submissions October 1, 2020
Webinar: Differential privacy and how to compete October 14, 2020 from 11:30 am - 12:30 pm EDT
Deadline for pre-screening submissions to be reviewed before Progressive Prizes October 20, 2020 by 10:00 pm EDT
Progressive Prize rankings determined October 23, 2020 at 10:00 pm EDT
Development phase closes November 9, 2020 by 8:00 pm EST
Final code submissions and write-ups due November 15, 2020 by 8:00 pm EST
Evaluation period November 15 - December 11, 2020
Winners announced January 6, 2021

Prizes

Place Prize Amount
1st $10,000
2nd $7,000
3rd $5,000
4th $2,000
5th $1,000

Final Scoring

Awarded at the end of the sprint based on evaluation of final code submissions and write-ups.

Prize Amount
Prize 1 $1,000
Prize 2 $1,000
Prize 3 $1,000
Prize 4 $1,000

Progressive Prizes

Awarded part-way through each sprint to four eligible teams with the best scores to date, with precedence given to solutions that are pre-screened as satisfying differential privacy.

Note: For this challenge, the Official Representative (individual or team lead) must be age 18 or older and a U.S. citizen or permanent resident of the United States or its territories in order to be eligible to receive a cash prize. See the Challenge Rules for full details and requirements.


This challenge is sponsored by NIST PSCR:

With support from NASA: