U.S. PETs Prize Challenge: Phase 3 (Red Teams)

Help unlock the potential of privacy-enhancing technologies (PETs) to combat global societal challenges. Test the privacy guarantees of blue teams' federated learning solutions. #privacy

$120,000 in prizes
Completed mar 2023
14 joined

Privacy-Enhancing Technology prize challenge for financial crime prevention and pandemic forecasting

PETs Prize Challenge: Advancing Privacy-Preserving Federated Learning


This is the third in a series of challenge phases with a total prize pool of $800,000! Each phase in the PETs Prize Challenge invites participants to test and apply their innovations in privacy-preserving federated learning. Phase 3 is for Red Teams to put the privacy claims of Blue Teams' solutions to the test.

Background


Privacy-enhancing technologies (PETs) have the potential to unlock more trustworthy innovation in data analysis and machine learning. Federated learning is one such technology that enables organizations to analyze sensitive data while providing improved privacy protections. These technologies could advance innovation and collaboration in new fields and help harness the power of data to tackle some of our most pressing societal challenges.

That’s why the U.S. and U.K. governments are partnering to deliver a set of prize challenges to unleash the potential of these democracy-affirming technologies to make a positive impact. In particular, this challenge will tackle two critical problems via separate data tracks: Data Track A will help with the identification of financial crime, while Data Track B will bolster pandemic responses.

By entering the prize challenges, innovators will have the opportunity to compete for cash prizes and engage with regulators and government agencies. Announced at the Summit for Democracy in December 2021, the prize challenges are a product of a collaboration between multiple government departments and agencies on both sides of the Atlantic. Winning solutions will have the opportunity to be profiled at the second Summit for Democracy, to be convened by President Joe Biden, in early 2023.

Objectives

The goal of this prize challenge is to mature federated learning approaches and build trust in their adoption. The challenge organizers hope to accelerate the development of efficient privacy-preserving federated learning solutions that leverage a combination of input and output privacy techniques to:

  • Drive innovation in the technological development and application of novel privacy-enhancing technologies
  • Deliver strong privacy guarantees against a set of common threats and privacy attacks
  • Generate effective models to accomplish a set of predictive or analytical tasks that support the use cases

Eligibility

At the time of entry, the Official Representative (individual or team lead, in the case of a group project) must be age 18 or older and a U.S. citizen or permanent resident of the United States or its territories. In the case of a private entity, the business shall be incorporated in and maintain a place of business in the United States or its territories.

Participants in this Challenge, whether they are individuals, entities, or team members, are prohibited from participating in the U.K. prize challenge, and participants in the U.K. prize challenge are likewise prohibited from participating in this Challenge. Individuals, entities, or team members who are found to have entered both the U.S. and the U.K. challenges will be disqualified from participating in either. If you wish to instead participate in the U.K. competition, please register here.

Any individual who is or has been associated with an active Blue Team is prohibited from participating in the Red Team Phase. A Blue Team may withdraw from Phase 2 of the challenge so that its members are eligible to participate in the Red Team Phase. To withdraw, all registered members of a Blue Team must each email info@drivendata.org and declare unanimous intent of the team's withdrawal.

If you're looking to find teammates for this challenge, the community forum or the cross-U.S.–U.K. public Slack channel are great places to start. You can request access to the Slack channel here.

For more details, please refer to the official rules.

Phase 3 Timeline and Prizes


Phase 3 Key Dates

Launch November 10, 2022
Registration Deadline December 2, 2022 at 11:59:59 PM UTC
Preparation Period December 9, 2022–February 13, 2023
Attack Period February 13–February 28, 2023
Winners Announced March 30, 2023

Phase 3 Prizes

Open to Red Team participants.

Top red teams will be evaluated for success and rigor and will be awarded prizes for their performance during this phase.

Prize Amount
1st $60,000
2nd $40,000
3rd $20,000

How to compete (Phase 3)

  1. Click the "Compete!" button in the sidebar to begin the registration process.
  2. Click on "Team" if you need to create or join a team with other participants. To find other participants to form a team with, check out our community forum or the cross-U.S.–U.K. public Slack channel. You can request access to the Slack channel here.
  3. Click on "Registration Submission" in the sidebar to fill out and submit the registration form. Only one submission is needed for a team. You're in!
  4. Get familiar with the problem through the data overview and problem description pages.
  5. During the preparation period, review the provided blue teams' concept papers to plan your privacy attacks.
  6. During the attack period, conduct privacy attacks on the blue team finalist solutions that you are assigned. Submit all required attack report materials by the deadline.

Note that registration closes on December 2, 2023 at 11:59 PM UTC. Your team must complete registration by this deadline by submitting the registration form (step #3 above).

Good luck!

Full Challenge Timeline and Prizes


Timeline Overview

There are three main phases in the challenge with two types of participants based on a red team/blue team approach. Blue Teams develop privacy-preserving solutions, while Red Teams act as adversaries to test those solutions.

  • Phase 1: Concept Development (Jul–Sept 2022): Blue Teams propose privacy-preserving federated learning solution concepts.
  • Phase 2: Solution Development (Oct 2022–Jan 2023): Blue Teams develop working prototypes of their solutions.
  • Phase 3: Red Teaming (Nov 2022–Feb 2023): Red Teams prepare and test privacy attacks on top blue team solutions from Phase 2.

Diagram of challenge timeline showing the three phases.

Prize Overview

Track Prize Pool
Phase 1: Concept Paper $55,000
Phase 2: Solution Development $575,000
Phase 3: Red Teaming $120,000
Open Source $140,000*
Total $800,000

* UPDATE March 22, 2023: Open Source award amounts have been increased to $20,000 per award and the number of awards have been increased to 7, for a total of up to $140,000. The increase is reallocated prize moneys that were not awarded in earlier phases, and the total pool of prize awards for the Challenge remains $800,000.

Additional Phase Details and Prizes

Place Prize Amount
1st $30,000
2nd $15,000
3rd $10,000

Phase 1: Concept Paper

July 20–September 19, 2022

Participants will produce an abstract and technical concept paper laying out their proposed solution. Concept papers will be evaluated by a panel of judges across a set of weighted criteria. Participants will be eligible to win prizes awarded to the top technical papers, ranked by points awarded. Participants must complete a paper in Phase 1 in order to be eligible to compete in Phase 2.

Open to Blue Team participants.

Data Track A: Financial Crime Prevention

Prize Amount
1st $100,000
2nd $50,000
3rd $25,000

Data Track B: Pandemic Response and Forecasting

Prize Amount
1st $100,000
2nd $50,000
3rd $25,000

Generalized Solutions

Prize Amount
1st $100,000
2nd $50,000
3rd $25,000

Special Recognition

Prize Amount
Pool $50,000

Phase 2: Solution Development

October 5, 2022–January 26, 2023

Put your papers into action! Registered teams from Phase 1 will develop working prototypes and submit them to a remote execution environment for federated training and evaluation. These solutions are expected to be functional, i.e., capable of training a model and predicting against the evaluation data set with measurement of relevant performance and accuracy metrics. Solutions will be evaluated by a panel of judges across a set of weighted criteria. The top solutions, ranked by points awarded, will have their final rankings determined by incorporating red team evaluation from the red teams from Phase 3.

Teams can qualify for one or multiple of Data Track A: Financial Crime Prevention, Data Track B: Pandemic Response and Forecasting, and Generalized Solutions prize categories depending on how their solutions address the two privacy-preserving federated learning tasks in the challenge.

A separate Special Recognition prize pool is set aside to award up to five solutions that do not win prizes from the three main prize categories but demonstrate excellence in specific areas of privacy innovation: novelty, advancement in a specific privacy technology, usability, and efficiency.

Open to Blue Team participants.

Prize Amount
Pool $140,000

Open Source

Submissions due April 20, 2023

Up to 7 of the final blue team winners from Phase 2 will be invited to release their solutions as open-source software. Each verified participating blue team will be awarded an Open Source prize of $20,000.

Open to top Blue Team winners from Phase 2.



This challenge is sponsored by the National Institute for Standards and Technology (NIST) and the National Science Foundation (NSF)


        


With additional collaboration from the Office of Science and Technology Policy (OSTP), NASA, SWIFT, and the University of Virginia's Biocomplexity Institute.

This prize challenge and its U.K. counterpart have been developed as part of a joint collaboration between the United Kingdom and the United States.