civic

Pushback to the Future: Predict Pushback Time at US Airports (Prescreened Arena)

CALLING PRESCREENED PARTICIPANTS! Accurate estimates of pushback time can help air traffic management systems more efficiently use the limited capacity of airports, runways and the National Airspace System. Use air traffic and weather data to automatically predict pushback time! #civic

$50,000 in prizes
apr 2023
49 joined

CALLING PRESCREENED PARTICIPANTS! Accurate estimates of pushback time can help air traffic management systems more efficiently use the limited capacity of airports, runways and the National Airspace System. Use air traffic and weather data to automatically predict pushback time!

Quick Facts

Participants
49
No. of Entries
403
Prize
$50,000
Winner

Team CDS

1st Place Team

Navigation

Code submission format

In the Prescreened Arena (you are here!), you will submit your trained model and the code to make predictions, and we will generate predictions for the evaluation dataset in a containerized runtime in our cloud environment. This score will determine the final rankings. The runtime repository contains the complete specification for the runtime.

What to submit

You will submit a predict function that generates predictions for a set of flights at a single airport and prediction time. Within the code execution environment, our code will iterate over each airport and prediction time in the test set. For each airport/time combination, our code will filter the features to the valid time range and call your predict function, passing it the valid features. Your predict function will use the provided features and your model to compute and return predictions for the provided set of flights. Our code will handle concatenating all of the individual time/airport predictions into a single file and submit it for scoring.

Here's a simplified version of the code involved:

model = load_model()  # <- your load_model

for prediction_time in prediction_times:
    for airport in airports:
        # ... we load the censored features

        partial_predictions = predict(  # <- your predict
            # ... features, model, etc.
        )

# we concatenate partial predictions

Your final submission should be a zip archive named with the extension .zip (for example, submission.zip). The root level of submission.zip must contain a script solution.py implementing the following functions:

predict: A function that makes a prediction for a set of flights at a single timestamp and a single airport. It should take the following inputs:

  • Features for the prediction time: A single data frame for each of the competition features (config, etd, first_position, lamp, mfs, runways, standtimes, tbfm, tfm). The features will already be censored to the valid time period from 30 hours before the prediction time to the prediction time. Therefore, you may use all of these input features (and you should only use the provided features).
  • airport: The departing airport for the predicted flights.
  • prediction_time: The assumed "current time" at which you are making your predictions.
  • partial_submission_format: A subset of the test submission format defining the set of flights to predict. It will have the columns gufi, timestamp, airport, and minutes_until_pushback (set to a placeholder value of 0). Each call to your predict function should output a data frame equal to the partial_submission_format except for minutes_until_pushback, which should contain your predictions for each flight.
  • model: Any assets returned by your load_model function (described below).
  • solution_directory: The path to your unzipped submission. You may read from this directory freely, although we recommend you pre-load assets using the load_model functionality to avoid needing to read from disk at each iteration.

load_model: A function that loads your model assets. The return value of that function will be passed to each call to predict. We recommend loading any large model weights here since doing so at each call to predict could add substantially to the time your submission takes to run.

For more detail on how to create and test your submission, visit the runtime repository. The repository also contains an example submission to help you get started. You can also read the benchmark blog post, which includes a working example of a simple baseline solution and how to translate that solution into a submission to the Prescreened Arena.

Runtime

Your code is executed within a container that is defined in our runtime repository. The limits are as follows:

  • Your submission must be written for Python (3.10.9) and must use the packages defined in the runtime repository.
  • The submission must complete execution in 10 hours or less. This limit is especially helpful to address non-terminating code, and we expect most submissions to complete more quickly. If you find yourself requiring more time than this limit allows, open a Github issue in the repository to let us know.
  • The container runtime has access to a single GPU. All of your code should run within the GPU environments in the container, even if actual computation happens on the CPU. (CPU environments are provided within the container for local debugging only.)
  • The container has access to 6 vCPUs powered by an Intel Xeon E5-2690 chip and 56GB RAM.
  • The container has 1 Tesla K80 GPU with 12GB of memory.
  • The container will not have network access. All necessary files (code and model assets) must be included in your submission.
  • The container execution will not have root access to the filesystem.

The machines for executing your code are a shared resource across competitors. Please be conscientious in your use of them. Please add progress information to your logs and cancel jobs that will run longer than the time limit. Canceled jobs won't count against your submission limit, and this means more available resources to score submissions that will complete on time.

Requesting package installations

Since the Docker container will not have network access, all packages must be pre-installed. We are happy to add packages as long as they do not conflict and can build successfully. Packages must be available through conda for Python 3.10.9. To request additional packages be added to the Docker image, follow the instructions in the runtime repository.

Happy building! Once again, if you have any questions or issues you can always head on over the user forum!

On this page