Encrypted | Login

An AAPM Grand Challenge

The aim of the OpenKBP Challenge is to advance fair and consistent comparisons of dose prediction methods for knowledge-based planning (KBP). Participants of the challenge will use a large dataset to train, test, and compare their prediction methods, using a set of standardized metrics, with those of other participants.

The Rationale?

KBP research is flourishing,[1] but many results are reported using institution-specific datasets and evaluation metrics. While many competing approaches have reported positive results, comparing them is difficult without a large open-source dataset, standardized metrics, and a platform to encourage collaboration, sharing, and benchmarking. Such an open and standardized approach is a staple in thriving machine learning-driven fields.

The Challenge

Competitors will predict a clinical quality dose distribution for a patient given only a contoured CT image. The challenge is divided into two streams: (1) the 3D Stream where full 3D dose distributions are evaluated, and (2) the Dose-Volume Histogram (DVH) Stream where only DVH metrics are evaluated. Anyone is welcome to participate in the challenge. Participants can choose to compete in either or both streams of the challenge.

Figure 1

Common Materials

For both streams of the challenge, contestants will be given CT images that are contoured with a standard set of organs-at-risk and targets that follow a common naming convention. All images are sourced from The Cancer Imaging Archive (TCIA), which hosts several open-source datasets. The images will be augmented with high-quality “synthetic” plans generated via a single automated planning method[2], itself trained on a dataset of 217 clinical plans from a single institution. The use of synthetic plans, which will be representative of clinical quality plans, ensures that the plans are created with a consistent planning protocol.

Submissions

In each stream, competitors must submit their predictions (i.e., dose distributions for the 3D Stream and DVHs for DVH Stream) for each patient, and a brief summary of their approach, using a template to be provided.

Challenge Datasets

The dataset is partitioned into a training, validation, and testing set.

Training (200 patients): Contains contoured CT images and their corresponding dose distributions. Contestants will train their models on this data.

Validation (40 patients): Contains only contoured CT images. Predictions will be used to populate a public leaderboard (one per stream) that ranks the submissions to date. These predictions will not be used to determine the winners of the challenge.

Testing (100 patients): Contains only contoured CT images, separate from the validation set. These predictions will be used to evaluate the final submissions and determine the winners.

Timeline

Prizes

The top performing team and two runners-up will be invited to present their methods and results at the AAPM OpenKBP webinar. Those same teams will also be invited to submit their work to the OpenKBP special issue in Medical Physics.

Register for Participation

Please complete the following two steps to participate in OpenKBP:

Host

OpenKBP is co-organized by Aaron Babier, Binghao Zhang, Rafid Mahmood, and Timothy Chan (University of Toronto, Canada); Andrea McNiven and Thomas Purdie (Princess Margaret Cancer Center, Canada); Kevin Moore (UC San Diego, USA).

Contact

Aaron Babier and Timothy Chan
openkbp@gmail.com

References

  1. Y. Ge, Q. J. Wu, “Knowledge-based planning for intensity-modulated radiation therapy: A review of data-driven approaches,” Medical Physics, Vol. 46, pp. 2760-2775, 2019.
  2. A. Babier, R. Mahmood, A.L. McNiven, A. Diamant, T.C.Y. Chan, “Knowledge-based automated planning with three-dimensional generative adversarial networks,” Medical Physics, accepted, 2019