The MArkerless Lung Target Tracking CHallenge (MATCH): An AAPM Grand Challenge

An AAPM Grand Challenge

A detailed description of the MATCH and the results of the main part of the challenge have now been published.
View the Article »

The MATCH challenge stands for Markerless Lung Target Tracking Challenge. The aim is to systematically investigate and benchmark the accuracy of various approaches for lung tumour motion tracking during radiation therapy in both a retrospective simulation study (Part A) and a prospective phantom experiment (Part B).

What is the Rationale?

The goal of the MATCH challenge is to explore the possibility of tracking lung tumour motion without the cost and risk of surgically inserted fiducial markers at the time that is needed most – in real-time during the radiation treatment.

Lung cancer stereotactic ablative body radiation therapy (SABR) is one of the cancer treatment success stories. The American Association of Physicists in Medicine (AAPM) Task Groups 76 ¹ (respiratory motion management) and 101² (stereotactic body radiation therapy) have highlighted the need for accurate target tracking during radiation therapy. To further improve patient safety for SABR, reduce margins and account for the breath-to-breath and day-to-day variation in lung tumour motion several commercial and academic markerless lung target tracking algorithms have been developed. These algorithms have yet to be benchmarked using a common prospective measurement methodology. This knowledge gap motivated the Markerless Lung Target Tracking Challenge.

The Challenge

The MATCH challenge consists of two parts: (Part A) in-silico study based on images with unknown motion and (Part B) experimental phantom measurements with patient-measured motion. The challenge is open to any participant, and participants can complete either one or both parts of the challenge.

Common materials

Common to both parts of MATCH are a lung phantom³, and a lung SABR planning protocol. The lung phantom is shown in Figure 2. The phantom is anthropomorphically realistic and therefore best suited to the task of evaluating markerless lung target tracking algorithms in a clinical environment. The phantom is a 3D printed anatomy taken from a real patient, including blood vessels, mediastinum with trachea and bony structures. In contrast to a real patient, the phantom can be moved with known displacements, offering a benchmark for the target position. The CT-scan and kV X-ray images of the phantom resemble closely those of a real patient. The phantom includes three targets, two of unit density and one of lower density. The lung SABR planning protocol was taken from RTOG0915⁴, using the 12Gy per fraction arm for expediency of measurements.

Common Metrics

The goal of markerless target tracking algorithms is to accurately and precisely reproduce the position of the target with time. For MATCH, the percentage of the values where the algorithm value was within 2mm of the ground truth was selected as a clinically relevant primary ranking metric. Additionally, the accuracy (mean difference in each dimension) and precision (standard deviation of difference in each dimension) will be recorded for each motion trace as well as the percentage of the values within 1 and 3mm of the ground truth.

Requirements

In-silico study (Part A): The participants are required to submit the tracking algorithm images together with the tracking results to support the integrity of the study.

Phantom study (Part B): The institution-specific materials include the markerless lung target tracking algorithm, a motion-platform or programable couch, a clinical CT scanner, a radiation therapy planning system and a radiation therapy delivery system. As for the experiments the ground truth is known, to support the integrity of the results the CT image set(s), plans, tracking algorithm images, treatment logs, treatment time, motion results, latency and imaging dose are required to be uploaded. Further, the participants may be asked to cover shipping costs for the phantom and additional equipment if required.

Host

The MATCH challenge is led by Marco Mueller and Paul Keall at the ACRF Image X Institute, a research institute under The University of Sydney. Collaborators who have contributed to the datasets included Per Poulsen from the Aarhus University Hospital, and Wilko Verbakel from the Amsterdam University Medical Centers. The study was designed in collaboration with Ross Berbeco (Harvard University, USA), Lei Wang (Stanford University, USA), Shinichiro Mori (National Institute of Radiological Sciences, Japan), Lei Ren (Duke University, USA), Per Poulsen and Rune Hansen (Aarhus University, Denmark), John Roeske (Loyola University Medical Center, USA) and Perry Zhang (Memorial Sloan Kettering Cancer Center, USA).

Challenge Datasets

MATCH Part A consists of multiple datasets representing the common imaging systems. The dataset composition will be announced soon

Timeline

November 2019 – Registration for participation opens.
November 2019 – Start of MATCH Part B.
January 2020 – Participants receive challenge datasets for MATCH Part A.
31 August 2020 – The deadline for submitting the results for MATCH Part A and Part B.
28 January 2021) –The results of the MATCH will be presented in an AAPM research webinar (WATCH RECORDING).

Prizes

Complimentary 2021 AAPM Annual Meeting registration for 1st and 2nd placed entries.

Download the challenge datasets

The dataset of the in-silico study can be downloaded here. The individual results of the participating MLTT approaches can be downloaded here. The analysis code can be downloaded from Github here.

Files in the download link include the MATCH part A datasets (test and training datasets) and MATLAB scripts to automatically quantify algorithm performance as conducted in the MATCH Challenge study. Instructions on how to use the datasets and the scripts are included. Anyone is welcome to use the MATCH Challenge datasets for their research. Any study that makes use of the MATCH Challenge dataset should cite the following reference.

Mueller, M, Poulsen, P, Hansen, R, et al. The markerless lung target tracking AAPM Grand Challenge (MATCH) results. Med. Phys. 2021; 00 1– 20. https://doi.org/10.1002/mp.15418

Contact

Marco Mueller

   www.linkedin.com/in/marcomueller-imagex/
   twitter.com/MScMarcoMueller
   marco.mueller@sydney.edu.au

References

Keall PJ, Mageras GS, Balter JM, et al. The management of respiratory motion in radiation oncology report of AAPM Task Group 76 [published online ahead of print 2006/11/09]. Med Phys. 2006;33(10):3874-3900.
Benedict SH, Yenice KM, Followill D, et al. Stereotactic body radiation therapy: The report of AAPM Task Group 101. Med Phys. 2010;37(8):4078-4101.
Hazelaar C, van Eijnatten M, Dahele M, et al. Using 3D printing techniques to create an anthropomorphic thorax phantom for medical imaging purposes. Med Phys. 2018;45(1):92-100.
Videtic, Gregory MM, et al. "A randomized phase 2 study comparing 2 stereotactic body radiation therapy schedules for medically inoperable patients with stage I peripheral non-small cell lung cancer: NRG Oncology RTOG 0915 (NCCTG N0927)." International Journal of Radiation Oncology* Biology* Physics 93.4 (2015): 757-764.