Encrypted | Login

An AAPM Grand Challenge

In partnership with the American Association of Physicists in Medicine (AAPM), the Center for Virtual Imaging Trials (CVIT) is offering the TrueCT Reconstruction Challenge. The Center will provide participants with simulated sinogram data of various anatomies and disease types for them to reconstruct into high quality CT slices using any type of algebraic, statistical, iterative, and ML-based algorithms as trained on clinical data. Reconstructions will be quantitatively evaluated versus the known truth inherent in the Center’s simulated data.


Overview

Increased use of “approximating” reconstruction methods in CT, including iterative reconstruction (IR) and deep learning (DL) methods, creates uncertainty in how much of the native information might be distorted in the reconstruction process. If one had access to ground truth, i.e., the true underlying anatomy and physiology, the precise limitation of the reconstruction process could be objectively quantified. However, such information is not available in patient images. Ground truth is known in physical phantoms, but these phantoms have limited utility for this purpose as they do not model patient complex anatomy, motion, abnormalities, or variability, factors that are known to impact the performance of non-linear, scene-dependent reconstruction methods. Virtual imaging toolsets consisting of realistic computational phantoms and accurate imaging simulation techniques can provide sets of imaging data with a digitally-defined ground truth with which CT reconstruction methods can be objectively and quantitatively evaluated and compared. This challenge will take advantage of the resources of the CVIT to create a dataset of realistic CT images of virtual patients with known ground truth to provide an objective evaluation of CT reconstruction methods.

Data

The data set will include 200 unique computational phantoms from varied virtual patients sampling the ranges of weight, age, and sex of adult patients. Each phantom will include a particular pathology or disease: 67 COPD, 67 lung nodule, and 66 abdominal cases. The data will include a mixture of subtle pathologies of undisclosed details. All cases will be virtually imaged with a representative CT system with ranges of radiation dose level to represent clinical variability in techniques. The nature of this challenge precludes access to a training dataset.

Data Access

All cases will be made available from a publicly available web site in sinogram format following the standardized format established by the Mayo Clinic. Participants, once registered, will download the dataset and supplemental data needed for reconstruction (namely the scan geometry and associated data needed for the reconstruction). They can then perform their reconstruction and upload the resulting CT image data in DICOM format. Detailed instructions will be provided on how to submit results, including required formats and file organization.

Evaluation

For all the analysis, the reconstructed data will be registered to the ground truth. Ground truth will be established based on the mono-energetic representation of the phantoms. Data will be evaluated based on the similarity to the ground truth of the phantoms. For ranking purposes, we will only use task-generic (e.g., structural similarity index, root mean squared error) metrics. In addition to task-generic metrics, we will also evaluate the images based on task-specific (e.g., detectability index, radiomics, density quantification) metrics, per case, per pathology type, and across the entire dataset. We will provide all the metrics to the participants and devise a collective score and report that in the coming publication. Detailed information on the acquired data and evaluation metrics will be shared along with the sinogram data.

Results

Results of the TrueCT Challenge will be presented as the AAPM Grand Challenge Symposium during the 2022 AAPM Annual Meeting in Washington, D.C. (July 11-14, 2022). Please note, an individual from each of the twop top-performing teams will be required to present on their results and methods during this session, and will be awarded a waiver of the meeting registration fee in order to attend. Top performing participants will also be offered the opportunity for their algorithm to be integrated with the Center for Virtual Imaging Trails resources, with a licensing option.

Publication

We will seek the publication of challenge results in the Medical Physics journal.

Get Started

  1. Visit the TrueCT Challenge site to register and get access to the Challenge data
  2. Download the sinogram data
  3. Apply your reconstruction techniques
  4. Submit your reconstructed images

Important Dates

Organizers

Contact

For further information, please contact: cvit-inquire@duke.edu