Synthetic X-ray or digitally reconstructed radiographs (DRRs) are simulated X-ray images projected from computed tomography (CT) data that are commonly used for CT and real X-Ray image registration. High-quality synthetic X-ray images can facilitate various applications such as guiding images for virtual reality (VR) simulation and training data for deep learning methods such as creating CT data from X-Ray images.
It is challenging to generate high-quality synthetic X-ray images from CT slices, especially in various view angles, due to gaps between CT slices, high computational cost, and the complexity of algorithms. Most synthetic X-ray generation methods use fast ray-tracing in a situation where the image quality demand is low. We aim to improve image quality while maintaining good accuracy and use two steps; 1) to generate synthetic X-ray images from CT data and 2) to increase the resolution of the synthetic X-ray images.
Our synthetic X-ray image generation method adopts a matrix-based projection method and dynamic multi-segment lookup tables, which shows better image quality and efficiency compared to conventional synthetic X-ray image generation methods. Our method is tested in a real-time VR training system for image-guided intervention procedures.
Then we proposed two novel approaches to raise the quality of synthetic X-ray images through deep learning methods. We use a reference-based super-resolution (RefSR) method as a base model to upsampling low-resolution images into higher resolution. Even though RefSR can produce fine details by utilizing the reference image, it inevitably generates some artifacts and noise. We propose texture transformer super-resolution with frequency domain (TTSR-FD) which introduces frequency domain loss as a constraint to improve the quality of the RefSR results with fine details and without apparent artifacts. To the best of our knowledge, this is the first work that utilizes frequency domain as a part of loss functions in the field of super-resolution (SR). We observe improved performance in evaluating TTSR-FD when tested on our synthetic X-ray and real X-ray image datasets.
A typical SR network is trained with paired high-resolution (HR) and low-resolution (LR) images, where LR images are created by downsampling HR images using a specific kernel. The same downsampling kernel is also used to create test LR images from HR images. As a result, most SR methods only perform well when the testing image is acquired using the same downsampling kernel used during the training process. We also propose TTSR-DMK, which uses multiple downsampling kernels during training to generalize the model and adopt a dual model that trains together with the main model. The dual model can form a closed-loop with the main model to learn the inverse mapping, which further improves the model’s performance. Our method works well for testing images produced by multiple kernels used during training. It can also help improve the model performance when testing images are acquired with kernels not used during training. To the best of our knowledge, we are the first to use the closed-loop method in RefSR.
We have achieved: (i) synthetic X-ray image generation from CT data, which is based on a matrix-based projection and lookup tables ; (ii) TTSR-FD: synthetic X-ray image super-resolution using a novel frequency domain loss ; (iii) TTSR-DMK: an adaptation network to overcome the performance drop for testing data which do not match to downsampling kernels used in training.
Our TTSR-FD results show improvements (PSNR from 37.953 to 39. 009) compared to the state-of-the-art methods TTSR. Our experiment with real X-Ray images using TTSR-FD can remove visible artifacts in the qualitative study even though PSNR is similar. Our proposed adaptation network, TTSR-DMK, improved model performance for multiple kernels even with unknown kernel situations.
| Identifer | oai:union.ndltd.org:uottawa.ca/oai:ruor.uottawa.ca:10393/42782 |
| Date | 04 October 2021 |
| Creators | Ma, Qing |
| Contributors | Lee, Wonsook |
| Publisher | Université d'Ottawa / University of Ottawa |
| Source Sets | Université d’Ottawa |
| Language | English |
| Detected Language | English |
| Type | Thesis |
| Format | application/pdf |
Page generated in 0.0026 seconds