"Developing Capsule Networks for Brain Image Segmentation" by Arman Avesta

Developing Capsule Networks for Brain Image Segmentation

Date of Award

Spring 2023

Document Type

Dissertation

Degree Name

Doctor of Philosophy (PhD)

Department

Investigative Medicine

First Advisor

Aneja, Sanjay

Abstract

Segmenting the brain anatomical structures and pathological lesions on brain images is an important step in the management of various neurological disorders. Segmenting a concept on an image includes finding the concept on the image and delineating its confines. Manual segmentation is impractical in the clinical workflow because it is time-consuming, requires radiologist-level expertise, and is prone to inter- and intra-operator variability. Current auto-segmentation methods, including UNet-based models, often fail to properly segment images that are not well represented in the training data. This issue is a common problem when space-occupying lesions such as brain tumors deform the brain anatomy. When a tumor deforms the shape and location of brain structures, these structures fall out of the distributions represented in the training data. Notably, the full distribution of the anatomical variations and deformations caused by brain tumors cannot be represented in any training data, no matter how large the training data might be. Therefore, the key to segmenting brain images in the presence of space-occupying lesions is to develop a model that can generalize beyond what it sees in the training data. Auto-segmentation using capsule networks (CapsNets) provides a solution to this problem. CapsNets can generalize beyond the spatial features that are present in the training data because they encode spatial information about the concepts that they detect on the image. If a concept rotates, changes in size, or undergoes other spatial changes, the capsule that encodes that concept can still recognize it while encoding the changed spatial properties. CapsNets can achieve this level of knowledge generalization without data augmentation. This level of knowledge generalization makes CapsNets remarkably more efficient. I developed 3D CapsNets and validated them for segmenting anatomical structures and pathological lesions on brain MRIs. I used two large datasets to train and test my models: the ADNI dataset with 3,400 brain MRIs, and the Yale Glioma Dataset with 755 brain MRIs. Using these datasets, I comprehensively benchmarked the performance of my 3D CapsNets compared to commonly used UNet-based models. I showed that 3D CapsNets can segment the anatomical structures and pathological lesions of the brain on MR images with high accuracy. I also showed that 3D CapsNets have better out-of-distribution generalization compared to UNet-based models. Beyond the segmentation accuracy, I also showed that 3D CapsNets are computationally more efficient compared to UNet-based models. CapsNets required less GPU memory by an order of magnitude compared to UNets, while achieving similar performance. CapsNets also converged faster during training compared to UNet-based models.To assess the added value of 3D CapsNets for the problem of brain image auto-segmentation, I compared the efficacy of 3D, 2.5D, and 2D auto-segmentation approaches across three distinct deep learning architectures: CapsNets, UNets, and nnUNets. I found that the 3D approach is more accurate, faster to train, and faster to deploy. Moreover, the 3D auto-segmentation approach maintained better performance in the setting of limited training data. I found the major disadvantage of 3D auto-segmentation approaches to be increased computational memory requirement compared to similar 2.5D and 2D auto-segmentation approaches. When an auto-segmentation model needs to be applied to a new segmentation task, multiple decisions should be made about the pre-processing steps and training hyperparameters. These decisions are cumbersome and require a high level of expertise. To remedy this problem, I developed self-configuring CapsNets (scCapsNets) that can scan the training data as well as the computational resources that are available, and then self-configure most of their design options. I showed that scCapsNets can segment brain tumor components with high accuracy, can outperform UNet-based models in the absence of data augmentation, are faster to train, and are computationally more efficient compared to UNet-based models. Finally, I explored how to bring my auto-segmentation models to the bedside by implementing them into the picture archiving and communication system (PACS) of our radiology department. I also explored how to overcome two central challenges in the implementation of auto-segmentation models into PACS: protecting patient privacy and handling brain MRIs that contain imaging artifacts.

This document is currently not available here.

Share

COinS