Pitfalls of Neuroimaging Predictive Models of Individual Differences

Date of Award

Spring 1-1-2025

Document Type

Dissertation

Degree Name

Doctor of Engineering (DEng)

Department

Biomedical Engineering (ENAS)

First Advisor

Scheinost, Dustin

Abstract

Advances in data collection efforts and computational methods have established neuroimaging-based predictive models as a fundamental area of human neuroscience research. Predictive models have successfully related individual differences in brain structure and function to a wide variety of phenotypes, but several pitfalls challenge the reliability of these models. In this thesis, I highlight several prominent limitations that need to be addressed to improve the reproducibility, replicability, and generalizability of neuroimaging predictive models of individual differences. In Chapters 2-3, I demonstrate how accidentally introducing information about the test data into the model training pipeline ("data leakage") leads to the misestimation of prediction performance. In Chapters 4-6, I discuss how neuroimaging-based predictive models are susceptible to minor data manipulations, which limits the trustworthiness of results in both research and clinical settings. Among Chapters 2-6, external validation - or the evaluation of models in independent datasets - emerged as a promising solution. Yet, in Chapters 7-8, I describe how current implementations of external validation are not conducive to improving the robustness of neuroimaging results due to poor statistical power. Notably, unlike traditional power calculations, power in external validation depends on two sample sizes: the training dataset sample size and the external dataset sample size. Together, the results presented in this thesis provide a strong foundation for advancing the robustness and reliability of neuroimaging predictive models, which brings the field one step closer to practical utility.

This document is currently not available here.

Share

COinS