Back to search

Cornell University

Synthesis of Brain Images using Deep Learning

Abstract

dc:description.abstract

Deep learning and neuroscience research have significantly progressed in the past few decades. While deep learning's conception and several key developments have been inspired by neuroscientific knowledge, the reverse osmosis of artificial neural network application in neuroscience has been limited. In this thesis, we tackle two important, but relatively under-explored tasks of brain images synthesis using deep learning. The first task bridges the gap between task-based and resting-state functional magnetic resonance imaging (fMRI), which represent the two most common experimental paradigms of functional neuroimaging. While resting-state offers a flexible and scalable approach for characterizing brain function, task-based techniques provide superior localization. We build on recent deep learning methods to create a model that predicts task-based contrast maps from resting-state fMRI scans. Specifically, we propose BrainSurfCNN, a surface-based fully-convolutional neural network model that works with a representation of the brain's cortical sheet. BrainSurfCNN achieves exceptional predictive accuracy on independent test data from the Human Connectome Project, which is on par with the repeat reliability of the measured subject-level contrast maps. Conversely, our analyses reveal that a previously published benchmark is no better than group-average contrast maps. We demonstrate that BrainSurfCNN can generalize remarkably well to novel domains with limited training data. In the second task, we apply deep learning methods in the field of neuroimaging coordinate-based meta-analysis. Neuroimaging studies are often limited by the number of subjects and cognitive processes that can be feasibly interrogated. However, a rapidly growing number of neuroscientific studies have collectively accumulated an extensive wealth of results. Digesting this growing literature and obtaining novel insights remains to be a major challenge, since existing meta-analytic tools are constrained to keyword queries. In this paper, we present \textit{Text2Brain}, an easy to use tool for synthesizing brain activation maps from open-ended text queries. Text2Brain was built on a transformer-based neural network language model and a coordinate-based meta-analysis of neuroimaging studies. Text2Brain combines a transformer-based text encoder and a 3D image generator, and was trained on variable-length text snippets and their corresponding activation maps sampled from 13,000 published studies. In our experiments, we demonstrate that Text2Brain can synthesize meaningful neural activation patterns from various free-form textual descriptions. The methodological advancements and results presented in this thesis are only the very first steps in their respective research directions but demonstrate the massive potential of deep learning in advancing neuroimaging applications and research.

Degree

thesis:*
Name thesis:degree_name
Ph. D., Electrical and Computer Engineering
Level thesis:degree_level
Doctor of Philosophy
Discipline thesis:degree_discipline
Electrical and Computer Engineering
Grantor
Cornell University
Year dc:date.issued
2022

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Ngo, Gia
Chair dc:contributor.chair
  • Sabuncu, Mert
Committee members dc:contributor.committeemember
  • Kuceyeski, Amy Frances
  • Weinberger, Kilian Quirin

Subjects

dc:subject × 5

Rights

dc:rights
Statement dc:rights
  • Attribution 4.0 International
Language dc:language.iso
en

Identifiers

dc:identifier.*
Dc Identifier Bibid
15578831
Dc Identifier Other
ProQuest Submission ID: 13160
ProQuest Publication ID: 29259227

Chain of custody

source
Harvested from
Cornell University
Base URL
ecommons.cornell.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Ngo, Gia. Synthesis of Brain Images using Deep Learning. Doctor of Philosophy thesis, Cornell University, 2022. https://hdl.handle.net/1813/112017