About me:
I am a PhD candidate at TU Munich. I currently conduct research to integrate heterogeneous data in deep learning models for medical image analysis. Closely related, I work on causal inference for such models. Previously, I was a researcher in the biomedical image analysis group at VRVis and the Laboratory for Artificial Intelligence in Medical Imaging at LMU. I completed my master’s degree in Biomedical Computing at TU Munich.
E-mail: tom_nuno.wolf@tum.de
Research interests:
- Medical Image Analysis
- Deep Learning on Heterogeneous Data
- Interpretability and Deconfounding of Neural Networks
- Unsupervised Learning and Meta-Learning
Publications
Wolf, Tom Nuno; Pölsterl, Sebastian; Wachinger, Christian DAFT: A Universal Module to Interweave Tabular Data and 3D Images in CNNs Journal Article In: NeuroImage, pp. 119505, 2022. @article{WOLF2022119505,Prior work on Alzheimer’s Disease (AD) has demonstrated that convolutional neural networks (CNNs) can leverage the high-dimensional image information for diagnosing patients. Beside such data-driven approaches, many established biomarkers exist and are typically represented as tabular data, such as demographics, genetic alterations, or laboratory measurements from cerebrospinal fluid. However, little research has focused on the effective integration of tabular data into existing CNN architectures to improve patient diagnosis. We introduce the Dynamic Affine Feature Map Transform (DAFT), a general-purpose module for CNNs that incites or represses high-level concepts learned from a 3D image by conditioning feature maps of a convolutional layer on both a patient’s image and tabular clinical information. This is achieved by using an auxiliary neural network that outputs a scaling factor and offset to dynamically apply an affine transformation to the feature maps of a convolutional layer. In our experiments on AD diagnosis and time-to-dementia prediction, we show that the DAFT is highly effective in combining 3D image and tabular information by achieving a mean balanced accuracy of 0.622 for diagnosis, and mean c-index of 0.748 for time-to-dementia prediction, thus outperforming all baseline methods. Finally, our extensive ablation study and empirical experiments reveal that the performance improvement due to the DAFT is robust with respect to many design choices. |
Pölsterl, Sebastian; Wolf, Tom Nuno; Wachinger, Christian Combining 3D Image and Tabular Data via the Dynamic Affine Feature Map Transform Conference Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2021. @conference{Poelsterl2021-daft,Prior work on diagnosing Alzheimer’s disease from magnetic resonance images of the brain established that convolutional neural networks (CNNs) can leverage the high-dimensional image information for classifying patients. However, little research focused on how these models can utilize the usually low-dimensional tabular information, such as patient demographics or laboratory measurements. We introduce the Dynamic Affine Feature Map Transform (DAFT), a general-purpose module for CNNs that dynamically rescales and shifts the feature maps of a convolutional layer, conditional on a patient’s tabular clinical information. We show that DAFT is highly effective in combining 3D image and tabular information for diagnosis and time-to-dementia prediction, where it outperforms competing CNNs with a mean balanced accuracy of 0.622 and mean c-index of 0.748, respectively. Our extensive ablation study provides valuable insights into the architectural properties of DAFT. Our implementation is available at https://github.com/ai-med/DAFT. |
You must be logged in to post a comment.