Frontiers in AI-Enabled Healthcare: Multimodal Data, Contextual Processing & Human-AI Collaboration

This talk presents three interconnected advances toward more effective AI systems in healthcare. We introduce HAIM, a comprehensive multimodal AI framework that integrates diverse clinical data types (tabular, time-series, text, and imaging data), demonstrating 6-33% performance improvements over single-modality approaches across 14,324 models trained on MIMIC-IV data. Building on this, we present TabText, a novel approach that enhances tabular clinical data representation using large language models, yielding up to 6% improvement in prediction accuracy. Finally, we examine how generative AI optimizes human-AI collaborative workflows in healthcare innovation evaluation, showing that AI assistance with narrative explanations increases decision consistency by 12 percentage points while maintaining human oversight. Together, these advances form a vision for next-generation healthcare AI systems that combine multimodal capabilities, contextual understanding, and effective human-AI collaboration. Léonard Boussioux Assistant Professor of Information Systems & Operations Management University of Washington (UW), Foster School of Business Adjunct Assistant Professor, UW Allen School of Computer Science and Engineering Professor Boussioux’s work combines operations research and artificial intelligence. He emphasizes multimodal frameworks and data-driven decision tools, especially in the fields of healthcare and sustainability. He earned his Ph.D. in Operations Research at the Massachusetts Institute of Technology. December 3, 2024 The University of Washington is committed to ensuring digital accessibility in our services, programs, and activities. If you encounter accessibility barriers using videos found on this channel, please contact UW Video at uwvideo [at] uw [dot] edu.