The Massive Multimodal Biomedical Understanding Challenge

By integrating rich visual information across diverse medical imaging modalities, anatomical scales, and disease contexts with complementary molecular, clinical, and scientific knowledge, multimodal large language models (MLLMs) hold immense potential to transform scientific discovery and precision medicine. Realizing this potential, however, requires biomedical MLLMs to possess two fundamental capabilities: perception (accurately recognizing and interpreting visual features) and cognition (integrating visual observations with prior knowledge to derive complex conclusions). While recent advances have emphasized reasoning, emerging evidence suggests that visual perception remains the primary bottleneck for reliable downstream reasoning. To address this problem, we launch the MMBU Challenge, a large-scale initiative to advance the visual perception capabilities of MLLMs across biomedicine. Using a large-scale peer-reviewed biomedical dataset curated by domain experts, we evaluate the open-ended visual question answering (VQA) capabilities of MLLMs, pushing them to recognize, localize, and characterize visual features across diverse biomedical images spanning imaging modalities, biological scales, anatomical regions, and clinical contexts.