Workshop `In the Picture: Medical Imaging Datasets´

Machine learning has shown promising results in medical image diagnosis, at times with claims of expert-level performance. However, algorithms with high reported performances do not always generalize to real-life settings, leading to incorrect and/or biased diagnoses. Two key datasets-related issues contribute to this challenge: (i) the presence of shortcuts, i.e. spurious correlations between artifacts in images and diagnostic labels, and (ii) the representativeness of the patients the algorithms were trained on, in terms of demographics and/or disease sub-type.  Our workshop's focus will be on challenges within medical imaging datasets that hinder the development of fair and robust AI algorithms. We will have several invited talks, but foster engagement and encourage collaboration, participants will mostly work in groups. Groups will work on various projects, including: (i) crafting tools to reviewing datasets, documenting metadata and identifying possible shortcuts, (ii) designing tools for generating living reviews, as traditional datasets reviews are static and cannot incorporate new evidences, (iii) formulating strategies to pursue additional funding opportunities such as COST.EU.