Universality

While a certain degree of diversity in the design and implementation of AI solutions in medicine is both expected and desirable to promote innovation and differentiation, the Universality principle recommends the definition and application of standards during algorithm development, evaluation and deployment. These standards, including technical, clinical, ethical and regulatory standards, will achieve at least three key objectives: (1) They will enable the development of AI technologies with increased interoperability and applicability across clinical centres, radiology units and geographical locations; (2) they will promote a culture of quality, safety and trust in medical AI based on well-proven, widely accepted frameworks; (3) they will facilitate co-creation and cooperation in medical AI between AI developers, manufacturers, radiologists, physicians, data managers and healthcare bodies based on unified language and common approaches. For increased universality in medical AI, we propose the following recommendations:

  1. Requirement definition: Requirements for universality should be compiled at the design phase for each AI tool. In particular, it should be defined whether the AI tool is intended to operate in multiple clinical sites, multiple countries, high-resource clinical settings, and/or low-resource clinical settings.
  2. Community-defined standards: To enable the deployment of medical AI technologies with increased interoperability across clinical sites, community-defined standards (g. clinical definitions, data annotations, ground truths, biomedical ontologies, evaluation criteria) should be reviewed and considered during AI development.
  3. Reference datasets: When possible, AI models should be evaluated on open-access public datasets that are representative of the real-world clinical cases, to enable more objective benchmarking.
  4. Third-party evaluation: Third-party clinical evaluation of the AI tool, independent of the AI developers, should be an integral part of the AI validation. This could be performed by dedicated institutions, networks, hospitals or companies against payment.
  5. Multi-site evaluation: Only if multi-centre usage is a requirement of the model. Requirements are normally defined at the beginning of a project, but can change over time. Hence, if there is a chance that the medical AI tool will be used in more than one site in the future, then it should be clinically evaluate for multiple sites (not necessarily “in” multiple sites).