Robustness

The Robustness principle refers to the ability of a medical AI tool to maintain its performance and accuracy under expected or unexpected variations in the input data. Existing research has shown that even small, imperceptible variations in the input data may lead AI models into incorrect decisions. Biomedical and health data can be subject to significant variations in the real world (both expected and unexpected), which can affect the performance of AI tools. Hence, it is important that medical AI tools are designed and developed to be robust against real-world variations, as well as evaluated and optimised accordingly.

To this end, three recommendations for Robustness are defined in the FUTURE-AI framework. At the design phase, the development team should first define robustness requirements for the medical AI application in question, by making an inventory of the potential sources of variation e.g. data-, equipment-, clinician-, patient- and centre-related variations (Robustness 1). Accordingly, the training datasets should be carefully selected, analysed and enriched to reflect these real-world variations as much as possible (Robustness 2). Subsequently, the robustness of the AI tool, as well as measures to enhance robustness, should be iteratively evaluated under conditions that reflect the variations of real-world clinical practice (Robustness 3).

Recommendation Practical steps Examples of approaches and methods Stage
Robustness 1. Define sources of data variations
  • Identify potential variations
  • Analyse impact on AI performance
  • Prioritise variations to address
  • Equipment differences
  • Technical faults
  • Data acquisition heterogeneities
  • Annotation variations
 Design
Robustness 2. Train with representative data
  • Collect diverse datasets
  • Analyse dataset representativeness
  • Augment datasets if needed
  • Validate dataset quality
  • Multi-centre data collection
  • Data augmentation techniques
  • Synthetic data generation
  • Data quality assessment tools
 Development
Robustness 3. Evaluate robustness
  • Design robustness tests
  • Perform stress testing
  • Assess repeatability
  • Implement robustness improvements
  • Adversarial testing
  • Data perturbation experiments
  • Reproducibility studies
  • Robustness enhancement techniques
 Evaluation