The development phase of FUTURE-AI framework focuses on implementing the design specifications and requirements through five key recommendations that emphasize data collection, privacy, and technical implementation. Table 4 provides a comprehensive breakdown of these development phase recommendations, detailing specific operations and examples for each recommendation, including representative data collection (robustness 2), collection of individual attributes (fairness 2), data privacy and security measures (general 2), risk mitigation implementation (general 3), and human-AI interaction mechanisms (usability 2), with varying compliance requirements between research (+) and deployable (++) AI tools.
The framework emphasizes stronger compliance requirements for deployable AI tools compared to research applications during the development phase, particularly in areas like human-AI interactions and oversight, reflecting the higher standards needed for real-world implementation. Only robustness 2 (training with representative real-world data) and general measures for privacy, security and risk mitigation are highly recommended (++) for both research and deployable tools.
| Recommendations | Operations | Examples |
|---|---|---|
| Collect representative training dataset (robustness 2) | Collect training data that reflect the demographic variations | According to age, sex, ethnicity, socioeconomics |
| Collect training data that reflect the clinical variations | Disease subgroups, treatment protocols, clinical outcomes, rare cases | |
| Collect training data that reflect variations in real world practice | Data acquisition protocols, data annotations, medical equipment, operational variations (eg, patient motion during scanning) | |
| Artificially enhance the training data to mimic real world conditions | Data augmentation, data synthesis (eg, low quality data, noise addition), data harmonisation, data homogenisation | |
| Collect information on individuals’ and data attributes (fairness 2) | Request approval for collecting data on personal attributes | Sex, age, ethnicity, socioeconomic status |
| Collect information on standard attributes of the individuals (if available and allowed) | Sex, age, nationality, education | |
| Include application specific information relevant for fairness analysis | Skin colour, breast density, presence of implants, comorbidity | |
| Estimate data distributions across subgroups | Male v female, across ethnic groups | |
| Collect information on data provenance | Data centres, equipment characteristics, data preprocessing, annotation processes | |
| Implement measures for data privacy and security (general 2) | Implement measures to ensure data privacy and security | Data deidentification, federated learning, differential privacy, encryption[ |
| Implement measures against malicious attacks | Firewalls, intrusion detection systems, regular security audits | |
| Adhere to applicable data protection regulations | General Data Protection Regulation, Health Insurance Portability and Accountability Act | |
| Define suitable data governance mechanisms | Access control, logging system | |
| Implement measures against identified AI risks (general 3) | Implement a baseline AI model and identify its limitations | Bias, lack of generalisability |
| Implement methods to enhance robustness to real world variations (if needed) | Regularisation, data augmentation, data harmonisation, domain adaptation | |
| Implement methods to enhance generalisability across settings (if needed) | Regularisation, transfer learning, knowledge distillation | |
| Implement methods to enhance fairness across subgroups (if needed) | Data resampling, bias free representation, equalised odds postprocessing | |
| Establish mechanisms for human-AI interactions (usability 2) | Implement mechanisms to standardise data preprocessing and labelling | Data preprocessing pipeline, data labelling plugin |
| Implement an interface for using the AI model | Application programming interface | |
| Implement interfaces for explainable AI | Visual explanations, heatmaps, feature importance bars | |
| Implement mechanisms for user centred quality control of the AI results | Visual quality control, uncertainty estimation | |
| Implement mechanism for user feedback | Feedback interface |