Healthcare practitioner involvement in data-driven clinical decision support development and evaluation: Critical narrative review of recommendations

Objective

While healthcare practitioner (HCP) involvement is widely acknowledged as essential for the development and evaluation of trustworthy data-driven clinical decision support systems (CDSS), practical guidance remains limited. This critical narrative review examines existing frameworks, highlights HCP-related considerations, and identifies areas where further guidance is warranted.

Methods

We combined searches of Ovid Medline, the EQUATOR Network Library, and relevant reviews to September 2024, seeking frameworks for developing or evaluating data-driven CDSS. Framework characteristics, coverage across the data-driven CDSS lifecycle, and details of HCP-related recommendations were extracted for analysis.

Results

165 publications were screened, and 32 met inclusion criteria. Nine frameworks made no recommendations relating to HCP involvement. In the other 23, HCP-related recommendations were found for most phases of the data-driven CDSS development and evaluation lifecycle. Recommendations relating to HCP end users included themes of acceptability, communication, and human-AI interaction. Expert clinical input was suggested for various phases, but not required by any reporting guidelines.

Discussion

Existing guidance lacks comprehensive methods for including HCPs throughout data-driven CDSS development and evaluation. Reporting guidelines do not position HCPs as experts, which may lead to clinical expertise being overlooked. Frameworks lack detail on complex challenges such as risk communication. No frameworks suggested HCP involvement in data preparation or post-market surveillance, yet HCPs could usefully contribute to these phases.

Conclusion

HCPs should be included in data-driven CDSS development and evaluation, but there is scope to better understand how to incorporate more clinical insight, and how this might improve trustworthiness of these tools.