A behavior-based model to validate electronic systems designed to collect patient-reported outcomes: model development and application

Background: The merits of technology have been adopted in capturing patient-reported outcomes (PROs) by incorporating PROs into electronic systems. Following the development of an electronic system, evaluation of system performance is crucial to ensuring the collection of meaningful data. In contemporary PRO literature, electronic system validation is overlooked, and evidence on validation methods is lacking.

Objective: This study aims to introduce a generalized concept to guide electronic patient-reported outcome (ePRO) providers in planning for system-specific validation methods.

Methods: Since electronic systems are essentially products of software engineering endeavors, electronic systems used to collect PRO should be viewed from a computer science perspective with consideration to the health care environment. On this basis, a testing model was blueprinted and applied to a newly developed ePRO system designed for clinical use in pediatric dentistry (electronic Personal Assessment Questionnaire-Paediatric Dentistry) to investigate its thoroughness.

Results: A behavior-based model of ePRO system validation was developed based on the principles of user acceptance testing and patient-centered care. The model allows systematic inspection of system specifications and identification of technical errors through simulated positive and negative usage pathways in open and closed environments. The model was able to detect 15 positive errors with 1 unfavorable response when applied to electronic Personal Assessment Questionnaire-Paediatric Dentistry system testing.

Conclusions: The application of the behavior-based model to a newly developed ePRO system showed a high ability for technical error detection in a systematic fashion. The proposed model will increase confidence in the validity of ePRO systems as data collection tools in future research and clinical practice.