The accuracy and reproducibility of video assessment in the pitch-side management of concussion in elite rugby

Objectives To investigate the accuracy and reliability of side-line video review of head impact events to aid identification of concussion in elite sport.

Design Diagnostic accuracy and inter-rater agreement study.

Methods Immediate care, match day and team doctors involved in the 2015 Rugby Union World Cup viewed 20 video clips showing broadcaster’s footage of head impact events occurring during elite Rugby matches. Subjects subsequently recorded whether any criteria warranting permanent removal from play or medical room head injury assessment were present. The accuracy of these ratings were compared to consensus expert opinion by calculating mean sensitivity and specificity across raters. The reproducibility of doctor’s decisions was additionally assessed using raw agreement and Gwets AC1 chance corrected agreement coefficient.

Results Forty rugby medicine doctors were included in the study. Compared to the expert reference standard overall sensitivity and specificity of doctors decisions were 77.5% (95% CI 73.1–81.5%) and 53.3% (95% CI 48.2–58.2%) respectively. Overall there was raw agreement of 67.8% (95% CI 57.9–77.7%) between doctors across all video clips. Chance corrected Gwets AC1 agreement coefficient was 0.39 (95% CI 0.17–0.62), indicating fair agreement.

Conclusions Rugby World Cup doctors’ demonstrated moderate accuracy and fair reproducibility in head injury event decision making when assessing video clips of head impact events. The use of real-time video may improve the identification, decision making and management of concussion in elite sports.