Background The use of telemedicine consultations in healthcare systems is rapidly increasing around the world, accelerated by the global COVID-19 pandemic. Digital cameras integrated in laptops, as well as smartphones and tablet computers, have provided an accessible method for patients to consult with clinicians. However, telemedicine consultations remain severely limited compared to standard faceto-face consultations, because clinicians cannot accurately examine patients remotely. This limitation, combined with the swift uptake of telemedicine globally, results in significant risks to patient safety. Thus, there is a growing and urgent need to extend the capabilities of technologies so that telemedicine consultations can better meet the needs of patients and clinicians. In all areas of healthcare, clinicians assess human movement throughout the life course of their patients as a fundamental part of clinical assessment. Currently, there are no methods to do this objectively via telemedicine.
Objectives Laptops are one of the most common devices used by patients for telemedicine consultations, but it is unclear whether videos from laptop cameras (with relatively low frame rate) can be used to accurately measure human movements at a range of speeds. The objectives of this study were to determine the validity and reliability of deep learning computer vision methods applied to video collected via a laptop camera to measure finger tapping, a well validated test of human movement.
Method Sixteen healthy adults (9 female, mean age 34.5 years; range 24-52) completed finger-tapping tests internally-paced ‘as big and fast as possible’ and externally-paced by an auditory metronome at frequencies of 0.5Hz, 1Hz, 2Hz and 3Hz. Hand movements were recorded simultaneously by a standard laptop camera at 30 frames per second (FPS) and by Optotrak, a high-speed 3D motion analysis system sampling at 250 FPS. Three DeepLabCut (deep learning-based artificial neural network) architectures were applied to the laptop video to track thumb-tip and index fingertip position and the extracted movement features of each method were compared to the ground truth Optotrak motion tracking.
Results The computer vision methods showed excellent validity and reliability compared to the Optotrak between 0.5Hz and 4Hz tapping frequency. Over 97% (552/538) of the computer vision measures were within +/-0.5Hz of the Optotrak measures. At higher tapping frequencies, there was a progressive decline in accuracy, attributed to motion blur associated with the laptop camera’s low FPS. This study shows that deep learning computer vision methods using laptop cameras hold strong potential for providing objective measurements of human movements during telemedicine consultations. This technology could substantially augment remote clinical assessments in healthcare systems but further developments will be required to accurately measure the fastest movements.
CORE (COnnecting REpositories)
CORE (COnnecting REpositories)