Pain Detection Through Automated Video Analysis

The study team has developed an algorithm for pain assessment based on automated video facial and body pose analysis. The investigators aim to assess the sensitivity of this algorithm in detecting pain in post-surgical patients and refine the algorithm to increase the sensitivity of pain detection in patients.

Post-surgical pain, if inadequately controlled, has deleterious short and long term consequences for the patient. Although most patients are able to report their pain scores, a minority are unable to do so and assessing their pain can prove to be a challenge for healthcare professionals. In recent years, facial recognition tools have been developed based on the premise that subtle facial variations signifies pain. However, changes in body, and head posture can also represent pain. As such, these tools are with their limitations and are only validated on certain groups of patients, thus may not be sensitive enough to detect pain in post-surgical patients. The first stage of the study will be conducted on 40 patients presenting for major gynaecological surgery, with the obtained data used to fine tune the algorithm. The patients will be video-taped pre-surgically in the pre-evaluation anaesthetic clinic and post-surgically in the ward. They will be asked to rate their pain scores on the numerical rating scale and fill in questionnaires on their psychological and quality of health status. The pain scores will be correlated with the results obtained from the pain assessment algorithm. The second phase will improve and enhance the model by (1) analysing body pose to improve the model performance; (2) validating the improved model by recruiting 200 patients undergoing surgical and pain procedures, inpatient and outpatient consultations to collect their videos before and after surgery and inpatient and outpatient pain consultations; (3) integrate the model into a standalone electronic application to improve its usability in both inpatient and outpatient settings. The third phase will recruit 130 male paediatric patients presenting for circumcision surgery to improve algorithm by i) Adding body posture analysis and other physiological measurement to further improve the performance of our model; ii) Developing our model for use in the pediatric population; and iii) Improving its usability in both clinical and non-clinical settings. Deidentified keypoints will be extracted from the videos to further validate the model.

First phase: Patients will complete questionnaire, pain score in pre-anaesthetic clinic. Their face and body pose are videotaped. Patients are reviewed on pain scores with videotaping at 12-36 hrs, 36 hrs till before discharge post-operatively. Second phase: Patients will complete questionnaires, pain score, videotaping in pre-operative/-procedural/inpatient/outpatient consultation waiting areas. After surgery/procedure/consultation, i) Surgery: Patients rate pain scores with videotaping at 12-36 hrs, 36 hrs till before discharge. ii) Procedure: Patients are asked on questionnaire, pain score, videotaping. iii) Consultation: Patients are asked on questionnaire, pain score, videotaping on next consultation. Third phase: Patients will complete questionnaire and pain score in preoperative areas. They are videotaped while wearing a physiological wristband to monitor heart rate and body temperature. After surgery, pain score is rated while being videotaped with wristband.

Before the videotaping, patients will be asked on their baseline pain scores. Their face and body pose from a frontal view will be videotaped via a mobile phone with no internet access. The collected videos will be further processed to extract key points, which will be the primary input for modelling algorithms and will further ensure anonymity of the patients in the video sequences.

Patients will be asked to fill in 1-2 questionnaires before surgery/procedure/consultation (Hospital Anxiety and Depression Scale (HADS) and/or EQ-5D-3L). After surgery/procedure/consultation, patients will be again asked to fill in HADS questionnaire (optional). For pediatrics patient, only Child Pain Anxiety Symptoms Scale (CPASS) will be administered before the surgery.

A medical grade wearable device, Empatica E4 wristband, will be used in phase 3 for pediatrics patients to monitor real-time physiological data on heart rate and body temperature.

Difference of Pain score before and after surgery/procedure/consultation. Pain scores (Numeric Rating Scale 0-10) will be asked, with zero being no pain, and 10 being the worst pain possible.

The collected videos will be further processed to extract key points, which will be the primary input for modelling algorithms and will further ensure anonymity of the patients in the video sequences. This will then be used for pain score prediction by correlated to outcome 1 (patients' reported pain score).

HADS Anxiety and Depression score before and after surgery/procedure/consultation. HADS is commonly used by doctors to determine the levels of anxiety and depression that a patient is experiencing. Each item on the questionnaire is scored from 0-3 and this means that a person can score between 0 and 21 for either anxiety or depression. The HADS uses a scale and therefore the data returned from the HADS is ordinal. For each subscore (anxiety/depression), 0-7 = Normal; 8-10 = Borderline abnormal (borderline case); and 11-21 = Abnormal (case).

EQ-5D-3L score before surgery/procedure/consultation. EQ-5D-3L is a standardized instrument for measuring generic health status. It is made up for two components; health state description and evaluation. The health status is measured in terms of five dimensions (5D); mobility, self-care, usual activities, pain/discomfort, and anxiety/depression; each dimension ranging from 1-3. From these five dimensions, EQ-5D index is calculated, having a value between 0-1. The evaluation part involves an analogue scale, asking to mark health status on the day of the interview on a 20 cm vertical scale with end points of 0 and 100. Zero corresponds to " the worst health you can imagine", and hundred corresponds to "the best health you can imagine".

CPASS score before surgery. The CPASS is a 20-item scale to evaluate pain anxiety in children. For each item, children are asked to rate the extent to which they think, act, or feel that way on a scale from 0 ("never think, act or feel that way") to 5 ("always think, act, or feel that way"). Total scores range from 0 to 100, with higher scores indicating higher levels of pain anxiety.

Inclusion Criteria Phases 1, 2: Patients undergoing surgical and pain procedures, inpatient and outpatient consultations American Society of Anaesthesiologists (ASA) physical status 1, 2 or 3 (ASA 1, 2 or 3) patients Phase 3: Male children undergoing laser circumcision in day surgery Exclusion Criteria Phases 1, 2: Pregnant patients; Medical problems/ medications: Psychiatric disorders (e.g. anxiety, depression) Neurological disorders (e.g. Cerebrovascular accident, Parkinson's Disease) Musculoskeletal limitations that result in gait abnormalities/limitations Phase 3: Developmental delay/ cognitive impairment Autism Spectrum disorder Attention-deficit/hyperactivity disorder Excessive anxiety or requires sedative premed Chronic pain conditions and/or medication Previous traumatic pain experience

Lee M, Kennedy L, Girgensohn A, Wilcox L, Lee JS, Tan CW, Sng BL. Pain Intensity Estimation from Mobile Video Using 2D and 3D Facial Keypoints. arXiv preprint arXiv:2006.12246. 2020 Jun 17.