Evaluation of Clinical Impacts and Costs of eHealth in Rwanda

Evaluation of the Clinical Impacts and Costs of eHealth in Rwanda Using Innovative Frameworks and Local Capacity Building

Centers for Disease Control and Prevention, Ministry of Health, Rwanda, Rwanda Biomedical Centre, Partners in Health, Innovative Support to Emergencies Diseases and Disasters, University of Pittsburgh, Jembi Health Systems, Brown University

This study will estimate the impact of a suite of clinical decision-support tools on structural, process, and clinical outcomes related to HIV care. The "enhanced EMR" package under investigation will include EMR monitoring tools, data quality control procedures and support, patient reports, alerts, and reminders about patient care. This intervention will be delivered by the Ministry of Health and Rwanda Biomedical Centre and monitored by the study team led by University of Rwanda's School of Public Health and Brown University.

Motivation of the study A previous cross sectional analysis of the national HIV program in Rwanda, described the HIV care continuum as a "multitrajectory pathway" with many opportunities for patients to exit and return to care between diagnosis and viral suppression. The authors concluded that the weakest point in the continuum is the transition from diagnosis to linkage to care where only half of newly diagnosed patients link to care within 6 months of receiving their diagnosis. This study also estimated that 82.2 percent of patients on ART achieve viral suppression. Overall, half of the HIV-positive population in Rwanda in 2013 was assumed to be virally suppressed. This estimate of viral suppression is based on an analysis of EMR data for a subset of 21,995 patients. Correspondence with one of the study authors clarified that 9,680 of these patients were eligible for viral load testing, and 3,066 of eligible patients had recorded viral load data. This suggests that two-thirds of patients eligible for viral load testing do not have viral load results recorded in the EMR. The study do not estimate any type of treatment failure (virologic, immunologic, clinical), and investigators are not aware of any such estimates for Rwanda. Studies in Botswana, Malawi, Uganda, South Africa, and Cameroon found that 15 to 25 percent of patients had recorded plasma HIV RNA concentrations in excess of 400 copies per mL within 3 years of starting first-line ART. More recently, kenyan study found that among a large cohort of Kenyan patients on ART, 11.6 percent had evidence of immunological treatment failure during the 12-month study period. In the Kenya study, investigators randomised 7 of 13 clinics using EMRs to an intervention group that received alerts and reminders about immunological treatment failure. The rate of appropriate clinical action in response to treatment failure increased from 30 percent in the control group to 54 percent in the intervention group. The authors also reported a 72 percent relative reduction in the time from the detection of treatment failure to appropriate clinical action. Investigators did not estimate the impact of the CDSS on treatment outcomes such as viral suppression and survival. With the proposed study in Rwanda, investigators see an opportunity to use low-cost decision support tools to increase the rate of linkage to care from diagnosis, improve data quality and completeness for laboratory data such as viral load, demonstrate the efficacy of these decision support tools for prompting timely clinical intervention following treatment failure, and demonstrate that early intervention can lead to positive clinical outcomes for patients. Intended/potential use of study findings The study findings will inform the Rwandan government on the performance, clinical impact and costs of the systems they have been implementing, and should help them decide on future eHealth investments for a variety of locations. The results will also help to inform such investments in a wide range of other low and middle income countries managing HIV and other diseases. Design/locations Investigators will conduct a cluster-randomized trial to estimate the treatment effect of the enhanced EMR packages on structural, process, and clinical outcomes related to HIV care in Rwanda. Research questions and outcomes Investigators will ask four primary research questions about the effect of the decision support intervention on process, structural, and clinical outcomes: Do alerts and reminders improve the linkage from HIV testing to care? Outcomes: a. Rate of linkage to care among HIV-positive patients within 3 months after diagnosis b. Time from HIV+ test result to linkage to care Do alerts and reminders improve the quality and completeness of routine lab results in the EMR? Outcomes: Percent of patients on ART completing their 6th month of treatment who have viral load results recorded in the EMR within 2 months of this initial milestone. Percent of patients on ART who get an annual VL test and have the results recorded in the EMR within 2 months of this annual milestone. Do alerts and reminders following treatment failure detected by CD4 or viral load improve clinical action? Outcomes: Percent of ART patients who have a recorded clinical action within 1 month of detected treatment failure Time from treatment failure to recorded clinical action 4. Do alerts and reminders following treatment failure detected by CD4 or viral load improve therapeutic outcomes such as viral suppression? Outcome: Percent of patients who experience treatment failure who are fully suppressed 4 months after the point of failure Hypotheses With the proposed study in Rwanda, investigators hypothesise that low-cost decision support tools can increase the rate of linkage to care from diagnosis, improve data quality and completeness for laboratory data such as viral load and CD4, and timely clinical intervention following treatment failure. Investigators will implement several levels of randomisation to answer different research questions mentioned above. I. Do alerts and reminders improve the linkage from HIV testing to care? Randomize included facilities to two arms: Intervention 1 (Int1) and Control (Ctrl1). Facilities assigned to the Ctrl1 will not receive any additional equipment, software tools, training or other forms of support. Facilities assigned to the enhanced package for Int1 will receive alerts and reminders to promote linkage from diagnosis to care. II. Do alerts and reminders improve the quality and completeness of lab results in the EMR? Randomize the Intervention 1 group into two additional arms: Intervention 2 (Int2) and Control (Ctrl2). Facilities assigned to Int2 will also receive alerts and reminders to improve lab reporting as part of their enhanced package. III. Do alerts and reminders following treatment failure detected by CD4 or viral load improve clinical action? Randomize the Intervention 2 group into two additional arms: Intervention 3 (Int3) or Control (Ctrl3). Facilities assigned to Int3 will also receive alerts and reminders to improve clinical response to the detection of treatment failure as part of their enhanced package. IV. Do alerts and reminders following treatment failure detected by CD4 or viral load improve clinical outcomes such as viral suppression? (no additional randomisation) Investigators believe that this cascading randomisation is needed because interventions designed to improve services at the beginning of the HIV care continuum could have downstream effects that might make it challenging to estimate the effect of each additional intervention in isolation. For instance, providing facilities with tools to improve the linkage from HIV testing to care (Int1) could improve a facility's data capture more generally and potentially improve ordering and recording of lab results (Int2), which would bias the results. Therefore, investigators propose to randomise to Int2 from within the subset of facilities assigned to Int1. For 90% power with alpha of 0.05, an ICC of 0.15, equal allocation to the final study arms, and 10 patients per cluster who experience treatment failure during the study, investigators could detect a shift in the percentage of patients who achieve viral suppression following treatment failure of 30 percentage points from 30% to 60%. These numbers are minimum targets and the investigators plan to enrol more sites if feasible to increase the power of the study. Definition of Primary Outcomes and Patient Cohorts 1a. Rate of linkage to care among HIV-positive patients Cohort: Every new adult patient (18 or older) who tests positive for HIV from the start of the trial through month 9. Outcomes for last "enrolled" patients measured in study month 12. Baseline situation: a study in Rwanda reported that 50% of diagnosed cases were linked to care within 3 months. Impact: Shift proportion from 50% to 75% b. Time from HIV+ test result to linkage to care Cohort: All adults with HIV positive test results recorded in the EMR at a study facility. Same timeline as 1a. Endpoint: Linked to care at a study facility within 3 months (N3 N4) Baseline situation: No data Impact: 50% decrease a. percentage of ART patients have viral load results in EMR (initial) Cohort: Every existing ART patient who completes their 6th month of treatment from the start of the trial until study month 10. Outcomes for last "enrolled" patients measured in study month 12. Baseline situation: Based on data presented in one study done in Rwanda and correspondence with one of the study authors, investigators estimate that approximately two-thirds of patients eligible for viral load testing do not have viral load results recorded in the EMR. Impact: 30% increase 2b. Percentage of ART patients have viral load results in EMR (annual) Cohort: Every existing ART patient who completes 12 months of treatment (annual) from the start of the trial until study month 10. Outcomes for last "enrolled" patients measured in study month 12. Baseline situation: Same as 2a Impact: 30% increase 3a. Percentage of ART patients with treatment failure experience clinical action Cohort: Every existing ART patient who has been on ART for at least 12 months and experiences treatment failure between the start of the sub-trial and study month 11. Outcomes for last "enrolled" patients measured in study month 12. Baseline situation: No data Impact: 50% increase 3b. Time from detection of treatment failure to clinical action Cohort: Every existing ART patient who has been on ART for at least 18 months and experiences treatment failure between the start of the trial and study month 11. Endpoint: Time in days from treatment failure (N6e) to recorded clinical action. Baseline situation: No data Impact: 50% decrease in time from treatment failure to clinical action 4. Percentage of patients who experience treatment failure who are fully suppressed 4 months after the point of failure Cohort: Every existing ART patient who has been on ART for at least 12 months and experiences treatment failure between the start of the sub-trial and study month 8. Outcomes for last "enrolled" patients measured in study month 12. Baseline situation: Assumed to be 30% in power calculation Impact: 30 percentage points from 30% to 60% Analysis Investigators will analyse the data using individual-level and cluster-level approaches: Individual-level Investigators will estimate intent-to-treat (ITT) treatment effects via logistic regression of the primary outcomes on cluster assignment to treatment (see contrasts in Table 1) blocking strata, and a vector of facility-level and patient-level baseline covariates. Standard errors will be clustered at the facility-level. Investigators will run sensitivity analyses with multilevel modelling approaches. Investigators will also use Kaplan-Meier methods to calculate time-to-event; to test the null hypothesis that there is no difference between the survival curves, investigators will use the log rank test. Cluster-level Investigators will estimate the ITT treatment effects via ordinary least squares regression of the primary outcomes on cluster assignment to treatment (see contrasts in Table 1) blocking strata, and a vector of facility-level covariates. All research questions, hypotheses and study endpoints recorded here have been approved by the IRBs in Rwanda and at CDC prior to 1/1/2018. Data Management All study facilities will have EMR systems by design. Therefore, most data will be collected by facility staff via routine care procedures. To gain access to this data, investigators will create automated scripts that create a study ID for each patient and extract de-identified data from the EMR. MOH EMR specialists will review the scripts to ensure that data are properly de-identified.

Facilities assigned to the enhanced package for Int1 will receive alerts and reminders to promote linkage of HIV positives from diagnosis to care.

Facilities assigned to the Ctrl1 will not receive any additional equipment, software tools, training or other forms of support.

Randomise the Intervention 1 group into two additional arms: Intervention 2 (Int2) and Control (Ctrl2). Facilities assigned to Int2 will also receive alerts and reminders to improve lab reporting as part of their enhanced package.

Facilities assigned to the Ctrl2 will not receive any additional equipment, software tools, training or other forms of support to improve lab reporting as part of their enhanced EMR.

Randomise the Intervention 2 group into two additional arms: Intervention 3 (Int3) or Control (Ctrl3). Facilities assigned to Int3 will receive alerts and reminders to improve clinical response to the detection of treatment failure as part of their enhanced package.

Facilities assigned to Ctrl3 will not receive alerts and reminders to improve clinical response to the detection of treatment failure as part of their enhanced package.

This intervention will consist of the following additions to the EMR package. A link on the clinician's homepage to enrol a new HIV+ patient in the EMR which will open a form for (1) entering patient demographics (2) adding the contact home address or description of area, phone number (if available), (3) the peer educator contacts (4) recording the HIV+ result and date. A report will be added that is run every week to identify HIV+ patients not linked to care. The patients identified will be checked with paper records to ensure they have definitely not visited, then contacted after one, 2 weeks and 4 weeks if he/she did not show up. After two attempted contacts, if the patient is not yet linked to care he/she will be visited at home by the health facility social worker using routine home visits by health care providers.

The data on availability of VL results in the EMR will come from a SQL statement to query the OpenMRS database. An alert will be fired if the patient has been enrolled for 8 months or more and does not have a viral load result in the EMR. The alert will be displayed on the patient summary and on the consult sheets, with text requesting the clinician orders a VL.

Alerts and reminders following treatment failure detected by CD4 or viral load improve clinical action

The data on VL results in the EMR showing detectable virus will come from a SQL statement to query the OpenMRS database. An alert will be fired if the patient has been enrolled for at least 12 months and the VL result in the EMR shows > 1000 copies/mm3. The alert will be displayed on the patient summary and on the consult sheets requesting actions to address treatment failure (change first line medication, start second line medication, repeat VL, counselling on treatment adherence). A report will also be added to regularly check for patients with high viral load.

Denominator: All adults (18 or older) with HIV positive test results recorded in the EMR at a study facility. Patients who die in the time between receiving a positive test result and the outcome measurement at 3 months will be excluded. Numerator: Subset of these patients who are linked to care at a study facility within 3 months

Denominator: Adult patients on ART completing their 6th month of treatment, thus becoming eligible for viral load monitoring. Numerator: Subset of these patients with VL results in the EMR 2 months after becoming eligible for testing

Denominator: Adult patients who have been on ART for at least 12 months and experience treatment failure: Virologic (viral load ≥ 1000 copies/ml) Immunological (>50% change in CD4 from highest previous value) Numerator: Subset of these patients who have a recorded clinical action in response to treatment failure within 1 month of the detected treatment failure.

Percentage of patients who experience treatment failure who are fully suppressed 4 months after the point of failure

Denominator: Adult patients who have been on ART for at least 12 months (first eligible for VL testing at 6 months, first expected result 8 months, retest after 4 months) and were found to have possible treatment failure. Numerator: Subset of these patients who are fully suppressed (viral load < 1000 copies /ml) 4 months after the point of treatment failure.

All adults with HIV positive test results recorded in the EMR at a study who are linked to care at a study facility within 3 months

Denominator: Adult patients on ART with at least 12th months of treatment, thus becoming eligible for annual viral load monitoring. Numerator: Subset of these patients with VL results in the EMR 2 months after becoming eligible for testing

Every existing ART patient who has been on ART for at least 18 months and experiences treatment failure between the start of the trial and study month 11

Inclusion Criteria: Is a health center with an average (3 month) monthly volume of 50-700 patients Is owned and operated by the public sector or faith-based institutions Has a power source Has network connectivity Has at least 3 computers and 1 printer Exclusion criteria: District hospitals (typically with high patient volume) Privately owned facilities Facilities operated by Partners in Health (who already run a version of the intervention) Facilities that only offer PMTCT services Facilities that run OpenMRS version 1.9 (rather than 1.6)

Public data sets will be accessible and shared to anyone who needs them upon writing request letter to the RBC HIV division through Principal Investigator and get approval written letter to access data.

Allen C, Jazayeri D, Miranda J, Biondich PG, Mamlin BW, Wolfe BA, Seebregts C, Lesh N, Tierney WM, Fraser HS. Experience in implementing the OpenMRS medical record system to support HIV treatment in Rwanda. Stud Health Technol Inform. 2007;129(Pt 1):382-6.

Amoroso CL, Akimana B, Wise B, Fraser HS. Using electronic medical records for HIV care in rural Rwanda. Stud Health Technol Inform. 2010;160(Pt 1):337-41.

Driessen J, Cioffi M, Alide N, Landis-Lewis Z, Gamadzi G, Gadabu OJ, Douglas G. Modeling return on investment for an electronic medical record system in Lilongwe, Malawi. J Am Med Inform Assoc. 2013 Jul-Aug;20(4):743-8. doi: 10.1136/amiajnl-2012-001242. Epub 2012 Nov 9.

Mamlin BW, Biondich PG, Wolfe BA, Fraser H, Jazayeri D, Allen C, Miranda J, Tierney WM. Cooking up an open source EMR for developing countries: OpenMRS - a recipe for successful collaboration. AMIA Annu Symp Proc. 2006;2006:529-33.

Oluoch T, Santas X, Kwaro D, Were M, Biondich P, Bailey C, Abu-Hanna A, de Keizer N. The effect of electronic medical record-based clinical decision support on HIV care in resource-constrained settings: a systematic review. Int J Med Inform. 2012 Oct;81(10):e83-92. doi: 10.1016/j.ijmedinf.2012.07.010. Epub 2012 Aug 24.

Oluoch T, Katana A, Kwaro D, Santas X, Langat P, Mwalili S, Muthusi K, Okeyo N, Ojwang JK, Cornet R, Abu-Hanna A, de Keizer N. Effect of a clinical decision support system on early action on immunological treatment failure in patients with HIV in Kenya: a cluster randomised controlled trial. Lancet HIV. 2016 Feb;3(2):e76-84. doi: 10.1016/S2352-3018(15)00242-8. Epub 2015 Dec 17.

Rosen S, Fox MP. Retention in HIV care between testing and treatment in sub-Saharan Africa: a systematic review. PLoS Med. 2011 Jul;8(7):e1001056. doi: 10.1371/journal.pmed.1001056. Epub 2011 Jul 19.

Nsanzimana S, Kanters S, Remera E, Forrest JI, Binagwaho A, Condo J, Mills EJ. HIV care continuum in Rwanda: a cross-sectional analysis of the national programme. Lancet HIV. 2015 May;2(5):e208-15. doi: 10.1016/S2352-3018(15)00024-7. Epub 2015 Mar 27.