Impact Evaluation of Community-Based Health Programs in Rwanda (CBEHPP)

National University, Rwanda, New York University, London School of Hygiene and Tropical Medicine, Innovations for Poverty Action, Bill and Melinda Gates Foundation, Emory University

Community Health Clubs to improve local sanitation, hygiene and health conditions have been implemented in several countries in Africa and Asia with some success. The Ministry of Health in Rwanda has committed to rolling out a program designed along similar lines, the Community Based Environmental Health Promotion Program (CBEHPP), in all 15000 villages across Rwanda. The main objective of the program is to achieve 'zero open defecation' in all villages of Rwanda, at least 80% hygienic latrine coverage and improvements in a range of health behaviors such as the use of mosquito nets, hand-washing with soap and the use of household water treatment. To evaluate the impact of the program on health, other socio-economic outcomes and community functioning, a single district has been chosen where 150 communities will be randomized to receive the intervention immediately or 18 months later. The evaluation is led by US based Innovation for Poverty Action (IPA) through researchers based at the National University of Rwanda, Georgetown University, London School of Hygiene and Tropical Medicine and New York University. The research team will work in close collaboration with the implementing team, which consists of the Ministry of Health, Rwanda and Africa AHEAD. The study will span three years, beginning in May 2013, and ending in late 2015.

Preventable diseases such as respiratory infections, diarrheal diseases and malaria account for a large share of mortality in low income economies. For example, diarrheal diseases kill nearly 2.5 million people worldwide and account for a considerable share of under five mortality in low income countries. Broad-based economic development that includes improvements in infrastructure and service delivery can address the challenges associated with the infectious disease burden. But for many countries, these improvements are far in the future and/or very costly. Behavior change of individuals can also address these challenges and more importantly can be done now and relatively cheaply. Understanding how to produce this behavior change in needy contexts is crucial to the short and medium term response to the high costs imposed by these diseases. This protocol outlines an evaluation of an intervention aimed at producing sustainable behavior change around health, hygiene and sanitation. The evaluation consists of two parts: firstly to evaluate the health and socio-economic effects of a well-designed and supported community health club intervention. Secondly to examine whether the intervention can deliver health and other benefits utilizing the resource base available in this context. The answer to this latter question informs the scalability of the intervention. The overall objective of the study is to evaluate whether and how community hygiene/health clubs are an effective and sustainable response to addressing broad health, sanitation and hygiene needs in low-income countries. A particular challenge associated with health interventions, is the extent to which free-rider problems mediate the adoption of improved behaviors. The community health club approach with regular meetings, is well placed to address these collective action challenges. Community health clubs provide a vehicle to inform and incentivize households to change their behavior in relation to a range of health outcomes. The information component arises from the six month long weekly courses covering a wide range of health, hygiene and sanitary subjects and facilitated by a community health worker. The incentives for sustainable behavior change follow from the social sanctions and rewards that club members can impose/bestow on members accordingly. As with other behavior change programs that have incorporated information components that emphasize the benefits of better health behavior, the evidence base for what works in this domain remains thin. The community health club approach is appealing largely because it mobilizes local resources and is plausibly cheaper and more sustainable than other top-down interventions. To evaluate the impact of the program on health, other socio-economic conditions and community functioning, a single district has been chosen where communities will be randomly assigned to receive the intervention immediately or 18 months later. The evaluation is led by US-based Innovation for Poverty Action (IPA) through researchers based at the National University of Rwanda, Georgetown University, London School of Hygiene and Tropical Medicine and New York University. The research team will work in close collaboration with the implementing team, which consists of the Ministry of Health, Rwanda and Africa AHEAD. The evaluation will measure impact heterogeneity with respect to key inputs that predict success of CHCs, including training and competence of CHWs, the quality of training materials, and community functioning. The evaluation will also document in detail the cost structure of the low- and high-resource versions of CHCs in order to inform the MOH about the most effective strategies for successful scale up of CHCs nationwide. More broadly, other governments, donor organizations and foundations will benefit from rigorous evidence on the effectiveness and cost-effectiveness of using health clubs to address health and sanitation challenges.

The standard "classic" approach will implement a total of 20 community health club sessions delivered through weekly education programs in the target communities as per the training manual. Community health workers (CHW) will receive careful training in the delivery of the CBEHPP instruction. High quality instructional materials (in color) will be used. Club members will each receive a membership card to be used to track attendance and compliance. Finally model home competitions and a graduation ceremony will be held. Monitoring of the clubs will be conducted by community health workers using mobile phones.

The "lite" trial arm will only implement 8 sessions covering all the WASH topics. It will be facilitated by CHWs receiving minimal training and using black/white photocopies of instructional materials. Members will not be issued with membership cards and will not have a graduation ceremony or home garden competitions. Minimal monitoring of this arm will be carried out by environmental health officers.

The control group is not enrolled in the CBEHPP. Because of the government's commitment for the national roll out to the CBEHPP, the control population will receive the intervention as soon as possible following the conclusion of the trial phase. Nevertheless, we will continue to evaluate the sustained impact of the intervention for two additional years by monitoring various behavioural outcomes and indicators and their impact on exposure outcomes (drinking water, hand hygiene, consumption, schooling and labour market participation etc.). We will use data from the RCT phase and clinical records to estimate the effect of any sustained impact on health. Long term impacts can be inferred by using data from the trial as well as data on long term behavioural outcomes.

The standard "classic" approach will implement a total of 20 community health club sessions delivered through weekly education programs in the target communities as per the training manual. Community health workers (CHW) will receive careful training in the delivery of the CBEHPP instruction. High quality instructional materials (in color) will be used. Club members will each receive a membership card to be used to track attendance and compliance. Finally model home competitions and a graduation ceremony will be held. Monitoring of the clubs will be conducted by community health workers using mobile phones.

The "lite" trial arm will only implement 8 sessions covering all the WASH topics. It will be facilitated by CHWs receiving minimal training and using black/white photocopies of instructional materials. Members will not be issued with membership cards and will not have a graduation ceremony or home garden competitions. Minimal monitoring of this arm will be carried out by environmental health officers.

The main outcome of the health impact evaluation trial will be 14 day recall self-reported diarrhoea in children under 5 years.

Height-for-age data will be collected from all children under 5 years old. Changes in Z-scores for each of these measures between baseline and endline will be compared across the two study arms and the control group.

weight-for-age data will be collected from all children under 5 years old. Changes in Z-scores for each of these measures between baseline and endline will be compared across the two study arms and the control group.

Water quality data will be collected at baseline and endline, and at an intermediate visit following implementation of the program. Water quality will be determined testing of drinking water; study households will provide a sample of drinking water for contamination tests.

A public goods game is played with a subset of community members. The researchers examine the impact of the treatment on the level of contributions.

Total counts of child diarrhea cases recorded in catchment area clinics and community health worker records

Total counts of child pneumonia cases recorded in catchment area clinics and community health worker records

Inclusion Criteria: -All households with children under 5 years in the study communities are eligible for socio-economic and health outcome evaluation. Exclusion Criteria: -All households are eligible to participate in the intervention. However, for the purposes of the study, households with no children under the age of 5 will not be selected for data collection purposes.

Sinharoy SS, Schmidt WP, Wendt R, Mfura L, Crossett E, Grepin KA, Jack W, Rwabufigiri BN, Habyarimana J, Clasen T. Effect of community health clubs on child diarrhoea in western Rwanda: cluster-randomised controlled trial. Lancet Glob Health. 2017 Jul;5(7):e699-e709. doi: 10.1016/S2214-109X(17)30217-6.