Back to resultsThe Doctors for Coronavirus Prevention Project Thanksgiving / Christmas Messaging Campaign
Primary Purpose
Covid19
Status
Completed
Phase
Not Applicable
Locations
United States
Study Type
Interventional
Intervention
Facebook Ads on the importance of staying safe during the Thanksgiving / Christmas holiday
Sponsored by
About this trial
This is an interventional prevention trial for Covid19
Eligibility Criteria
Inclusion Criteria:
- Individuals viewing the ads must be Facebook users, 18 years and older. Facebook will decide specifically which users receive the messages within a target geographical area (zip or county).
Exclusion Criteria:
-
Sites / Locations
- JPAL North America
Arms of the Study
Arm 1
Arm 2
Arm Type
Experimental
No Intervention
Arm Label
High-Intensity Treatment Arm
Low-Intensity Control Arm
Arm Description
Treatment: Individuals received approximately 3 Facebook ads over a 2-week period. Each ad contained a short video recorded by a physician using a script that discusses staying safe during the holidays by considering not traveling and using a mask when appropriate. Randomization to treatment: The 13 states which centrally report COVID-19 cases at the ZCTA level contained 829 counties. At Thanksgiving, 9 counties were excluded due to data limitations. At Christmas, 62 counties were excluded due to data limitations and potential negative impacts in rural, conservative areas resulting from polarization in the wake of the 2020 presidential election. Approximately half of the counties were randomized to high-intensity treatment with the remaining randomized to low-intensity treatment. In high-intensity counties, 3/4 of ZCTAs were treated (i.e., Facebook users in those ZCTAs received ads) and 1/4 were not. In low-intensity counties, 1/4 of ZCTAs were treated and 3/4 were not.
Control: Individuals did not receive ads containing short physician-recorded videos. Randomization to treatment: The 13 states which centrally report COVID-19 cases at the ZCTA level contained 829 counties. At Thanksgiving, 9 counties were excluded due to data limitations. At Christmas, 62 counties were excluded due to data limitations and potential negative impacts in rural, conservative areas resulting from polarization in the wake of the 2020 presidential election. Approximately half of the counties were randomized to high-intensity treatment with the remaining randomized to low-intensity treatment. In high-intensity counties, 3/4 of ZCTAs were treated (i.e., Facebook users in those ZCTAs received ads) and 1/4 were not. In low-intensity counties, 1/4 of ZCTAs were treated and 3/4 were not.
Outcomes
Primary Outcome Measures
Facebook Movement Metric
The change in movement metric is the percent change in distance covered in a day compared to the same day of the week in the benchmark period of February 2-29, 2020, by people who started the day in a particular location. We define holiday travel as travel during the three days preceding each holiday, since the available data does not allow us to compute the impact of the intervention on the return travel (after the holiday). The reason is that the mobility data describes the behavior throughout the day, for people who were in each county that morning. Since the campaign was targeted based on home location, we can only capture its impact on travel away from home, not back home.
Percentage Leaving Home on Day of Holiday
The Facebook stay put metric is the percentage of people who stay within a small geographical area (a "tile" of 600m*600m in which they started the day). We use it to compute the share of people leaving home on the day of the holiday (i.e. this variable = 1 - stay put on the day of the holiday).
Inverse Hyperbolic Sine of COVID-19 Cases
Inverse hyperbolic sine of COVID-19 cases during a 14 day period starting 5 days after each holiday
Secondary Outcome Measures
Knowledge of COVID-19 Prevention Message (Recall of ad, Intent to Travel, Mask Wearing and Beliefs About Travel)
Facebook will ask a set of users in treated zip codes to answer four survey questions (each user answers only one question each). A small within-zip control group will be held from being treated, enabling comparisons between those who see the videos and those who don't. These questions will be asked a few days after the ad is shown on the User's feed and will include 1) recall of the ad; 2) intention of traveling over holiday; 3) intention of wearing a mask; and 4) beliefs about whether people should travel over holiday.
Average Number of Tiles People Occupy (Mobility Measure)
this measures each day "how much people move around by counting the number of level-16 Bing tiles they are seen in within a day. People seen in more tiles are probably moving around more, while people seen in fewer are probably moving around less. Each day take eligible people in a given region and compute the number of distinct tiles they were seen in." This is aggregated to the county level.
Percentage of Eligible Participants Only Observed in a Single Level-16 Bing Tile (no Change in Movement)
Facebook "Stay-put" data: this measures each day "the percentage of eligible people who are only observed in a single level-16 Bing tile (600m x 600m) during the course of a day" aggregated to they county level.
Full Information
NCT ID
NCT04644328
First Posted
November 13, 2020
Last Updated
December 15, 2021
Sponsor
National Bureau of Economic Research, Inc.
Collaborators
Massachusetts Institute of Technology, Harvard University, Massachusetts General Hospital, Yale University, Stanford University
1. Study Identification
Unique Protocol Identification Number
NCT04644328
Brief Title
The Doctors for Coronavirus Prevention Project Thanksgiving / Christmas Messaging Campaign
Official Title
The Doctors for Coronavirus Prevention Project Thanksgiving / Christmas Messaging Campaign
Study Type
Interventional
2. Study Status
Record Verification Date
December 2021
Overall Recruitment Status
Completed
Study Start Date
November 12, 2020 (Actual)
Primary Completion Date
February 11, 2021 (Actual)
Study Completion Date
February 11, 2021 (Actual)
3. Sponsor/Collaborators
Responsible Party, by Official Title
Principal Investigator
Name of the Sponsor
National Bureau of Economic Research, Inc.
Collaborators
Massachusetts Institute of Technology, Harvard University, Massachusetts General Hospital, Yale University, Stanford University
4. Oversight
Studies a U.S. FDA-regulated Drug Product
No
Studies a U.S. FDA-regulated Device Product
No
Data Monitoring Committee
No
5. Study Description
Brief Summary
Facebook ads with physician-delivered videos were shown before the Thanksgiving and Christmas holidays and focused on staying safe during the COVID pandemic by limiting travel and mask-wearing.
Detailed Description
The investigators used Facebook ads to show a 20 second video clip recorded by Massachusetts General Hospital, Harvard and Lynn Community health center doctors (6 people in all) to approximately 20,000,000 Facebook users. The ads will be shown before the Thanksgiving and Christmas holiday and will focus on staying safe - limiting travel and mask-wearing. The investigators will randomize exposure to the ad campaign at the ZIP code or county level to ask:
Do the videos change mobility and Thanksgiving and Christmas holiday travel? Do they reduce the spread of COVID-19?
Are there spillover impacts from the video messages? For example, if individuals decide to stay home, then do the geographical regions that tend to be visited by people from treated areas experience any effects, either through information spillovers or through a reduction in travel?
6. Conditions and Keywords
Primary Disease or Condition Being Studied in the Trial, or the Focus of the Study
Covid19
7. Study Design
Primary Purpose
Prevention
Study Phase
Not Applicable
Interventional Study Model
Sequential Assignment
Model Description
A total of 13 states, comprising 829 counties, centrally report COVID-19 cases at the ZCTA level. Two separate experiments were run within the study, one around Thanksgiving and one around Christmas. For each experiment, approximately half of the counties were randomized to high-intensity treatment with the remaining randomized to low-intensity treatment. In high-intensity counties, 3/4 of ZCTAs were treated and 1/4 were not. In low-intensity counties, 1/4 of ZCTAs were treated and 3/4 were not. In treated ZCTAs, Facebook users received ads with messages from physicians and other health personnel urging them to stay home and wear masks to stop the spread of COVID-19. Users in non-treated ZCTAs received no such ads. After exclusions, 820 counties were randomized to high- or low-intensity treatment during Thanksgiving and 767 counties were during Christmas.
Masking
None (Open Label)
Allocation
Randomized
Enrollment
820 (Actual)
8. Arms, Groups, and Interventions
Arm Title
High-Intensity Treatment Arm
Arm Type
Experimental
Arm Description
Treatment: Individuals received approximately 3 Facebook ads over a 2-week period. Each ad contained a short video recorded by a physician using a script that discusses staying safe during the holidays by considering not traveling and using a mask when appropriate.
Randomization to treatment: The 13 states which centrally report COVID-19 cases at the ZCTA level contained 829 counties. At Thanksgiving, 9 counties were excluded due to data limitations. At Christmas, 62 counties were excluded due to data limitations and potential negative impacts in rural, conservative areas resulting from polarization in the wake of the 2020 presidential election. Approximately half of the counties were randomized to high-intensity treatment with the remaining randomized to low-intensity treatment. In high-intensity counties, 3/4 of ZCTAs were treated (i.e., Facebook users in those ZCTAs received ads) and 1/4 were not. In low-intensity counties, 1/4 of ZCTAs were treated and 3/4 were not.
Arm Title
Low-Intensity Control Arm
Arm Type
No Intervention
Arm Description
Control: Individuals did not receive ads containing short physician-recorded videos.
Randomization to treatment: The 13 states which centrally report COVID-19 cases at the ZCTA level contained 829 counties. At Thanksgiving, 9 counties were excluded due to data limitations. At Christmas, 62 counties were excluded due to data limitations and potential negative impacts in rural, conservative areas resulting from polarization in the wake of the 2020 presidential election. Approximately half of the counties were randomized to high-intensity treatment with the remaining randomized to low-intensity treatment. In high-intensity counties, 3/4 of ZCTAs were treated (i.e., Facebook users in those ZCTAs received ads) and 1/4 were not. In low-intensity counties, 1/4 of ZCTAs were treated and 3/4 were not.
Intervention Type
Behavioral
Intervention Name(s)
Facebook Ads on the importance of staying safe during the Thanksgiving / Christmas holiday
Intervention Description
Facebook ads to show a 15 second video clip recorded by MGH, Harvard and Lynn Community health center and other health care professionals. The ads will be shown before the Thanksgiving/Christmas holiday and will focus on staying safe - limiting travel and mask-wearing. The investigators will randomize exposure to the ad campaign at the ZIP code or county level.
Primary Outcome Measure Information:
Title
Facebook Movement Metric
Description
The change in movement metric is the percent change in distance covered in a day compared to the same day of the week in the benchmark period of February 2-29, 2020, by people who started the day in a particular location. We define holiday travel as travel during the three days preceding each holiday, since the available data does not allow us to compute the impact of the intervention on the return travel (after the holiday). The reason is that the mobility data describes the behavior throughout the day, for people who were in each county that morning. Since the campaign was targeted based on home location, we can only capture its impact on travel away from home, not back home.
Time Frame
November 26, 2020 (Thanksgiving); December 24-25, 2020 (Christmas); February 2-29, 2020 (Baseline benchmark)
Title
Percentage Leaving Home on Day of Holiday
Description
The Facebook stay put metric is the percentage of people who stay within a small geographical area (a "tile" of 600m*600m in which they started the day). We use it to compute the share of people leaving home on the day of the holiday (i.e. this variable = 1 - stay put on the day of the holiday).
Time Frame
November 26, 2020 (Thanksgiving); December 24-25, 2020 (Christmas)
Title
Inverse Hyperbolic Sine of COVID-19 Cases
Description
Inverse hyperbolic sine of COVID-19 cases during a 14 day period starting 5 days after each holiday
Time Frame
December 1-14 (Thanksgiving arms) and January 1-14 (Christmas arms)
Secondary Outcome Measure Information:
Title
Knowledge of COVID-19 Prevention Message (Recall of ad, Intent to Travel, Mask Wearing and Beliefs About Travel)
Description
Facebook will ask a set of users in treated zip codes to answer four survey questions (each user answers only one question each). A small within-zip control group will be held from being treated, enabling comparisons between those who see the videos and those who don't. These questions will be asked a few days after the ad is shown on the User's feed and will include 1) recall of the ad; 2) intention of traveling over holiday; 3) intention of wearing a mask; and 4) beliefs about whether people should travel over holiday.
Time Frame
up to one month
Title
Average Number of Tiles People Occupy (Mobility Measure)
Description
this measures each day "how much people move around by counting the number of level-16 Bing tiles they are seen in within a day. People seen in more tiles are probably moving around more, while people seen in fewer are probably moving around less. Each day take eligible people in a given region and compute the number of distinct tiles they were seen in." This is aggregated to the county level.
Time Frame
up to one month
Title
Percentage of Eligible Participants Only Observed in a Single Level-16 Bing Tile (no Change in Movement)
Description
Facebook "Stay-put" data: this measures each day "the percentage of eligible people who are only observed in a single level-16 Bing tile (600m x 600m) during the course of a day" aggregated to they county level.
Time Frame
up to one month
10. Eligibility
Sex
All
Minimum Age & Unit of Time
18 Years
Accepts Healthy Volunteers
Accepts Healthy Volunteers
Eligibility Criteria
Inclusion Criteria:
- Individuals viewing the ads must be Facebook users, 18 years and older. Facebook will decide specifically which users receive the messages within a target geographical area (zip or county).
Exclusion Criteria:
-
Facility Information:
Facility Name
JPAL North America
City
Cambridge
State/Province
Massachusetts
ZIP/Postal Code
02138
Country
United States
12. IPD Sharing Statement
Plan to Share IPD
Undecided
IPD Sharing Plan Description
The pre-analysis plan will be posted publicly. The ability to share IPD will be based on discussions with our partner organization.
Citations:
PubMed Identifier
34413518
Citation
Breza E, Stanford FC, Alsan M, Alsan B, Banerjee A, Chandrasekhar AG, Eichmeyer S, Glushko T, Goldsmith-Pinkham P, Holland K, Hoppe E, Karnani M, Liegl S, Loisel T, Ogbu-Nwobodo L, Olken BA, Torres C, Vautrey PL, Warner ET, Wootton S, Duflo E. Effects of a large-scale social media advertising campaign on holiday travel and COVID-19 infections: a cluster randomized controlled trial. Nat Med. 2021 Sep;27(9):1622-1628. doi: 10.1038/s41591-021-01487-3. Epub 2021 Aug 19.
Results Reference
result
Learn more about this trial
The Doctors for Coronavirus Prevention Project Thanksgiving / Christmas Messaging Campaign
We'll reach out to this number within 24 hrs