search
Back to results

The Impact Integrated Data System for Quality and Outcomes Tracking of Prevention Programs (Impact)

Primary Purpose

Trauma, Psychological, Social Skills, Family Dysfunctional

Status
Completed
Phase
Not Applicable
Locations
United States
Study Type
Interventional
Intervention
Impact
Sponsored by
3-C Institute for Social Development
About
Eligibility
Locations
Arms
Outcomes
Full info

About this trial

This is an interventional health services research trial for Trauma, Psychological focused on measuring Evidence-based Prevention Programs, Youth Prevention Programs, Outcomes Data, Continuous Quality Improvement

Eligibility Criteria

21 Years - undefined (Adult, Older Adult)All SexesAccepts Healthy Volunteers

Inclusion Criteria:

  • Must be an Evidence-based Prevention Program Provider
  • Regularly implement one of the three supported EBPPs to youth in a school or community setting
  • Commit to implementing that EBPP over the study period (approximately one school year)
  • Serve a minimum of 10 youth over the study period
  • Be willing to allow 3C researchers to collect data from their setting

Sites / Locations

  • 3C Institute

Arms of the Study

Arm 1

Arm 2

Arm Type

Experimental

Experimental

Arm Label

Naive EBP Providers

Experienced EBP Providers

Arm Description

Naive Providers (providers with no prior experience implementing the EBP) will attend a one-hour orientation meeting via web-based meeting software (e.g., WebEx) or in person to receive training on research procedures and implementation methods for the study. Participants will complete online surveys via 3C's secure system for any measures not collected directly via Impact. Providers will receive secure login information to the survey system and automatic email alerts with links to surveys to prompt completion. Surveys will be completed at PRE and POST time points to assess feasibility, usability, and value. 3C research staff will be available to assist providers (via email and phone) throughout the pilot field study.

Experienced EBP Providers (providers with prior experience implementing the EBP) will attend a one-hour orientation meeting via web-based meeting software (e.g., WebEx) or in person to receive training on research procedures and implementation methods for the study. Participants will complete online surveys via 3C's secure system for any measures not collected directly via Impact. Providers will receive secure login information to the survey system and automatic email alerts with links to surveys to prompt completion. Surveys will be completed at PRE and POST time points to assess feasibility, usability, and value. 3C research staff will be available to assist providers (via email and phone) throughout the pilot field study.

Outcomes

Primary Outcome Measures

Fidelity to the Program Model
Providers will complete a Fidelity form, and will receive a total score ranging from 0 to 100, with 80 or higher representing better outcomes. Scores are calculated by converting responses (1 to 5) to percentage (0 to 100) and averaging item percentages.

Secondary Outcome Measures

Attitudes toward EBP Tracking
Providers will complete the 22-item Attitudes Towards Standardized Assessment scale which measures provider perceptions of using data in practice. Three subscale scores will be calculated by averaging responses of each item within that subscale. Scores range from 1=Strongly Agree to 5=Strongly Disagree with 5 being the better outcome.
Skilled Use of Data
Providers will rate their understanding of and skill using data for an EBP using an 8-item Skilled Use of Data scale, measuring self-reported skill level for using data related to an EBP. Scores range from 1=Minimal to 5=Advanced, with 5 being the better outcome. A total score is calculated by averaging responses for each item.
Satisfaction with Impact assessed by Acceptability Scale
Providers will rate their satisfaction with using Impact as a method to enter and track data using an 80-item Acceptability scale across 10 domains. A subscale score will be calculated for each domain by averaging scores for each item within that domain. Scores range from 1=Definitely No to 5=Definitely Yes, with 5 being the better outcome.
Practicality of Impact
Providers will complete a Practicality evaluation scale (80 items across 10 domains). A score will be calculated for each domain, for a total of 10 subscale scores. Each subscale score ranges from 1=Very Low to 7=Very High, with 7 being the better outcome. Subscale scores are calculated by averaging the item responses within each subscale.
Integration of Impact
Providers will complete an 8-item evaluation scale (Integration of Impact) measuring how well Impact works with other systems. A total score for this scale will be calculated by averaging item responses, and scores range from 1=Strongly Disagree to 7=Strongly Agree, where 7 is the better outcome.
Usability of Impact
Providers will complete a 20-item Usability scale adapted from the Post-Study System Usability Questionnaire (PSSUQ). Three subscale scores will be calculated, and each ranges from 7=Strongly Disagree to 1=Strongly Agree, with 1 being the better outcome. Scores are calculated by averaging item responses within each subscale.
Value of Impact
Providers will complete the 8-item Value of Impact scale, and a total score will be calculated by averaging item responses. Scores range from 1=Poor to 5=Excellent, with 5 being the better outcome.
Commitment to Change
Providers will complete the 18-item Commitment to Change scale, which measures how committed providers are to use Impact in their organization. Three subscale scores are calculated, and each score ranges from 1=Strongly Disagree to 7=Strongly Agree, with 7 being the better outcome. Scores are calculated by averaging item responses within each subscale.
Time Spent Using Impact
Providers will report the amount of time it takes them to complete all activities (e.g., data entry, generating reports) using Impact. Time will be reported in minutes.

Full Information

First Posted
July 10, 2018
Last Updated
July 11, 2022
Sponsor
3-C Institute for Social Development
search

1. Study Identification

Unique Protocol Identification Number
NCT03622138
Brief Title
The Impact Integrated Data System for Quality and Outcomes Tracking of Prevention Programs
Acronym
Impact
Official Title
The Impact Integrated Data System for Quality and Outcomes Tracking of Prevention Programs
Study Type
Interventional

2. Study Status

Record Verification Date
July 2022
Overall Recruitment Status
Completed
Study Start Date
June 8, 2018 (Actual)
Primary Completion Date
July 31, 2021 (Actual)
Study Completion Date
July 31, 2021 (Actual)

3. Sponsor/Collaborators

Responsible Party, by Official Title
Sponsor
Name of the Sponsor
3-C Institute for Social Development

4. Oversight

Studies a U.S. FDA-regulated Drug Product
No
Studies a U.S. FDA-regulated Device Product
No
Data Monitoring Committee
No

5. Study Description

Brief Summary
There has been substantial progress in developing evidence-based prevention programs (EBPPs) for preventing risky behaviors and promoting positive outcomes for youth. Unfortunately, their adoption, quality implementation, long-term sustainability, and scalability in real world service settings, continues to lag far behind resulting in an inability to achieve the intended broad scale public impact. This SBIR Phase II project will build on the findings and feedback gathered from stakeholders in the Phase I project to fully develop and test the Impact implementation support system. Impact will provide an easy to use, cost- and time-efficient technology platform that is capable of (a) gathering relevant process and outcomes data in a feasible way within real-world service settings; (b) producing real-time analytics (e.g., graphical displays) and meaningful reports at the provider, service setting, and broader levels (e.g., state-wide); and (c) linking data analytics to continuous quality improvement (CQI) feedback to help providers achieve best practices and targeted youth outcomes.
Detailed Description
Describe the objective(s) of the proposed research including purpose, research question and design, anticipated length of the entire study, hypothesis and relevant background information. There has been substantial progress in developing evidence-based prevention programs (EBPPs) for preventing risky behaviors and promoting positive outcomes for youth. Unfortunately, their adoption, quality implementation, long-term sustainability, and scalability in real world service settings, continues to lag far behind resulting in an inability to achieve the intended broad scale public impact. This SBIR Phase II project will build on the findings and feedback gathered from stakeholders in the Phase I project to fully develop and test the Impact implementation support system. Impact will provide an easy to use, cost- and time-efficient technology platform that is capable of (a) gathering relevant process and outcomes data in a feasible way within real-world service settings; (b) producing real-time analytics (e.g., graphical displays) and meaningful reports at the provider, service setting, and broader levels (e.g., state-wide); and (c) linking data analytics to continuous quality improvement (CQI) feedback to help providers achieve best practices and targeted youth outcomes. For this Phase II project, 3C will create three instantiations (or 'instances') of Impact, i.e., one instance for each of three EBPPs. These EBPPs explicitly target one or more mental health outcomes for youth, and provide a test of the application of Impact to a variety of service delivery models, including small group-based, universal classroom, and parent training: Cognitive Behavioral Intervention for Trauma in Schools (CBITS) is an evidence-based program delivered by mental health providers in the school setting. CBITS has been shown to reduce posttraumatic stress disorder (PTSD) symptoms, depression, and anxiety for youth in grades 5-12 as well as improve psychosocial functioning. The 10-session program teaches cognitive behavioral skills using a mixture of didactic presentations, relaxation training, and strategies to combat negative thoughts, reduce avoidance, develop a trauma narrative, and build social problem solving skills. CBITS also includes 1-3 individual child sessions and 2 optional parent sessions. Social Skills Group Intervention K-2 (S.S. GRIN K-2) is an evidence-based program delivered by mental health providers in the school setting to kindergarten through 2nd grade children. S.S. GRIN has been shown to improve self-esteem and self-efficacy, as well as reduce social anxiety and, in aggressive children, reduce aggression and bullying behavior. The intervention includes 10 small group sessions, delivered in an in-person format. The third EBP that will be developed and tested for this Phase II research is Strengthening Families, which has been shown to improve youth outcomes related to substance use, aggressive and hostile behaviors, and risky sexual behaviors, as well as parent protective factors such as affective quality and general child management. The program includes 10-14 sessions, each of which consists of a separate parent and child meeting followed by a family meeting, designed to help families improve communication, conflict resolution, appropriate disciplinary factors, and child involvement in the family. Strengthening Families is implemented in schools, community centers, and mental health facilities by two to four mental health providers. The proposed three-year project will accomplish three specific aims: (1) fully develop the Impact software platform for the three Blueprints-supported EBPPs listed above, (2) conduct a pilot field study to test whether and in what ways Impact is superior to currently used process and outcomes tracking methods for each EBPP, and (3) prepare Impact for commercialization. Impact's technology infrastructure will be able to accommodate a wide array of EBPPs, providing a much needed cost-effective mechanism to support broad-scale dissemination and use in service settings. In addition, administrators will be able to track implementation, thus enhancing accountability and providing on-going quality assurance, while also providing data analytics to quickly and effectively monitor impact and cost-effectiveness, thereby increasing the likelihood a program's core features are adhered to over time and maximizing treatment benefits for youth.

6. Conditions and Keywords

Primary Disease or Condition Being Studied in the Trial, or the Focus of the Study
Trauma, Psychological, Social Skills, Family Dysfunctional
Keywords
Evidence-based Prevention Programs, Youth Prevention Programs, Outcomes Data, Continuous Quality Improvement

7. Study Design

Primary Purpose
Health Services Research
Study Phase
Not Applicable
Interventional Study Model
Parallel Assignment
Masking
None (Open Label)
Allocation
Randomized
Enrollment
203 (Actual)

8. Arms, Groups, and Interventions

Arm Title
Naive EBP Providers
Arm Type
Experimental
Arm Description
Naive Providers (providers with no prior experience implementing the EBP) will attend a one-hour orientation meeting via web-based meeting software (e.g., WebEx) or in person to receive training on research procedures and implementation methods for the study. Participants will complete online surveys via 3C's secure system for any measures not collected directly via Impact. Providers will receive secure login information to the survey system and automatic email alerts with links to surveys to prompt completion. Surveys will be completed at PRE and POST time points to assess feasibility, usability, and value. 3C research staff will be available to assist providers (via email and phone) throughout the pilot field study.
Arm Title
Experienced EBP Providers
Arm Type
Experimental
Arm Description
Experienced EBP Providers (providers with prior experience implementing the EBP) will attend a one-hour orientation meeting via web-based meeting software (e.g., WebEx) or in person to receive training on research procedures and implementation methods for the study. Participants will complete online surveys via 3C's secure system for any measures not collected directly via Impact. Providers will receive secure login information to the survey system and automatic email alerts with links to surveys to prompt completion. Surveys will be completed at PRE and POST time points to assess feasibility, usability, and value. 3C research staff will be available to assist providers (via email and phone) throughout the pilot field study.
Intervention Type
Other
Intervention Name(s)
Impact
Intervention Description
Impact is an easy to use, cost- and time-efficient technology platform that is capable of (a) gathering relevant process and outcomes data in a feasible way within real-world service settings; (b) producing real-time analytics (e.g., graphical displays) and meaningful reports at the provider, service setting, and broader levels (e.g., state-wide); and (c) linking data analytics to continuous quality improvement (CQI) feedback to help providers achieve best practices and targeted youth outcomes.
Primary Outcome Measure Information:
Title
Fidelity to the Program Model
Description
Providers will complete a Fidelity form, and will receive a total score ranging from 0 to 100, with 80 or higher representing better outcomes. Scores are calculated by converting responses (1 to 5) to percentage (0 to 100) and averaging item percentages.
Time Frame
To assess change, this measure will be taken each time providers enter data into the Impact system after each session (once per week for up to 12 weeks).
Secondary Outcome Measure Information:
Title
Attitudes toward EBP Tracking
Description
Providers will complete the 22-item Attitudes Towards Standardized Assessment scale which measures provider perceptions of using data in practice. Three subscale scores will be calculated by averaging responses of each item within that subscale. Scores range from 1=Strongly Agree to 5=Strongly Disagree with 5 being the better outcome.
Time Frame
To assess change, this measure will be taken (1) at baseline (PRE) and (2) following completion of providers' use of Impact, up to 9 months (POST).
Title
Skilled Use of Data
Description
Providers will rate their understanding of and skill using data for an EBP using an 8-item Skilled Use of Data scale, measuring self-reported skill level for using data related to an EBP. Scores range from 1=Minimal to 5=Advanced, with 5 being the better outcome. A total score is calculated by averaging responses for each item.
Time Frame
To assess change, this measure will be taken (1) at baseline (PRE) and (2) following completion of providers' use of Impact, up to 9 months (POST).
Title
Satisfaction with Impact assessed by Acceptability Scale
Description
Providers will rate their satisfaction with using Impact as a method to enter and track data using an 80-item Acceptability scale across 10 domains. A subscale score will be calculated for each domain by averaging scores for each item within that domain. Scores range from 1=Definitely No to 5=Definitely Yes, with 5 being the better outcome.
Time Frame
This evaluation measure will be assessed following completion of providers' use of Impact, up to 9 months (POST).
Title
Practicality of Impact
Description
Providers will complete a Practicality evaluation scale (80 items across 10 domains). A score will be calculated for each domain, for a total of 10 subscale scores. Each subscale score ranges from 1=Very Low to 7=Very High, with 7 being the better outcome. Subscale scores are calculated by averaging the item responses within each subscale.
Time Frame
This evaluation measure will be assessed following completion of providers' use of Impact, up to 9 months (POST).
Title
Integration of Impact
Description
Providers will complete an 8-item evaluation scale (Integration of Impact) measuring how well Impact works with other systems. A total score for this scale will be calculated by averaging item responses, and scores range from 1=Strongly Disagree to 7=Strongly Agree, where 7 is the better outcome.
Time Frame
This evaluation measure will be assessed following completion of providers' use of Impact, up to 9 months (POST).
Title
Usability of Impact
Description
Providers will complete a 20-item Usability scale adapted from the Post-Study System Usability Questionnaire (PSSUQ). Three subscale scores will be calculated, and each ranges from 7=Strongly Disagree to 1=Strongly Agree, with 1 being the better outcome. Scores are calculated by averaging item responses within each subscale.
Time Frame
This evaluation measure will be assessed following completion of providers' use of Impact, up to 9 months (POST).
Title
Value of Impact
Description
Providers will complete the 8-item Value of Impact scale, and a total score will be calculated by averaging item responses. Scores range from 1=Poor to 5=Excellent, with 5 being the better outcome.
Time Frame
This evaluation measure will be assessed following completion of providers' use of Impact, up to 9 months (POST).
Title
Commitment to Change
Description
Providers will complete the 18-item Commitment to Change scale, which measures how committed providers are to use Impact in their organization. Three subscale scores are calculated, and each score ranges from 1=Strongly Disagree to 7=Strongly Agree, with 7 being the better outcome. Scores are calculated by averaging item responses within each subscale.
Time Frame
This evaluation measure will be assessed following completion of providers' use of Impact, up to 9 months (POST).
Title
Time Spent Using Impact
Description
Providers will report the amount of time it takes them to complete all activities (e.g., data entry, generating reports) using Impact. Time will be reported in minutes.
Time Frame
This measure will be assessed each week while providers are implementing the EBP and using the Impact system (once per week, up to 12 weeks).

10. Eligibility

Sex
All
Minimum Age & Unit of Time
21 Years
Accepts Healthy Volunteers
Accepts Healthy Volunteers
Eligibility Criteria
Inclusion Criteria: Must be an Evidence-based Prevention Program Provider Regularly implement one of the three supported EBPPs to youth in a school or community setting Commit to implementing that EBPP over the study period (approximately one school year) Serve a minimum of 10 youth over the study period Be willing to allow 3C researchers to collect data from their setting
Overall Study Officials:
First Name & Middle Initial & Last Name & Degree
Melissa E DeRosier, PhD
Organizational Affiliation
3C Institute
Official's Role
Principal Investigator
Facility Information:
Facility Name
3C Institute
City
Durham
State/Province
North Carolina
ZIP/Postal Code
27713
Country
United States

12. IPD Sharing Statement

Plan to Share IPD
Undecided

Learn more about this trial

The Impact Integrated Data System for Quality and Outcomes Tracking of Prevention Programs

We'll reach out to this number within 24 hrs