Examining the Effectiveness of the Early Start Denver Model in Community Programs Serving Young Autism

Examining the Effectiveness of the Early Start Denver Model in Community Programs Serving Young Autism

This study tests the effectiveness of the Early Start Denver Model (ESDM) in community agencies that serve young autistic children. The feasibility, usability and acceptability of the model will be explored. Understanding effectiveness of new evidence-based models will increase the quality of autism care available to more diverse children and families in more geographic locations.

The rising number of children identified as autistic has led to exponential growth in for-profit applied behavior analysis (ABA) agencies and their use of highly structured approaches that may not be developmentally appropriate for young children.1 This has led to growing public health concerns regarding limited effectiveness data combined with high cost of services. Newer research has led to development of evidence-based autism interventions (EBI) called naturalistic developmental behavior interventions (NDBIs),30 supported by multiple clinical trials.31-34 NDBIs integrate theory and strategies from ABA and developmental science,30 are considered best practice for young autistic children,35 and are supported by systematic reviews and meta-analyses.9,36 However, NDBI effectiveness has not been tested in the community and there is also a need to test the variables that moderate outcomes, and the mechanisms of treatment action.2 The lack of effectiveness data regarding NDBI use in community-based agencies (CBAs) contributes to limited funding as payors are more likely to recognize older methods. The Early Start Denver Model (ESDM) is a comprehensive NDBI shown to improve social communication and language for autistic children in multiple controlled efficacy studies.6,31 ESDM engages social motivation and caregiver use of strategies as mechanisms to increase child engagement in social learning opportunities in the environment, resulting in increased rates of learning.65 ESDM is a manualized approach that includes assessment and data collection methods that meet funder requirements and a tested community training model. The transportability of ESDM is evidenced by two recent community pilot studies.63,64 This proposal addresses a critical need to understand ESDM effectiveness and whether the same treatment mechanisms operating in efficacy trials also operate in community implementation with diverse samples. Answering these critical scientific questions will determine the potential of NDBIs to meet public health goals of improving access to quality care for young autistic children. In addition to the challenge of determining effectiveness within communities are challenges of implementation and scale-up. CBAs have grown exponentially in number and size since changes in insurance regulations allow funding for such services. The nine largest CBAs operate over 300 centers and employ thousands of therapists generating $1.07 billion this year, outpacing prescription drugs for autism symptoms. The fast growth in CBA service delivery highlights a large research gap between efficacy and clinical effectiveness findings for current community practices. Given the number of children, families, and the costs involved in this public health challenge, using hybrid effectiveness-implementation designs can accelerate scalability of effective NDBI for community settings by ensuring fit, feasibility and acceptability for CBAs and diverse families. The investigators propose to use the Exploration, Preparation, Implementation and Sustainment (EPIS) framework to identify multi-level factors that affect implementation of ESDM in the community.95 This project will use a hybrid type 1 randomized controlled design to examine ESDM effectiveness and to gather data on implementation determinants. The specific aims and hypotheses are to: Test the effectiveness of ESDM for improving social communication and language outcomes in a diverse community sample of autistic children using a randomized controlled trial of Community Based Agencies. Compared to treatment as usual (TAU): a) Children in the ESDM condition will demonstrate significantly increased growth rates in social communication and language (primary); b) caregivers in the ESDM condition will have greater increases in use of ESDM strategies (secondary). Examine engagement of the treatment mechanisms of social motivation and caregiver fidelity within both treatment groups. The investigators predict that: (a) increased social motivation and better caregiver fidelity will act as mechanisms of change in social communication and language in both ESDM and TAU and (b) children in the ESDM group will demonstrate greater changes in social motivation than children in TAU. Examine moderating variables on ESDM treatment effects. The investigators predict that (a) lower caregiver education and child racial/ethnic diversity will have larger negative effects on child growth rates in TAU than ESDM; and that (b) CBA provider adherence to ESDM fidelity will have positive effects on child rate of growth. Exploratory AIM: Use the EPIS framework to gather data on ESDM Implementation outcomes including acceptability, feasibility, appropriateness and cultural responsivity, CBA provider ESDM fidelity, and caregiver engagement. a) participants will find ESDM to be acceptable, feasible, appropriate and culturally responsive for young autistic children; b) CBA providers will demonstrate ESDM fidelity; c) caregivers receiving ESDM will have higher attendance, parenting competence, and satisfaction than those in TAU. Impact: As indicated by Autism Interagency Coordinating Committee goals, understanding the effectiveness of an intervention like ESDM, the variables that mediate and moderate child outcomes, and engagement of its mechanisms of action in community use, has the potential to increase access to high quality, effective intervention for all young autistic children, especially those from diverse backgrounds who depend on public services. Understanding implementation determinants will support scale-up of effective models throughout a broad range communities and service systems.

The primary aim of the proposed study is to test the effectiveness of ESDM using a hybrid type 1 (effectiveness/implementation) randomized controlled trial in which community based agency (CBA) regions will be randomized to receive training in the Early Start Denver Model (ESDM) or continue treatment as usual (TAU). The investigators decided to randomize by region to address potential contamination across providers and children.

Early Start Denver Model (ESDM) is a comprehensive model that integrates principles of applied behavior analysis (ABA), relationship-based, developmental, and play-based orientations into an individualized and yet manualized treatment. Core features include the use of naturalistic ABA strategies, sensitivity to typical developmental sequence, caregiver involvement, a focus on interpersonal exchange and positive affect within everyday activities. Providers in the ESDM condition will receive training in caregiver coaching strategies and will be asked to conduct caregiver coaching in the strategies at least monthly.

Treatment as usual will vary based on the agency. However, a majority of CBAs use Discrete trial teaching (DTT) based on the Lovaas model.19 DTT involves 10 components described in numerous research publications: capturing child physical and visual attention, adult presentation of the stimuli and instruction (antecedent), child behavior, adult reinforcement, correction procedures, 3-5 second interstimulus interval between trials, behavior-specific praise, and data recording. Most CBAs include caregivers in some way as caregiver involvement is required by most funders. Often, caregivers observe treatment sessions or learn the teaching approach being used with their child. Providers often provide consultation on addressing behavioral concerns.

The Early Start Denver Model focuses on teaching inside children's play and care activities, carried out within a joint activity structure. Adults follow children's leads into activities, embed teaching objectives inside the play, use the play as the reward, and build targeted skills following developmental science and ABA principles.

the APPL operationalizes research-based language development stages. Language phases are derived from spoken language or augmentative communication systems and standardized assessments. Language samples will be obtained from transcriptions of child-caregiver interactions recorded at each timepoint and coded by naive observers

Standardized parent interview measuring the use of adaptive communication. The Vineland Adaptive Behavior Scales-3rd Edition communication domain provides a standardized score with a mean of 100 and a standard deviation of 15. Higher scores mean better outcomes.

The Vineland Adaptive Behavior Scales-3rd Edition consists of four domains of adaptive behavior: communication, daily living skills, socialization, and motor skills. Overall adaptive behavior composite will be used in analyses. The Vineland Adaptive Behavior Scales-3rd Edition adaptive behavior composite provides a standardized score with a mean of 100 and a standard deviation of 15. Higher scores mean better outcomes.

The CC comprehensively assesses the skills of toddlers and preschoolers with autism across developmental domains based on specific skill use. Video recordings of the CC will be coded by trained research staff naïve to treatment timing and condition

The Caregiver Quality of Life Instrument assesses perceived caregiver quality of life across seven dimensions for informal caregivers. Minimum score is 0 and maximum score is 14 where higher scores indicate increased caregiving burden (worse outcomes).

The PedsQL assesses children's quality of life across four domains based on caregiver report and has been validated in an autism population. The PedsQL is scored on a scale of 0 to 100, with higher numbers correlating with better quality of life.

The Parenting Sense of Competence (PSOC) is a 17-item caregiver questionnaire that measures and assesses caregiver self-efficacy in working with their child. Parents will complete this at each time point. Scores range from 17 (min) to 102 (max). A higher score indicates a higher parenting sense of competency.

The Brief Observation of Social Change consists of 15 items that are coded on a 6-point scale and results in Social Communication (SC; i.e., eye contact, facial expressions, gestures, vocalizations, integration of vocal and non-vocal communication, frequency/function of social overtures, frequency/quality of social responses, engagement in activities/interaction, and play with objects) and Restricted and Repetitive Behavior (RRB) domain totals (unusual sensory interests, hand/finger or other complex mannerisms, and unusually repetitive interests/stereotyped behaviors). The Core total combines the SC and RRB scores. I

The ESDM Fidelity Checklist will assess use of ESDM practices. The ESDM Fidelity Checklist consists of 13 items: (a) management of child attention; (b) ABC teaching format; (c) instructional techniques; (d) Modulating child affect/arousal; (e) management of unwanted behavior; (f) use of turn-taking/dyadic engagement; (g) child motivation is optimized; (h) adult use of positive affect; (i) adult sensitivity and responsivity; (j) multiple varied communicative functions; (k) adult language; (l) joint activity and elaboration; and (m) transition between activities.

This measure was developed to capture common elements across NDBI interventions. This measure has adequate reliability, sensitivity to change, and concurrent, convergent, and discriminative validity. The investigators will use the total score as well as examine differences by strategy type, responsive and directive consistent with recent studies

The Implementation Climate Scale measures employees' shared perceptions of the policies, practices, procedures, and behaviors that are expected, rewarded, and supported in order to facilitate effective EBI implementation. The scale includes 18 items capturing six dimensions (three items per dimension). The response scale ranges from 0 ("not at all") to 4 ("to a very great extent"). Scores for each dimension are created by averaging the three subscale items, and the composite score is created by calculating the mean of the subscale scores. Scores range from 0 (minimum) to 4 (maximum). High scores indicate better climate for implementation.

The Autism Self-Efficacy Scale for Teachers is a 30-item scale that measures provider beliefs about their ability to implement appropriate strategies when working with autistic children. Scores range from 0 (minimum) to 100 (maximum) with hither scores reflecting higher self-efficacy.

The Provider Report of Sustainment Scale captures provider report of continued use of an intervention. Each of 3 items is rated on a Likert scale from 0-4. An average score across items is used for analyses, with a minimum score of 0 and a maximum score of 4. Higher scores indicate increased sustainment.

The Acceptability of Intervention Measure determines the extent to which a participant believes an intervention is acceptable. Each of 3 items is rated on a Likert scale from 0-4. An average score across items is used for analyses, with a minimum score of 0 and a maximum score of 4. Higher scores indicate better outcomes.

The Intervention Appropriateness Measure determines the extent to which a participant believes an intervention is appropriate for their population or practice. Each of 3 items is rated on a Likert scale from 0-4. An average score across items is used for analyses, with a minimum score of 0 and a maximum score of 4. Higher scores indicate better outcomes.

The Feasibility of Intervention Measure determines the extent to which a participant believes an intervention is feasible to use in their program. Each of 3 items is rated on a Likert scale from 0-4. An average score across items is used for analyses, with a minimum score of 0 and a maximum score of 4. Higher scores indicate better outcomes.

The investigators will collect data from leaders, providers and clients in participating autism CBAs. The investigators will collect data from 20 Regional Managers, 100 supervisors, and 200 technicians working with autistic children. Participants will include 300 children ages 1-5 years with a diagnosis of autism spectrum disorder living in the US and being served by participating treatment team. Inclusion criteria for Autism CBAs include: Serve at least 10 children with autism under age 5 annually and have at least 2 regions that can be randomized. Accept Medicaid or equivalent payment (e.g., funding for low income families through public service systems). Inclusion criteria for supervisors are as follows: employed as a program supervisor at participating region plans to be employed for at least the next 12 months serves children with autism under age 5 has not has previous training in ESDM supervises at least two technicians Inclusion criteria for technicians is as follows: supervised by a participating supervisor plans to be employed for at least the next 12 months serves children with autism under age 5 has not had previous ESDM training Inclusion criteria for children are as follows: child age 1-5 years has a current autism spectrum disorder (ASD) diagnosis on record or is being served as at-risk for ASD if under age 3 family speaks English or Spanish child expected to be in therapy for at least 7 months

The investigators will allow for data sharing via the National Data Archive. Descriptive/raw data will be submitted semi-annually; submission of all other data will be done at the time of publication and /or prior to the end of the grant. The investigators are open to accepting requests for data. If request are within the scope of the research team to support so that appropriate analyses can be conducted, a data sharing agreement will be established between the institutions (and investigators) requesting and holding the research data. No personally identifying information will be shared or combinations of data that would have the potential for identification of individual research participants.

Therefore, the investigators will make the data and associated documentation available to users only under a data-sharing agreement that provides for: (1) a commitment to using the data only for research purposes and not to identify any individual participant; (2) a commitment to securing the data using appropriate computer technology; and (3) a commitment to destroying or returning the data after analyses are completed.