Neurocognitive Factors in EdTech Intervention Response

Effectiveness and Predictors of Intervention Response for a Technology-based Reading Intervention in the Home

Child Mind Institute, Haskins Laboratories, Georgia State University, Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD)

The current project will carry out a large-scale, randomized controlled trial (RCT) to examine the effectiveness of a home-administered technology-based treatment for reading disability (GraphoLearn) in a diagnostically diverse children with reading disability (ages 6.0-10.00). To accomplish this rapidly and with minimal cost, the experimenters will leverage the Healthy Brain Network [HBN], an ongoing study of mental health and learning disorders in children ages 5.0-21.0 whose family have one or more concerns about behavior and/or learning (target n = 10,000; current enrollment = 3000+). The HBN includes comprehensive psychiatric, cognitive, electroencephalogram [EEG] and multimodal magnetic resonance imaging [MRI] characterizations for all participants, providing the present work rich data to build from. The present work will recruit 450 children (ages 6.0-10.0) with reading difficulty from the HBN. In order to evaluate GraphoLearn effectiveness the experimenters will compare reading (and related language skills) before and after a 12-week GraphoLearn reading intervention relative to an active (math) control. The experimenters also assess the stability of the reading gains by including a 12 week retention period ( with pre and post retention assessment). The experimenters hypothesize that they will observe significant gains in reading (and related language) skills relative to the math control conditions, but that these gains will be variable and predicted by participant and environment level factors (predictive models are explored under Aim 2). This evaluation will involve a 3 to 4 visit between groups longitudinal study with cross over elements to evaluate GraphoLearn in struggling readers ages 6-10 using and pre-post behavioral and EEG assessment.

The overarching goal of this proposal is to address limitations inherent in current remedial approaches utilized in reading disability (RD). Specifically, the experimenters aim to evaluate the effectiveness of a technology-based intervention for children with RD, using a randomized controlled trial. Nature of the Problem. Reading Disability (RD) is the most common learning disability, affecting 10 - 15% of school age children. It incurs major functional impairments at all stages of life. A wealth of data documents lifelong disadvantages in educational and occupational attainment. Problematically, current evidence-based reading interventions largely rely on services by trained specialists, either in well-resourced classrooms or clinical settings. As such, under-resourced schools (or countries) often are unable to provide reading interventions for their students. This potentially tragic situation has fostered the proposed project that aims to evaluate an affordable, promising, technology-based, treatment, which has twofold advantages, that of being home-based and, also, to be implemented without direct expert supervision, thereby minimizing resource demands. Recent studies have supported the efficacy of technology-based reading interventions, particularly GraphoLearn, which focuses on teaching phonological skills and letter sound correspondence, though with notable variation in effect sizes. As highlighted in a recent meta-analysis, sample sizes tend to be small, and there is marked variation in key methodological components across studies, particularly gameplay dosage (which is well below expert recommendations in most studies), control group (e.g., active versus passive), and level of adult involvement in administration. Most relevant, the utility of GraphoLearn has not been adequately examined outside the confines of controlled school-settings, thus failing to inform on the value of home administration - the major positive feature of the GraphoLearn. Similarly, possible predictors of negative outcomes for home administration are unexplored (e.g., compliance with prescribed gameplay dosage, which, as noted in our preliminary data, can vary). Overview for Aim 1. The proposed project addresses limitations inherent in remedial interventions by implementing a large-scale, randomized controlled trial (RCT) of home-administered GraphoLearn in 450 RD children (boys and girls, ages 6.0-10.0). To accomplish this goal rapidly and with minimal cost, the experimenters will recruit participants enrolled in the Healthy Brain Network [HBN], an ongoing study of mental health and learning disorders in children, ages 5.0-21.0, whose family have concerns about behavior and/or learning (target n = 10,000; current n = 3000+). The HBN includes comprehensive psychiatric, cognitive, EEG and multimodal MRI characterizations for all participants, providing the proposed study with rich descriptive data. Reading disabilities will be identified using the TOWRE (standard score < 85). Children will be assigned to a 12-week/20-hour program of GraphoLearn or an "active" control condition, group matched for IQ and reading-level. The control will consist of a math game, similar in style to GraphoLearn, to control for non-specific effects of the intervention, such as time spent on the program, attention, and other factors. Children randomized to the control group will be offered GraphoLearn following the study due to ethical considerations. Consistent with real-world conditions, non-study related interventions during the course of the study period will be permitted. They will be documented for comprehensive sample description, and for analyzing predictors of response (Aim 2). Aim 1: Evaluate the effectiveness of GraphoLearn in a large sample of children with RD. Children's reading level will be assessed before, during (week 6), and at the end of, the active study (week 12). Extended effects will be evaluated 12 weeks post treatment (week 24). The experimenters hypothesize statistically significant differences in reading level in favor of the GraphoLearn group over the control group at weeks 12 and 24. Should the hypothesis be supported, it will have important public health implications for children with RD (there is no claim that the program will be substitute for individualized, and expert-supervised, reading remediation). Overview for Aim 2. The identification of participant-level and environment-level factors that predict treatment response is an important topical goal but has not been considered for GraphoLearn. For specialist-based interventions, a consistent finding is that older children, and children with language or cognitive impairments, are less likely to benefit. Here, the experimenters leverage the diversity of co-morbid clinical presentations among poor readers in HBN to identify putative predictors of treatment response. In line with recent studies suggesting the added value of brain-based features for prediction of intervention outcomes, the experimenters also leverage the availability of anatomical MRI data in the development of predictive models. Aim 2. Identify participant-related and environment-level factors that are significantly associated with GraphoLearn outcomes. The extensive phenotypic and imaging characterizations included in the HBN sample are uniquely suited for exploring an extensive set of participant and environmental factors that may predict treatment outcomes. Random Forest Regression models will be used to predict differences in intervention response to GraphoLearn, based on an a priori set of baseline demographic, cognitive, emotional, brain and home environment variables. Assuming that compliance (i.e., actual time spent playing GraphoLearn relative to prescribed) is found to be a strong predictor of response, exploratory analyses will also look at predictors of compliance to further increase real world value of the proposed work.

This is a parallel intervention design with crossover elements (control group is offered the intervention after completion of the control intervention). We will evaluate a reading intervention of interest in comparison to a math control.

The only masking is done at the level of data analysis, as it is readily evident to participants and those monitoring which group the participants are in.

GraphoLearn is a research-based treatment, delivered as an engaging computer game. Players match auditory targets (e.g., phonemes, rimes) to visual targets (single letters, letter sequences, words). The complexity of the items within each level is ordered such that at each level, the most frequent and regular mappings are introduced first based on measures such as orthographic/phonological neighborhood size and morphological family size. GraphoLearn allows the children to practice and reinforce lessons at their own individual trial pace and provides a record of performance progress that can be used to guide analyses.

We selected an active control to maximize the specificity of the treatment outcomes; to this end, math games are among the most commonly used. Game sessions support learning numerical mathematical skills and cognition related to mathematical skills. In addition to math, this game contains training tasks for visuospatial working memory, spatial visualization and visuospatial reasoning. The overall theme of the game and feedback style are similar to those in GraphoLearn.

The TWRE is a composite of phonemic decoding efficiency (PDE) and sight word efficiency (SWE) -- subtests from the TOWRE II -- these are timed measures of word and nonword reading. The range of scores for the TWRE is 53- 147. Higher scores indicate better performance. Greater change equals better outcome/response.

Assessment is given before and after the intervention, ~12 weeks apart for the primary outcome; the assessment will also be re-administered after a 12 week retention period

Woodcock Johnson Tests of Achievement (WJ-IV), Letter Word identification (ID) and Word Attack subtests.

untimed tests of word and nonword reading. The range of raw scores for the WA = 0-32 and the range of raw scores for the ID is 0-78; higher scores indicate better performance. Greater change equals better outcome/response.

Assessment is given before and after the intervention, ~12 weeks apart; the assessment will also be re-administered after a 12 week retention period.

Phonological Awareness (PA), Phonological Memory, (PM) and Rapid Naming (RN), which are composite scores and will be assessed with the Comprehensive Test of Phonological Processing (CTOPP-II)

These are measures of phonological awareness and processing. For PA, the range of scores is 43 - 165; for the PM, the range of scores is 46-162; for the RN, the range of scores is 46-162. Higher scores indicate better performance. Greater change equals better outcome/response.

Assessment is given before and after the intervention, ~12 weeks apart; the assessment will also be re-administered after a 12 week retention period.

Neurobiological measures of sensitivity to phonological information and degree of letter sound integration. Score range and better/worse designation are not possible for this outcome measure.

Inclusion Criteria: Must be 6.0 to 10.0 years old Must be currently enrolled in Healthy Brain Network (HBN) Must have a standard score <85, on the Test of Word Reading Efficiency (TOWRE), Total Word Reading Efficiency (TWRE) composite Must have full Scale IQ (FSIQ) on the Wechsler Intelligence Scale for Children (WISC V) > 75 Must be a native speaker of English Exclusion Criteria: Vision or hearing impairment, per the Snellen Test and audiometer measurement Moderate to severe autism (levels 2-3) Major depressive disorder Suicidal or homicidal ideation Psychotic disorder Manic episode Any neurologic impairment that limits the ability to use a keyboard or mouse, or to touch a screen

Consistent with the policies established by one of the principal investigators (PIs), PI Milham and colleagues on the International Neuroimaging Data-sharing Initiative (INDI), all data will be anonymized prior to being placed in the data repository. All phenotypic data collected during the proposed work, including daily GraphoLearn longitudinal training data, will be shared following completion of data collection, which is anticipated to occur prior to the end of year 4. EEG data obtained through the proposed work will be shared through INDI as well.