Development of a Game-supported Intervention

Development of a Game-supported Intervention to Improve Learning and Study Strategies Among At-Risk Students

The investigators will develop and pilot test an innovative computer-assisted behavior intervention (CABI) for middle school students with attention-deficit/hyperactivity disorder (ADHD). The CABI will deliver evidence-based behavior interventions for ADHD in a manner that is acceptable to teachers and motivating for students. The key element of the CABI is a serious game that will teach and encourage rehearsal of ADHD coping skills in an engaging game environment. Transfer of those skills to authentic education settings will be facilitated by teacher interventionists in consultation with school or clinical psychologists. The investigators will test the CABI condition (n = 18) against traditional teacher consultation for ADHD (n = 18) on measures of consultation fidelity and student academic and behavioral outcomes.

An estimated 5% to 9.5% of school-age children in the United States have attention deficit hyperactivity disorder (ADHD), affecting 2.5 to 4.8 million public school students. ADHD is a costly disorder that often goes untreated, especially among adolescents. School-based behavior interventions can be efficacious, but there are persistent implementation challenges in those settings, resulting in a research-practice gap. Computer-based programs have emerged to potentially strengthen school-based intervention, but most target the neuropsychological correlates of ADHD (e.g., working memory deficits) rather than its impairments, with disappointing results. Designers have assumed that gains in neuropsychological functioning can indirectly alleviate ADHD impairments in real-world settings; the research, however, does not support this assumption. As an alternative, serious games might teach coping skills that directly remediate ADHD impairments, and then those new skills might be transferred to a target setting with the active assistance of interventionists. To date, game designers have not explored the latter approach. This study will be the first to test an effort to actively transfer ADHD coping skills learned in a serious game to an authentic education setting. The game, called ATHEMOS, is based on the Challenging Horizons Program, which was inducted into SAMHSA's National Registry of Evidence-based Programs and Practices in 2010. The broad research aims are to (a) develop the serious game (ATHEMOS) that will teach evidence-based ADHD coping skills to adolescents; (b) test and revise initial versions of the game using feedback from focus groups and feasibility case studies; and (c) determine the degree to which these innovations achieve the benefits predicted. The last aim is the focus of this clinical trial registration, and will be achieved in a pilot study whereby middle school students with ADHD will be randomly assigned to receive the computer-assisted behavior intervention (CABI) or traditional school consultation for ADHD. The pilot study will occur at two sites (North Carolina and Ohio), with each site recruiting 18 participants. The investigators will verify ADHD diagnosis in an intake evaluation, randomly assign to conditions, and then begin school consultation with an identified teacher "mentor" for each case in the participating schools. After up to 16-weeks of intervention, the investigators will conduct an end-of-intervention evaluation. The data will be analyzed in a regression framework. Outcome measures at post-test will be regressed on pretest scores and a dummy variable representing condition. The slope of this dummy variable represents how change from pre to post-test differs across conditions. This approach provides a more powerful test of group differences in change than simple difference scores or analysis of variance. If after random assignment, one condition differs significantly from the other on an intake measure, the investigators may use that indicator as a covariate to adjust for this difference in subsequent analyses. The investigators will also assess clinical significance for all symptom and impairment measures to preliminarily compare the results to the relevant treatment literature.

Eligible families will be randomly assigned to either the CABI or traditional consultation condition. The investigators will use a two-wave cohort design, so that one-half of the sample will be randomly assigned to participate in the first semester (September to December) and the second half in the second semester (January to May), assuring that the proportion of students in each condition do not exceed a 5:4 ratio each semester. Participation will end with a post-intervention evaluation in December or May (depending on wave), thereby providing three total measurement occasions.

A serious game paired with teacher consultation to address common needs related to ADHD in the classroom.

The CABI includes a serious game called "ATHEMOS" that is designed to help students rehearse skills that ameliorate impairments associated with ADHD, including organization, scheduling, and note taking strategies. Once students have some experience with the game, a teacher consultee will help students actively transfer skills learned in the game to real-world classroom demands.

The Challenging Horizons Program is an established psychosocial intervention package for young adolescents with ADHD, focusing on academic and behavioral impairments.

The Classroom Performance Survey (CPS) is a 15-item instrument that assesses the unique performance demands of secondary schools using 5-point Likert-type response formats. Research shows that the CPS is comprised of two subscales-referred to as academic competence (10 items; range = 10 - 50; M = 23.7; SD = 10.8) and interpersonal competence (5 items; range = 5 - 25; M = 9.5; SD = 4.8)-with high internal reliability (α = .98 and .91, respectively). Criterion-related validity with the Impairment Rating Scale, an established measure of functional impairment, concluded that the CPS was significantly and meaningfully related to that criterion (rs = .47 to .73) (Brady et al., 2012). Higher scores suggest greater degrees of academic impairment.

The School Functioning Scale (SFS) is a 10-item instrument that assesses school-related impairments in secondary schools using 5- or 6-point Likert type response formats (range = 9 to 47). Psychometric properties suggest that the SFS is a valid and reliable measure of school performance, with a single overall score (general factor; M = 38.2; SD = 7.0) accounting for the largest percent of variance, strongest reliability, and highest factor loading when compared to 2- and 3-factor models. Proposed subscales include a 5-item academic (M = 16.5; SD = 3.2), 3-item behavior (M = 6.2; SD = 3.1), and 2-item social factors (M = 9.9; SD = 2.2) (DuPaul et al., 2019). Higher scores represent more effective school functioning.

The Children's Organizational Skills Scale--Parent Report (COSS-P) is a 66-item instrument that assesses children's organization, time management, and planning using 4-point Likert-type response formats. The COSS-P is comprised of psychometrically validated subscales, including Task Planning (6-items; range = 6 to 24), Organized Actions (10-items; range = 10 to 40), and Memory and Materials Management (10 items; range = 10 to 40). High scores on these subscales suggest greater levels of organization. In addition, an impairment scale is computed from 8 items (range = 8 to 32), where high scores suggest greater levels of impairment. An overall total score is derived from the 56 non-impairment items (range = 56 to 224), where high scores suggest greater levels of organization. The COSS-P was normed with a representative sample of parents, allowing a conversion from raw scores to T-scores (Molitor et al., 2017).

The Working Alliance Inventory - Short Revised (WAI-SR) is a 12-item self-report measure of working alliance between therapists and client that will be modified for the present study to measure the relationship between students and their teacher mentors. The WAI-SR uses a 5-point Likert-type response format, producing a total score ranging from 12 to 60. The instrument has demonstrated adequate psychometric properties (e.g., Munder et al., 2010). The investigators will modify the language slightly for the purposes of the present study (e.g., changing "therapist" to "teacher mentor"). Higher scores represent better working alliance.

Report card data from the participants, as provided by the participating schools. Grades are scaled on a 5-point scale, from A (4.0) to F (0.0). Any variation in that format will be rescaled to make the scores comparable across sites.

An 8-item satisfaction questionnaire used in previous studies of the Challenging Horizons Program administered to teachers, parents, and students. Each item uses a 4-point Likert-type response format, and total scores range from 8 to 32. Higher scores represent higher program satisfaction. Although the instrument has not been examined psychometrically, the feedback will be used to identify elements of the program that participants valued.

A satisfaction questionnaire to assess the acceptability of a new technology, adapted to inquire specifically about the computer-assisted behavior intervention. Will be administered to teachers who participate in the experimental arm of the present study. The Technology Acceptance Model Instrument-Fast Form is comprised of 12-items, and each item uses a 7-point Likert-type response format ranging from -3 to +3. Total scores range from -36 to +36. Higher scores represent higher levels of instrument acceptability.

Inclusion Criteria: All participants must meet DSM-5 criteria for ADHD (any presentation); A history of academic impairment (regardless of medication status) Exclusion Criteria: Estimated full-scale IQ less than 80; Plans to leave the school district during the study timeframe; Psychiatric conditions ill-suited to the proposed intervention (e.g., psychosis, bipolar disorder)

Munder T, Wilmers F, Leonhart R, Linster HW, Barth J. Working Alliance Inventory-Short Revised (WAI-SR): psychometric properties in outpatients and inpatients. Clin Psychol Psychother. 2010 May-Jun;17(3):231-9. doi: 10.1002/cpp.658.

DuPaul GJ, Evans SW, Allan D, Puzino K, Xiang J, Cooper J, Owens JS. High school teacher ratings of academic, social, and behavioral difficulties: Factor structure and normative data for the School Functioning Scale. Sch Psychol. 2019 Sep;34(5):479-491. doi: 10.1037/spq0000323. Epub 2019 May 2.

Molitor SJ, Langberg JM, Evans SW, Dvorsky MR, Bourchtein E, Eddy LD, Smith ZR, Oddo LE. Evaluating the Factor Validity of the Children's Organizational Skills Scale in Youth with ADHD. School Ment Health. 2017 Jun;9(2):143-156. doi: 10.1007/s12310-016-9205-0. Epub 2016 Dec 27.