Optimizing BCI-FIT: Brain Computer Interface - Functional Implementation Toolkit (BCI-FIT)

This project adds to non-invasive BCIs for communication for adults with severe speech and physical impairments due to neurodegenerative diseases. Researchers will optimize & adapt BCI signal acquisition, signal processing, natural language processing, & clinical implementation. BCI-FIT relies on active inference and transfer learning to customize a completely adaptive intent estimation classifier to each user's multi-modality signals simultaneously. 3 specific aims are: 1. develop & evaluate methods for on-line & robust adaptation of multi-modal signal models to infer user intent; 2. develop & evaluate methods for efficient user intent inference through active querying, and 3. integrate partner & environment-supported language interaction & letter/word supplementation as input modality. The same 4 dependent variables are measured in each SA: typing speed, typing accuracy, information transfer rate (ITR), & user experience (UX) feedback. Four alternating-treatments single case experimental research designs will test hypotheses about optimizing user performance and technology performance for each aim.Tasks include copy-spelling with BCI-FIT to explore the effects of multi-modal access method configurations (SA1.3a), adaptive signal modeling (SA1.3b), & active querying (SA2.2), and story retell to examine the effects of language model enhancements. Five people with SSPI will be recruited for each study. Control participants will be recruited for experiments in SA2.2 and SA3.4. Study hypotheses are: (SA1.3a) A customized BCI-FIT configuration based on multi-modal input will improve typing accuracy on a copy-spelling task compared to the standard P300 matrix speller. (SA1.3b) Adaptive signal modeling will allow people with SSPI to typing accurately during a copy-spelling task with BCI-FIT without training a new model before each use. (SA2.2) Either of two methods of adaptive querying will improve BCI-FIT typing accuracy for users with mediocre AUC scores. (SA3.4) Language model enhancements, including a combination of partner and environmental input and word completion during typing, will improve typing performance with BCI-FIT, as measured by ITR during a story-retell task. Optimized recommendations for a multi-modal BCI for each end user will be established, based on an innovative combination of clinical expertise, user feedback, customized multi-modal sensor fusion, and reinforcement learning.

For each specific aim, the development of new assistive technology BCI access methods will be evaluated in one or more experiments using alternating-treatments single-case research design (SCRD) with healthy controls and/or participants with SSPI. SCRD is ideal for examining small, heterogenous populations such as individuals with SSPI. It allows for detailed examination of performance trends and changes over time, and for participant-specific modifications to the intervention as part of an iterative design process. Because each participant serves as their own control, a sample size of five is sufficient to demonstrate and replicate an initial effect. Please see the Statistical Design and Power section for additional information about SCRD and data visualization and analysis. A total of 60 participants will evaluate the BCI advancements; 15 individuals with SSPI and 45 controls. Participants with SSPI who currently have a reliable means of communication, either using speech and/or an AAC device, will be enrolled. All participants will be within the ages of 18-89 years (NIH-defined adults), with an equal number of men and women. Healthy controls will be matched for age, gender, and education level. In SCRD studies, each participant serves as their own control, so participants will experience all of the baseline and intervention conditions included in each individual study, as described below. Condition order will be randomized in the alternating-treatments, controlled such that each participant completes an equal number of sessions with each intervention, with no more than two consecutive sessions with the same intervention. Blinding is not possible as each subject must know their condition in an alternating treatment design. All study visits with people with SSPI will be conducted in participants' homes by OHSU staff. Study visits with healthy controls will take place at the OHSU BCI laboratory. For all typing tasks, participants are seated approximately 75cm from an LCD display, set up for the BCI-FIT system. Depending on the user's customized BCI-FIT configuration (procedures described in SA1.1), one or more of the following control signals will be used in each typing session: EEG (ERP, Code or SSVEP), eye movements (gaze position or velocity), or binary switches. The experiments for SA1.3a, SA1.3b, and SA2.2 all involve copy-spelling tasks, in which participants will copy five common 5-letter English words of approximately equal typing difficulty (according to LM input), and correct mistakes by choosing the backspace character when appropriate. Individual signal models will be initialized to population models and will be personalized and refined with each acquired copy-spelling task data set. The experiment for SA3.4 involves a story-retell task, described below in the paragraph about that experiment. Experiment 1.3a will test the hypothesis that a customized BCI-FIT configuration based on multi-modal input will improve typing accuracy on a copy-spelling task compared to a standard P300 matrix speller. We will pilot test new multi-modal input features with control participants before every SCRD with participants who present with SSPI. It will include five participants with SSPI in an alternating-treatments SCRD and will concentrate on typing accuracy as the primary DV. An initial baseline phase will involve weekly copy-spelling sessions with each participant's existing access method. Three or more baseline sessions will be conducted until stable performance is observed, then the alternating-treatments phase will begin. Treatments consist of two different BCI-FIT configurations: 1) a multi-modal configuration chosen by a combination of the approaches described in SA1.1. (clinically-supported and performance data-supported) and 2) a standard P300 matrix speller. In weekly data-collection visits, participants will complete copy-spelling sessions with each BCI-FIT configuration, with counterbalanced session order. Participants complete at least five sessions with each configuration, more if needed to achieve stable performance. In Experiment 1.3b, it is hypothesized that adaptive individualized signal modeling will allow people with SSPI to type accurately during a copy-spelling task with BCI-FIT without training a new model for each use. This experiment will also include five participants with SSPI in an alternating-treatments SCRD with typing accuracy as the primary DV. In this study, no baseline is planned, as the comparison of interest is between versions of BCI-FIT with and without adaptive signal modeling. Initially, each participant will complete system optimization procedures described in SA1.1 and SA1.2 to identify their customized BCI-FIT configuration. During each visit, in the alternative treatments experiment, the participant will attempt three copy-spelling sessions with their customized BCI-FIT configuration, using three different model types: (1) a single calibration completed by the same user immediately before copy spelling; (2) multiple calibrations completed by the same user on previous days; (3) multiple calibrations completed by other users. Data will be graphed and analyzed separately (following procedures in the Statistical Design and Power section) to evaluate effects on performance with both system versions. The experiment in SA2.2 will test the hypothesis that either of two methods of adaptive querying will improve BCI-FIT typing accuracy for users with mediocre AUC scores. It will include five controls and five participants with SSPI, each with an AUC score in the range of 70-80%. (Based on pilot testing, adaptive querying is expected to provide the most benefit to users with this level of baseline performance.) The experiment will follow an alternating-treatments SCRD. In the baseline phase, participants will complete weekly copy-spelling sessions with BCI-FIT without adaptive querying. Each weekly visit will include two copy-spelling sessions with BCI-FIT either with or without adaptive querying techniques. Condition order will be counterbalanced such that conditions occur in random order (with no more than two instances of the same condition in a row) and participants will experience each condition an equal number of times (at least five times each, until stable performance is achieved). The experiment in SA3.4 will use an alternating-treatments SCRD experiment to test the hypothesis that language model enhancements, including a combination of partner and environmental input and word completion during typing, will improve typing performance with BCI-FIT, as measured by ITR during a story-retell task. This experiment will include five controls and five participants with SSPI, each paired with a partner to provide partner input (total enrollment of 10 dyads). In each weekly data-collection visit, participants will engage in two structured story-retell activities, one with and one without the enhanced language model features. Condition order will be counterbalanced such that conditions occur in random order (with no more than two instances of the same condition in a row) and participants will experience each condition an equal number of times (at least five times each, until stable performance is achieved). The story-retell activity will involve the participant watching a short video along with a communication partner, then using BCI-FIT to answer questions posed by a third person. The primary DV in this experiment will be ITR.

Amyotrophic Lateral Sclerosis, Brainstem Stroke, Muscular Dystrophies, Parkinson's Disease and Parkinsonism, Multiple System Atrophy, Brain Tumor Adult, Spinal Cord Injuries, Locked-in Syndrome

Single case research design with: Alternating treatments without baseline for experiments 1.3a, 2.2; Alternating treatments without baseline for experiments 1.3b and 3.4

In single case research design, each participant is their own control. The proposed intervention is behavioral and study personnel are aware of each data collection condition.

For this single case research design with alternating treatments without baseline, 5 participants with severe speech and physical impairment will complete copy spelling tasks with a standard P300 matrix speller layout and with the multi-modal configurations optimized from the BCI-FIT algorithms. Outcome measures are typing accuracy, typing speed and user experience.

For this single case research design with alternating treatments without baseline, 5 participants with severe speech and physical impairment will complete copy spelling tasks with 3 signal adaptive modeling configurations. Outcome measures are typing accuracy, typing speed and user experience.

For this single case research design with alternating treatments without baseline, 5 control volunteers and 5 participants with severe speech and physical impairment who have AUC scores between 70-80% will complete copy spelling tasks with BCI-FIT active querying technique on and with BCI-FIT active querying technique off. Outcome measures are typing accuracy, typing speed and user experience.

For this single case research design with alternating treatments, 5 control volunteers and 5 participants with severe speech and physical impairment, each with a control partner for partner input will complete a story retell task with BCI-FIT language modeling features on and with BCI-FIT language modeling features off. Outcome measures are information transfer rate and user experience.

Adding a personalized multi-modal access protocol to customize a BCI-FIT access method configuration for each individual end user, based on a combination of user characteristics, clinical expertise, user feedback, and system performance data in the software.

Adding a BCI-FIT adaptive signal modeling that employs transfer learning and on-line model adaptation techniques with noisy labels in the software of this brain-computer interface to eliminate the need for data collection exclusively for model calibration, as well as to address model drift issues associated with drowsiness, fatigue, and other human and environmental factors.

Adding BCI-FIT active querying techniques which are software-based optimal action control policies in the brain-computer interface developed with active and reinforcement learning techniques in order to perform efficient user intent inference to improve the entire speed-accuracy trade-off curve for alternative communication.

Adding vocabulary and location information (called partner and environmental input) to the language models in the brain-computer interface from a user's communication partner.

Time-averaged mutual information between intended and typed symbols from the alphabet, computed using probability distributions in accordance with a language model

Responses to 10 items on the NASA TLX questionnaire about comfort, workload and satisfaction using the brain-computer interface system during all typing tasks

Inclusion Criteria: Controls Able to read and communicate in English Capable of participating in study visits lasting 1-3 hours Adequate visuospatial skills to select letters, words, or icons to copy or generate messages Live within a 2-hour drive of OHSU or is willing to travel to OHSU Participants with severe speech and physical impairment: Adults between 18-89 years of age SSPI that may result from a variety of degenerative or neurodevelopmental conditions, including but not limited to: Duchenne muscular dystrophy, Rett Syndrome, ALS, brainstem CVA, SCI, and Parkinson-plus disorders (MSA, PSP) Able to read and communicate in English with speech or AAC device Capable of participating in study visits lasting 1-3 hours Adequate visuospatial skills to select letters, words or icons to copy or generate basic messages Life expectancy greater than 6 months Able to give informed consent or assent according to IRB approved policy Exclusion Criteria: Participants with severe speech and physical impairment: Unstable medical conditions (fluctuating health status resulting in multiple hospitalizations within a 6 week interval) Unable to tolerate weekly data collection visits Photosensitive seizure disorder Presence of implanted hydrocephalus shunt, cochlear implant or deep brain stimulator High risk of skin breakdown from contact with data acquisition hardware.

Three types of information will be available to other researchers. The Python code (called BciPy) that runs the BCI-FIT system is open sourced and available to other laboratories that are building and implementing non-invasive brain-computer interfaces. The datasets of neurophysiological data (EEG, EOG, EMG) collected during use of BciPy in different experimental configurations will be made available. All data are de-identified and maintained in an OHSU-secure BOX folder, an OHSU REDCap database and OHSU approved and compliant human subjects research repository. 3. The typing speed, typing accuracy and user experience data from the four single case research studies will be de-identified and stored in an OHSU REDCap database and OHSU approved and compliant human subjects research repository.

A bcipy.github.io website will be built to share the BCI Python code that is used to collect data and run the brain-computer interface. It is expected that the website will be available in June, 2021 until June, 2025 (during years 2-5 of this award).

Other researchers will have access to neurophysiologic data and outcomes data from the different experimental arms under a data-sharing agreement that provides for: (1) a commitment to using the data only for research purposes and not to identify any individual participant; (2) a commitment to securing the data using appropriate computer technology; and (3) a commitment to destroying or returning the data after analyses are completed.