Effects of Clear Speech on Listening Effort and Memory in Sentence Processing

Understanding the Effects of Listening Effort on Sentence Processing and Memory in Sensorineural Hearing Loss: Evidence From Simultaneous Electrophysiology and Pupillometry (Study 2)

Sensorineural hearing loss (SNHL) is among the most prevalent chronic conditions in aging and has a profoundly negative effect on speech comprehension, leading to increased social isolation, reduced quality of life, and increased risk for the development of dementia in older adulthood. Typical audiological tests and interventions, which focus on measuring and restoring audibility, do not explain the full range of cognitive difficulties that adults with hearing loss experience in speech comprehension. For example, adults with SNHL have to work disproportionally harder to decode acoustically degraded speech. That additional effort is thought to diminish shared executive and attentional resources for higher-level language processes, impacting subsequent comprehension and memory, even when speech is completely intelligible. This phenomenon has been referred to as listening effort (LE). There is a growing understanding that these cognitive factors are a critical and often "hidden effect" of hearing loss. At the same time, the effects of LE on the neural mechanisms of language processing and memory in SNHL are currently not well understood. In order to develop evidence-based assessments and interventions to improve comprehension and memory in SNHL, it is critical that we elucidate the cognitive and neural mechanisms of LE and its consequences for speech comprehension. In this project, we adopt a multi-method approach, combining methods from clinical audiology, psycholinguistics, and cognitive neuroscience to address this gap of knowledge. Specifically, we adopt a novel and innovative method of co-registering pupillometry (a reliable physiological measure of LE) and language-related event-related brain potential (ERP) measures during real-time speech processing to characterize the effects of clear speech (i.e., a listener-oriented speaking style that is spontaneously adopted to improve intelligibility when speakers are aware of a perception difficulty on behalf of the listener) on high-level language processes (e.g., semantic retrieval, syntactic integration) and subsequent speech memory in older adults with SNHL. This innovative work addresses a time-sensitive gap in the literature regarding the identification of objective and reliable markers of specific neurocognitive processes impacted by speech clarity and LE in age-related SNHL.

This experiment is a Basic Experimental Study with Humans (BESH) Trial. All participants are exposed to all experimental conditions (i.e., "interventions") in a complete factorial 2 x 3 within-subjects experimental design. Participants will consist of 80 older adults between the ages of 60 - 90 who will be recruited from the Salt Lake metro community. Participants will be recruited through the Utah Senior Ears database, the Utah Center on Aging, through flyers placed throughout the community (e.g., bulletin boards, waiting area in the Ear Nose and Throat (ENT) clinic at the University of Utah), through on-line advertisements (e.g., Facebook) and via word-of-mouth. We will recruit for an approximately equal number of participants into two hearing groups based on their pure-tone average (PTA) thresholds: normal hearing (PTA of < 25 decibels hearing level (dB HL), 1-4kHz) and clinically-relevant hearing loss (PTA of > 25 dBHL 1-4kHz). While we adopt this dichotomous grouping for recruitment, we will treat hearing level as continuous to increase statistical power. We will follow all American Psychological Association (APA) guidelines with respect to the treatment of human subjects. All participants will provide informed consent after study procedures are explained to them and the voluntary nature of participation will be emphasized. No identifying information (e.g., names) will be obtained from participants and the only information connected to their data files will be a unique, arbitrary code. Study procedures will be conducted within a single session, lasting between 3-4 hours. Following informed consent, participants will complete a standardized hearing assessment, neuropsychological assessment, audibility control assessment, and then participant in the EEG/pupillometry experiment, each of which is described below. All data will be stored on password-protected computers in the PI's laboratory. All material from participants will be collected specifically for research purposes. The materials presented in these studies have no known potential to stress, embarrass, stigmatize, or incriminate experimental participants. Prior to the beginning of the experiment, we will conduct two audiometric tests via a Maico MA-41 audiometer via RadioEar IP-30 insert air-conduction earphones. First, pure-tone thresholds will be measured using the modified Hughson- Westlake at octave frequencies from 250 to 8000 Hz for each ear. Second, we will test speech recognition thresholds (SRTs) using a recorded Central Institute for the Deaf (CID) W-1 spondee word list Near visual acuity was also tested for both the right and left eye using the Rosenbaum visual acuity test. Participants will then complete a brief battery of cognitive assessments. (a) the Montreal Cognitive Assessment (MoCA). Although the appropriate cut-off score for cognitive impairment in the MoCA is variable across samples, individuals scoring below 20 are generally considered to be at increased risk for mild cognitive impairment (MCI). Therefore, we use this as our conservative cut-off. Participants will also complete the F-A-S phonemic fluency task as a measure of verbal fluency, via the short- form computerized version of the reading span task as a measure of verbal working memory, and the extended range vocabulary tests from the Educational Testing Services Kit of Factor Referenced Cognitive Tests, as a measure of verbal ability. Participants will then complete an audibility control task. For this task, the same native speaker of American English will be used to record the stimuli at the same +3dB. Participants will hear nine different test sentences (e.g., "Don't touch the wet paint) and will be tasked with "shadowing" each sentence by repeating out loud each word as it was heard. The immediate repetition is done to reduce the contribution of any memory components to task performance. Following the audiological and neuropsychological testing, we will begin the primary experiment. Electrophysiological data (EEG), pupil dilation measurements (pupillometry), and behavioral recordings will be made from participants. EEG will be recorded from 64 silver-silver chloride electrodes embedded in an EasyCap (Electro-Cap, Inc), following a 10-20 montage. In addition, an electrode will placed on the left infraorbital ridge to monitor for vertical eye movements and blinks, and a virtual bipolar horizontal electrooculogram channel will be created offline for monitoring horizontal eye movements by calculating a difference between two fronto-temporal electrodes FT10-FT9 that sit posterior to the outer canthus of each eye. The continuous EEG will be amplified with a BrainAmp DC amplifier (Brain-Vision, LLC, Morrisville, NC) (bandwidth filtered: 0.02-250 Hz) and recorded to hard disk at a sampling rate of 1000 Hz. Electrode impedances will be kept below 5 kOhms. During the listening task, pupil size measurements will be continuously recorded from the right eye using an Eyelink 1000 Plus desktop mounted infrared eye tracker camera distributed by SR Research (SR Research Ltd., Ottawa, ON, Canada). Continuous pupil size measurements will be recorded at a rate of 1000 Hz using Eyelink software and will be downsampled offline to 50 Hz. See Statistical Analysis and Power for information on pupil data processing and cleaning. Participants will be tested in a quiet sound-attenuated testing room. Participants will be seated 85 cm from a 24-in high-performance LCD monitor that will be used to present instructions and cues to the participant (e.g., when to take a break). Speech stimuli will be presented through the sound card of the stimulus presentation computer and routed to a Maico MA-41 audiometer via the auxiliary channel, allowing for direct control of stimulus intensity. The audiometer will route the speech to the participant via RadioEar IP-30 insert air-conduction earphones. Stimuli will be presented presented at 65 dB HL for all participants to their better hearing ear (based on PTA thresholds between 1-4 kHz). All hardware is calibrated to standard in our testing room by a certified technician. The experiment will be programmed in Python via the PsychoPy open source platform. There will be 360 trials. On each trial, participants will listen to a sentence that contains a single target word that will either be a normal plausible continuation, a semantic violation, or a syntactic violation (see Table 1 in Research Plan). In addition, speech will be produced either using a conversational speaking style or a hyperarticulated clear speech speaking style (see Research Plan for more information), in a 2 x 3 within-subjects factorial design. Clear and conversational speech samples will be elicited from a single male talker. In the conversational speech condition, which will be recorded first, the talker will be instructed to read the sentences as they would in everyday conversation. In the clear speech condition, the talker will be instructed to produce the sentences as if they would if they were talking to a person with hearing loss . Stimuli will be counterbalanced across experimental conditions following a Latin square design, with each participant randomized to one of six experimental presentation lists. This ensures that each critical word appears equally often across experimental conditions and that no participants will see the same critical word or sentence more than once. All sentences will be presented in +3 decibel (dB) signal-to-noise ratio (SNR) stationary speech-shaped background noise in induce acoustic challenge. EEG will be measured continuously during speech processing. After the end of the 360 sentences, a delayed recognition memory task will be administered. Participants will be visually presented with 360 test sentence frames on a tablet computer, each with the target word missing. They will be instructed to mark whether or not they recognized each sentence as one that they had heard during the experimental task. For the sentences they reported as having heard previously, they will be asked to recall the target word to the best of their ability by typing their response. There will be no time limit on the memory test. 180 of the sentences will be old items that they had heard during the task and the other 180 will be semantic foils. The 180 sentences that were ones heard previously will be taken evenly from each of the six experimental conditions, such that there will be 30 sentences from each condition. Each semantic foil will be created by taking 2 to 4 of the meaning-bearing words from a sentence that the participant had actually heard and these will be used to create a new semantically similar sentence. For example, if the participant had heard the sentence "Dan recognized John even though he had grown a beard since the last time they saw each other.", a semantic foil may be: "No one at the reunion recognized Dan because he had grown a _____ since the last time everyone met." The inclusion of foils are used to make the recognition task more challenging in order to reduce the likelihood of ceiling performance. This approach has been validated in past studies on listening effort and recognition memory. Although the primary memory analyses concern sentence recognition memory, we will also report performance on the cued word recall task as well, as we have done in prior work. Upon completion of the memory assessment, the study procedures will be complete. Participants will be debriefed and will have the opportunity to ask any questions of the study team.

This basic science experiment meets the NIH criteria of a BESH (Basic Experimental Study with Humans) Trial. All participants are exposed to all experimental conditions (i.e., "interventions") in a complete factorial 2 x 3 within-subjects experimental design. "Interventions" involve presenting various speech stimuli to participants.

The study is a within-subjects 2 x 3 factorial design. All participants are exposed to all experimental conditions or "interventions"

The sentences are designed such that a specific target word is either (a) Plausible (normal), (b) a semantic pragmatic violation, or (c) a morpho-syntactic violation. Example: e.g., "Every morning at breakfast, the boys would (A: EAT)/ (B.PLANT )/ (C. EATS ) eggs..."

The sentences will be presented either in A. conversational speaking style, or B. hyper-articulated clear speech style. Clear and conversational speech samples will be elicited from a single male talker. In the conversational speech condition, which will be recorded first, the talker will be instructed to read the sentences as they would in everyday conversation. In the clear speech condition, the talker will be instructed to produce the sentences as if they would if they were talking to a person with hearing loss.

The N400 mean amplitude will be measured from the ERP waveform for each condition, using standard procedures for ERP research.

The N400 onset latency will be measured from the ERP waveform for each condition, using standard procedures for ERP research.

The P600 mean amplitude will be measured from the ERP waveform for each condition, using standard procedures.

The P600 onset latency will be measured from the ERP waveform for each condition, using standard procedures.

The pupil dilation response (average proportion change from baseline) will be measured from the continuous pupillometry time series, measured prior to target word onset, and compared between speech samples presented in clear and conversational speech.

Recognition memory will be measured using both standard accuracy and signal detection measures from the recognition memory task

Inclusion Criteria: Age 60-90 Right-handed Native English speaker Scores in the normal range (> or = 25 points) on Montreal Cognitive Assessment (MoCA) For adults with hearing loss, a pure-tone average score of > 25 dB HL (between 1 - 4kHz) Exclusion Criteria: Left-handed (language-related electrophysiological responses of left-handed subjects differ from those of right-handed subjects) History of psychiatric or neurological illnesses (including skull fractures, as this is known to alter electrophysiological response at the scalp) Score of < 25 points on the MOCA Use of certain prescription and non-prescription drugs known to alter brain function and the autonomic nervous system, including pupil dilation (e.g., anti-depressants, attention deficit hyperactivity disorder drugs) Any eye disease that would impair the ability to measure pupil dilation (e.g., cataracts, nystagmus, amblyopia) Scores on speech shadowing audibility control task below 50%, suggesting poor intelligibility A display of behavior that would significantly interfere with the validity of data collection or safety during the study;

All experimental code and associated stimuli will be posted online. Experimental protocols, analysis plans, and materials will be made publically available via the Center for Open Science: Open Science Framework (osf.io). Once core findings are published, we will make all de-identified data available to the scientific community via Open Science Framework (OSF) as well. This will include raw data, along with a readme documentation detailing the meaning of the columns used to store the data. Format of the data will be under the form of general .txt or .csv, and .ascii (alpha- numeric character) documents suitable for reanalysis. This data set will contain no personally identifying information, and the website will be linked from our respective laboratory webpages and posted in all publications.

Data will be available upon completion of the first manuscript from this paper, with an anticipated date of one year from end of study.