Computational Neuroscience of Language Processing in the Human Brain

Language is a signature human cognitive skill, but the precise computations that support language understanding remain unknown. This study aims to combine high-quality human neural data obtained through intracranial recordings with advances in computational modeling of human cognition to shed light on the construction and understanding of speech.

The neural architecture of language is the foundation for the highest form of human interaction. Prior work has identified a network of frontal and temporal brain areas that selectively support language processing, but the precise computations that underlie our ability to extract meaning from sequences of words have remained unknown. The standard approaches in human cognitive neuroscience lack the spatial and temporal resolution necessary for precise comparisons to computational models. To bridge this gap in knowledge, neural responses to language stimuli will be collected from epileptic patients undergoing intracranial monitoring. Overall, these data will be used to identify cortical maps of different linguistic manipulations and to better understand properties of the human language network.

Patients with pharmaco-resistant epilepsy undergoing intracranial monitoring involving the left cerebral hemisphere.

Participants will listen to sentences and stories while neural data are recorded through electrodes placed for clinical purposes.

By using sEEG intracranial recordings of the brain, EEG power in frequency bands will reflect cortical maps of responses to different linguistic manipulations, informing the functional organization of the human language system. Power is measured in arbitrary units; higher power reflects greater activity at the investigated frequency.

Time-courses of neural response to language across diverse parts of the language network. These data will be used to predict across-time variation in response strength from the properties of linguistic input.

Human neural data will be compared to ANN language models to test how well these models predict human responses to language and why. There are no minimum or maximum scores. Higher values mean better model predictivity (i.e., a better match between model representations and neural responses).

Inclusion Criteria: clinical indications to proceed with intracranial monitoring involving the left cerebral hemisphere, as determined by a multidisciplinary epilepsy surgery team the ability to comply with test directions and provide informed consent between ages 18 - 85 Exclusion Criteria: inability to understand or perform the task outlined in the protocol, or who are unwilling or unable to participate

Blank I, Balewski Z, Mahowald K, Fedorenko E. Syntactic processing is distributed across the language system. Neuroimage. 2016 Feb 15;127:307-323. doi: 10.1016/j.neuroimage.2015.11.069. Epub 2015 Dec 5.

Blank IA, Fedorenko E. No evidence for differences among language regions in their temporal receptive windows. Neuroimage. 2020 Oct 1;219:116925. doi: 10.1016/j.neuroimage.2020.116925. Epub 2020 May 11.

Fedorenko E, Behr MK, Kanwisher N. Functional specificity for high-level linguistic processing in the human brain. Proc Natl Acad Sci U S A. 2011 Sep 27;108(39):16428-33. doi: 10.1073/pnas.1112937108. Epub 2011 Sep 1.

Fedorenko E, Blank IA. Broca's Area Is Not a Natural Kind. Trends Cogn Sci. 2020 Apr;24(4):270-284. doi: 10.1016/j.tics.2020.01.001. Epub 2020 Feb 20.

Fedorenko E, Duncan J, Kanwisher N. Language-selective and domain-general regions lie side by side within Broca's area. Curr Biol. 2012 Nov 6;22(21):2059-62. doi: 10.1016/j.cub.2012.09.011. Epub 2012 Oct 11.

Fedorenko E, Hsieh PJ, Nieto-Castanon A, Whitfield-Gabrieli S, Kanwisher N. New method for fMRI investigations of language: defining ROIs functionally in individual subjects. J Neurophysiol. 2010 Aug;104(2):1177-94. doi: 10.1152/jn.00032.2010. Epub 2010 Apr 21.

Fedorenko E, Nieto-Castanon A, Kanwisher N. Lexical and syntactic representations in the brain: an fMRI investigation with multi-voxel pattern analyses. Neuropsychologia. 2012 Mar;50(4):499-513. doi: 10.1016/j.neuropsychologia.2011.09.014. Epub 2011 Sep 17.

Fedorenko E, Scott TL, Brunner P, Coon WG, Pritchett B, Schalk G, Kanwisher N. Neural correlate of the construction of sentence meaning. Proc Natl Acad Sci U S A. 2016 Oct 11;113(41):E6256-E6262. doi: 10.1073/pnas.1612132113. Epub 2016 Sep 26.

Mollica F, Siegelman M, Diachek E, Piantadosi ST, Mineroff Z, Futrell R, Kean H, Qian P, Fedorenko E. Composition is the Core Driver of the Language-selective Network. Neurobiol Lang (Camb). 2020 Mar 1;1(1):104-134. doi: 10.1162/nol_a_00005. eCollection 2020.

Nieto-Castanon A, Fedorenko E. Subject-specific functional localizers increase sensitivity and functional resolution of multi-subject analyses. Neuroimage. 2012 Nov 15;63(3):1646-69. doi: 10.1016/j.neuroimage.2012.06.065. Epub 2012 Jul 8.

Norman-Haignere S, Kanwisher NG, McDermott JH. Distinct Cortical Pathways for Music and Speech Revealed by Hypothesis-Free Voxel Decomposition. Neuron. 2015 Dec 16;88(6):1281-1296. doi: 10.1016/j.neuron.2015.11.035.

Pereira F, Lou B, Pritchett B, Ritter S, Gershman SJ, Kanwisher N, Botvinick M, Fedorenko E. Toward a universal decoder of linguistic meaning from brain activation. Nat Commun. 2018 Mar 6;9(1):963. doi: 10.1038/s41467-018-03068-4.

Shain C, Blank IA, van Schijndel M, Schuler W, Fedorenko E. fMRI reveals language-specific predictive coding during naturalistic sentence comprehension. Neuropsychologia. 2020 Feb 17;138:107307. doi: 10.1016/j.neuropsychologia.2019.107307. Epub 2019 Dec 24.

Siegelman M, Blank IA, Mineroff Z, Fedorenko E. An Attempt to Conceptually Replicate the Dissociation between Syntax and Semantics during Sentence Comprehension. Neuroscience. 2019 Aug 10;413:219-229. doi: 10.1016/j.neuroscience.2019.06.003. Epub 2019 Jun 11.