Academic Journal
View in EDS
PDF Full Text
HTML Full Text

A constitutive model for coal gangue coarse-grained subgrade filler incorporating particle breakage

Bibliographic Details
Title: A constitutive model for coal gangue coarse-grained subgrade filler incorporating particle breakage
Authors: Zong-Tang Zhang, An Liu, Si-Chun Yu, Shun-Kai Liu, Chuan-Feng Fang, Ze Liu, Wei Hu
Source: Scientific Reports, Vol 15, Iss 1, Pp 1-18 (2025)
Publisher Information: Nature Portfolio, 2025.
Publication Year: 2025
Collection: LCC:Medicine
LCC:Science
Subject Terms: Coal gangue subgrade, Subgrade filler, Particle breakage, Coarse-grained soil, Constitutive model, Medicine, Science
More Details: Abstract The accumulation and discharge amount of coal gangue are substantial, occupying significant land resources over time. Utilizing coal gangue as subgrade filler can generate notable economic and social benefits. Coal gangue coarse-grained soil (CGSF) was used to conduct a series of large-scale vibration compaction tests and large-scale triaxial tests. The results indicate that the maximum dry density of CGSF initially increases and then decreases with the increase in fractal dimension. The stress–strain curves of the samples exhibit a distinct nonlinear growth pattern. Analysis of the compaction effect suggests that the compaction degree of CGSF should not be lower than 93%. As the confining pressure increases, the extent of failure strength improvement due to increased compaction decreases. Additionally, the failure strength of samples initially increases and then decreases with the increase in coarse particle content. A modified quadratic polynomial fractal model gradation equation was proposed to describe the gradation of samples after particle breakage. Based on this, a new quantitative index for particle breakage was established. Analysis of particle breakage in samples revealed that higher confining pressure and greater coarse particle content lead to increased particle breakage. The breakage exhibited a significant size effect, and the impact of particle gradation on sample breakage was greater than that of confining pressure. The stress–strain relationship of CGSF was analyzed by using a logarithmic constitutive model, and the correlation between model parameters and the newly derived particle breakage index was generated. A constitutive model incorporating particle breakage for CGSF was established, and its accuracy was validated.
Document Type: article
File Description: electronic resource
Language: English
ISSN: 2045-2322
Relation: https://doaj.org/toc/2045-2322
DOI: 10.1038/s41598-025-85979-z
Access URL: https://doaj.org/article/ebb586c13bcb4e87881eed8b7cc372c8
Accession Number: edsdoj.bb586c13bcb4e87881eed8b7cc372c8
Database: Directory of Open Access Journals
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHjPtM4BHU3ZchRwgzYmadcigk49r9CVlbU7V5F6lgH7WwFcMANj47BgtCMMrXvn5zV6AAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDMUS3VrwYGOpU-aeigIBEICBmpIYrHNAhA0Z-5Axg3bdWthDgM8UfqAGJ79Sb6-LZB-zRH94Nw_elJ9ZF4F21FrGTMi2kIlWO2rPYrO8CS_Qn__fIsFis2rBEDC6oCAPral3Tfs-xWWjjyVpxvbMkyQUhUmUG-eS8bVsuggk1PuM3qR3MoFwb-8OoDsdK5E3SNv38aV_fNgQ8In72h5_p-4GpSSoj6SvAa-Vx4I=
Text:
  Availability: 1
  Value: <anid>AN0182076388;[fkqs]02jan.25;2025Jan09.01:57;v2.2.500</anid> <title id="AN0182076388-1">Brain-model neural similarity reveals abstractive summarization performance </title> <p>Deep language models (DLMs) have exhibited remarkable language understanding and generation capabilities, prompting researchers to explore the similarities between their internal mechanisms and human language cognitive processing. This study investigated the representational similarity (RS) between the abstractive summarization (ABS) models and the human brain and its correlation to the performance of ABS tasks. Specifically, representational similarity analysis (RSA) was used to measure the similarity between the representational patterns (RPs) of the BART, PEGASUS, and T5 models' hidden layers and the human brain's language RPs under different spatiotemporal conditions. Layer-wise ablation manipulation, including attention ablation and noise addition was employed to examine the hidden layers' effect on model performance. The results demonstrate that as the depth of hidden layers increases, the models' text encoding becomes increasingly similar to the human brain's language RPs. Manipulating deeper layers leads to more substantial decline in summarization performance compared to shallower layers, highlighting the crucial role of deeper layers in integrating essential information. Notably, the study confirms the hypothesis that the hidden layers exhibiting higher similarity to human brain activity play a more critical role in model performance, with their correlations reaching statistical significance even after controlling for perplexity. These findings deepen our understanding of the cognitive mechanisms underlying language representations in DLMs and their neural correlates, potentially providing insights for optimizing and improving language models by aligning them with the human brain's language-processing mechanisms.</p> <p>Keywords: Deep language models; Representational similarity analysis; Abstractive summarization; Electroencephalography; Neural correlates; Psychology and Cognitive Sciences Psychology Cognitive Sciences Medical and Health Sciences Neurosciences</p> <hd id="AN0182076388-2">Introduction</hd> <p>A key goal of Natural Language Processing (NLP) is to enable machines to understand and generate language like humans. Recent advances in Deep Learning (DL) techniques have led to remarkable progress in Deep Language Models (DLMs) across various NLP tasks, with these models demonstrating language understanding and generation capabilities comparable to those of humans[<reflink idref="bib1" id="ref1">1</reflink>],[<reflink idref="bib2" id="ref2">2</reflink>]. This phenomenon has prompted researchers to investigate the similarities and differences between the internal mechanisms of DLMs and human language cognitive processing, attempting to explain the connection between Artificial Intelligence(AI) and biological intelligence in language processing[<reflink idref="bib3" id="ref3">3</reflink>], [<reflink idref="bib4" id="ref4">4</reflink>]–[<reflink idref="bib5" id="ref5">5</reflink>]. DLMs have provided a new computational framework for studying the cognitive neural mechanisms of human language understanding[<reflink idref="bib5" id="ref6">5</reflink>], [<reflink idref="bib6" id="ref7">6</reflink>]–[<reflink idref="bib7" id="ref8">7</reflink>], while drawing on knowledge from cognitive neuroscience has also offered new perspectives for optimizing and improving DLMs[<reflink idref="bib8" id="ref9">8</reflink>], [<reflink idref="bib9" id="ref10">9</reflink>]–[<reflink idref="bib10" id="ref11">10</reflink>].</p> <p>Numerous studies have demonstrated that DLMs exhibit similarities in Language Representation (LR) to the human brain and can, to a certain extent, explain the brain's activation patterns in response to language[<reflink idref="bib11" id="ref12">11</reflink>], [<reflink idref="bib12" id="ref13">12</reflink>]–[<reflink idref="bib13" id="ref14">13</reflink>]. Pioneering research has found that Transformer-based DLMs (such as GPT and BERT) can predict functional Magnetic Resonance Imaging (fMRI) and Electrocorticography (ECoG) activation in language-related human brain regions within a noise tolerance range, substantially outperforming traditional Recurrent Neural Network (RNN) models[<reflink idref="bib14" id="ref15">14</reflink>].[<reflink idref="bib15" id="ref16">15</reflink>] discovered that a model's performance on the next-word prediction task is highly correlated with its ability to explain brain activation. This finding suggests that the human brain may primarily rely on a predictive coding mechanism to understand language, continuously predicting the likely upcoming words during the comprehension process[<reflink idref="bib16" id="ref17">16</reflink>],[<reflink idref="bib17" id="ref18">17</reflink>]. Recent empirical studies have provided further support for this theory. Research[<reflink idref="bib18" id="ref19">18</reflink>] has provided strong evidence for hierarchical predictive processing in natural language understanding. Brain responses to words were found to be modulated by predictions at multiple levels, from phonemes to meaning, supporting theories of hierarchical predictive coding in language processing. Additional investigations demonstrated that enhancing language model activations with long-range predictions improved their mapping to brain responses, revealing a hierarchical organization of language predictions in the cortex. Frontoparietal cortices were shown to predict higher-level, longer-range, and more contextual representations compared to temporal cortices, emphasizing the importance of context in language understanding[<reflink idref="bib19" id="ref20">19</reflink>].Expanding on these findings, researchers have delved deeper into the intrinsic connection between the internal representations of DLMs and human brain language processing. In terms of semantic representation,[<reflink idref="bib5" id="ref21">5</reflink>],[<reflink idref="bib6" id="ref22">6</reflink>],[<reflink idref="bib20" id="ref23">20</reflink>] found that the contextual word embeddings of DLMs like GPT-2 can capture the geometric structure of activation in the human brain's language areas (such as the left Inferior Frontal Gyrus, IFG), outperforming traditional static word embeddings (like GloVe). This result indicates that DLMs have learned context-dependent lexical semantic representations similar to the human brain's.</p> <p>Regarding factors influencing similarity,[<reflink idref="bib21" id="ref24">21</reflink>] discovered that the activation similarity between GPT-2 and the human brain in semantic-related brain regions (such as the left temporal lobe) increases with a person's level of semantic understanding, and this similarity is most evident in the deeper layers of the model[<reflink idref="bib22" id="ref25">22</reflink>]. DLMs exhibit hierarchical representation characteristics, with shallow layers of models like BERT encoding surface information, middle layers encoding syntactic features, and deep layers integrating long-distance, high-level global semantics[<reflink idref="bib23" id="ref26">23</reflink>]. When using DLMs to predict human brain activation, it was found that different layers of different models have varying predictive abilities for human brain language processing. For example, in GPT-2, the long-distance dependency representations in the middle layers best predict human language understanding, while the short-distance attention in the shallow layers best predicts speech encoding[<reflink idref="bib24" id="ref27">24</reflink>]; in BERT, the middle layers perform best in predicting human brain activation[<reflink idref="bib10" id="ref28">10</reflink>]. These studies have uncovered the close connection between the internal mechanisms of DLMs and human language understanding, offering valuable insights for further investigating their similarities.</p> <p>While many current studies focus on investigating the factors contributing to the similarities between DLMs and the human brain, few have explored how the degree of such similarity relates to model performance. Some research has found that the higher the similarity between language models and the human brain, the better their performance on various benchmarks[<reflink idref="bib12" id="ref29">12</reflink>]. However, other studies have indicated that this correlation has limited predictive power for model performance. For example,[<reflink idref="bib14" id="ref30">14</reflink>] discovered that a model's neural fit (brain score) only significantly correlates with its performance on the next-word prediction task. Furthermore, current research on the internal mechanisms of DLMs mainly focuses on elemental semantic composition at the lexical or sentence level. In contrast, exploration of discourse comprehension mechanisms is relatively insufficient[<reflink idref="bib25" id="ref31">25</reflink>],[<reflink idref="bib26" id="ref32">26</reflink>]. Contextual information is crucial for language understanding[<reflink idref="bib27" id="ref33">27</reflink>],[<reflink idref="bib28" id="ref34">28</reflink>], yet existing studies have not fully considered the impact of context on model LR. The task of Abstractive Summarization (ABS) provides an ideal research scenario for investigating the discourse comprehension mechanisms of DLMs. Automatic text summarization, which extracts critical knowledge from vast amounts of information, alleviates information overload[<reflink idref="bib29" id="ref35">29</reflink>]. It has broad application prospects and research value[<reflink idref="bib30" id="ref36">30</reflink>],[<reflink idref="bib31" id="ref37">31</reflink>]. Compared to other NLP tasks, the ABS task places higher demands on a model's language understanding ability, as it requires deep encoding and integration of the semantics of an entire text. This task primarily employs encoder-decoder architecture models, such as BART[<reflink idref="bib32" id="ref38">32</reflink>], PEGASUS[<reflink idref="bib33" id="ref39">33</reflink>] as well as T5[<reflink idref="bib34" id="ref40">34</reflink>],[<reflink idref="bib35" id="ref41">35</reflink>], which can be scaled up to Large Language Models (LLMs) with sufficient parameters[<reflink idref="bib36" id="ref42">36</reflink>]. The encoder is responsible for hierarchical semantic encoding of the original text, while the decoder generates text summaries based on the encoded information. Considering the encoder's crucial role in semantic understanding and integration, exploring the neural correlates of the ABS model's encoder can help reveal the cognitive mechanisms of the human brain in understanding and integrating discourse semantics.</p> <p>Regarding the ABS task and models, this study raises two pressing questions. First, the roles of different layers of ABS models in discourse semantic understanding and the Representational Patterns (RPs) of syntactic and semantic information remain unclear. Research has found that the human brain processes semantics and syntax with different spatiotemporal characteristics[<reflink idref="bib37" id="ref43">37</reflink>], and the two exhibit partially overlapping distributed representations in the human brain[<reflink idref="bib38" id="ref44">38</reflink>],[<reflink idref="bib39" id="ref45">39</reflink>]. Consequently, we aim to investigate the following question: What roles do the different layers of the ABS model's encoder play in integrating discourse semantics, and how are grammatical and semantic information represented within these layers? Exploring the model's RPs can help us understand ABS models' hierarchical language encoding mechanisms and their neural basis. Second, the correlation between the similarity of ABS models to human brain LRs and their task performance merits further investigation. Language is a unique, high-level cognitive function of humans, and the human brain has evolved highly complex and efficient language processing mechanisms over a long period. Therefore, the closer a DLM is to the human brain in structure and function, the stronger its language understanding and generation capabilities may be. So, for the more complex ABS task, can the degree of similarity between the model and human brain representations also predict the quality of its generated summaries?</p> <p>Based on the above analysis of the relationship between DLMs and human brain language processing, we hypothesize that, in the context of the ABS task, the hidden layers of the model that exhibit a higher degree of Representational Similarity (RS) to the human brain's activation patterns may play a more critical role in the model's performance on this task. To test this hypothesis, we designed an experiment involving human participants and employed two methods: Representational Similarity Analysis (RSA) and layer-wise ablation. This study used the ABS models PEGASUS and BART as the research objects to investigate the similarity between their encoder hidden layer representations and those of the human brain (i.e., neural similarity) and the contribution of each layer to the summarization task. RSA can reveal the similarity of representational structures between systems (such as DLMs and the human brain) by calculating the correlation between their response patterns to the same set of inputs, overcoming differences in representational forms[<reflink idref="bib13" id="ref46">13</reflink>],[<reflink idref="bib40" id="ref47">40</reflink>],[<reflink idref="bib41" id="ref48">41</reflink>]. Many different stimuli are input into the target system to compute representational structure, i.e., the geometric properties of the representational space. The activity patterns evoked by each stimulus are measured, and the similarity of responses between all stimulus pairs is calculated. This process quantifies the system's representational structure. Layer-wise ablation, on the other hand, removes the model's learned attention layer by layer, examining the contribution of each layer to the ABS task by intervening with the model's internal representations[<reflink idref="bib10" id="ref49">10</reflink>],[<reflink idref="bib42" id="ref50">42</reflink>].</p> <p>Specifically, the N400, a negative-going Event-Related Potential (ERP) component peaking around 400 ms post-stimulus onset, is a neural signature of semantic processing[<reflink idref="bib43" id="ref51">43</reflink>]. In contrast, the P600, a positive-going ERP component peaking around 600 ms, is associated with syntactic processing and integration[<reflink idref="bib44" id="ref52">44</reflink>]. This study first calculated the similarity between the model's hidden layer representations and human brain representations under different spatiotemporal conditions. It aimed to reveal the model's hierarchical language processing mechanism and potential neural correlates. We defined four spatiotemporal conditions: Full time window (using the complete time window and all channels), Early window (using the time window and electrode positions typically associated with the N400 component), Late window (using the time window and electrode positions typically associated with the P600 component) and Baseline (using a pre-stimulus time window and all channels as a control). Then, we performed layer-wise attention ablation and noise addition on each layer, and evaluate the resulting changes in summary quality to investigate the role of each layer in integrating semantics and grasping the text's overall meaning. Finally, this study connects the RS of each hidden layer of the model to the human brain and its impact on model performance, attempting to test the hypothesis we proposed. Figure 1 illustrates the overall research framework.</p> <p>Graph: Figure 1 Schematic diagram of the research framework.</p> <p>Through a series of experiments and analyses, this study makes the following contributions:</p> <p></p> <ulist> <item> This study employed RSA to compare LRs in DLMs and the human brain. Results showed increasing similarity with model depth, reaching statistical significance, indicating DLMs' ability to acquire brain-like LRs. Control experiments revealed the crucial role of model training in this process.</item> <p></p> <item> The layer manipulation experiment reveals that each encoder layer contributes to the ABS task to a different extent. Ablating the attention mechanism or adding noise in deeper hidden layers leads to a more substantial decline in summarization performance, indicating that deeper layers play a more crucial role in integrating critical information from the original text and understanding its comprehensive theme.</item> <p></p> <item> This study reveals a significant correlation between hidden layers' similarity to human brain activity patterns and their impact on model summarization performance. Examining BART, PEGASUS, and T5 models, we found this correlation consistent across diverse spatiotemporal brain activity conditions and layer manipulation methods. The correlation's persistence after controlling for perplexity further supports our findings' robustness.</item> </ulist> <p>These findings deepen the understanding of the internal working mechanisms of DLMs. They provide valuable insights for optimizing and improving DLMs from a brain-inspired perspective.</p> <hd id="AN0182076388-3">Methods</hd> <p></p> <hd id="AN0182076388-4">EEG signal acquisition and processing</hd> <p></p> <hd id="AN0182076388-5">Experimental design</hd> <p>Fifteen healthy adult participants were recruited for this study, of which 13 were effective, including nine males and four females, with a mean age of 24.31 ± 2.72 years. All participants were right-handed, had normal or corrected-to-normal vision, and had no neurological or psychiatric disorders history. All participants provided written informed consent prior to their participation in the study.</p> <p>The BEATs system[<reflink idref="bib45" id="ref53">45</reflink>] was utilized to collect 28-channel EEG signals, as shown in Fig. 3b. The electrodes were placed in an elastic cap according to the standard positions of the international 10-20 system. The 28 electrodes used for recording EEG signals are highlighted in yellow. Based on the research[<reflink idref="bib46" id="ref54">46</reflink>], the electrodes sensitive to the N400 ERP component include Cz, C3, C4, CP1, CP2, Pz, P3, and P4, marked with blue borders in Fig. 3b. The time window sensitive to the N400 component was set to 250-450ms. According to the study[<reflink idref="bib47" id="ref55">47</reflink>], the channels sensitive to the P600 ERP component include C3, Cz, C4, P3, Pz, and P4, marked with red borders in Fig. 3b. The time window sensitive to the P600 component was set to 500-800ms. Throughout the recording process, the sampling rate was set to 250Hz, and the impedance of each electrode was maintained below 10 K <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="normal">Ω</mi></math> </ephtml> to ensure signal quality.</p> <p>The experimental program was written using PsychoPy (version 2022.2.5)[<reflink idref="bib48" id="ref56">48</reflink>],[<reflink idref="bib49" id="ref57">49</reflink>], requiring participants to read a text that appeared word by word continuously. The reading materials were selected from 8 articles in the Factuality Evaluation Benchmark (FRANK) dataset[<reflink idref="bib50" id="ref58">50</reflink>], containing 62 sentences and 1427 words. We present the details of the selected materials in Supplementary Table S1 in the appendix.</p> <p>Before the experiment, all participants were required to read and voluntarily sign an informed consent form. The experiment was conducted in a soundproof laboratory, with participants seated approximately 60cm from the computer screen. They were required to maintain a horizontal gaze and minimize head movements. The procedure of the EEG signal acquisition experiment is shown in Fig. 2.</p> <p>Graph: Figure 2 Procedure of the EEG experiment.</p> <p>First, participants fixated on a central fixation point (a cross) presented on the screen for 5000ms to record baseline EEG signals. Subsequently, participants read articles presented word by word, with each word appearing in the center of the screen for 1500ms, during which the EEG signals were recorded. Finally, after each article was played, participants were required to answer three judgment questions related to the content to ensure they were focused on the reading task. To ensure data quality, only participants who achieved an accuracy rate exceeding 50% for all questions were included in the analysis. After answering the questions, participants had a 2-minute break to avoid potential fatigue effects before starting the experiment with the following article.</p> <hd id="AN0182076388-6">Data processing</hd> <p>A series of preprocessing steps were performed to remove noise and artifacts from the EEG signals, including filtering, segmentation, baseline correction, artifact removal, and re-referencing. First, the continuous EEG data were bandpass filtered between 0.5Hz and 30Hz to remove low-frequency drift and high-frequency noise. Then, based on the 1459 tokens obtained by PEGASUS's tokenizer, the continuous data were segmented into discrete time windows. Tokens derived from the same word shared the EEG signals recorded during the presentation of that word. The duration of the EEG signal segment corresponding to each token was 1700ms, including 1500ms of EEG signals during each event (word presentation) and 200ms of EEG signals preceding the event. To avoid signal drift, we performed baseline correction by subtracting the average potential of the 200ms preceding each event from the entire epoch. Next, we identified noisy or faulty channels through visual inspection of the raw EEG data, focusing on channels exhibiting persistent high-frequency noise, flat lines, or excessive artifacts. Then, bad channels were repaired using spherical spline interpolation, and Independent Component Analysis (ICA) was employed to remove artifacts such as eye movements and muscle activity. Finally, a whole-brain re-referencing approach, which means using the mean voltage across all electrodes as the reference, was performed to obtain EEG signals relative to the global average potential. We used Python 3.7.9 and SPSS version 26 for EEG data processing and statistical analysis.</p> <hd id="AN0182076388-7">Experiments on ABS models</hd> <p>This experiment explores the intrinsic mechanisms of discourse comprehension in ABS models. Given the crucial role of the encoder in text comprehension within the model and the focus of EEG experiments primarily on human language comprehension rather than generation, this study centers on analyzing the characteristics of the encoder. The study utilizes three prevalent DLMs that have been fine-tuned for ABS tasks: BART, PEGASUS, and T5[<reflink idref="bib35" id="ref59">35</reflink>]. Specifically, the BART-large model is fine-tuned on the CNN/DM dataset[<reflink idref="bib51" id="ref60">51</reflink>], while the PEGASUS-large model on the XSum dataset[<reflink idref="bib52" id="ref61">52</reflink>], and the T5-large model on a combined XSum-CNN/DM dataset. CNN/DM and XSum are benchmark datasets used for training and evaluating ABS models, containing news articles paired with human-written summaries. Each model, featuring distinct architectures and varying numbers of layers, is fine-tuned on different datasets, enhancing the robustness of the experimental findings.</p> <p>Take the PEGASUS-large model as an instance, the encoder consists of 16 hidden layers, each generating a unique representation of the input discourse text. To analyze the impact of the attention mechanism in different hidden layers of the encoder on the model's performance, we designed an experiment to remove the attention in each layer individually. This approach eliminates the influence of the learned attention in a specific encoder layer while keeping the parameters of the other layers fixed, allowing us to observe its contribution to the model's performance. The attention is calculated as shown in Equation 1[<reflink idref="bib53" id="ref62">53</reflink>], where <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>d</mi><mi>k</mi></msub></math> </ephtml> is the dimension of the keys, <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>Q</mi><mo>=</mo><mi>X</mi><msup><mrow><mi>W</mi></mrow><mi>Q</mi></msup></mrow></math> </ephtml> is the query matrix, <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi><mo>=</mo><mi>X</mi><msup><mrow><mi>W</mi></mrow><mi>K</mi></msup></mrow></math> </ephtml> is the key matrix, and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>V</mi><mo>=</mo><mi>X</mi><msup><mrow><mi>W</mi></mrow><mi>V</mi></msup></mrow></math> </ephtml> is the value matrix.</p> <p>1 <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mtable><mtr><mtd columnalign="right"><mrow><mi>A</mi><mi>t</mi><mi>t</mi><mi>e</mi><mi>n</mi><mi>t</mi><mi>i</mi><mi>o</mi><mi>n</mi><mrow><mo stretchy="false">(</mo><mi>Q</mi><mo>,</mo><mi>K</mi><mo>,</mo><mi>V</mi><mo stretchy="false">)</mo></mrow><mo>=</mo><mi>S</mi><mi>o</mi><mi>f</mi><mi>t</mi><mi>m</mi><mi>a</mi><mi>x</mi><mfenced close=")" open="("><mfrac><mrow><mi>Q</mi><msup><mi>K</mi><mi>T</mi></msup></mrow><msqrt><msub><mi>d</mi><mi>k</mi></msub></msqrt></mfrac></mfenced><mi>V</mi></mrow></mtd></mtr></mtable></mrow></math> </ephtml></p> <p>Graph</p> <p> <emph>X</emph> represents the input to the layer, and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msup><mrow><mi>W</mi></mrow><mi>Q</mi></msup></math> </ephtml> , <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msup><mrow><mi>W</mi></mrow><mi>K</mi></msup></math> </ephtml> , <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msup><mrow><mi>W</mi></mrow><mi>V</mi></msup></math> </ephtml> are pre-trained parameter matrices. The uniformization of the attention mechanism is achieved by modifying the matrices <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msup><mrow><mi>W</mi></mrow><mi>Q</mi></msup></math> </ephtml> , <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msup><mrow><mi>W</mi></mrow><mi>K</mi></msup></math> </ephtml> , and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msup><mrow><mi>W</mi></mrow><mi>V</mi></msup></math> </ephtml> to ensure that the attention produces equal probabilities for the values in matrix <emph>V</emph>, thereby eliminating the influence of the layer's attention on matrix <emph>V</emph>[<reflink idref="bib10" id="ref63">10</reflink>]. Specifically, the <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msup><mrow><mi>W</mi></mrow><mi>Q</mi></msup></math> </ephtml> and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msup><mrow><mi>W</mi></mrow><mi>K</mi></msup></math> </ephtml> matrices are set to zero matrices, and the <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msup><mrow><mi>W</mi></mrow><mi>V</mi></msup></math> </ephtml> matrix is set to an identity matrix. This modification allows for removing the learned attention effect of the layer during model propagation, facilitating the analysis of the contribution of the encoder's layer to the model's performance. In subsequent sections, we refer to the method of setting attention to a uniform distribution as attention ablation. Additionally, we employ another approach to influence model performance called noise addition. This method involves introducing random perturbations drawn from a Gaussian distribution to the weight matrices of each encoder layer, allowing us to examine the impact of controlled noise on model performance.</p> <p>To assess the impact of layer manipulation on model performance in the ABS task, we employ two different layer-wise manipulations on the BART, PEGASUS, and T5 models: attention ablation and noise addition. We then generate summaries using these manipulated models on the complete test set. The quality of these summaries, which reflects the models' performance on the ABS task, is assessed using three variants of Rouge (Recall-Oriented Understudy for Gisting Evaluation): Rouge1, Rouge2, and RougeL. These metrics evaluate the overlap between generated and reference summaries at different levels. Rouge1 measures word-level overlap by calculating the ratio of words in the reference summary that appear in the generated summary. Rouge2 assesses the overlap of consecutive word pairs (bigrams) between the generated and reference summaries. RougeL evaluates the sequence similarity based on the longest common subsequence, capturing the overall structural similarity. This comprehensive evaluation allows us to assess summary quality regarding individual words, phrase structures, and overall content alignment. By applying these metrics to summaries generated after manipulating each encoder layer, we can quantify the impact of attention mechanisms and noise introduction on the models' ABS capabilities.</p> <hd id="AN0182076388-8">Representational similarity analysis</hd> <p>Due to the substantial differences in how language is represented in artificial neural network models and the human brain, directly comparing the correspondence between the two at the word or token level is challenging. Therefore, this study employed the RSA method to compare the activation patterns of models and the human brain in response to input text, i.e., the geometric structure of the representational space. The overall process of RSA is shown in Fig. 3.</p> <p>Graph: Figure 3 Schematic of representational similarity analysis.</p> <p>For the activation patterns of the model, the cosine similarity was used to calculate the pairwise similarity between the feature vectors corresponding to all tokens in each hidden layer of the encoder, yielding an <emph>n</emph><ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mo>×</mo></math> </ephtml><emph>n</emph> similarity matrix, as shown in Fig. 3a (<emph>n</emph> is the total number of tokens after the tokenizer processes the text; in the example shown, <emph>n</emph> = 5). This matrix characterizes the representational structure of the input text within the model. Specifically, the Representational Similarity Matrix (RSM) <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msubsup><mi mathvariant="bold">M</mi><mrow><mi mathvariant="italic">ij</mi></mrow><mi>l</mi></msubsup></math> </ephtml> of the hidden layer can be represented as Equation 2, where <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msubsup><mi mathvariant="bold">h</mi><mi>i</mi><mi>l</mi></msubsup></math> </ephtml> represents the feature vector of the <emph>i-th</emph> token in the <emph>l-th</emph> hidden layer, and <emph>n</emph> is the total number of tokens.</p> <p>2 <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mtable><mtr><mtd columnalign="right"><mrow><msubsup><mi mathvariant="bold">S</mi><mrow><mi mathvariant="italic">ij</mi></mrow><mi>l</mi></msubsup><mo>=</mo><mfrac><mrow><msubsup><mi mathvariant="bold">h</mi><mi>i</mi><mi>l</mi></msubsup><mo>·</mo><msubsup><mi mathvariant="bold">h</mi><mi>j</mi><mi>l</mi></msubsup></mrow><mrow><mrow><mo stretchy="false">|</mo></mrow><msubsup><mi mathvariant="bold">h</mi><mi>i</mi><mi>l</mi></msubsup><mrow><mo stretchy="false">|</mo><mo stretchy="false">|</mo></mrow><msubsup><mi mathvariant="bold">h</mi><mi>j</mi><mi>l</mi></msubsup><mrow><mo stretchy="false">|</mo></mrow></mrow></mfrac><mo>,</mo><mi>i</mi><mo>,</mo><mi>j</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo><mo>...</mo><mo>,</mo><mi>n</mi></mrow></mtd></mtr></mtable></mrow></math> </ephtml></p> <p>Graph</p> <p>Regarding the activation patterns of the human brain, the EEG signals corresponding to each token were similarly represented as a vector and calculated as follows. The EEG experiment recorded the participants' electrical signals of the human brain while they read all the tokens. After aligning each token with its corresponding word, each token corresponded to the 1500ms EEG signal from the appearance to the disappearance of the word. Given that the sampling rate of the EEG signals was 250Hz, each token had 375 sampling points within the 1500ms time window, accumulating 10500 data points (375 <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mo>×</mo></math> </ephtml> 28) across the 28 electrodes. By concatenating the signals from the 28 electrodes, each token corresponded to a 10500-dimensional EEG feature vector. Similar to the model, the similarity of all token pairs in the EEG feature space was computed, and an <emph>n</emph><ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mo>×</mo></math> </ephtml><emph>n</emph> similarity matrix was generated for each participant, as shown in Fig. 3b (<emph>n</emph> = 5 in the example, the EEG signal placed at the right of each word ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>e</mi><mi>i</mi></msub></math> </ephtml> ) represents the concatenation of all electrode data for that word), reflecting the representational structure of the human brain during language processing.</p> <p>We established four spatiotemporal conditions for EEG signals in language comprehension. The condition of Full time window uses the complete time window and all electrodes, providing a comprehensive view of brain activity during language processing. Early window and Late window conditions employ time windows and electrode locations associated with the N400 and P600 components, respectively, capturing brain activity in these specific spatiotemporal windows. Finally, Baseline condition (control group) uses the time window from -200 to 0 ms before stimulus onset and all electrodes as a comparison baseline. Traditional ERP components inspire these conditions; we posit that EEG signals under these conditions may contain information relatively sensitive to human brain semantic (Early window) or syntactic (Late window) processing. However, we do not assume a direct, exclusive mapping between these components and specific linguistic processes. The selection of these conditions primarily aims to observe brain activity patterns during language processing from multiple perspectives, thereby enabling the examination of RS across different spatiotemporal windows. The composition of the EEG vectors under the four conditions is shown in Table 1.</p> <p>Table 1 Composition of the three EEG vector types.</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p>EEG vector</p></th><th align="left"><p>Selected electrodes</p></th><th align="left"><p>Time windows (ms)</p></th><th align="left"><p>Sampling points</p></th><th align="left"><p>Vector dimension</p></th></tr></thead><tbody><tr><td align="left"><p>Full time window</p></td><td align="left"><p>All electrodes</p></td><td align="left"><p>0 to 1500</p></td><td align="left"><p>375</p></td><td align="left"><p>10500</p></td></tr><tr><td align="left"><p>Early window</p></td><td align="left"><p>C3, Cz, C4, P3, Pz, P4, CP1, CP2</p></td><td align="left"><p>250 to 450</p></td><td align="left"><p>50</p></td><td align="left"><p>400</p></td></tr><tr><td align="left"><p>Late window</p></td><td align="left"><p>C3, Cz, C4, P3, Pz, P4</p></td><td align="left"><p>500 to 800</p></td><td align="left"><p>75</p></td><td align="left"><p>450</p></td></tr><tr><td align="left"><p>Baseline</p></td><td align="left"><p>All electrodes</p></td><td align="left"><p>− 200 to 0</p></td><td align="left"><p>50</p></td><td align="left"><p>400</p></td></tr></tbody></table> </ephtml> </p> <p>The similarity matrix <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msubsup><mi mathvariant="bold">B</mi><mrow><mi mathvariant="italic">ij</mi></mrow><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msubsup></math> </ephtml> of the human brain's representation of text can be represented as Equation 3, where <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msubsup><mi mathvariant="bold">e</mi><mi>i</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msubsup></math> </ephtml> represents the EEG vector recorded for the <emph>k-th</emph> participant while reading the <emph>i-th</emph> token, based on the representation type t, which can be overall representation, semantic representation, or syntactic representation.</p> <p>3 <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mtable><mtr><mtd columnalign="right"><mrow><msubsup><mi mathvariant="bold">B</mi><mrow><mi mathvariant="italic">ij</mi></mrow><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msubsup><mo>=</mo><mfrac><mrow><msubsup><mi mathvariant="bold">e</mi><mi>i</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msubsup><mo>·</mo><msubsup><mi mathvariant="bold">e</mi><mi>j</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msubsup></mrow><mrow><mrow><mo stretchy="false">|</mo></mrow><msubsup><mi mathvariant="bold">e</mi><mi>i</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msubsup><mrow><mo stretchy="false">|</mo><mo stretchy="false">|</mo></mrow><msubsup><mi mathvariant="bold">e</mi><mi>j</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msubsup><mrow><mo stretchy="false">|</mo></mrow></mrow></mfrac><mo>,</mo><mi>i</mi><mo>,</mo><mi>j</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo><mo>...</mo><mo>,</mo><mi>n</mi></mrow></mtd></mtr></mtable></mrow></math> </ephtml></p> <p>Graph</p> <p>Using the n <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mo>×</mo></math> </ephtml> n RSM, this study characterizes the language comprehension process of the model and the human brain in a unified manner. Based on these RSMs, we calculate the correlation between them to investigate the RS between different hidden layers of the model and the brain's overall, semantic-specific, and syntactic-specific representations. The Spearman rank correlation coefficient was used to calculate the correlation between the RSMs of the model and the human brain, yielding the RSA score:</p> <p>4 <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mtable><mtr><mtd columnalign="right"><mrow><msup><mi>ρ</mi><mrow><mi>l</mi><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msup><mo>=</mo><mfrac><mrow><msubsup><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mrow><mi>n</mi><mo stretchy="false">(</mo><mi>n</mi><mo>-</mo><mn>1</mn><mo stretchy="false">)</mo><mo stretchy="false">/</mo><mn>2</mn></mrow></msubsup><mspace width="4pt" /><mrow><mo stretchy="false">(</mo><msubsup><mi>r</mi><mi>i</mi><mi>l</mi></msubsup><mo>-</mo><msup><mrow><mover><mi>r</mi><mo>¯</mo></mover></mrow><mi>l</mi></msup><mo stretchy="false">)</mo></mrow><mrow><mo stretchy="false">(</mo><msubsup><mi>r</mi><mi>i</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msubsup><mo>-</mo><msup><mrow><mover><mi>r</mi><mo>¯</mo></mover></mrow><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msup><mo stretchy="false">)</mo></mrow></mrow><mrow><msqrt><mrow><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mrow><mi>n</mi><mo stretchy="false">(</mo><mi>n</mi><mo>-</mo><mn>1</mn><mo stretchy="false">)</mo><mo stretchy="false">/</mo><mn>2</mn></mrow></munderover><mspace width="4pt" /><msup><mrow><mo stretchy="false">(</mo><msubsup><mi>r</mi><mi>i</mi><mi>l</mi></msubsup><mo>-</mo><msup><mrow><mover><mi>r</mi><mo>¯</mo></mover></mrow><mi>l</mi></msup><mo stretchy="false">)</mo></mrow><mn>2</mn></msup></mrow></msqrt><msqrt><mrow><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mrow><mi>n</mi><mo stretchy="false">(</mo><mi>n</mi><mo>-</mo><mn>1</mn><mo stretchy="false">)</mo><mo stretchy="false">/</mo><mn>2</mn></mrow></munderover><mspace width="4pt" /><msup><mrow><mo stretchy="false">(</mo><msubsup><mi>r</mi><mi>i</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msubsup><mo>-</mo><msup><mrow><mover><mi>r</mi><mo>¯</mo></mover></mrow><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msup><mo stretchy="false">)</mo></mrow><mn>2</mn></msup></mrow></msqrt></mrow></mfrac></mrow></mtd></mtr></mtable></mrow></math> </ephtml></p> <p>Graph</p> <p>In Equation 4, <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msubsup><mi>r</mi><mi>i</mi><mi>l</mi></msubsup></math> </ephtml> and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msubsup><mi>r</mi><mi>i</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msubsup></math> </ephtml> represent the ranks of the upper (lower) triangular part (excluding the diagonal) of <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msubsup><mi mathvariant="bold">S</mi><mrow><mi mathvariant="italic">ij</mi></mrow><mi>l</mi></msubsup></math> </ephtml> and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msubsup><mi mathvariant="bold">B</mi><mrow><mi mathvariant="italic">ij</mi></mrow><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msubsup></math> </ephtml> , respectively, when sorted by similarity values. <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msup><mrow><mover><mi>r</mi><mo>¯</mo></mover></mrow><mi>l</mi></msup></math> </ephtml> and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msup><mrow><mover><mi>r</mi><mo>¯</mo></mover></mrow><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msup></math> </ephtml> are the corresponding average ranks. Since the similarity matrix is symmetric about the diagonal and the values on the diagonal are always 1, the Spearman rank correlation coefficient calculation is based on the upper triangular part of the matrix, excluding the diagonal elements, as shown in Fig. 3c. Furthermore, this study first calculates the correlation between each participant's EEG RSM and the RSM of each hidden model layer. Then, it averages the correlations across participants to record the correlation between the representation of that hidden layer and the human brain.</p> <hd id="AN0182076388-9">Experimental results</hd> <p></p> <hd id="AN0182076388-10">RS between human brain and models</hd> <p>To systematically investigate the RPs across the human brain and DLMs, we first conducted comprehensive visualization analyses of their RSMs using rank-based correlation heatmaps. Figure 4 presents these matrices, where color intensity denotes the strength of RS: darker red indicates more substantial similarity and darker blue indicates weaker similarity. We constructed these heatmaps based on rank ordering rather than absolute values to align with our subsequent Spearman rank correlation analyses.</p> <p>Graph: Figure 4 Visualization of RSMs.</p> <p>Visual analyses of these matrices revealed distinct architectural features in the representational spaces. A characteristic cross-shaped pattern emerged consistently across the human brain and model RSMs, indicating systematic similarities in specific RPs. In human brain RSMs (excluding baseline conditions), the core of the cross-pattern is localized in the lower-left quadrant of the heatmaps. Notably, BART's RSMs demonstrated remarkable architectural convergence with human brain RPs, resembling this characteristic lower-left positioning of the cross-pattern core. In contrast, PEGASUS and T5 initially exhibited divergent organizational principles, with their cross-pattern cores predominantly concentrated in the upper-right quadrant. This difference aligns with our later findings that BART achieved higher performance in the RSA.</p> <p>Figure 5 illustrates the relationship between the degree of RS (RSA scores) of the hidden layers in BART, PEGASUS, and T5 models and human brain activation during language processing. The error bars represent the <emph>standard error</emph> (<emph>SE</emph>), serving as uncertainty estimators and provide a visual representation of data dispersion. For each model, we calculated seven sets of RSA scores: Full time, Early, Late window and Baseline from Table 1, and three control groups for the former three. We randomly initialized the model parameters in the control groups and processed the same text input as the standard ABS models. Then, we calculated the RSA scores between these models and human brain RPs. The control group results, represented by thin lines without markers in Fig. 5, help demonstrate the effect of language model training and fine-tuning on representational capabilities.</p> <p>Graph: Figure 5 RSA scores between each hidden layer of the DLMs and the human brain.</p> <p>While RSA's effect sizes (correlation coefficients) appear relatively small, we suggest they are meaningful and interpretable for several reasons. First, our visualization analysis of the RSMs reveals systematic similarities in RPs between the human brain and DLM representations, particularly the characteristic cross-shaped structures in blue. These geometric configuration matches substantiate the meaningful relationships underlying the low correlation coefficients. Second, given the scale of our RSMs ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>1459</mn><mo>×</mo><mn>1459</mn></mrow></math> </ephtml> ), the actual sample size for correlation computation reaches the order of one million, making even small effect sizes statistically reliable at such large scales[<reflink idref="bib54" id="ref64">54</reflink>]. Finally, the core strength of RSA lies in capturing the overall geometric structure of representational spaces rather than superficial linear relationships. Therefore, despite modest effect sizes, RSA effectively reveals the fundamental similarities in information encoding patterns between human brains and DLMs.</p> <p>We employed the Jonckheere–Terpstra test to assess the presence of a statistically significant monotonic trend in the seven sets of RSA scores for each model as the number of layers increased. This statistical analysis was performed using SPSS (version 26), and the results are recorded in Table 2. The sign of <emph>Z</emph> indicates the trend direction, with positive and negative values representing increasing and decreasing trends. The <emph>p</emph>-value < 0.05 indicates statistical significance.</p> <p>Table 2 Jonckheere–Terpstra test results of RSA scores under different conditions.</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" rowspan="2"><p><bold>RSA conditions</bold></p></th><th align="left" colspan="2"><p><bold>BART</bold></p></th><th align="left" colspan="2"><p><bold>PEGASUS</bold></p></th><th align="left" colspan="2"><p><bold>T5</bold></p></th></tr><tr><th align="left"><p><bold><italic>Z</italic></bold></p></th><th align="left"><p><bold><italic>p</italic></bold></p></th><th align="left"><p><bold><italic>Z</italic></bold></p></th><th align="left"><p><bold><italic>p</italic></bold></p></th><th align="left"><p><bold><italic>Z</italic></bold></p></th><th align="left"><p><bold><italic>p</italic></bold></p></th></tr></thead><tbody><tr><td align="left"><p>Full time window</p></td><td align="left"><p>2.843</p></td><td align="left"><p>0.004**</p></td><td align="left"><p>5.694</p></td><td align="left"><p><0.001***</p></td><td align="left"><p>7.578</p></td><td align="left"><p><0.001***</p></td></tr><tr><td align="left"><p>Early window</p></td><td align="left"><p>2.858</p></td><td align="left"><p>0.004**</p></td><td align="left"><p>4.837</p></td><td align="left"><p><0.001***</p></td><td align="left"><p>8.120</p></td><td align="left"><p><0.001***</p></td></tr><tr><td align="left"><p>Late window</p></td><td align="left"><p>1.643</p></td><td align="left"><p>0.100</p></td><td align="left"><p>3.331</p></td><td align="left"><p>0.001**</p></td><td align="left"><p>8.062</p></td><td align="left"><p><0.001***</p></td></tr><tr><td align="left"><p>Baseline</p></td><td align="left"><p>− 1.861</p></td><td align="left"><p>0.063</p></td><td align="left"><p>− 0.054</p></td><td align="left"><p>0.957</p></td><td align="left"><p>3.413</p></td><td align="left"><p>0.001**</p></td></tr><tr><td align="left"><p>Control (rand.init., Full)</p></td><td align="left"><p>− 2.975</p></td><td align="left"><p>0.003**</p></td><td align="left"><p>− 0.831</p></td><td align="left"><p>0.406</p></td><td align="left"><p>0.058</p></td><td align="left"><p>0.954</p></td></tr><tr><td align="left"><p>Control (rand.init., Early)</p></td><td align="left"><p>− 3.313</p></td><td align="left"><p>0.001**</p></td><td align="left"><p>− 0.649</p></td><td align="left"><p>0.516</p></td><td align="left"><p>0.026</p></td><td align="left"><p>0.980</p></td></tr><tr><td align="left"><p>Control (rand.init., Late)</p></td><td align="left"><p>− 3.969</p></td><td align="left"><p><0.001***</p></td><td align="left"><p>− 0.919</p></td><td align="left"><p>0.358</p></td><td align="left"><p>0.030</p></td><td align="left"><p>0.976</p></td></tr></tbody></table> </ephtml> </p> <p>* <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>p</mi><mo><</mo><mo>.</mo><mn>05</mn></mrow></math> </ephtml> ; ** <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>p</mi><mo><</mo><mo>.</mo><mn>01</mn></mrow></math> </ephtml> ; *** <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>p</mi><mo><</mo><mo>.</mo><mn>001</mn></mrow></math> </ephtml> .</p> <p>For the three experimental groups (Full time, Early and Late window), the Jonckheere–Terpstra test revealed that the PEGASUS and T5 models exhibited a statistically significant monotonic increase in RSA scores as the number of layers deepened (<emph>Z</emph>> 0, <emph>p</emph> < 0.05). The BART model showed a statistically significant monotonic increase in RSA scores for Full time and Early window (<emph>Z</emph> > 0, <emph>p</emph> < 0.05), but for Late window, the increasing trend did not reach statistical significance (<emph>Z</emph> = 1.643, <emph>p</emph> = 0.100). Theses findings indicate that the neural networks of DLMs progressively learn LRs that more closely resemble those of the human brain. Through successive layers of abstraction and integration, these deep neural networks encode text in a way that increasingly parallels human cognitive processes. This evolution may be attributed to the models' shift from extracting superficial lexical features to integrating complex semantic information, aligning more closely with the mechanisms of human language comprehension[<reflink idref="bib23" id="ref65">23</reflink>].</p> <p>For the four control groups, the Jonckheere–Terpstra test showed no statistically significant monotonic trend in RSA scores as the layers deepened for the BART and PEGASUS models under Baseline. The T5 model exhibited a statistically significant monotonic increase (<emph>Z</emph> = 3.413, <emph>p</emph> = 0.001). Still, Fig. 5 reveals that its RSA scores were close to 0 with minimal fluctuation compared to other groups, indicating no apparent learning effect or similarity to human brain representations despite statistical significance. For the randomly initialized control groups (Control (rand.init., Full time, Early and Late)), the BART model showed a statistically significant monotonic decrease in RSA scores (<emph>Z</emph> < 0, <emph>p</emph> < 0.05), while the PEGASUS and T5 models showed no significant trend. This finding suggests that without specific training, deepening layers in DLMs leads to either decreased (BART) or unchanged (PEGASUS and T5) similarity between the model's hierarchical representations and human brain representations, underscoring the importance of appropriate training for learning brain-like representations.</p> <p>We conducted Friedman tests to determine if there were differences among the RSA scores of the three experimental groups (Full time, Early and Late window) for each model. The results revealed statistically significant differences for all models: BART ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msup><mi>χ</mi><mn>2</mn></msup><mrow><mo stretchy="false">(</mo><mn>2</mn><mo stretchy="false">)</mo></mrow></mrow></math> </ephtml> = 22.17, <emph>p</emph> < 0.001), PEGASUS ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msup><mi>χ</mi><mn>2</mn></msup><mrow><mo stretchy="false">(</mo><mn>2</mn><mo stretchy="false">)</mo></mrow></mrow></math> </ephtml> = 12.88, <emph>p</emph> = 0.002), and T5 ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msup><mi>χ</mi><mn>2</mn></msup><mrow><mo stretchy="false">(</mo><mn>2</mn><mo stretchy="false">)</mo></mrow></mrow></math> </ephtml> = 48.00, <emph>p</emph> < 0.001).</p> <p>Subsequently, we performed post hoc analysis using Wilcoxon signed-rank tests with a Bonferroni correction for multiple comparisons. This statistical analysis was also performed using SPSS (version 26). For the BART model, there were statistically significant differences in RSA scores between all conditions: Full time (<emph>Mdn</emph> = 0.0326) and Early window (<emph>Mdn</emph> = 0.0199) (<emph>p</emph> = 0.002), Full time and Late window (<emph>Mdn</emph> = 0.0226) (<emph>p</emph> = 0.002), and Early and Late window (<emph>p</emph> = 0.005). For the PEGASUS model, post hoc analysis revealed no statistically significant difference in RSA scores between Full time (<emph>Mdn</emph> = 0.0391) and Early window (<emph>Mdn</emph> = 0.0397) (<emph>p</emph> = 0.196). However, there were statistically significant differences between Full time and Late window (<emph>Mdn</emph> = 0.0238) (<emph>p</emph> = 0.013) and between Early and Late window (<emph>p</emph> = 0.001). Finally, for the T5 model, there were statistically significant differences in RSA scores between Full time window (<emph>Mdn</emph> = -0.0024 and Early window (<emph>Mdn</emph> = 0.0024) (<emph>p</emph> < 0.001), between Full time and Late window(<emph>Mdn</emph> = -0.0061) (<emph>p</emph> < 0.001), and between Early and Late window (<emph>p</emph> < 0.001).</p> <p>The results revealed distinct processing patterns across language models for various spatiotemporal windows of linguistic information. BART exhibited the highest similarity to human brain activity in comprehensive information processing (Full time window), followed by the P600-associated window (Late window), and lastly, the N400-associated window (Early window). This finding possibly indicates that BART excels in capturing overall language processing and potentially syntax-related information. PEGASUS demonstrated similar processing for comprehensive and N400-associated information but diverged significantly in the P600-associated window. This pattern may indicate consistent handling of general and potentially semantic-related information, with a distinct approach to syntax-related information. T5 showed the highest similarity in the N400-associated window, followed by the comprehensive condition, and lowest in the P600-associated window. This finding suggests that T5's processing patterns align more closely with human brain activity in spatiotemporal windows associated with the N400 component. These disparities likely reflect the architecture, pretraining methods, and task adaptability differences of DLMs. For instance, BART's sequence-to-sequence pretraining may enhance structural information processing, while PEGASUS's summarization-focused design may prioritize semantic information. The findings provide insights into how various language models process different components of human language comprehension.</p> <p>We also employed Wilcoxon signed-rank tests to determine whether significant differences existed between each model's three experimental conditions and their respective control conditions. The results are presented in Table 3. A positive <emph>Z</emph>-score indicates that the experimental condition surpassed the control condition. At the same time, a <emph>p</emph>-value < 0.05 denotes a statistically significant difference between the two.</p> <p>Table 3 Wilcoxon signed-rank test results for differences between experimental and control conditions across models.</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" rowspan="2"><p><bold>Comparison pairs</bold></p></th><th align="left" colspan="2"><p><bold>BART</bold></p></th><th align="left" colspan="2"><p><bold>PEGASUS</bold></p></th><th align="left" colspan="2"><p><bold>T5</bold></p></th></tr><tr><th align="left"><p><bold><italic>Z</italic></bold></p></th><th align="left"><p><bold><italic>p</italic></bold></p></th><th align="left"><p><bold><italic>Z</italic></bold></p></th><th align="left"><p><bold><italic>p</italic></bold></p></th><th align="left"><p><bold><italic>Z</italic></bold></p></th><th align="left"><p><bold><italic>p</italic></bold></p></th></tr></thead><tbody><tr><td align="left"><p>Full time window—baseline</p></td><td align="left"><p>3.059</p></td><td align="left"><p>0.002**</p></td><td align="left"><p>2.327</p></td><td align="left"><p>0.020*</p></td><td align="left"><p>2.857</p></td><td align="left"><p>0.004**</p></td></tr><tr><td align="left"><p>Full time window—control (rand.init., Full)</p></td><td align="left"><p>2.934</p></td><td align="left"><p>0.003**</p></td><td align="left"><p>3.516</p></td><td align="left"><p><0.001***</p></td><td align="left"><p>1.229</p></td><td align="left"><p>0.219</p></td></tr><tr><td align="left"><p>Early window—baseline</p></td><td align="left"><p>3.059</p></td><td align="left"><p>0.002**</p></td><td align="left"><p>2.689</p></td><td align="left"><p>0.007**</p></td><td align="left"><p>1.829</p></td><td align="left"><p>0.067</p></td></tr><tr><td align="left"><p>Early window—control (rand.init., Early)</p></td><td align="left"><p>3.059</p></td><td align="left"><p>0.002**</p></td><td align="left"><p>3.464</p></td><td align="left"><p>0.001**</p></td><td align="left"><p>4.286</p></td><td align="left"><p><0.001***</p></td></tr><tr><td align="left"><p>Late window—baseline</p></td><td align="left"><p>3.059</p></td><td align="left"><p>0.002**</p></td><td align="left"><p>1.813</p></td><td align="left"><p>0.056</p></td><td align="left"><p>3.743</p></td><td align="left"><p><0.001***</p></td></tr><tr><td align="left"><p>Late window—control (rand.init., Late)</p></td><td align="left"><p>2.981</p></td><td align="left"><p>0.003**</p></td><td align="left"><p>3.516</p></td><td align="left"><p><0.001***</p></td><td align="left"><p>3.771</p></td><td align="left"><p><0.001***</p></td></tr></tbody></table> </ephtml> </p> <p>* <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>p</mi><mo><</mo><mo>.</mo><mn>05</mn></mrow></math> </ephtml> *; ** <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>p</mi><mo><</mo><mo>.</mo><mn>01</mn></mrow></math> </ephtml> ; *** <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>p</mi><mo><</mo><mo>.</mo><mn>001</mn></mrow></math> </ephtml> .</p> <p>The results demonstrate that all experimental conditions for each model exceeded their corresponding control conditions (<emph>Z</emph> > 0), with the majority achieving statistical significance (<emph>p</emph> < 0.05). This finding suggests that trained language models can acquire LRs more akin to those of the human brain and, in most cases, significantly differentiate between language processing states (Full time, Early and Late window) and baseline states (Baseline). These findings underscore the crucial role of model training in attaining brain-like LRs.</p> <p>The RSA score comparisons that do not reach statistical significance include the PEGASUS model's Late window vs. Baseline, the T5 model's Full time window vs. Control (rand.init., Full), and Early window vs. Baseline. As illustrated in Fig. 5, in the shallower layers of each model, the experimental conditions' RSA scores are lower than those of the control conditions. However, as the layer depth increases, the experimental conditions' RSA scores consistently surpass those of the control conditions from a particular layer onwards. This phenomenon indicates that as the network deepens, the model progressively learns representations that better distinguish between language processing and baseline states. Baseline, serving as the baseline before stimulus presentation, represents a non-specific language processing state. As the layer depth increases, the models' similarity to language processing-related conditions (Full time, Early and Late window) increases, while the difference from the baseline condition becomes more apparent, suggesting that the models are learning features similar to human brain language processing. Concurrently, this phenomenon indicates that the effects of training are more pronounced in the deeper layers of the model, emphasizing that more profound layers of network structures play a more critical role in capturing the LRs of the human brain.</p> <p>Finally, we conducted two additional exploratory analyses. First, we performed an RSA across different encoder layers of the 3 DLMs. Specifically, we calculated the RSA scores between each pair of encoder layers for BART, PEGASUS, and T5 models. We generated three heatmap matrices based on these RSA scores, as illustrated in Supplementary Fig. S1. Supplementary Fig. S1a and Fig. S1c correspond to the inter-layer similarities between BART-PEGASUS, BART-T5, and PEGASUS-T5. We normalized the matrices to a square format to facilitate a comparison between models with different numbers of layers. This visualization approach revealed several interesting trends. For BART and PEGASUS, the RS between hidden layers initially increased with depth but decreased in the deeper layers. The representational similarity between BART and T5 hidden layers showed an increasing trend as the layer depth increased. Conversely, the RS between PEGASUS and T5 hidden layers exhibited a decreasing trend with increasing layer depth. These findings offer valuable insights that could inform future comparative analyses of representational patterns across different models.</p> <p>We performed two exploratory RSA-based analyses to comprehensively interpret the experimental results.</p> <p>In the first analysis, we calculated the RSA scores between each pair of encoder layers across the BART, PEGASUS, and T5 models. We visualized these scores as three heatmap matrices in Supplementary Fig. S1, normalizing them to a square format to facilitate comparison between models with different numbers of layers. The visualization revealed extensive regions of similarity (depicted in blue), indicating potential convergence in RPs across these DLMs. Notably, PEGASUS and T5 demonstrate higher inter-model RS than their respective similarities with BART, consistent with the visualization patterns observed in Figure 4. Furthermore, our analysis revealed significant correlations between the deeper hidden layers of all three DLMs and human brain activity patterns. These results support the Platonic Representation Hypothesis[<reflink idref="bib55" id="ref66">55</reflink>], which suggests that, given adequate training data and model capacity, neural networks naturally converge toward a common statistical representation of reality.</p> <p>In the second analysis, we calculated the correlation between the RSMs obtained under three conditions for each participant and then averaged the results across all participants. The results reveal a Spearman correlation of 0.439 between Full time and Early window, 0.466 between Full time and Late window, and 0.273 between Early and Late window. All 13 participants demonstrate statistically significant correlations (<emph>p</emph> < 0.05) across the three comparisons. These findings indicate high correlations among the RPs of EEG under the three conditions, notably higher than the RSA scores between human brain activity and DLMs. We posit that this outcome is reasonable. Firstly, it relates to the construction method of the EEG vectors. Full time window effectively encompasses all the information from Early and Late window, including time durations and electrode locations. Moreover, Early and Late window partially overlap, sharing specific EEG electrodes and being temporally separated by only 50ms. This inherent interconnection in the data structure may contribute to the high similarity between conditions. Secondly, this analysis compares different RPs within the same neural system (the human brain). In contrast, we compare patterns between two independent systems when we conduct RSA between human brain activity and DLMs. Therefore, it is reasonable that RSA scores across different EEG conditions are higher than those between brain activity and models. This reasoning also applies to the observation that inter-layer RSA scores within DLMs are higher than those between brain activity and DLMs, as all three DLMs are fundamentally based on the Transformer architecture, and we have extracted representations from their encoder layers. Lastly, previous research[<reflink idref="bib41" id="ref67">41</reflink>] analyzing the RS between human EEG signals and model embeddings has reported RSA scores (correlation coefficients) on the order of <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msup><mn>10</mn><mrow><mo>-</mo><mn>2</mn></mrow></msup></math> </ephtml> . To some extent, this finding supports the notion that this magnitude of correlation between human brain activity and model representations is indeed meaningful and valid.</p> <hd id="AN0182076388-11">Impact of layer manipulation on model performance</hd> <p>Tables 4, 5 and 6 present the Rouge metrics for three DLMs after conducting layer-wise attention ablation or noise addition, reflecting the models' performance on the ABS task. The notation "w/o" indicates that no manipulation was applied to any encoder layer, while "all" signifies that all encoder layers were processed. The bolded values represent the Rouge scores that showed the greatest performance decline after attention ablation or noise addition. Perplexity is a control variable for the correlation analysis in the following subsection. In language model evaluation, a lower perplexity score indicates that the model is better at predicting the text.</p> <p>Table 4 BART model performance after layer manipulation.</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p><bold>Ablated</bold><bold>layer</bold></p></th><th align="left"><p><bold>Rouge1</bold></p></th><th align="left"><p><bold>Rouge2</bold></p></th><th align="left"><p><bold>RougeL</bold></p></th><th align="left"><p><bold>Perplexity</bold></p></th><th align="left"><p><bold>Noisy</bold><bold>layer</bold></p></th><th align="left"><p><bold>Rouge1</bold></p></th><th align="left"><p><bold>Rouge2</bold></p></th><th align="left"><p><bold>RougeL</bold></p></th><th align="left"><p><bold>Perplexity</bold></p></th></tr></thead><tbody><tr><td align="left"><p>w/o</p></td><td align="left"><p>0.4570</p></td><td align="left"><p>0.2172</p></td><td align="left"><p>0.4839</p></td><td align="left"><p>6.5545</p></td><td align="left"><p>w/o</p></td><td align="left"><p>0.4570</p></td><td align="left"><p>0.2172</p></td><td align="left"><p>0.4839</p></td><td align="left"><p>6.5545</p></td></tr><tr><td align="left"><p>1</p></td><td align="left"><p>0.4536</p></td><td align="left"><p>0.2172</p></td><td align="left"><p>0.4213</p></td><td align="left"><p>6.9029</p></td><td align="left"><p>1</p></td><td align="left"><p>0.4571</p></td><td align="left"><p>0.2223</p></td><td align="left"><p>0.4249</p></td><td align="left"><p>6.5728</p></td></tr><tr><td align="left"><p>2</p></td><td align="left"><p>0.4476</p></td><td align="left"><p>0.2144</p></td><td align="left"><p>0.4159</p></td><td align="left"><p>7.4548</p></td><td align="left"><p>2</p></td><td align="left"><p>0.4557</p></td><td align="left"><p>0.2219</p></td><td align="left"><p>0.4239</p></td><td align="left"><p>6.6231</p></td></tr><tr><td align="left"><p>3</p></td><td align="left"><p>0.4497</p></td><td align="left"><p>0.2142</p></td><td align="left"><p>0.4171</p></td><td align="left"><p>7.7634</p></td><td align="left"><p>3</p></td><td align="left"><p>0.4545</p></td><td align="left"><p>0.2214</p></td><td align="left"><p>0.4224</p></td><td align="left"><p>6.6872</p></td></tr><tr><td align="left"><p>4</p></td><td align="left"><p>0.4514</p></td><td align="left"><p>0.2178</p></td><td align="left"><p>0.4194</p></td><td align="left"><p>6.8744</p></td><td align="left"><p>4</p></td><td align="left"><p>0.4577</p></td><td align="left"><p>0.2227</p></td><td align="left"><p>0.4256</p></td><td align="left"><p>6.5961</p></td></tr><tr><td align="left"><p>5</p></td><td align="left"><p>0.4443</p></td><td align="left"><p>0.2114</p></td><td align="left"><p>0.4127</p></td><td align="left"><p>7.3242</p></td><td align="left"><p>5</p></td><td align="left"><p>0.4551</p></td><td align="left"><p>0.2205</p></td><td align="left"><p>0.4227</p></td><td align="left"><p>6.7478</p></td></tr><tr><td align="left"><p>6</p></td><td align="left"><p>0.4349</p></td><td align="left"><p>0.2039</p></td><td align="left"><p>0.4041</p></td><td align="left"><p>7.6296</p></td><td align="left"><p>6</p></td><td align="left"><p>0.4507</p></td><td align="left"><p>0.2180</p></td><td align="left"><p>0.4188</p></td><td align="left"><p>6.9318</p></td></tr><tr><td align="left"><p>7</p></td><td align="left"><p>0.4417</p></td><td align="left"><p>0.2107</p></td><td align="left"><p>0.4102</p></td><td align="left"><p>7.2417</p></td><td align="left"><p>7</p></td><td align="left"><p>0.4540</p></td><td align="left"><p>0.2183</p></td><td align="left"><p>0.4218</p></td><td align="left"><p>6.8882</p></td></tr><tr><td align="left"><p>8</p></td><td align="left"><p>0.4301</p></td><td align="left"><p>0.2044</p></td><td align="left"><p>0.3993</p></td><td align="left"><p>7.5065</p></td><td align="left"><p>8</p></td><td align="left"><p>0.4501</p></td><td align="left"><p>0.2158</p></td><td align="left"><p>0.4182</p></td><td align="left"><p>6.9067</p></td></tr><tr><td align="left"><p>9</p></td><td align="left"><p>0.4356</p></td><td align="left"><p>0.2021</p></td><td align="left"><p>0.4039</p></td><td align="left"><p>7.8835</p></td><td align="left"><p>9</p></td><td align="left"><p>0.4507</p></td><td align="left"><p>0.2182</p></td><td align="left"><p>0.4192</p></td><td align="left"><p>6.7484</p></td></tr><tr><td align="left"><p>10</p></td><td align="left"><p>0.4409</p></td><td align="left"><p>0.2071</p></td><td align="left"><p>0.4090</p></td><td align="left"><p>7.2498</p></td><td align="left"><p>10</p></td><td align="left"><p>0.4517</p></td><td align="left"><p>0.2162</p></td><td align="left"><p>0.4207</p></td><td align="left"><p>6.3082</p></td></tr><tr><td align="left"><p>11</p></td><td align="left"><p>0.4337</p></td><td align="left"><p>0.1984</p></td><td align="left"><p>0.4029</p></td><td align="left"><p>6.5815</p></td><td align="left"><p>11</p></td><td align="left"><p><bold>0.4425</bold></p></td><td align="left"><p><bold>0.2106</bold></p></td><td align="left"><p><bold>0.4110</bold></p></td><td align="left"><p>6.4026</p></td></tr><tr><td align="left"><p>12</p></td><td align="left"><p><bold>0.3840</bold></p></td><td align="left"><p><bold>0.1677</bold></p></td><td align="left"><p><bold>0.3572</bold></p></td><td align="left"><p>6.9424</p></td><td align="left"><p>12</p></td><td align="left"><p>0.4491</p></td><td align="left"><p>0.2133</p></td><td align="left"><p>0.4174</p></td><td align="left"><p>5.9276</p></td></tr><tr><td align="left"><p>All</p></td><td align="left"><p>0.0549</p></td><td align="left"><p>0.0024</p></td><td align="left"><p>0.0527</p></td><td align="left"><p>1.1475</p></td><td align="left"><p>all</p></td><td align="left"><p>0.2653</p></td><td align="left"><p>0.0763</p></td><td align="left"><p>0.2444</p></td><td align="left"><p>10.0152</p></td></tr></tbody></table> </ephtml> </p> <p>Table 5 PEGASUS model performance after layer manipulation.</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p><bold>Ablated</bold><bold>layer</bold></p></th><th align="left"><p><bold>Rouge1</bold></p></th><th align="left"><p><bold>Rouge2</bold></p></th><th align="left"><p><bold>RougeL</bold></p></th><th align="left"><p><bold>Perplexity</bold></p></th><th align="left"><p><bold>Noisy</bold><bold>layer</bold></p></th><th align="left"><p><bold>Rouge1</bold></p></th><th align="left"><p><bold>Rouge2</bold></p></th><th align="left"><p><bold>RougeL</bold></p></th><th align="left"><p><bold>Perplexity</bold></p></th></tr></thead><tbody><tr><td align="left"><p>w/o</p></td><td align="left"><p>0.4835</p></td><td align="left"><p>0.2485</p></td><td align="left"><p>0.3978</p></td><td align="left"><p>4.4455</p></td><td align="left"><p>w/o</p></td><td align="left"><p>0.4835</p></td><td align="left"><p>0.2485</p></td><td align="left"><p>0.3978</p></td><td align="left"><p>4.4455</p></td></tr><tr><td align="left"><p>1</p></td><td align="left"><p>0.4839</p></td><td align="left"><p>0.2482</p></td><td align="left"><p>0.3981</p></td><td align="left"><p>3.8162</p></td><td align="left"><p>1</p></td><td align="left"><p>0.4843</p></td><td align="left"><p>0.2487</p></td><td align="left"><p>0.3982</p></td><td align="left"><p>5.8081</p></td></tr><tr><td align="left"><p>2</p></td><td align="left"><p>0.4837</p></td><td align="left"><p>0.2482</p></td><td align="left"><p>0.3975</p></td><td align="left"><p>4.6588</p></td><td align="left"><p>2</p></td><td align="left"><p>0.4836</p></td><td align="left"><p>0.2484</p></td><td align="left"><p>0.3979</p></td><td align="left"><p>4.5298</p></td></tr><tr><td align="left"><p>3</p></td><td align="left"><p>0.4816</p></td><td align="left"><p>0.2463</p></td><td align="left"><p>0.3943</p></td><td align="left"><p>4.4042</p></td><td align="left"><p>3</p></td><td align="left"><p>0.4833</p></td><td align="left"><p>0.2479</p></td><td align="left"><p>0.3973</p></td><td align="left"><p>4.4933</p></td></tr><tr><td align="left"><p>4</p></td><td align="left"><p>0.4818</p></td><td align="left"><p>0.2462</p></td><td align="left"><p>0.3955</p></td><td align="left"><p>4.6315</p></td><td align="left"><p>4</p></td><td align="left"><p>0.4838</p></td><td align="left"><p>0.2485</p></td><td align="left"><p>0.3976</p></td><td align="left"><p>4.5074</p></td></tr><tr><td align="left"><p>5</p></td><td align="left"><p>0.4827</p></td><td align="left"><p>0.2478</p></td><td align="left"><p>0.3967</p></td><td align="left"><p>4.4057</p></td><td align="left"><p>5</p></td><td align="left"><p>0.4844</p></td><td align="left"><p>0.2488</p></td><td align="left"><p>0.3980</p></td><td align="left"><p>4.4795</p></td></tr><tr><td align="left"><p>6</p></td><td align="left"><p>0.4813</p></td><td align="left"><p>0.2466</p></td><td align="left"><p>0.3957</p></td><td align="left"><p>4.5811</p></td><td align="left"><p>6</p></td><td align="left"><p>0.4834</p></td><td align="left"><p>0.2488</p></td><td align="left"><p>0.3974</p></td><td align="left"><p>4.5506</p></td></tr><tr><td align="left"><p>7</p></td><td align="left"><p>0.4828</p></td><td align="left"><p>0.2481</p></td><td align="left"><p>0.3974</p></td><td align="left"><p>4.8670</p></td><td align="left"><p>7</p></td><td align="left"><p>0.4837</p></td><td align="left"><p>0.2490</p></td><td align="left"><p>0.3984</p></td><td align="left"><p>4.5029</p></td></tr><tr><td align="left"><p>8</p></td><td align="left"><p>0.4816</p></td><td align="left"><p>0.2472</p></td><td align="left"><p>0.3959</p></td><td align="left"><p>4.7064</p></td><td align="left"><p>8</p></td><td align="left"><p>0.4824</p></td><td align="left"><p>0.2477</p></td><td align="left"><p>0.3966</p></td><td align="left"><p>4.5299</p></td></tr><tr><td align="left"><p>9</p></td><td align="left"><p>0.4813</p></td><td align="left"><p>0.2465</p></td><td align="left"><p>0.3957</p></td><td align="left"><p>4.2938</p></td><td align="left"><p>9</p></td><td align="left"><p>0.4817</p></td><td align="left"><p>0.2468</p></td><td align="left"><p>0.3955</p></td><td align="left"><p>4.5912</p></td></tr><tr><td align="left"><p>10</p></td><td align="left"><p>0.4819</p></td><td align="left"><p>0.2469</p></td><td align="left"><p>0.3961</p></td><td align="left"><p>4.7257</p></td><td align="left"><p>10</p></td><td align="left"><p>0.4813</p></td><td align="left"><p>0.2466</p></td><td align="left"><p>0.3956</p></td><td align="left"><p>4.9480</p></td></tr><tr><td align="left"><p>11</p></td><td align="left"><p><bold>0.4790</bold></p></td><td align="left"><p><bold>0.2440</bold></p></td><td align="left"><p>0.3926</p></td><td align="left"><p>4.6511</p></td><td align="left"><p>11</p></td><td align="left"><p>0.4803</p></td><td align="left"><p>0.2447</p></td><td align="left"><p>0.3942</p></td><td align="left"><p>4.6066</p></td></tr><tr><td align="left"><p>12</p></td><td align="left"><p>0.4811</p></td><td align="left"><p>0.2459</p></td><td align="left"><p>0.3950</p></td><td align="left"><p>4.7428</p></td><td align="left"><p>12</p></td><td align="left"><p>0.4812</p></td><td align="left"><p>0.2460</p></td><td align="left"><p>0.3950</p></td><td align="left"><p>4.5174</p></td></tr><tr><td align="left"><p>13</p></td><td align="left"><p>0.4801</p></td><td align="left"><p>0.2448</p></td><td align="left"><p>0.3938</p></td><td align="left"><p>4.7486</p></td><td align="left"><p>13</p></td><td align="left"><p>0.4793</p></td><td align="left"><p>0.2449</p></td><td align="left"><p>0.3934</p></td><td align="left"><p>4.4523</p></td></tr><tr><td align="left"><p>14</p></td><td align="left"><p>0.4794</p></td><td align="left"><p>0.2442</p></td><td align="left"><p><bold>0.3926</bold></p></td><td align="left"><p>4.5026</p></td><td align="left"><p>14</p></td><td align="left"><p>0.4797</p></td><td align="left"><p>0.2448</p></td><td align="left"><p>0.3938</p></td><td align="left"><p>4.4388</p></td></tr><tr><td align="left"><p>15</p></td><td align="left"><p>0.4809</p></td><td align="left"><p>0.2461</p></td><td align="left"><p>0.3940</p></td><td align="left"><p>4.5126</p></td><td align="left"><p>15</p></td><td align="left"><p>0.4788</p></td><td align="left"><p>0.2433</p></td><td align="left"><p>0.3916</p></td><td align="left"><p>4.4305</p></td></tr><tr><td align="left"><p>16</p></td><td align="left"><p>0.4801</p></td><td align="left"><p>0.2452</p></td><td align="left"><p>0.3935</p></td><td align="left"><p>4.6827</p></td><td align="left"><p>16</p></td><td align="left"><p><bold>0.4774</bold></p></td><td align="left"><p><bold>0.2419</bold></p></td><td align="left"><p><bold>0.3910</bold></p></td><td align="left"><p>4.5789</p></td></tr><tr><td align="left"><p>All</p></td><td align="left"><p>0.1790</p></td><td align="left"><p>0.0275</p></td><td align="left"><p>0.1351</p></td><td align="left"><p>4.2754</p></td><td align="left"><p>all</p></td><td align="left"><p>0.4364</p></td><td align="left"><p>0.2011</p></td><td align="left"><p>0.3459</p></td><td align="left"><p>6.5883</p></td></tr></tbody></table> </ephtml> </p> <p>Table 6 T5 model performance after layer manipulation.</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p><bold>Ablated</bold><bold>layer</bold></p></th><th align="left"><p><bold>Rouge1</bold></p></th><th align="left"><p><bold>Rouge2</bold></p></th><th align="left"><p><bold>RougeL</bold></p></th><th align="left"><p><bold>Perplexity</bold></p></th><th align="left"><p><bold>Noisy</bold><bold>layer</bold></p></th><th align="left"><p><bold>Rouge1</bold></p></th><th align="left"><p><bold>Rouge2</bold></p></th><th align="left"><p><bold>RougeL</bold></p></th><th align="left"><p><bold>Perplexity</bold></p></th></tr></thead><tbody><tr><td align="left"><p>w/o</p></td><td align="left"><p>0.3600</p></td><td align="left"><p>0.1432</p></td><td align="left"><p>0.2881</p></td><td align="left"><p>4.2509</p></td><td align="left"><p>w/o</p></td><td align="left"><p>0.3600</p></td><td align="left"><p>0.1432</p></td><td align="left"><p>0.2881</p></td><td align="left"><p>4.2509</p></td></tr><tr><td align="left"><p>1</p></td><td align="left"><p>0.3579</p></td><td align="left"><p>0.1443</p></td><td align="left"><p>0.2892</p></td><td align="left"><p>4.4911</p></td><td align="left"><p>1</p></td><td align="left"><p>0.3598</p></td><td align="left"><p>0.1428</p></td><td align="left"><p>0.2880</p></td><td align="left"><p>5.2915</p></td></tr><tr><td align="left"><p>2</p></td><td align="left"><p>0.3620</p></td><td align="left"><p>0.1442</p></td><td align="left"><p>0.2901</p></td><td align="left"><p>6.5301</p></td><td align="left"><p>2</p></td><td align="left"><p>0.3596</p></td><td align="left"><p>0.1425</p></td><td align="left"><p>0.2879</p></td><td align="left"><p>4.6982</p></td></tr><tr><td align="left"><p>3</p></td><td align="left"><p>0.3586</p></td><td align="left"><p>0.1415</p></td><td align="left"><p>0.2865</p></td><td align="left"><p>4.9183</p></td><td align="left"><p>3</p></td><td align="left"><p>0.3597</p></td><td align="left"><p>0.1424</p></td><td align="left"><p>0.2877</p></td><td align="left"><p>4.8546</p></td></tr><tr><td align="left"><p>4</p></td><td align="left"><p>0.3575</p></td><td align="left"><p>0.1408</p></td><td align="left"><p>0.2859</p></td><td align="left"><p>5.0020</p></td><td align="left"><p>4</p></td><td align="left"><p>0.3591</p></td><td align="left"><p>0.1425</p></td><td align="left"><p>0.2874</p></td><td align="left"><p>4.7872</p></td></tr><tr><td align="left"><p>5</p></td><td align="left"><p>0.3605</p></td><td align="left"><p>0.1432</p></td><td align="left"><p>0.2887</p></td><td align="left"><p>4.8580</p></td><td align="left"><p>5</p></td><td align="left"><p>0.3589</p></td><td align="left"><p>0.1423</p></td><td align="left"><p>0.2873</p></td><td align="left"><p>4.7402</p></td></tr><tr><td align="left"><p>6</p></td><td align="left"><p>0.3596</p></td><td align="left"><p>0.1425</p></td><td align="left"><p>0.2877</p></td><td align="left"><p>4.7825</p></td><td align="left"><p>6</p></td><td align="left"><p>0.3588</p></td><td align="left"><p>0.1423</p></td><td align="left"><p>0.2871</p></td><td align="left"><p>4.9763</p></td></tr><tr><td align="left"><p>7</p></td><td align="left"><p>0.3591</p></td><td align="left"><p>0.1423</p></td><td align="left"><p>0.2875</p></td><td align="left"><p>4.9920</p></td><td align="left"><p>7</p></td><td align="left"><p>0.3596</p></td><td align="left"><p>0.1429</p></td><td align="left"><p>0.2883</p></td><td align="left"><p>4.8143</p></td></tr><tr><td align="left"><p>8</p></td><td align="left"><p>0.3583</p></td><td align="left"><p>0.1422</p></td><td align="left"><p>0.2870</p></td><td align="left"><p>4.6951</p></td><td align="left"><p>8</p></td><td align="left"><p>0.3581</p></td><td align="left"><p>0.1423</p></td><td align="left"><p>0.2871</p></td><td align="left"><p>4.7670</p></td></tr><tr><td align="left"><p>9</p></td><td align="left"><p>0.3624</p></td><td align="left"><p>0.1453</p></td><td align="left"><p>0.2909</p></td><td align="left"><p>4.5493</p></td><td align="left"><p>9</p></td><td align="left"><p>0.3597</p></td><td align="left"><p>0.1427</p></td><td align="left"><p>0.2878</p></td><td align="left"><p>4.7384</p></td></tr><tr><td align="left"><p>10</p></td><td align="left"><p>0.3603</p></td><td align="left"><p>0.1434</p></td><td align="left"><p>0.2889</p></td><td align="left"><p>4.5590</p></td><td align="left"><p>10</p></td><td align="left"><p>0.3581</p></td><td align="left"><p>0.1418</p></td><td align="left"><p>0.2865</p></td><td align="left"><p>4.9308</p></td></tr><tr><td align="left"><p>11</p></td><td align="left"><p>0.3600</p></td><td align="left"><p>0.1430</p></td><td align="left"><p>0.2880</p></td><td align="left"><p>4.8998</p></td><td align="left"><p>11</p></td><td align="left"><p>0.3596</p></td><td align="left"><p>0.1430</p></td><td align="left"><p>0.2879</p></td><td align="left"><p>4.8856</p></td></tr><tr><td align="left"><p>12</p></td><td align="left"><p>0.3607</p></td><td align="left"><p>0.1434</p></td><td align="left"><p>0.2886</p></td><td align="left"><p>5.1357</p></td><td align="left"><p>12</p></td><td align="left"><p>0.3589</p></td><td align="left"><p>0.1427</p></td><td align="left"><p>0.2875</p></td><td align="left"><p>4.6917</p></td></tr><tr><td align="left"><p>13</p></td><td align="left"><p>0.3582</p></td><td align="left"><p>0.1413</p></td><td align="left"><p>0.2865</p></td><td align="left"><p>4.7659</p></td><td align="left"><p>13</p></td><td align="left"><p>0.3575</p></td><td align="left"><p>0.1413</p></td><td align="left"><p>0.2864</p></td><td align="left"><p>4.9292</p></td></tr><tr><td align="left"><p>14</p></td><td align="left"><p>0.3590</p></td><td align="left"><p>0.1425</p></td><td align="left"><p>0.2877</p></td><td align="left"><p>4.9346</p></td><td align="left"><p>14</p></td><td align="left"><p>0.3574</p></td><td align="left"><p>0.1414</p></td><td align="left"><p>0.2862</p></td><td align="left"><p>4.7829</p></td></tr><tr><td align="left"><p>15</p></td><td align="left"><p>0.3574</p></td><td align="left"><p>0.1414</p></td><td align="left"><p>0.2859</p></td><td align="left"><p>5.0197</p></td><td align="left"><p>15</p></td><td align="left"><p>0.3587</p></td><td align="left"><p>0.1424</p></td><td align="left"><p>0.2869</p></td><td align="left"><p>5.0375</p></td></tr><tr><td align="left"><p>16</p></td><td align="left"><p>0.3567</p></td><td align="left"><p>0.1401</p></td><td align="left"><p>0.2853</p></td><td align="left"><p>5.0489</p></td><td align="left"><p>16</p></td><td align="left"><p><bold>0.3571</bold></p></td><td align="left"><p>0.1413</p></td><td align="left"><p>0.2860</p></td><td align="left"><p>4.8089</p></td></tr><tr><td align="left"><p>17</p></td><td align="left"><p>0.3553</p></td><td align="left"><p>0.1398</p></td><td align="left"><p>0.2842</p></td><td align="left"><p>5.1154</p></td><td align="left"><p>17</p></td><td align="left"><p>0.3580</p></td><td align="left"><p>0.1415</p></td><td align="left"><p>0.2864</p></td><td align="left"><p>4.8560</p></td></tr><tr><td align="left"><p>18</p></td><td align="left"><p>0.3568</p></td><td align="left"><p>0.1408</p></td><td align="left"><p>0.2852</p></td><td align="left"><p>5.5020</p></td><td align="left"><p>18</p></td><td align="left"><p>0.3582</p></td><td align="left"><p>0.1417</p></td><td align="left"><p>0.2864</p></td><td align="left"><p>5.1116</p></td></tr><tr><td align="left"><p>19</p></td><td align="left"><p>0.3536</p></td><td align="left"><p>0.1385</p></td><td align="left"><p>0.2823</p></td><td align="left"><p>5.4765</p></td><td align="left"><p>19</p></td><td align="left"><p>0.3579</p></td><td align="left"><p>0.1412</p></td><td align="left"><p>0.2866</p></td><td align="left"><p>4.9381</p></td></tr><tr><td align="left"><p>20</p></td><td align="left"><p><bold>0.3535</bold></p></td><td align="left"><p><bold>0.1379</bold></p></td><td align="left"><p><bold>0.2818</bold></p></td><td align="left"><p>5.2662</p></td><td align="left"><p>20</p></td><td align="left"><p>0.3595</p></td><td align="left"><p>0.1423</p></td><td align="left"><p>0.2878</p></td><td align="left"><p>4.9578</p></td></tr><tr><td align="left"><p>21</p></td><td align="left"><p>0.3549</p></td><td align="left"><p>0.1392</p></td><td align="left"><p>0.2837</p></td><td align="left"><p>4.5478</p></td><td align="left"><p>21</p></td><td align="left"><p>0.3581</p></td><td align="left"><p>0.1418</p></td><td align="left"><p>0.2866</p></td><td align="left"><p>4.5155</p></td></tr><tr><td align="left"><p>22</p></td><td align="left"><p>0.3556</p></td><td align="left"><p>0.1394</p></td><td align="left"><p>0.2842</p></td><td align="left"><p>4.3239</p></td><td align="left"><p>22</p></td><td align="left"><p>0.3565</p></td><td align="left"><p><bold>0.1405</bold></p></td><td align="left"><p><bold>0.2851</bold></p></td><td align="left"><p>5.3488</p></td></tr><tr><td align="left"><p>23</p></td><td align="left"><p>0.3551</p></td><td align="left"><p>0.1387</p></td><td align="left"><p>0.2836</p></td><td align="left"><p>4.9647</p></td><td align="left"><p>23</p></td><td align="left"><p>0.3595</p></td><td align="left"><p>0.1427</p></td><td align="left"><p>0.2877</p></td><td align="left"><p>4.6696</p></td></tr><tr><td align="left"><p>24</p></td><td align="left"><p>0.3581</p></td><td align="left"><p>0.1395</p></td><td align="left"><p>0.2864</p></td><td align="left"><p>5.6963</p></td><td align="left"><p>24</p></td><td align="left"><p>0.3593</p></td><td align="left"><p>0.1423</p></td><td align="left"><p>0.2876</p></td><td align="left"><p>4.9935</p></td></tr><tr><td align="left"><p>All</p></td><td align="left"><p>0.1427</p></td><td align="left"><p>0.0139</p></td><td align="left"><p>0.1111</p></td><td align="left"><p>3.7422</p></td><td align="left"><p>all</p></td><td align="left"><p>0.3396</p></td><td align="left"><p>0.1294</p></td><td align="left"><p>0.2717</p></td><td align="left"><p>5.0026</p></td></tr></tbody></table> </ephtml> </p> <p>Figure 6 graphically demonstrates the impact of ablating attention and noise addition at each layer on the ABS task, with negative numbers indicating performance improvements. This subsection primarily describes experimental results to lay the groundwork for analysis in the following subsection. For BART, attention ablation in any encoder layer decreases all Rouge metrics, most significantly in the final layer. Noise introduction causes the most significant decline at layer 11. Interestingly, layer 4 shows slight increases in Rouge1 and RougeL. From layer 3 onwards, attention ablation degrades performance in PEGASUS, with deeper layers having a more significant impact. The 11th layer's ablation most significantly reduces Rouge1 and Rouge2. The 14th layer's uniform attention distribution causes the most significant RougeL decrease. Noise introduction from layer 8 onwards reduces all Rouge metrics, peaking at the final layer. For T5, performance degradation due to attention ablation begins at layer 13. Earlier layers show performance fluctuations, with some improving all Rouge metrics. From layers 13 to 24, performance consistently declines, maximally at layer 20. Noise introduction generally decreases all Rouge metrics (except RougeL in the lower seven layers), with peak drops at layers 16 (Rouge1) and 22 (Rouge2 and RougeL).</p> <p>Graph: Figure 6 Impact of layer-wise attention ablation and noise addition on ABS task performance.</p> <p>Notably, the changes in Rouges for the PEGASUS and T5 models are considerably more minor than those observed in the BART model, suggesting that the increased number of layers in PEGASUS and T5 enhances performance stability. However, this does not imply that more layers invariably leads to better performance. For instance, after setting the attention of layer 9 in the T5 model to a uniform distribution, a considerable performance improvement was observed. This suggests that the attention mechanism in this layer may have previously interfered with the effective extraction and integration of crucial information, indicating that not every hidden layer positively contributes to the task of ABS.</p> <hd id="AN0182076388-12">Correlation between RS and model performance</hd> <p>This subsection presents the experimental results based on a correlation analysis conducted to test the research hypothesis using data from the previous two subsections. Figure 7 and Supplementary Fig. S2 and Fig. S3 illustrate the correlation between model performance (Rouge1/2/L) decline and the RSA scores of each encoder hidden layer for the three ABS models. Each subplot represents the results of correlation analysis under a specific experimental condition of RSA. Specifically, each subplot contains two sets of regression lines and corresponding scatter plots: solid lines with solid scatter points represent the effects of attention ablation (setting each layer's attention to a uniform distribution). In contrast, dashed lines with diamond scatter points depict the effects of adding noise to each layer. Both sets of regression lines share the same x-axis (RSA scores) but utilize different y-axis. The left y-axis corresponds to the former, representing the performance decline due to attention ablation (Rouge1/2/L decline (a)). The right y-axis corresponds to the latter, indicating the performance decline from noise addition (Rouge1/2/L decline (n)). For example, with its 12 encoder layers, the BART model is represented by 24 data points: 12 circle points for attention ablation and 12 diamond points for noise addition, each corresponding to a specific layer. The shaded areas surrounding each regression line represent the 95% confidence intervals (95%<emph>CI</emph>). the Spearman correlation coefficients presented in this subsection were calculated using Python 3.7.9. The '<emph>r</emph>' value in the figure represents the Spearman correlation coefficient, and '<emph>p</emph>' denotes the significance level of the correlation, with a <emph>p</emph>-value < 0.05 signifying a statistically significant correlation. To better focus on the main findings of our study, we have opted to include only the figures and tables related to Rouge1 in the main text. The Supplementary Fig. S2 and Fig. S3 and tables showing results for Rouge2 and RougeL can be found in the Appendix.</p> <p>Figure 7 demonstrates that for the three DLMs, the RSA scores calculated under three conditions show a significant positive correlation (<emph>p</emph> < 0.05) with the model performance decline (Rouge1) caused by two types of encoder layer operations. Similar results are observed for Rouge2 and RougeL in Supplementary Fig. S2 and Fig. S3. In other words, hidden layers with LRs more closely aligned to the human brain exhibit more pronounced model performance deterioration when manipulated. This finding supports our hypothesis that the hidden layers of the model that exhibit a higher degree of RS to the human brain's activation patterns may play a more critical role in the model's performance on the ABS task.</p> <p>Graph: Figure 7 Correlation between RSA scores and Rouge1 declines in BART, PEGASUS, and T5 models.</p> <p>To enhance the preciseness of our findings, we used approximate non-parametric partial correlation analysis to control for perplexity's influence. We first calculated Spearman correlations among Rouge, RSA, and perplexity. To remove perplexity's impact, we computed partial correlations between Rouge and RSA using linear regression to estimate perplexity's effect, then calculated residuals. Finally, we determined Spearman correlations between these residuals, yielding partial correlation coefficients. This method excluded potential confounding effects from perplexity, allowing for a more accurate assessment of relationships between variables.</p> <p>Table 7 presents the results of partial correlation analysis for Rouge1 scores. The study reveals that, even when controlling for perplexity, the RSA scores maintain a significant positive correlation (<emph>p</emph> < 0.05) with the performance decline caused by two types of encoder layer operations across all three experimental conditions for the three DLMs. Supplementary Table S2 and Table S3 show essentially consistent results for Rouge2 and RougeL, except the positive correlation between RSA scores and Rouge2 decline for the T5 model in Full time window narrowly missed statistical significance (<emph>p</emph> = 0.051). These findings unveil a deep structure-function relationship that transcends the models' general language understanding capabilities while emphasizing the crucial role of specific neural network layers in the ABS task. These layers may perform functions analogous to the human brain in language compression and key information extraction processes.</p> <p>Table 7 Partial correlation analysis of RSA scores and Rouge1 performance decline.</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" rowspan="2"><p>Conditions</p></th><th align="left" colspan="2"><p>Full time window</p></th><th align="left" colspan="2"><p>Early window</p></th><th align="left" colspan="2"><p>Late window</p></th></tr><tr><th align="left"><p><italic>r'</italic></p></th><th align="left"><p><italic>p'</italic></p></th><th align="left"><p><italic>r'</italic></p></th><th align="left"><p><italic>p'</italic></p></th><th align="left"><p><italic>r'</italic></p></th><th align="left"><p><italic>p'</italic></p></th></tr></thead><tbody><tr><td align="left"><p>BART(a)</p></td><td align="left"><p>0.804</p></td><td align="left"><p>0.002**</p></td><td align="left"><p>0.790</p></td><td align="left"><p>0.002**</p></td><td align="left"><p>0.734</p></td><td align="left"><p>0.007**</p></td></tr><tr><td align="left"><p>BART(n)</p></td><td align="left"><p>0.790</p></td><td align="left"><p>0.002**</p></td><td align="left"><p>0.776</p></td><td align="left"><p>0.003**</p></td><td align="left"><p>0.776</p></td><td align="left"><p>0.003**</p></td></tr><tr><td align="left"><p>PEGASUS(a)</p></td><td align="left"><p>0.900</p></td><td align="left"><p><0.001***</p></td><td align="left"><p>0.935</p></td><td align="left"><p><0.001***</p></td><td align="left"><p>0.897</p></td><td align="left"><p><0.001***</p></td></tr><tr><td align="left"><p>PEGASUS(n)</p></td><td align="left"><p>0.744</p></td><td align="left"><p><0.001***</p></td><td align="left"><p>0.788</p></td><td align="left"><p><0.001***</p></td><td align="left"><p>0.718</p></td><td align="left"><p>0.002**</p></td></tr><tr><td align="left"><p>T5(a)</p></td><td align="left"><p>0.723</p></td><td align="left"><p><0.001***</p></td><td align="left"><p>0.737</p></td><td align="left"><p><0.001***</p></td><td align="left"><p>0.722</p></td><td align="left"><p><0.001***</p></td></tr><tr><td align="left"><p>T5(n)</p></td><td align="left"><p>0.434</p></td><td align="left"><p>0.034*</p></td><td align="left"><p>0.454</p></td><td align="left"><p>0.026*</p></td><td align="left"><p>0.447</p></td><td align="left"><p>0.029*</p></td></tr></tbody></table> </ephtml> </p> <p>Notably, this correlation persists across different model architectures (BART, PEGASUS, T5), various conditions of human brain LR (Full time, Early and Late window), different perturbation methods (attention ablation, noise injection), and multiple performance metrics (Rouge1, 2, L). This consistency and robustness suggest the possible existence of universal neural computational principles independent of specific model implementations.</p> <p>To comprehensively evaluate model performance, we invite five native speakers to manually score summaries generated by BART and PEGASUS models. We randomly select ten articles from the test set to assess the model summaries after attention ablation. The evaluation metrics include informativeness, readability, and factual consistency, with scores ranging from 1 to 5, where higher scores indicate better performance. Detailed evaluation criteria are presented in Supplementary Table S4. We do not evaluate the T5 model, as it generates 25 summaries per article, which could lead to evaluator fatigue and introduce errors. We calculate the correlation between the three metrics of model summaries and the RSA scores of affected hidden layers and the human brain, with results shown in Table 8 (BART) and Table 9 (PEGASUS). For BART, the three metrics show negative correlations with RSA scores under all three conditions, although these correlations do not reach statistical significance. This result suggests that model performance tends to decline when hidden layers more similar to human brain representations are disrupted. PEGASUS exhibits identical trends in Full time and Early window, while in Late window, RSA scores are only moderately negatively correlated with readability.</p> <p>Table 8 Correlation Between the RSA Scores and Manual Ratings of BART Model.</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" rowspan="2"><p>Conditions</p></th><th align="left" colspan="2"><p>Informativeness</p></th><th align="left" colspan="2"><p>Readability</p></th><th align="left" colspan="2"><p>Factual consistency</p></th></tr><tr><th align="left"><p><italic>r</italic></p></th><th align="left"><p><italic>p</italic></p></th><th align="left"><p><italic>r</italic></p></th><th align="left"><p><italic>p</italic></p></th><th align="left"><p><italic>r</italic></p></th><th align="left"><p><italic>p</italic></p></th></tr></thead><tbody><tr><td align="left"><p>Overall</p></td><td align="left"><p>-0.455</p></td><td align="left"><p>0.137</p></td><td align="left"><p>-0.336</p></td><td align="left"><p>0.285</p></td><td align="left"><p>-0.343</p></td><td align="left"><p>0.276</p></td></tr><tr><td align="left"><p>N400</p></td><td align="left"><p>-0.438</p></td><td align="left"><p>0.155</p></td><td align="left"><p>-0.336</p></td><td align="left"><p>0.285</p></td><td align="left"><p>-0.336</p></td><td align="left"><p>0.286</p></td></tr><tr><td align="left"><p>P600</p></td><td align="left"><p>-0.328</p></td><td align="left"><p>0.301</p></td><td align="left"><p>-0.235</p></td><td align="left"><p>0.463</p></td><td align="left"><p>-0.266</p></td><td align="left"><p>0.404</p></td></tr></tbody></table> </ephtml> </p> <p>Table 9 Correlation Between the RSA Scores and Manual Ratings of PEGASUS Model.</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" rowspan="2"><p>Conditions</p></th><th align="left" colspan="2"><p>Informativeness</p></th><th align="left" colspan="2"><p>Readability</p></th><th align="left" colspan="2"><p>Factual Consistency</p></th></tr><tr><th align="left"><p><italic>r</italic></p></th><th align="left"><p><italic>p</italic></p></th><th align="left"><p><italic>r</italic></p></th><th align="left"><p><italic>p</italic></p></th><th align="left"><p><italic>r</italic></p></th><th align="left"><p><italic>p</italic></p></th></tr></thead><tbody><tr><td align="left"><p>Overall</p></td><td align="left"><p>-0.112</p></td><td align="left"><p>0.679</p></td><td align="left"><p>-0.308</p></td><td align="left"><p>0.245</p></td><td align="left"><p>-0.091</p></td><td align="left"><p>0.736</p></td></tr><tr><td align="left"><p>N400</p></td><td align="left"><p>-0.131</p></td><td align="left"><p>0.628</p></td><td align="left"><p>-0.358</p></td><td align="left"><p>0.173</p></td><td align="left"><p>-0.109</p></td><td align="left"><p>0.687</p></td></tr><tr><td align="left"><p>P600</p></td><td align="left"><p>0.003</p></td><td align="left"><p>0.991</p></td><td align="left"><p>-0.279</p></td><td align="left"><p>0.295</p></td><td align="left"><p>0.016</p></td><td align="left"><p>0.952</p></td></tr></tbody></table> </ephtml> </p> <p>Overall, the manual evaluation results align with the trends observed in Rouge evaluations (Rouge uses decreased values in evaluation, hence the positive correlation). Although the correlations in the manual assessment do not reach significant levels, they also support our hypothesis: hidden layers with representations similar to human brain language patterns play a crucial role in summarization tasks.</p> <hd id="AN0182076388-13">Discussion</hd> <p>This study proposes a central hypothesis: the hidden layers of a Deep Language Model (DLM) that exhibit a higher degree of Representational Similarity (RS) to the human brain's activation patterns may play a more critical role in the model's performance. We employed EEG experiments and Representational Similarity Analysis (RSA) to systematically investigate the similarity between the Language Representations (LRs) of hidden layers in Abstractive Summarization (ABS) models and human language understanding processes to validate this hypothesis. We further explored the correlation between this brain-model similarity and model performance through layer-wise attention ablation and noise addition. The experimental results consistently supported our hypothesis across diverse testing conditions, demonstrating robust evidence for our claim.</p> <p>Through the RSA of three representative ABS models, we derived several findings that hold significant implications for understanding the internal LR mechanisms of DLMs and their relationship to human language cognition. We observed a notable trend: as the model layers deepened, the similarity between encoder layers and human brain language processing generally showed an upward trend, reaching statistical significance (<emph>p</emph> < 0.05) under most experimental conditions. However, this finding differs from some previous studies. For instance, research[<reflink idref="bib56" id="ref68">56</reflink>] found that the 9th layer (out of 12) of GPT-2 was most similar to human brain activity. At the same time[<reflink idref="bib10" id="ref69">10</reflink>], it was discovered that the middle layers (6 or 7 out of 12) of the BERT model showed the highest similarity to human brain activity. This apparent discrepancy reflects several issues worthy of in-depth exploration. Firstly, although our experimental results show a statistically significant monotonic increase in RSA scores between the model and the human brain as layer depth increases, this does not preclude the possibility of local fluctuations in intermediate hidden layers. For example, under Full time window, the BART model showed a temporary decrease starting from the 6th layer, and the difference in RSA scores between the 6th layer and the 12th (final) layer was insignificant (<emph>p</emph> = 0.385). Focusing on individual layer comparisons rather than the overall trend might lead to different conclusions. Secondly, our study employed an encoder-decoder model fine-tuned on the ABS task to analogize the complete process of human language understanding and generation. In contrast, GPT-2 and BERT use decoder-only and encoder-only architectures, respectively. This task-specific architectural difference may result in different representation learning patterns. Moreover, the ABS task demands more robust semantic integration and information compression capabilities from the model, potentially driving deeper networks to form high-level LRs more akin to the human brain. Lastly, control experiments using randomly initialized models indicate that the pre-training and fine-tuning processes play a crucial role in shaping the model's representational characteristics. This finding underscores the importance of task-specific training in molding internal model representations. Thus, specific fine-tuning for the ABS task may alter the LRs of DLMs, causing deeper networks to focus more on language feature extraction relevant to the ABS task. Our findings help to reveal the complex interplay between DLM architecture, task-specific training, and models' similarity to human brain language processing. This study underscores the importance of considering these factors when comparing DLMs to human cognition, providing a foundation for future research in developing more human-like language models.</p> <p>The core findings of this study reveal that hidden layers in DLMs with higher similarity to human brain LRs have a more significant impact on model performance. This correlation demonstrates remarkable robustness across multiple dimensions: various DLM types (BART, PEGASUS, T5), different human brain LR conditions (Full time, Early and Late window), diverse layer manipulation methods (attention ablation, noise addition), and persists even after controlling for perplexity. The consistency and robustness of these findings warrant in-depth discussion from multiple perspectives. From the architectural standpoint of DLMs, although BART, PEGASUS, and T5 models differ in the number of hidden layers, training objectives, and fine-tuning datasets, they all share the encoder-decoder Transformer architecture as a common foundation. The shared underlying structure may preserve similar information processing mechanisms and representation learning patterns, potentially as a critical factor in the observed cross-model similarities. Considering the human brain LR conditions, while Full time, Early and Late window focus on different time windows and electrode locations, they likely capture partially overlapping language processing information. This spatiotemporal overlap in neural signals may contribute to the consistency across conditions. Moreover, DLMs may simultaneously learn LRs corresponding to multiple stages of brain processing, enabling them to exhibit similarities with human brain patterns across different conditions. From the perspective of layer-wise operations, layers with LRs more closely aligned to human brain language patterns may perform specific functions crucial to the ABS task, such as high-level semantic integration or contextual understanding. Consequently, whether through attention ablation or noise addition, directly impacting the model's ability to process critical linguistic information leads to significant performance degradation. Regarding the control variable, perplexity primarily reflects a model's general language modeling capability. However, the ABS task requires not only basic language comprehension but also the ability to extract and compress critical information. LRs in DLMs similar to those in the human brain may excel at this advanced information processing and abstraction, which is not fully captured by the perplexity metric. These findings highlight a crucial link between the neural representations in DLMs and human brain language processing, suggesting that the brain-like layers are also critical for high-level language tasks. This convergence, robust across model architectures and experimental conditions, implies that DLMs may be capturing fundamental aspects of human language processing.</p> <p>Throughout our research process, we have identified several limitations in the current study, which provide directions for future research. Firstly, this study did not conduct an in-depth comparative analysis of the experimental results among the three DLMs. Our research design selected BART, PEGASUS, and T5 models, which vary in type, hidden layer count, and fine-tuning dataset, to enhance the consistency and robustness of our hypotheses across diverse model types. However, this approach limited our ability to isolate the effects of individual factors. Future research could adopt more rigorous variable control strategies to investigate these factors independently. For example, comparing BART and PEGASUS models with identical layer counts and fine-tuning datasets could isolate the influence of model type. Similarly, examining T5 variants that differ only in parameter count while keeping other conditions constant could focus on the impact of model scale on performance. These targeted comparisons would provide more nuanced insights into the specific contributions of each factor to the observed effects. Secondly, this study primarily focused on the overall trends of RSA scores across different experimental conditions without delving into changes at the individual hidden layer level. Therefore, future experiments could be designed to cross-validate RSA scores with findings from other interpretability research methods, such as probing tasks or feature visualization, to understand the model's internal representations better. Furthermore, although we set three EEG signal conditions, this approach still faces challenges separating semantic and syntactic representations within EEG signals. However, studies[<reflink idref="bib18" id="ref70">18</reflink>],[<reflink idref="bib19" id="ref71">19</reflink>],[<reflink idref="bib56" id="ref72">56</reflink>] have, in turn, provided approaches for separating syntactic and semantic representations in GPT-2 (decoder-only). Future research could attempt to adapt and transfer this method to encoder architectures, enabling separation and analysis of syntactic and semantic representations within encoder layers. Lastly, the recent emergence of decoder-only open-source Large Language Models (LLMs), such as LLaMA[<reflink idref="bib57" id="ref73">57</reflink>], Vicuna[<reflink idref="bib58" id="ref74">58</reflink>], and Mistral[<reflink idref="bib59" id="ref75">59</reflink>], has captured significant attention from academia and industry. Despite lacking explicit encoder structures, these models demonstrate remarkable language understanding capabilities and show potential for application in ABS tasks. Future research could explore the relationships between these LLMs and human language cognition. Specifically, comparative analyses could focus on aspects such as parameter scale, pre-training strategies, and performance on downstream tasks. Such investigations could provide valuable insights into the similarities and differences between these advanced AI models and human language processing, potentially bridging the gap between artificial and biological approaches to language understanding.</p> <hd id="AN0182076388-14">Conclusion</hd> <p>This study investigated the similarity between the deep language models' encoder hidden layer representations and human brain activity during language processing using EEG-based experiments and representational similarity analysis. Subsequently, we employed a hierarchical manipulation approach to analyze the correlation between this brain-model similarity and model performance. Our core hypothesis was verified: the hidden layers of a model that exhibit a higher degree of representational similarity to the human brain's activation patterns play a more critical role in the model's performance on the abstractive summarization task. This correlation manifested and attained statistical significance across various experimental conditions. These conditions encompassed diverse deep language model types (BART, PEGASUS, T5), multiple spatiotemporal windows representing distinct phases of human language representation, various hierarchical manipulation techniques including attention ablation and noise addition, and an array of model performance evaluation metrics (Rouge1, Rouge2, RougeL). Notably, this correlation persisted even after accounting for the confounding influence of perplexity on model performance. The heterogeneity of these experimental conditions bolsters the robustness and validity of our conclusions, yielding profound insights into the alignment between deep language models and human language processing mechanisms.</p> <hd id="AN0182076388-15">Acknowledgements</hd> <p>We are immensely grateful to the Beijing University of Posts and Telecommunications students who participated in this work and provided us with their invaluable support.</p> <hd id="AN0182076388-16">Author contributions</hd> <p>Conceptualization, L.L. and L.Z.; methodology, Z.Z. and S.G.; validation, Z.Z. and Y.Z.; formal analysis, Z.Z., S.G., W.Z. and Y.L.; investigation, Z.Z. and S.G.; resources, L.L.; data curation, Z.Z. and Y.L.; writing-original draft preparation, Z.Z. and S.G.; writing-review & editing, Z.Z., S.G., W.Z., L.L. and L.Z.; visualization, Z.Z.; supervision, L.L., L.Z. and Y.Z.; project administration, L.L.; funding acquisition, L.L.</p> <hd id="AN0182076388-17">Funding</hd> <p>This research was funded by National Natural Science Foundation of China (NSFC) [grant number 62176024]; and BUPT innovation and entrepreneurship support program [grant number 2024-YC-A126]; and Engineering Research Center of Information Networks, Ministry of Education.</p> <hd id="AN0182076388-18">Data availability</hd> <p>The EEG data and model hidden states are available at https://drive.google.com/drive/folders/1-C_e_HqdS6MIf1gGVehG7DW6B0MQguco?usp=sharing.</p> <hd id="AN0182076388-19">Code availability</hd> <p>The code for RSA (with data), layer manipulation, and correlation computation (with data) are also available at https://drive.google.com/drive/folders/1-C_e_HqdS6MIf1gGVehG7DW6B0MQguco?usp=sharing.</p> <hd id="AN0182076388-20">Declarations</hd> <p></p> <hd id="AN0182076388-21">Competing interests</hd> <p>The authors declare no competing interests.</p> <hd id="AN0182076388-22">Ethical approval declarations</hd> <p>The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Ethics Committee of the Beijing University of Posts and Telecommunications (Ethic approval code: 202302003).</p> <hd id="AN0182076388-23">Supplementary Information</hd> <p>Graph: Supplementary Information.</p> <hd id="AN0182076388-24">Supplementary Information</hd> <p>The online version contains supplementary material available at https://doi.org/10.1038/s41598-024-84530-w.</p> <hd id="AN0182076388-25">Publisher's note</hd> <p>Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.</p> <ref id="AN0182076388-26"> <title> References </title> <blist> <bibl id="bib1" idref="ref1" type="bt">1</bibl> <bibtext> Chang Y. A survey on evaluation of large language models. ACM Trans. Intell. Syst. Technol. 2024; 15: 1-45. 10.1145/3641289. 1081.65535</bibtext> </blist> <blist> <bibl id="bib2" idref="ref2" type="bt">2</bibl> <bibtext> Zhao, W. X. et al. A survey of large language models, https://doi.org/10.48550/arXiv.2303.18223 (2023). arXiv: 2303.18223.</bibtext> </blist> <blist> <bibl id="bib3" idref="ref3" type="bt">3</bibl> <bibtext> Antonello, R, Vaidya, A. & Huth, A. Scaling laws for language encoding models in fmri, in Oh, A. et al. (eds.) Advances in Neural Information Processing Systems, vol. 36 21895–21907 (2023).</bibtext> </blist> <blist> <bibl id="bib4" idref="ref4" type="bt">4</bibl> <bibtext> Arana, S, Pesnot Lerousseau, J. & Hagoort, P. Deep learning models to study sentence comprehension in the human brain. Lang. Cognit. Neurosci. https://doi.org/10.1080/23273798.2023.2198245 (2023).</bibtext> </blist> <blist> <bibl id="bib5" idref="ref5" type="bt">5</bibl> <bibtext> Goldstein, A. et al. Alignment of brain embeddings and artificial contextual embeddings in natural language points to common geometric patterns. Nat. Commun. https://doi.org/10.1038/s41467-024-46631-y (2024).</bibtext> </blist> <blist> <bibl id="bib6" idref="ref7" type="bt">6</bibl> <bibtext> Goldstein, A. et al. Thinking ahead: spontaneous prediction in context as a keystone of language in humans and machines. bioRxiv[SPACE]https://doi.org/10.1101/2020.12.02.403477 (2021).</bibtext> </blist> <blist> <bibl id="bib7" idref="ref8" type="bt">7</bibl> <bibtext> Goldstein A. Shared computational principles for language processing in humans and deep language models. Nat. Neurosci. 2022; 25: 369-380. 1:CAS:528:DC%2BB38XmsFajt7s%3D. 10.1038/s41593-022-01026-4. 35260860. 8904253. 1040.76069</bibtext> </blist> <blist> <bibl id="bib8" idref="ref9" type="bt">8</bibl> <bibtext> Ding, X, Chen, B, Du, L, Qin, B. & Liu, T. CogBERT: Cognition-guided pre-trained language models, in Calzolari, N. et al. (eds.) Proceedings of the 29th International Conference on Computational Linguistics 3210–3225 (International Committee on Computational Linguistics, 2022).</bibtext> </blist> <blist> <bibl id="bib9" idref="ref10" type="bt">9</bibl> <bibtext> Hollenstein, N. et al. Decoding eeg brain activity for multi-modal natural language processing. Front. Hum. Neurosci. https://doi.org/10.3389/fnhum.2021.659410 (2021).</bibtext> </blist> <blist> <bibtext> Toneva, M. & Wehbe, L. Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain), in Wallach, H. et al. (eds.) Advances in Neural Information Processing Systems, vol. 32 (2019).</bibtext> </blist> <blist> <bibtext> Hale JT. Neurocomputational models of language processing. Annu. Rev. Linguist. 2022; 8: 427-446. 10.1146/annurev-linguistics-051421-020803. 0445.62083</bibtext> </blist> <blist> <bibtext> Mischler, G, Li, Y. A, Bickel, S, Mehta, A. D. & Mesgarani, N. Contextual feature extraction hierarchies converge in large language models and the brain, https://doi.org/10.48550/arXiv.2401.17671. arXiv: 2401.17671 (2024).</bibtext> </blist> <blist> <bibtext> Wingfield, C. et al. On the similarities of representations in artificial and brain neural networks for speech recognition. Front. Comput. Neurosci. https://doi.org/10.3389/fncom.2022.1057439 (2022).</bibtext> </blist> <blist> <bibtext> Schrimpf, M. et al. The neural architecture of language: Integrative modeling converges on predictive processing. Proc. Natl. Acad. Sci. https://doi.org/10.1073/pnas.2105646118 (2021).</bibtext> </blist> <blist> <bibtext> Hosseini EA. Artificial neural network language models predict human brain responses to language even after a developmentally realistic amount of training. Neurobiol. Lang. 2024; 5: 43-63. 10.1162/nol_a_00137. 1336.62124</bibtext> </blist> <blist> <bibtext> Barthel M, Tomasello R, Liu M. Conditionals in context: Brain signatures of prediction in discourse processing. Cognition. 2024; 242. 10.1016/j.cognition.2023.105635. 37883821. 07803972105635</bibtext> </blist> <blist> <bibtext> Nour Eddine S, Brothers T, Wang L, Spratling M, Kuperberg GR. A predictive coding model of the n400. Cognition. 2024; 246. 10.1016/j.cognition.2024.105755. 38428168. 07886245105755</bibtext> </blist> <blist> <bibtext> Heilbron, M, Armeni, K, Schoffelen, J.-M, Hagoort, P. & de Lange, F. P. A hierarchy of linguistic predictions during natural language comprehension. Proc. Natl. Acad. Sci.119, https://doi.org/10.1073/pnas.2201968119 (2022).</bibtext> </blist> <blist> <bibtext> Caucheteux C, Gramfort A, King J-R. Evidence of a predictive coding hierarchy in the human brain listening to speech. Nat. Hum. Behav. 2023; 7: 430-441. 10.1038/s41562-022-01516-2. 36864133. 10038805</bibtext> </blist> <blist> <bibtext> Goldstein A. Correspondence between the layered structure of deep language models and temporal structure of natural language processing in the human brain. bioRxiv. 2022. 10.1101/2022.07.11.499562. 35702156. 9196117. 07595244</bibtext> </blist> <blist> <bibtext> Caucheteux, C. & King, J.-R. Brains and algorithms partially converge in natural language processing. Commun. Biol.5, https://doi.org/10.1038/s42003-022-03036-1 (2022).</bibtext> </blist> <blist> <bibtext> Caucheteux, C, Gramfort, A. & King, J.-R. Long-range and hierarchical language predictions in brains and algorithms, https://doi.org/10.48550/arXiv.2111.14232. arXiv:2111.14232. (2021)</bibtext> </blist> <blist> <bibtext> Jawahar, G, Sagot, B. & Seddah, D. What does bert learn about the structure of language? in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, https://doi.org/10.18653/v1/p19-1356 (2019).</bibtext> </blist> <blist> <bibtext> Caucheteux, C, Gramfort, A. & King, J.-R. Deep language algorithms predict semantic comprehension from brain activity. Sci. Rep.12, https://doi.org/10.1038/s41598-022-20460-9 (2022).</bibtext> </blist> <blist> <bibtext> Golan T, Siegelman M, Kriegeskorte N, Baldassano C. Testing the limits of natural language models for predicting human language judgements. Nat. Mach. Intell. 2023; 5: 952-964. 10.1038/s42256-023-00718-1</bibtext> </blist> <blist> <bibtext> Sun, K. & Wang, R. Computational sentence-level metrics predicting human sentence comprehension, https://doi.org/10.48550/arXiv.2403.15822. arXiv: 2403.15822 (2024).</bibtext> </blist> <blist> <bibtext> Deniz F, Tseng C, Wehbe L, Dupré la Tour T, Gallant JL. Semantic representations during language comprehension are affected by context. J. Neurosci. 2023; 43: 3144-3158. 1:CAS:528:DC%2BB3sXhtFemsL%2FN. 10.1523/jneurosci.2459-21.2023. 36973013. 10146529</bibtext> </blist> <blist> <bibtext> Yang S, Feng D, Liu Y, Li D. Distant context aware text generation from abstract meaning representation. Appl. Intell. 2021; 52: 1672-1685. 10.1007/s10489-021-02431-1. 1539.82289</bibtext> </blist> <blist> <bibtext> Wang, T. et al. A study of extractive summarization of long documents incorporating local topic and hierarchical information. Sci. Rep.14, https://doi.org/10.1038/s41598-024-60779-z (2024).</bibtext> </blist> <blist> <bibtext> Zhang M, Zhou G, Yu W, Huang N, Liu W. A comprehensive survey of abstractive text summarization based on deep learning. Comput. Intell. Neurosci. 2022; 1–21: 2022. 10.1155/2022/7132226. 07259268</bibtext> </blist> <blist> <bibtext> Huang Y. Abstractive document summarization via multi-template decoding. Appl. Intell. 2022; 52: 9650-9663. 10.1007/s10489-021-02607-9. 1073.81060</bibtext> </blist> <blist> <bibtext> Lewis, M. et al. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension, in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, https://doi.org/10.18653/v1/2020.acl-main.703 (2020).</bibtext> </blist> <blist> <bibtext> Zhang, J, Zhao, Y, Saleh, M. & Liu, P. PEGASUS: Pre-training with extracted gap-sentences for abstractive summarization, in III, H. D. & Singh, A. (eds.) Proceedings of the 37th International Conference on Machine Learning, vol. 119 of Proceedings of Machine Learning Research, 11328–11339 (2020).</bibtext> </blist> <blist> <bibtext> Raffel C. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 2020; 21: 1-67. 4138124. 0988.68648</bibtext> </blist> <blist> <bibtext> Goodwin, T, Savery, M. & Demner-Fushman, D. Flight of the pegasus? comparing transformers on few-shot and zero-shot multi-document abstractive summarization, in Proceedings of the 28th International Conference on Computational Linguistics, https://doi.org/10.18653/v1/2020.coling-main.494 (International Committee on Computational Linguistics, 2020).</bibtext> </blist> <blist> <bibtext> Chung HW. Scaling instruction-finetuned language models. J. Mach. Learn. Res. 2024; 25: 1-53. 07953549</bibtext> </blist> <blist> <bibtext> Friederici AD, Opitz B, von Cramon DY. Segregating semantic and syntactic aspects of processing in the human brain: an fmri investigation of different word types. Cereb. Cortex. 2000; 10: 698-705. 1:STN:280:DC%2BD3czptFKltQ%3D%3D. 10.1093/cercor/10.7.698. 10906316</bibtext> </blist> <blist> <bibtext> Pasquiou A, Lakretz Y, Thirion B, Pallier C. Information-restricted neural language models reveal different brain regions' sensitivity to semantics, syntax, and context. Neurobiol. Lang. 2023; 4: 611-636. 10.1162/nol_a_00125</bibtext> </blist> <blist> <bibtext> Reddy, A. J. & Wehbe, L. Can fmri reveal the representation of syntactic structure in the brain? In Ranzato, M, Beygelzimer, A, Dauphin, Y, Liang, P. & Vaughan, J. W. (eds.) Advances in Neural Information Processing Systems, vol. 34 9843–9856 (2021).</bibtext> </blist> <blist> <bibtext> Karamolegkou, A, Abdou, M. & Søgaard, A. Mapping brains with language models: A survey, in Findings of the Association for Computational Linguistics: ACL 2023, https://doi.org/10.18653/v1/2023.findings-acl.618 (2023).</bibtext> </blist> <blist> <bibtext> He, T, Boudewyn, M. A, Kiat, J. E, Sagae, K. & Luck, S. J. Neural correlates of word representation vectors in natural language processing models: Evidence from representational similarity analysis of event-related brain potentials. Psychophysiology. https://doi.org/10.1111/psyp.13976 (2022).</bibtext> </blist> <blist> <bibtext> OOTA, S, Gupta, M. & Toneva, M. Joint processing of linguistic properties in brains and language models, in Oh, A. et al. (eds.) Advances in Neural Information Processing Systems, vol. 36 18001–18014 (2023).</bibtext> </blist> <blist> <bibtext> Kutas M, Hillyard SA. Reading senseless sentences: Brain potentials reflect semantic incongruity. Science. 1980; 207: 203-205. 1980Sci.207.203K. 1:STN:280:DyaL3c%2FotFGntg%3D%3D. 10.1126/science.7350657. 7350657</bibtext> </blist> <blist> <bibtext> Osterhout L, Holcomb PJ. Event-related brain potentials elicited by syntactic anomaly. J. Mem. Lang. 1992; 31: 785-806. 10.1016/0749-596x(92)90039-z</bibtext> </blist> <blist> <bibtext> Zou B. Beats: An open-source, high-precision, multi-channel eeg acquisition tool system. IEEE Trans. Biomed. Circuits Syst. 2022; 16: 1287-1298. 10.1109/tbcas.2022.3230500. 1504.93232</bibtext> </blist> <blist> <bibtext> Dimigen O, Sommer W, Hohlfeld A, Jacobs AM, Kliegl R. Coregistration of eye movements and eeg in natural reading: Analyses and review. J. Exp. Psychol. Gen. 2011; 140: 552-572. 10.1037/a0023885. 21744985</bibtext> </blist> <blist> <bibtext> Coderre EL, Cohn N. Individual differences in the neural dynamics of visual narrative comprehension: The effects of proficiency and age of acquisition. Psychon. Bull. Rev. 2023; 31: 89-103. 10.3758/s13423-023-02334-x. 37578688. 10866750</bibtext> </blist> <blist> <bibtext> Peirce J. Psychopy2: Experiments in behavior made easy. Behav. Res. Methods. 2019; 51: 195-203. 10.3758/s13428-018-01193-y. 30734206. 6420413. 0507.73027</bibtext> </blist> <blist> <bibtext> Peirce JW. Psychopy-psychophysics software in python. J. Neurosci. Methods. 2007; 162: 8-13. 10.1016/j.jneumeth.2006.11.017. 17254636. 2018741. 0507.73027</bibtext> </blist> <blist> <bibtext> Pagnoni, A, Balachandran, V. & Tsvetkov, Y. Understanding factuality in abstractive summarization with frank: A benchmark for factuality metrics, in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, https://doi.org/10.18653/v1/2021.naacl-main.383 (2021).</bibtext> </blist> <blist> <bibtext> Hermann, K. M. et al. Teaching machines to read and comprehend, in Cortes, C, Lawrence, N, Lee, D, Sugiyama, M. & Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 28 (Curran Associates, Inc, 2015).</bibtext> </blist> <blist> <bibtext> Narayan, S, Cohen, S. B. & Lapata, M. Don't give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization, in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, https://doi.org/10.18653/v1/d18-1206 (Association for Computational Linguistics, 2018).</bibtext> </blist> <blist> <bibtext> Vaswani, A. et al. Attention is all you need, in Guyon, I. et al. (eds.) Advances in Neural Information Processing Systems, vol. 30 (2017).</bibtext> </blist> <blist> <bibtext> Funder, D. C. & Ozer, D. J,. Evaluating effect size in psychological research: Sense and nonsense. Adv. Methods Pract. Psychol. Sci.2, 156–168. https://doi.org/10.1177/2515245919847202 (2019).</bibtext> </blist> <blist> <bibtext> Huh, M, Cheung, B, Wang, T. & Isola, P. The platonic representation hypothesis, https://doi.org/10.48550/arXiv.2405.07987, arXiv: 2405.07987. (2024)</bibtext> </blist> <blist> <bibtext> Caucheteux, C, Gramfort, A. & King, J.-R. Disentangling syntax and semantics in the brain with deep networks, in Meila, M. & Zhang, T. (eds.) Proceedings of the 38th International Conference on Machine Learning, vol. 139 of Proceedings of Machine Learning Research, 1336–1348 (PMLR, 2021).</bibtext> </blist> <blist> <bibtext> Touvron, H. et al. Llama: Open and efficient foundation language models, https://doi.org/10.48550/arXiv.2302.13971. arXiv: 2302.13971 (2023).</bibtext> </blist> <blist> <bibtext> Chiang, W.-L. et al.Vicuna: An Open-Source Chatbot Impressing gpt-4 with 90%* Chatgpt Quality (2023).</bibtext> </blist> <blist> <bibtext> Jiang, A. Q. et al.Mistral 7b, https://doi.org/10.48550/arXiv.2310.06825 (2023). arXiv: 2310.06825.</bibtext> </blist> </ref> <aug> <p>By Zhejun Zhang; Shaoting Guo; Wenqing Zhou; Yingying Luo; Yingqi Zhu; Lin Zhang and Lei Li</p> <p>Reported by Author; Author; Author; Author; Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib10" firstref="ref11"></nolink> <nolink nlid="nl2" bibid="bib11" firstref="ref12"></nolink> <nolink nlid="nl3" bibid="bib12" firstref="ref13"></nolink> <nolink nlid="nl4" bibid="bib13" firstref="ref14"></nolink> <nolink nlid="nl5" bibid="bib14" firstref="ref15"></nolink> <nolink nlid="nl6" bibid="bib15" firstref="ref16"></nolink> <nolink nlid="nl7" bibid="bib16" firstref="ref17"></nolink> <nolink nlid="nl8" bibid="bib17" firstref="ref18"></nolink> <nolink nlid="nl9" bibid="bib18" firstref="ref19"></nolink> <nolink nlid="nl10" bibid="bib19" firstref="ref20"></nolink> <nolink nlid="nl11" bibid="bib20" firstref="ref23"></nolink> <nolink nlid="nl12" bibid="bib21" firstref="ref24"></nolink> <nolink nlid="nl13" bibid="bib22" firstref="ref25"></nolink> <nolink nlid="nl14" bibid="bib23" firstref="ref26"></nolink> <nolink nlid="nl15" bibid="bib24" firstref="ref27"></nolink> <nolink nlid="nl16" bibid="bib25" firstref="ref31"></nolink> <nolink nlid="nl17" bibid="bib26" firstref="ref32"></nolink> <nolink nlid="nl18" bibid="bib27" firstref="ref33"></nolink> <nolink nlid="nl19" bibid="bib28" firstref="ref34"></nolink> <nolink nlid="nl20" bibid="bib29" firstref="ref35"></nolink> <nolink nlid="nl21" bibid="bib30" firstref="ref36"></nolink> <nolink nlid="nl22" bibid="bib31" firstref="ref37"></nolink> <nolink nlid="nl23" bibid="bib32" firstref="ref38"></nolink> <nolink nlid="nl24" bibid="bib33" firstref="ref39"></nolink> <nolink nlid="nl25" bibid="bib34" firstref="ref40"></nolink> <nolink nlid="nl26" bibid="bib35" firstref="ref41"></nolink> <nolink nlid="nl27" bibid="bib36" firstref="ref42"></nolink> <nolink nlid="nl28" bibid="bib37" firstref="ref43"></nolink> <nolink nlid="nl29" bibid="bib38" firstref="ref44"></nolink> <nolink nlid="nl30" bibid="bib39" firstref="ref45"></nolink> <nolink nlid="nl31" bibid="bib40" firstref="ref47"></nolink> <nolink nlid="nl32" bibid="bib41" firstref="ref48"></nolink> <nolink nlid="nl33" bibid="bib42" firstref="ref50"></nolink> <nolink nlid="nl34" bibid="bib43" firstref="ref51"></nolink> <nolink nlid="nl35" bibid="bib44" firstref="ref52"></nolink> <nolink nlid="nl36" bibid="bib45" firstref="ref53"></nolink> <nolink nlid="nl37" bibid="bib46" firstref="ref54"></nolink> <nolink nlid="nl38" bibid="bib47" firstref="ref55"></nolink> <nolink nlid="nl39" bibid="bib48" firstref="ref56"></nolink> <nolink nlid="nl40" bibid="bib49" firstref="ref57"></nolink> <nolink nlid="nl41" bibid="bib50" firstref="ref58"></nolink> <nolink nlid="nl42" bibid="bib51" firstref="ref60"></nolink> <nolink nlid="nl43" bibid="bib52" firstref="ref61"></nolink> <nolink nlid="nl44" bibid="bib53" firstref="ref62"></nolink> <nolink nlid="nl45" bibid="bib54" firstref="ref64"></nolink> <nolink nlid="nl46" bibid="bib55" firstref="ref66"></nolink> <nolink nlid="nl47" bibid="bib56" firstref="ref68"></nolink> <nolink nlid="nl48" bibid="bib57" firstref="ref73"></nolink> <nolink nlid="nl49" bibid="bib58" firstref="ref74"></nolink> <nolink nlid="nl50" bibid="bib59" firstref="ref75"></nolink>
CustomLinks:
  – Url: https://resolver.ebsco.com/c/xy5jbn/result?sid=EBSCO:edsdoj&genre=article&issn=20452322&ISBN=&volume=15&issue=1&date=20250101&spage=1&pages=1-18&title=Scientific Reports&atitle=A%20constitutive%20model%20for%20coal%20gangue%20coarse-grained%20subgrade%20filler%20incorporating%20particle%20breakage&aulast=Zong-Tang%20Zhang&id=DOI:10.1038/s41598-025-85979-z
    Name: Full Text Finder (for New FTF UI) (s8985755)
    Category: fullText
    Text: Find It @ SCU Libraries
    MouseOverText: Find It @ SCU Libraries
  – Url: https://doaj.org/article/ebb586c13bcb4e87881eed8b7cc372c8
    Name: EDS - DOAJ (s8985755)
    Category: fullText
    Text: View record from DOAJ
    MouseOverText: View record from DOAJ
Header DbId: edsdoj
DbLabel: Directory of Open Access Journals
An: edsdoj.bb586c13bcb4e87881eed8b7cc372c8
RelevancyScore: 1064
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 1064.22692871094
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: A constitutive model for coal gangue coarse-grained subgrade filler incorporating particle breakage
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Zong-Tang+Zhang%22">Zong-Tang Zhang</searchLink><br /><searchLink fieldCode="AR" term="%22An+Liu%22">An Liu</searchLink><br /><searchLink fieldCode="AR" term="%22Si-Chun+Yu%22">Si-Chun Yu</searchLink><br /><searchLink fieldCode="AR" term="%22Shun-Kai+Liu%22">Shun-Kai Liu</searchLink><br /><searchLink fieldCode="AR" term="%22Chuan-Feng+Fang%22">Chuan-Feng Fang</searchLink><br /><searchLink fieldCode="AR" term="%22Ze+Liu%22">Ze Liu</searchLink><br /><searchLink fieldCode="AR" term="%22Wei+Hu%22">Wei Hu</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: Scientific Reports, Vol 15, Iss 1, Pp 1-18 (2025)
– Name: Publisher
  Label: Publisher Information
  Group: PubInfo
  Data: Nature Portfolio, 2025.
– Name: DatePubCY
  Label: Publication Year
  Group: Date
  Data: 2025
– Name: Subset
  Label: Collection
  Group: HoldingsInfo
  Data: LCC:Medicine<br />LCC:Science
– Name: Subject
  Label: Subject Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Coal+gangue+subgrade%22">Coal gangue subgrade</searchLink><br /><searchLink fieldCode="DE" term="%22Subgrade+filler%22">Subgrade filler</searchLink><br /><searchLink fieldCode="DE" term="%22Particle+breakage%22">Particle breakage</searchLink><br /><searchLink fieldCode="DE" term="%22Coarse-grained+soil%22">Coarse-grained soil</searchLink><br /><searchLink fieldCode="DE" term="%22Constitutive+model%22">Constitutive model</searchLink><br /><searchLink fieldCode="DE" term="%22Medicine%22">Medicine</searchLink><br /><searchLink fieldCode="DE" term="%22Science%22">Science</searchLink>
– Name: Abstract
  Label: Description
  Group: Ab
  Data: Abstract The accumulation and discharge amount of coal gangue are substantial, occupying significant land resources over time. Utilizing coal gangue as subgrade filler can generate notable economic and social benefits. Coal gangue coarse-grained soil (CGSF) was used to conduct a series of large-scale vibration compaction tests and large-scale triaxial tests. The results indicate that the maximum dry density of CGSF initially increases and then decreases with the increase in fractal dimension. The stress–strain curves of the samples exhibit a distinct nonlinear growth pattern. Analysis of the compaction effect suggests that the compaction degree of CGSF should not be lower than 93%. As the confining pressure increases, the extent of failure strength improvement due to increased compaction decreases. Additionally, the failure strength of samples initially increases and then decreases with the increase in coarse particle content. A modified quadratic polynomial fractal model gradation equation was proposed to describe the gradation of samples after particle breakage. Based on this, a new quantitative index for particle breakage was established. Analysis of particle breakage in samples revealed that higher confining pressure and greater coarse particle content lead to increased particle breakage. The breakage exhibited a significant size effect, and the impact of particle gradation on sample breakage was greater than that of confining pressure. The stress–strain relationship of CGSF was analyzed by using a logarithmic constitutive model, and the correlation between model parameters and the newly derived particle breakage index was generated. A constitutive model incorporating particle breakage for CGSF was established, and its accuracy was validated.
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: article
– Name: Format
  Label: File Description
  Group: SrcInfo
  Data: electronic resource
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 2045-2322
– Name: NoteTitleSource
  Label: Relation
  Group: SrcInfo
  Data: https://doaj.org/toc/2045-2322
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1038/s41598-025-85979-z
– Name: URL
  Label: Access URL
  Group: URL
  Data: <link linkTarget="URL" linkTerm="https://doaj.org/article/ebb586c13bcb4e87881eed8b7cc372c8" linkWindow="_blank">https://doaj.org/article/ebb586c13bcb4e87881eed8b7cc372c8</link>
– Name: AN
  Label: Accession Number
  Group: ID
  Data: edsdoj.bb586c13bcb4e87881eed8b7cc372c8
PLink https://login.libproxy.scu.edu/login?url=https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&scope=site&db=edsdoj&AN=edsdoj.bb586c13bcb4e87881eed8b7cc372c8
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1038/s41598-025-85979-z
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 18
        StartPage: 1
    Subjects:
      – SubjectFull: Coal gangue subgrade
        Type: general
      – SubjectFull: Subgrade filler
        Type: general
      – SubjectFull: Particle breakage
        Type: general
      – SubjectFull: Coarse-grained soil
        Type: general
      – SubjectFull: Constitutive model
        Type: general
      – SubjectFull: Medicine
        Type: general
      – SubjectFull: Science
        Type: general
    Titles:
      – TitleFull: A constitutive model for coal gangue coarse-grained subgrade filler incorporating particle breakage
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Zong-Tang Zhang
      – PersonEntity:
          Name:
            NameFull: An Liu
      – PersonEntity:
          Name:
            NameFull: Si-Chun Yu
      – PersonEntity:
          Name:
            NameFull: Shun-Kai Liu
      – PersonEntity:
          Name:
            NameFull: Chuan-Feng Fang
      – PersonEntity:
          Name:
            NameFull: Ze Liu
      – PersonEntity:
          Name:
            NameFull: Wei Hu
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 20452322
          Numbering:
            – Type: volume
              Value: 15
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Scientific Reports
              Type: main
ResultId 1