NAISS
SUPR
NAISS Projects
SUPR
Cognitive Foundations of Language Modeling
Dnr:

NAISS 2026/3-816

Type:

NAISS Medium

Principal Investigator:

Sharid Loáiciga

Affiliation:

Göteborgs universitet

Start Date:

2026-10-01

End Date:

2027-10-01

Primary Classification:

10208: Natural Language Processing

Secondary Classification:

10210: Artificial Intelligence

Tertiary Classification:

20208: Computer Vision and learning System (Computer Sciences aspects in 10207)

Allocation

Abstract

The goal of the project is to develop and evaluate cognitively inspired language models that more closely reflect human language learning and processing. Current language models achieve impressive performance but rely on amounts of training data and computation that differ substantially from the conditions under which humans acquire language. Human learning takes place with considerably less linguistic input and in rich contexts that combine information from multiple modalities. Our project uses insights from human linguistic and cognitive behavior to investigate more efficient learning methods and to better understand how current models represent and use contextual information. The project continues along two complementary research directions. First, we investigate **efficient language model pre-training under cognitively plausible data constraints**. Building on our work on Active Curriculum Language Modeling in the BabyLM framework, we study whether selecting and organizing training data according to its informativeness can improve learning from limited data. We investigate different model architectures, training configurations, and curriculum criteria, including surprisal- and semantic-similarity-based selection. The broader objective is to understand how effective language models can be trained with less data and computation. Second, we investigate **how humans and language models represent and use context**, particularly in multimodal settings where language is grounded in visual information. We examine whether the fluency of current language and vision-language models is accompanied by human-like representations of discourse and narrative context. Our experiments investigate narrative coherence, coreference and discourse organization, character grounding and persistence, and sensitivity to controlled changes in visual context. Comparisons across model families, architectures, and parameter scales allow us to identify which properties of contextual processing are robust and where models systematically differ from humans. Together, these research directions address complementary aspects of cognitively informed language modeling: how models can learn more efficiently and how the representations they acquire compare with human language processing. The project combines controlled experimentation, language model pre-training, large-scale inference, and systematic evaluation across linguistic and multimodal tasks.