The project aims to evaluate Large Language Model (LLM) decision-making through the Theory of Response Sampling, which posits that LLMs' responses are driven by a concept's statistical reality (descriptive) with its idealized version (prescriptive). The project aims to investigate how input language affects the concept distribution, specifically investigating the effects of larger reasoning models. This will be done by generating sampling distributions of average, sample, and ideal prompts (N=10) and then performing statistical tests, mechanistic analysis of logits, and faithfulness analysis of thinking tokens. The goal of using Arrhenius is to run larger LLMs, such as Gemma-4-31B and Qwen-3.6-35B, to generate sampling distributions, which requires access to more GPU resources. This will be repeated across multiple languages to measure differences in responses among the LLMs.
The goal is to publish the results in an academic conference exploring cross-cultural and linguistic bias in a similar setup as Sivaprasad et al. (2025) paper from ACL 2025 on response sampling bias.