NAISS
SUPR
NAISS Projects
SUPR
Uncertainty Communication for Safe Escalation in Mixed-Scale LLM Agent Systems
Dnr:

NAISS 2026/4-1595

Type:

NAISS Small

Principal Investigator:

Yuxi Xia

Affiliation:

Kungliga Tekniska högskolan

Start Date:

2026-09-15

End Date:

2027-10-01

Primary Classification:

10210: Artificial Intelligence

Webpage:

Allocation

Abstract

This project studies how an LLM agent should communicate uncertainty about a potentially unsafe action to a stronger verifying agent, so that the final safety decision improves without escalation costs or over-refusal destroying usability. GPU access is essential for the experiments: the uncertainty estimators we compare require token-level logits and hidden states, which inference APIs do not expose. We host small agents of 1–30B parameters (Qwen and Llama families) at several scales, each fitting on 1/2 GPUs. Phase 1 is inference-dominated, which includes a factorial sweep over message formats, sender reliability levels, and escalation thresholds over trajectories from execution-grounded safety benchmarks. Phase 2 is contingent and involves a diagnostic at the end of Phase 1 that distinguishes whether the weak agent lacks a usable risk signal or expresses one poorly, and only in the latter case do we fine-tune (LoRA on the small 1–8B models) and re-run the evaluation.