NAISS
SUPR
AI for Formally-Verified code generation
Dnr:

NAISS 2026/3-960

Type:

NAISS Medium

Principal Investigator:

Marco Chiesa

Affiliation:

Kungliga Tekniska högskolan

Start Date:

2026-11-01

End Date:

2027-05-01

Primary Classification:

10210: Artificial Intelligence

Secondary Classification:

10213: Formal Methods

Tertiary Classification:

10205: Software Engineering

Webpage:

Allocation

Abstract

AI code generators now write a large share of new software, including for critical systems, faster than humans can review it and often with security vulnerabilities. Formal verification offers the strongest available guarantee: a mathematical proof, checked by a small trusted verifier, that a program meets a precise specification. Writing such proofs still takes expert effort. This project develops AI methods that make formal verification of AI-generated code fast and reliable, so that code is accepted based on a machine-checked proof rather than trust in the model that wrote it. The project builds on our work on AI-based code verification (*Dissect-and-Restore*), which raised verification success on complex programs from 25% to 87%, and is part of the WASP project *Automated Software Verification with Expert-Driven Reasoning*, funded by the Knut and Alice Wallenberg Foundation. Three research lines share one code base, model stack and verification toolchain: Task 1. Understanding and steering model reasoning. We study how open-weight models internally represent programs, specifications and proof obligations, and how these representations relate to verification success. Methods include probing, activation patching, sparse feature dictionaries and activation steering. We also investigate how reasoning patterns emerge during downstream fine-tuning, alignment and reinforcement learning with verifier rewards, whether this emergence can be detected early and accelerated. The goal is to identify unreliable proof attempts, explain failures and steer models towards correct proofs. Task 2. Better use of verification tools by AI models. Verifiers such as Dafny and Verus translate proofs into queries for the Z3 SMT solver, which can miss simple facts, causing proofs to fail unless hand-written "gap" assertions are added. In preliminary work, nine sequence laws added to the verifiers' built-in axioms repair 99 of 320 Dafny and 107 of 475 Verus benchmark tasks at about 6% extra solver time, and let an open 14B model solve 18 of 32 tasks with a missing loop invariant, versus 3 of 32 without. We will extend this to further verifier gaps and train open models, including reinforcement learning with verifier verdicts as rewards, to exploit strengthened verifiers and repair failed proofs using solver feedback. Task 3. Agentic, multi-round reasoning for verification. We study agents that alternate between reasoning, tool calls and verifier feedback over several rounds, across verification languages (e.g. Dafny, Verus, Lean) and mathematical reasoning tasks. We evaluate how reasoning budgets, feedback formats and search strategies affect success, and train models for multi-round use. All tasks require open-weight models with full access to weights and internal activations, unavailable through commercial APIs. We study model families from about 1B to over 100B parameters, including Qwen3 (14B, 32B, Coder-30B-A3B), gpt-oss (20B, 120B), DeepSeek-R1-distilled, Llama, Gemma, Mistral/Devstral, and proof-specialised models such as ReForm and DeepSeek-Prover. GPUs support large-scale inference with multiple attempts and feedback rounds, activation extraction, and supervised and reinforcement-learning fine-tuning. Benchmarks, models and tools will be released as open source.