This project studies video unlearning for vision-language models (VLMs). Video models can memorize sensitive training content such as faces, license plates, private locations, or other identifiable visual information. Removing that content without full retraining is important for privacy, but is still largely unexplored compared to text-only machine unlearning.
Building on prior privacy research on type-aware unlearning for language models, this project asks whether similar ideas apply to video-language settings. Different visual content types may be stored in different parts of a VLM (vision encoder, temporal modules, projector, language backbone) and may therefore require different forgetting strategies rather than one global correction strength.
The project will:
1) inject synthetic canary content into VLMs and measure memorization with targeted prompts and exposure-style metrics;
2) compute saliency patterns across model components for different content types;
3) develop type-aware task-arithmetic unlearning for video models;
4) evaluate both forgetting of targeted content and preservation of utility on standard video QA benchmarks.
Experiments will use open vision-language models with PyTorch and Hugging Face on NAISS Arrhenius, scaling to larger models as the project progresses. The goal is an initial method and benchmark for privacy-preserving unlearning in video-language models.
This project studies video unlearning for vision-language models (VLMs). Video models can memorize sensitive training content such as faces, license plates, private locations, or other identifiable visual information. Removing that content without full retraining is important for privacy, but is still largely unexplored compared to text-only machine unlearning.
Building on prior privacy research on type-aware unlearning for language models, this project asks whether similar ideas apply to video-language settings. Different visual content types may be stored in different parts of a VLM (vision encoder, temporal modules, projector, language backbone) and may therefore require different forgetting strategies rather than one global correction strength.
The project will:
1) inject synthetic canary content into VLMs and measure memorization with targeted prompts and exposure-style metrics;
2) compute saliency patterns across model components for different content types;
3) develop type-aware task-arithmetic unlearning for video models;
4) evaluate both forgetting of targeted content and preservation of utility on standard video QA benchmarks.
Experiments will use open vision-language models with PyTorch and Hugging Face on NAISS Arrhenius, scaling to larger models as the project progresses. The goal is an initial method and benchmark for privacy-preserving unlearning in video-language models.
Main supervisor: Fredrik Heintz, Linköping University, Department of Computer and Information Science (IDA).