NAISS
SUPR
NAISS Projects
SUPR
Resource-Aware Subnetwork Allocation for Heterogeneous Decentralized Learning
Dnr:

NAISS 2026/4-1351

Type:

NAISS Small

Principal Investigator:

Zhuojun Tian

Affiliation:

Kungliga Tekniska högskolan

Start Date:

2026-08-06

End Date:

2027-09-01

Primary Classification:

10214: Networked, Parallel and Distributed Computing

Webpage:

Allocation

Abstract

Heterogeneous device capabilities create severe synchronization bottlenecks in synchronous decentralized learning, making predefined model configurations inefficient across participating nodes. This letter proposes a resource-aware retention-ratio allocation framework that determines heterogeneous subnetworks before decentralized training by combining offline utility profiling with online resource-aware latency prediction. The resulting optimization minimizes the bottleneck latency under memory and aggregate learning-utility constraints. Although the learning utility is generally non-monotonic with respect to the retention ratio, we prove that the associated latency-feasibility problem remains monotonic, enabling an efficient binary-search solution. Experimental results verify that the proposed framework improves resource utilization, alleviates synchronization bottlenecks, and reduces overall training time while maintaining comparable learning performance.