Machine learning and AI are quickly becoming ubiquitous, and yet the development of learning algorithms is often based on heuristics and empirically investigated best practices. Moreover, we often lack theory to explain the algorithms’ properties and performance, which makes designing and optimising the algorithms challenging. Recent attempts at explaining learning algorithms has included the inductive bias, induced either implicitly or explicitly, by optimisers, architecture, and loss functions. In NAISS project 2023/22-844, we induced an explicit inductive bias on the signal space representations. The project utilised well-developed tools from communication theory and related fields, which enabled mathematical analysis. In this project, we aim to extend this line of work from supervised learning to a wider range of topics in representation learning. Of particular interest is to investigate the effect of semantics in the data, as well as other intersections between communication theory, as well as signal processing and information theory, with representation learning. As pointed out in our activity report, the resource allocation in our previous NAISS project 2023/22-844 was too small for handling some of the state-of-the-art benchmarks in machine learning. Therefore, to validate all proposed schemes on the current state-of-the-art benchmark tasks, we want to use larger datasets which are common in the literature. For this aim, GPUs will be important. This proposal is related to Vetenskapsrådet’s project VR-2021-05266, and extends on NAISS project 2023/22-844.