Machine learning and AI are quickly becoming ubiquitous, and yet the development of learning algorithms is often based on heuristics and empirically investigated best practices. Moreover, we often lack theory to explain the algorithms’ properties and performance, which makes designing and optimising the algorithms challenging. Recent attempts at explaining learning algorithms has included the inductive bias implicitly induced by optimisers, architecture, and loss functions. In this project, we aim to investigate learning algorithms from an information-theoretic and coding-theoretic perspective: by designing suitable signal space representations, informed by well-developed theoretical tools like signal processing, communication theory, and information theory, we hope to induce signal space representations which both work well in practice, but also are suitable for mathematical analysis. By increasing the applicability of existing theory we hope to decrease the influence of inductive bias, since this is hard to analyse. We wish to validate our theory-based signal space representations through simulations; in particular, GPUs will be used to train modern machine learning models on larger datasets, both of which are common in publications. This proposal is related to Vetenskapsrådet’s project VR-2021-05266.