Machine learning and AI are quickly becoming ubiquitous, and yet the development of learning algorithms is often based on heuristics and empirically investigated best practices. Moreover, we often lack theory to explain the algorithms’ properties and performance, which makes designing and optimising the algorithms challenging. Recent attempts at explaining learning algorithms has included the inductive bias, induced either implicitly or explicitly, by optimisers, architectures, and loss functions.
This project proposal is an extension proposal to NAISS projects 2023/22-844 and 2024/22-1101. In NAISS project 2023/22-844, we induced an explicit inductive bias on the signal space representations. The project utilised well-developed tools from communication theory and related fields, which enabled mathematical analysis. In our ongoing NAISS project 2024/22-1101, we are proposing a new approach which extends our previous results by taking into account data semantics. We have validated this novel approach on certain large-scale computer vision benchmarks, namely image classification on ImageNet. The preliminary results are promising, and we are now looking to extend them further. In particular, we want to other necessary benchmarks on ImageNet classification, as well as new necessary benchmarks in a self-supervised setting.
This project is also funded by Vetenskapsrådet’s project VR-2021-05266.