This project investigates the fundamental interplay between data structure and representation learning in deep neural networks (DNNs), with a focus on understanding the dynamics of transfer learning. While transfer learning is a cornerstone of modern AI, a deeper theoretical understanding of how learned representations transfer between tasks, especially in relation to the structural properties of the training data, is still needed. Our research aims to bridge this gap by systematically analysing how features such as input geometry and input-output correlations shape the representations learned by DNNs and determine their reusability across different tasks. Our methodology is rooted in large-scale, controlled computational experiments. We train a wide range of architectures, including classic convolutional networks (e.g., ResNet, VGG) and modern Vision Transformers (ViTs), on a suite of benchmark datasets (MNIST, CIFAR-10, CelebA, and ImageNet). Critically, our investigation extends to numerous structurally modified variants of these datasets, which are designed to isolate specific data properties. The experimental pipeline involves repeatedly training models across a combinatorial set of conditions to generate a rich body of data on representation dynamics. The subsequent analysis phase employs quantitative metrics like Centered Kernel Alignment (CKA) and intrinsic dimensionality to dissect the learned representations. The expected outcomes will provide a more robust theoretical and practical foundation for transfer learning, contributing to the development of more efficient and reliable AI systems. The results will be disseminated through publications in top-tier machine learning venues, building on our group's prior work in this domain.