The aim of this project is to understand from a new point of view the ability of deep neural networks to successfully learn from data and generalize. There is a tendency of neural networks (trained in presence or absence of explicit regularization) to represent a given input-output relation in a structurally “simple” manner, in the sense that many parts of the network encode for the same, most informative feature(s) of the input. We have found evidence that this degree of simplicity, which is related to the ability of the models to learn the high-information modes of data and disregard the noise, can be quantified for trained neural networks using a graph theoretic measure called frustration. We want to study how this quantity behaves during training in real large scale neural networks of different types, such as convolutional, recurrent and Transformer neural networks, and under different conditions, such as large and small initialization of network weights, and varying optimization settings.