Neural networks have shown great promise in solving complex tasks, where the weights of artificial neural networks are learned using optimization schemes. Today, networks are often overparametrized to increase the possibilities of finding good solutions through optimization. Such overparameterization is, however, costly in terms of both training and inference time, limiting deployment on resource-constrained devices. This project aims at studying and developing metrics and algorithms to reduce the size of neural networks by gradual pruning; gradually removing weights during training. With gradual pruning, the network architecture is learnt simultaneously as the weights resulting in more compact and robust models. We especially study image analysis applications and metrics originating from the Taylor approximation of the change in loss when removing weights. Successful solutions will not only streamline and optimize AI model designs, they will also reduce energy, emissions, and effort currently needed to optimize architectures. The resulting models can also be smaller and faster, enabling more powerful applications on embedded devices. Supervisor: Prof. Orcun Göksel, Uppsala University