This project focuses on the design and analysis of optimizers for training neural networks. The main objective is to develop efficient optimizers with improved convergence behavior, robustness, and computational efficiency for large-scale nonconvex optimization problems arising in deep learning. The proposed methods will be evaluated on neural network training tasks and compared with widely used optimization algorithms such as SGD, momentum methods, Adam, Muon, etc. The numerical experiments will investigate convergence speed, training loss, generalization performance, sensitivity to hyperparameters, and computational efficiency across different neural network architectures and datasets. GPU resources are essential for these experiments because training modern neural networks requires large-scale tensor and matrix operations, repeated gradient evaluations, and extensive hyperparameter studies. The computational results will be used to validate the theoretical analysis, assess the practical performance of the proposed optimizers, and support research publications in mathematical optimization and machine learning.