Large-scale optimization plays a crucial role in various domains and industries, enabling organizations to make informed decisions, improve operational efficiency, and maximize resource utilization. This project proposal aims to investigate and develop advanced optimization techniques specifically tailored for tackling large-scale problems. The primary objective is to address the challenges posed by complex optimization scenarios, such as high-dimensional search spaces, large amounts of data, and diverse constraints. Here are some of the projects/proposals: 1. Investigate efficient large-scale optimal transportation based on GPU: it solves the large-scale linear program that is beneficial for lots of applications like generative models. 2. Decentralized optimization: it investigates the decentralized algorithms for training the neural networks based on GPUs, which may improve in terms of less communication cost and more system robustness compared with the centralized schemes.