Cyber-Physical Systems (CPS) have become integral to modern infrastructures, with smart grids serving as a cornerstone for energy distribution in the digital age. Ensuring the safety, efficiency, and robustness of these systems in dynamic and uncertain environments presents significant challenges. This project explores the application of advanced multi-agent reinforcement learning (MARL) to optimize the performance and resilience of CPS, focusing on smart grid operations. The proposed framework leverages MARL techniques to enable decentralized decision-making among heterogeneous agents, representing components of the smart grid, such as power generators, distribution nodes, and consumers. Agents learn to cooperate and compete dynamically, adapting their strategies based on system constraints, environmental uncertainties, and safety requirements. The primary objective is to balance energy demand and supply efficiently while minimizing risks such as system failures, overloads, or cyber-attacks. To address these goals, this project introduces a novel reward structure and policy design tailored to CPS environments. Emphasis is placed on ensuring safety by integrating constraints directly into the learning algorithms, utilizing safety envelopes, and adopting robust adversarial training to mitigate vulnerabilities. Efficiency is further enhanced by incorporating model-based MARL approaches to accelerate convergence and reduce computational overhead. The framework is validated through simulations and case studies in realistic smart grid scenarios. Metrics such as system reliability, energy efficiency, computational scalability, and compliance with safety standards are used to evaluate performance. The outcomes aim to provide actionable insights into deploying MARL for real-world CPS applications, emphasizing adaptability to varying scales and conditions. By combining state-of-the-art reinforcement learning techniques with domain-specific requirements of CPS, this project contributes to advancing the safe and efficient operation of smart grids, paving the way for broader applications in critical infrastructure management.