Heat pumps are widely recognized as a key technology for decarbonizing both space heating and domestic hot water sectors on a global scale. When powered by renewable electricity, such as solar PV, heat pumps can significantly reduce reliance on fossil fuels, contributing to more sustainable, affordable, and resilient energy systems. This research aims to develop and evaluate smart data-driven approaches to provide innovative services by heat pump systems such as grid flexibility, cost-saving measures, and predictive maintenance. More specifically, the project aims to develop a digital twin of a heat pump system which is capable of dynamic interaction with its physical twin and support it by compensating missing information and assist in control optimization. It also aims to facilitate heat pump integration to smart grids through data post-processing and communicating with the controller of the physical system. One key aspect of the project in terms of control is “trade-off management” where the controller will pursue actions that account for trade-offs among and between competing objectives of comfort, cost, and environmental impact. The method selected for controlling with trade-off management is reinforcement learning (RL) since it is rooted in trade-off exploration/exploitation. A custom training environment with a functional mock-up unit (FMU) is built, which is used to train a policy / an agent represented by a deep neural network. The training and evaluation of the agent is intended to be performed with the help of a supercomputer to speed up the research process, while the deployment of the trained controller will be done in a physical system at the Granryd Laboratory. The entire workflow is implemented in Python with libraries FMPy, Gymnasium, and StableBaselines 3, among others. In the course of the project, the trade-offs among objectives by performing control actions shall be investigated, with particular interest in understanding and quantifying user-side effects of grid flexibility provision. Performance of the controller shall also be evaluated across different building types (which may include energy storage technologies) as well as scaled up in the context of energy communities (i.e., multiple systems). Main supervisor: Hatef Madani, Professor, Department of Energy Technology