With the growing interest in Machine Learning applications and the rapid expansion of dispersed edge devices, leveraging a large number of edge devices has become increasingly critical. Federated Learning is a distributed learning technique that allows model training in a large decentralized network while avoiding the exchange of sensitive local user data. Reinforcement Learning has made significant development in recent years and is currently utilized not just in simulators and games, but also in embedded systems as another software-intensive field. In this project, we would like to 1) analyze those two techniques and solve the challenges which prevent them to be deployed into real-world systems 2) investigate the possibility to combine those emerging techniques and enable self-improving in the edge. Imagine that all of the edge devices can autonomously improve without human interaction and share the knowledge with others, we expect that the prediction performance of each local model can be significantly improved.