Machine learning (ML) is becoming pivotal in life science research, offering powerful tools for interpreting complex biological data. In particular, explainable ML provides insights into the reasoning behind model predictions, highlighting the data features that drove the model outcome. Our work focuses on building explainable ML models for microscopy images. These models not only classify cell fates but also reveal the underlying data patterns and features influencing these classifications. Specifically, we have developed models to classify individual lung cancer cell fates, such as proliferation and death, from live-cell microscopy data. By leveraging explainable ML techniques, we gained insights into the decision-making process of these models, revealing the key cellular markers that determine whether a cell would proliferate or die. The combination of ML and specialised image acquisition enabled us to address specific biological questions and uncover novel insights about underlying cellular mechanisms. This work demonstrates the potential of explainable ML in enhancing our understanding of complex biological processes, and how we can gain novel knowledge from images.