Anomaly detection (AD) aims to identify data points that deviate significantly from the majority of instances, with applications spanning manufacturing, finance, and healthcare. Complex data types, such as images and text, introduce additional challenges for AD due to the reduced distance between data objects in high-dimensional spaces. This project addresses these challenges by focusing on three key objectives: 1) Developing robust methods for anomaly detection in contaminated data environments, 2) Creating explainable AD methods that offer anomaly localization, and 3) Building robust systems to detect attacks through AD methods. For the first objective, we will evaluate state-of-the-art methods under increasing data contamination and devise approaches that effectively manage noisy data. The second objective involves developing anomaly scoring techniques that provide insights into anomaly localization, explaining why specific instances are identified as anomalies. The third objective explores introducing backdoor attacks in NLP and vision foundation models, studying performance variations, and proposing AD-based methodologies to detect such attacks. This project primarily focuses on working with benchmark datasets in image, text, and multimodal formats.