Foundation models have recently shown promising results in the different fields such as computer vision and natural language processing. Given their high performance, researchers of other fields such as neuroscience have also started investigating these models on their data. In this project, we are going to evaluate electroencephalogram (EEG) foundation models on how they represent visually evoked EEG responses. Specifically, we take paired EEG-image datasets and compare EEG embeddings created by an EEG foundation model with image embeddings created by image foundation models. The representation can be evaluated based on their performance on downstream tasks such as image retrieval and image reconstruction. We then evaluate to what extent finetuning the EEG encoder can improve the performance.