Computer Vision algorithm development for environmental monitoring applications

NAISS 2024/5-84


NAISS Medium Compute

Principal Investigator:

Josephine Sullivan


Kungliga Tekniska högskolan

Start Date:


End Date:


Primary Classification:

10599: Other Earth and Related Environmental Sciences

Secondary Classification:

10207: Computer Vision and Robotics (Autonomous Systems)

Tertiary Classification:

20703: Remote Sensing



This project develops high-level computer vision algorithms to be applied to challenging environmental monitoring applications. In particular for detection/prediction of wild fire progression from satellite imagery and generalized category discovery for the plankton images. Wild fires are increasing each year in their intensity and frequency as climate changes accelerate. These wild fires can have catastrophic effects both on the local populations and the environment. Thus there is a critical need to develop better algorithms to predict their temporal progress to help manage their progress and lessen their impact. In this project we will investigate and develop methods for the prediction wild fire progress imaged with low spatial resolution but high temporal resolution satellites in combination with weather data and topographic descriptors. We will focus on harnessing auto-regression deep learning approaches such, as diffusion models, to make these predictions. In the second half of the project we will also investigate the possibility of upgrading low-spatial resolution predictions to higher-spatial resolution ones. Plankton has a fundamental impact on fish stocks and on the carbon cycle, and ultimately the climate. Certain plankton can also produce harmful toxins that bioaccumulate in seafood while others can form nuisance blooms. Both are to the detriment of human well-being. Understanding the factors controlling the growth and distribution of different plankton species is needed to help better tackle the above issues. This requires monitoring of plankton at a high spatio-temporal resolution. Routine monitoring of phyto- and microzooplankton is today largely based on manual microscopy. This is a time-consuming process (~4h/sample) relying heavily on the skills of the taxonomist, and cross-comparisons between datasets can be difficult. Deep learning techniques can make the process more efficient and reliable! Thus we focus on generalized category discovery for images of plankton. Generalized category discovery assumes one has access to two distinct sets of training data - a labelled set and an unlabelled one. In the unlabelled dataset there are images from potentially both known and unknown categories. The goal then is to discover the new categories present in the unlabelled dataset frequently with some form of implicit or explicit clustering. Most state-of-the-art methods rely on ideas from self- and semi-supervised learning to incorporate unlabelled data into the training process and discover new classes.