SUPR
Neural representation of massive 3D data
Dnr:

NAISS 2023/22-751

Type:

NAISS Small Compute

Principal Investigator:

Jens Sjölund

Affiliation:

Uppsala universitet

Start Date:

2023-08-01

End Date:

2024-08-01

Primary Classification:

10207: Computer Vision and Robotics (Autonomous Systems)

Webpage:

Allocation

Abstract

X-ray microtomography employs X-rays to generate cross-sectional images of a physical object, enabling the creation of a non-destructive virtual model (3D model). The prefix micro signifies that the pixel sizes of the cross-sections fall within the micrometer range. Because of this high resolution, the virtual model is massive; e.g. a 1 cubic centimeter volume depicting a mouse tumor requires about 50 GB of storage. The sheer size of the data presents a major challenge for subsequent processing steps using, for instance, neural networks. This project investigates whether such massive 3D data can be compressed using an implicit neural representation. The vision is to determine a neural representation of the full 3D data from the micro-CT that fits on a single GPU. Either of the following two capabilities would be of great use: 1. an interactive viewer capable of rendering arbitrary views (slices through the data) and resolutions, i.e. supporting zoom functionality. 2. an interface for downstream applications based on e.g. convolutional neural networks (CNNs). One example would be a 3D CNN that predicts the diffusion MRI signal from the neural representation. Another example would be a CNN that performs semantic segmentation of the micro-CT data.