SUPR
3D visual reconstruction from Internet videos in the wild
Dnr:

NAISS 2024/22-1248

Type:

NAISS Small Compute

Principal Investigator:

Shuo Sun

Affiliation:

Örebro universitet

Start Date:

2024-10-01

End Date:

2024-12-01

Primary Classification:

10207: Computer Vision and Robotics (Autonomous Systems)

Webpage:

Allocation

Abstract

This project aims at 3D reconstruction from videos in the wild (such as those on YouTube). Although we have some mature 3D reconstruction pipelines such as COLMAP and Reality Capture, which can reconstruct from collected images, they rely on good image correspondence to get satisfactory reconstruction quality. At the same time, the reconstruction process is time-consuming (for example, for 1K images, often >24 hours are needed). Given the shortcomings of the current reconstruction methods, applying them directly to Internet videos is impossible to get good results. Meanwhile, Internet videos are good resources for 2D information. If we can run reconstruction successfully on Internet videos, it has great potential to provide large data for many other visual tasks such as visual localization, generative AI etc. To summary, the goal of the project is to find a good way to reconstruct directly from Internet videos, and successfully estimate the camera poses and scene structure. the impact of the project would be to provide lots of reconstruction data for many other visual tasks. Of course, to avoid the potential copyright and privacy risks, we will only use those videos with approvals.