SUPR
Deep Learning Models for Robot Manipulation
Dnr:

NAISS 2023/22-551

Type:

NAISS Small Compute

Principal Investigator:

Ruiyu Wang

Affiliation:

Kungliga Tekniska högskolan

Start Date:

2023-05-11

End Date:

2024-06-01

Primary Classification:

10299: Other Computer and Information Science

Webpage:

Allocation

Abstract

Robot manipulation is a challenging task that requires precise control and perception in order to manipulate objects in complex and dynamic environments. Deep learning models have shown great promise in this field, by enabling robots to learn from large datasets of sensory input and perform complex tasks such as grasping, pushing, and lifting objects. In this project, we propose to study deep learning models for robot manipulation, with a particular focus on diffusion models and transformers. Diffusion models provide a powerful framework for modeling complex distributions, while transformers have shown great success in natural language processing and computer vision tasks, and have recently been applied to robotics. We will explore the use of diffusion models and transformers for training robots to perform diverse tasks in cluttered environments. We will provide the robot with high-dimensional sensory input such as camera images and joint angles, and use deep neural networks to map this input to actions. We will evaluate the performance of our models on a variety of benchmark tasks, and compare their performance to existing approaches in the field. We will also investigate the transferability of our models to new environments and objects, by testing them on a variety of real-world scenarios. We will analyze the interpretability and explainability of our models to gain insights into the decision-making process of the robots. By studying diffusion models and transformers for robot manipulation, we hope to contribute to the development of more intelligent and capable robots, that can perform complex tasks in a variety of environments.