Dobb·E is an open-source framework designed for teaching robots household tasks through a process known as imitation learning. The framework aims to address the limitations of current home robotics by providing a cheap and ergonomic solution for collecting demonstrations.

This is achieved through the use of a tool called the Stick, which is constructed using a $25 Reacher-grabber stick, 3D printed parts, and an iPhone.Dobb·E leverages the Stick to collect data from a dataset called Homes of New York (HoNY), which consists of 13 hours of interactions at 22 different homes in New York City.

The dataset includes RGB and depth videos, as well as action annotations for the gripper’s 6D pose and opening angle.Using the collected data, Dobb·E trains a representation learning model called Home Pretrained Representations (HPR).

This model, based on ResNet-34 architecture and trained using self-supervised learning objectives, initializes a robot policy for performing new tasks in novel environments.Dobb·E has demonstrated the ability to achieve an 81% average success rate in solving novel tasks within 15 minutes, based on five minutes of collected data in a new home.The framework provides access to pre-trained models, code, and documentation through GitHub.

Additionally, an open-access paper titled “On Bringing Robots Home” provides further insights into the methodology and results of Dobb·E.