mattd@allenai.org

Matt Deitke

Hi! I work on the PRIOR team at the Allen Institute for AI and am also an undergraduate in the Allen School at the University of Washington, Seattle. My research interests are in computer vision, deep learning, and embodied AI.

Previously, I spent several teenage years working on computer graphics, interface design, and visualization in the Department of Athletics at The Ohio State University, the University of Cincinnati, and a variety of other organizations. Later in high school, I studied machine learning and deep learning at Georgia Tech.

Preprints

[3]

🏘️ ProcTHOR: Large-Scale Embodied AI Using Procedural Generation

Matt Deitke, Eli VanderBilt, Alvaro Herrasti, Luca Weihs, Jordi Salvador, Kiana Ehsani, Winson Han, Eric Kolve, Ali Farhadi, Aniruddha Kembhavi, Roozbeh Mottaghi

In Submission.

TLDR

We built a platform to procedurally generate realistic, interactive, simulated 3D environments to dramatically scale up the diversity and size of training data in Embodied AI. We find that it helps significantly with performance on many tasks.

PDF

arXiv

Publications

[2]

Visual Room Rearrangement

Luca Weihs, Matt Deitke, Aniruddha Kembhavi, Roozbeh Mottaghi

CVPR 2021 Oral Presentation

TLDR

We built a pre-training task where the agent's goal is to interactively rearrange objects in a room from one state to another. For instance, the agent may have to open the Fridge and move the Lettuce to the CounterTop. Modern deep-RL struggles.

[1]

RoboTHOR: An Open Simulation-to-Real Embodied AI Platform

Matt Deitke*, Winson Han*, Alvaro Herrasti*, Aniruddha Kembhavi*, Eric Kolve*, Roozbeh Mottaghi*, Jordi Salvador*, Dustin Schwenk*, Eli VanderBilt*, Matthew Wallingford*, Luca Weihs*, Mark Yatskar*, Ali Farhadi

CVPR 2020

TLDR

We rent office buildings in Seattle and turn them into apartment studios with many possible furniture and wall layouts. Each apartment layout is then computationally remodeled by hand to enable a simulated robot to interact with it in video-game-like context. We study how well a robot trained purely in the simulated environments can transfer to reality.

Featured Software

AI2-THOR

AI2-THOR consists of real and simulated environments for interactive robot learning.

Workshops

Embodied AI

I've co-organized the Embodied AI workshops at CVPR. Our goal is to bring together researchers to share and discuss the current state of intelligent agents that can see, talk, act, and reason.