Matt Deitke
Matt Deitke
AI Researcher at AI2 and UW CSE
mattd@allenai.org
Seattle, WA
CV

Matt Deitke

Hi! I work on the PRIOR team at the Allen Institute for AI and am also an undergraduate in the Allen School at the University of Washington, Seattle. My research interests are in computer vision, deep learning, and embodied AI.

Previously, I spent several teenage years working on computer graphics, interface design, and visualization in the Department of Athletics at The Ohio State University, the University of Cincinnati, and a variety of other organizations. Later in high school, I studied machine learning and deep learning at Georgia Tech.

News

Images generated with DALL-E.
Avatar
Extremely excited about the release of ProcTHOR! We use procedural generation to scale up the diversity of data in Embodied AI and achieve remarkable generalization results.
Jun, 2022
Avatar
Giving an invited talk at UW's Vision Lunch titled, "Scaling Embodied AI with ProcTHOR: Where We Are and What's Next."
Jun, 2022
Avatar
Co-Organizing the Embodied AI Workshop and the AI2-THOR Rearrangement Challenge at CVPR 2022 in New Orleans.
Jun, 2022
Avatar
The 2nd edition of Rick Szeliski's Computer Vision textbook was published! Ecstatic to have contributed!
Jan, 2022
Avatar
We released an updated revision of the AI2-THOR paper covering its impact and new features!
Aug, 2022
Avatar
Serving as a reviewer for ICLR 2023!
Aug, 2022

Publications

ProcTHOR

🏘️ ProcTHOR: Large-Scale Embodied AI Using Procedural Generation

Matt Deitke, Eli VanderBilt, Alvaro Herrasti, Luca Weihs, Jordi Salvador, Kiana Ehsani, Winson Han, Eric Kolve, Ali Farhadi, Aniruddha Kembhavi, Roozbeh Mottaghi

NeurIPS 2022
TLDR
We built a platform to procedurally generate realistic, interactive, simulated 3D environments to dramatically scale up the diversity and size of training data in Embodied AI. We find that it helps significantly with performance on many tasks.
0
PDF
Semantic Scholar
arXiv
Features in AI2-THOR

AI2-THOR: An Interactive 3D Environment for Visual AI

Eric Kolve, Roozbeh Mottaghi, Winson Han, Eli VanderBilt, Luca Weihs, Alvaro Herrasti, Matt Deitke, Kiana Ehsani, Daniel Gordon, Yuke Zhu, Aniruddha Kembhavi, Abhinav Gupta, Ali Farhadi

ArXiv 2022
TLDR
We introduce The House Of inteRactions (THOR), a framework for visual AI research. AI2-THOR consists of near photo-realistic 3D indoor scenes, where AI agents can navigate in the scenes and interact with objects to perform tasks. It has enabled research in many areas of AI.
479
PDF
Semantic Scholar
GitHub
arXiv
Room Rearrangement in AI2-THOR

Visual Room Rearrangement

Luca Weihs, Matt Deitke, Aniruddha Kembhavi, Roozbeh Mottaghi

CVPR 2021 Oral Presentation
TLDR
We built a pre-training task where the agent's goal is to interactively rearrange objects in a room from one state to another. For instance, the agent may have to open the Fridge and move the Lettuce to the CounterTop. Modern deep-RL struggles.
32
PDF
Semantic Scholar
GitHub
arXiv
Video
RoboTHOR

RoboTHOR: An Open Simulation-to-Real Embodied AI Platform

Matt Deitke*, Winson Han*, Alvaro Herrasti*, Aniruddha Kembhavi*, Eric Kolve*, Roozbeh Mottaghi*, Jordi Salvador*, Dustin Schwenk*, Eli VanderBilt*, Matthew Wallingford*, Luca Weihs*, Mark Yatskar*, Ali Farhadi

CVPR 2020
TLDR
We rent office buildings in Seattle and turn them into apartment studios with many possible furniture and wall layouts. Each apartment layout is then computationally remodeled by hand to enable a simulated robot to interact with it in video-game-like context. We study how well a robot trained purely in the simulated environments can transfer to reality.
93
PDF
Semantic Scholar
GitHub
arXiv

Software

AI2-THOR

AI2-THOR consists of real and simulated environments for interactive robot learning.

allenai/ai2thor
Framework
Interactive Simulated Environments for Embodied AI
Unity
692
170
Procedurally Generate Houses for Embodied AI Training
Python
10
2
A Framework for Training Embodied-AI Agents
PyTorch
198
42
Code for Running the Visual Room Rearrangement task
PyTorch
49
12
Python Package for Distributing Datasets and Models
Python
7
0
Run AI2-THOR with Google Colab
Colab
9
1
The ProcTHOR-10K Houses Dataset
Python
5
0
Evaluation tasks for ObjectNav models
Python
1
0
The Website for the Embodied AI Workshop at CVPR
React
5
2
Explore Trending Papers at CVPR
React
37
3

Workshops

Embodied AI

I've co-organized the Embodied AI workshops at CVPR. Our goal is to bring together researchers to share and discuss the current state of intelligent agents that can see, talk, act, and reason.