Spatial data company Matterport will collaborate with Facebook AI Research (FAIR) to create the largest-ever dataset of 3D indoor spaces, which can be used for applications including training robots and AI assistants at scale.
The collaboration involves the Habitat-Matterport 3D Research Dataset (HM3D), an unprecedented collection of digital twins, will be made available exclusively for academic and non-commercial uses.
According to the project, the HM3D is an unprecedented collection of 1,000 high-resolution Matterport digital twins made up of residential, commercial, and civic spaces generated precisely from real-world environments.
HM3D is designed to play a role in advancing embodied AI research which seeks to teach robots and virtual AI assistants to understand and interact with the complexities of the physical world.
“Until now, this rich spatial data has been glaringly absent in the field, so HM3D has the potential to change the landscape of embodied AI and 3D computer vision,” said Dhruv Batra, research scientist at FAIR.
“Our hope is that the 3D dataset brings researchers closer to building intelligent machines, to do for embodied AI what pioneers before us did for 2D computer vision and other areas of AI.”
HM3D is free and available now for academic, non-commercial research. Researchers can use it with FAIR’s Habitat simulator to train embodied agents, such as home robots and AI assistants.
Matterport added that HM3D is a foundational step towards helping these agents navigate through real-world environments and better understand the variations of spaces such as bedrooms, bathrooms, kitchen and hallways, as well as the different configurations of those rooms within every structure.
Furthermore, it can assist robots in recognising how objects within rooms are typically arranged so that instructions are correctly understood. This research could one day be used in production applications like robots that can retrieve medicine from a bedroom nightstand or AR glasses that can help people remember where they left their keys.
“We are excited to collaborate with Facebook as we provide the academic and research communities access to this unique spatial dataset that is sure to impact how we work and live,” said Conway Chen, vice president of business development and alliances at Matterport.
“With more than five million spaces captured with the Matterport platform, we are the only company that can offer a diverse library of high-resolution, data-rich digital twins of various styles, sizes, and complexities from across the world. HM3D can also be used more broadly by academia, and we can’t wait to see what innovations emerge.”
