Nvidia’s new AI magic turns 2D photos into 3D graphics

Nvidia has made another attempt to add depth to shallow images.

After converting 2D images to 3D scenesmodelsand videosthe company has shifted its focus to editing.

The GPU giant has a new one today AI method that converts still photos into 3D objects that creators can easily modify.

Greetings, Humanoids

Sign up for our newsletter now for a weekly roundup of our favorite AI stories delivered to your inbox.

Dubbed 3D MoMa, the technique could give game studios an easy way to change images and scenes. This usually relies on time-consuming photogrammetry, which involves taking measurements from photos.

3D MoMa speeds up the task through inverse rendering. This process uses AI to estimate the physical characteristics of a scene — from geometry to lighting — by analyzing still images. The photos are then reconstructed into a realistic 3D form.

David Luebke, Nvidia’s VP of graphics research, describes the technique as “a holy grail that unites computer vision and computer graphics.”

“By formulating each part of the inverse rendering problem as a GPU-accelerated differentiable component, the NVIDIA 3D MoMa rendering pipeline uses the machinery of modern AI and the raw computational power of NVIDIA GPUs to quickly produce 3D objects that creators can import , edit, and unlimited extend into existing tools,” says Lubeke.

3D MoMa generates objects as triangular meshes — a format that is easy to edit with commonly used tools. The models are made in under an hour on a single NVIDIA Tensor Core GPU.

Materials can then be placed on the mesh as skins. The lighting of the scene is also predicted, allowing creators to change the effects on the objects.

Triangle meshes define the shapes used in 3D graphics and modeling.