DreamFusion is Google’s next-gen, and it is here with its AI-powered text-to-3D-image generator!
DreamFusion is an evolution of Dream Fields, a text-to-3D-image generator introduced by Google back in 2021. Similar to Dream Fields, DreamFusion creates its 3D images by combining a Neural Radiance Field (NeRF) — or a neural network that can develop synthetic 3D scenes using partial 2D datasets — with a pre-trained text-to-image prompt model.
Dream Fields used OpenAI’s CLIP technology as a pre-trained model. However, DreamFusion now uses its own: Imagen, Google’s DALL-E 2 competitor.
So, basically, Google booted Elon Musk’s OpenAI tech and figured out how to use its own.
“Happy to announce DreamFusion, our new method for Text-to-3D!” Ben Poole, a research scientist at Google Brain and co-author of the proof-of-concept paper, wrote on Twitter. “We optimize a NeRF from scratch using a pre-trained text-to-image diffusion model. No 3D data needed!”
While the DreamFusion models aren’t totally realistic, they’re still quite impressive. Its makers state that they are “coherent, with high-quality normals, surface geometry, and depth, and are relightable with a Lambertian shading model.”
This means that they might not be as convincingly realistic as some of those photorealistic DALL-E 2 images, they have all of the right elements. The proportions are correct, the depth is good, and so on. This next version of the tech is certainly a visual improvement from its first iteration.
It is yet to be revealed that when DreamFusion will be available to the public. According to Twitter, it’s already been used to 3D-print a ghost eating a hamburger!
 — or a neural network that can develop synthetic 3D […]&media=https://wonderfulengineering.com/wp-content/uploads/2022/10/imgonline-com-ua-resize-ojYbjS6QwnzmX.jpg){kind=link}