Wu and colleagues introduced 3D-GAN, which sampled a latent vector and used a volumetric convolutional network to generate an object's shape as a grid of occupied voxels. The model learned to generate new shapes from a collection of existing ones.
3D Generation
Also known as: Text-to-3D · Image-to-3D · 3D Asset Generation · 3D Generation Models
Plain English
3D generation creates digital objects or scenes from prompts such as text, a single image, or several reference views. The result can be a textured mesh for editing in 3D tools or a representation designed for rendering from new viewpoints. With one reference image, the model has to infer what the hidden surfaces look like.
Technical Overview
Some methods use a pretrained image diffusion model to guide repeated renders while optimizing each asset. Others predict 3D directly from an image in a forward pass, or sample a learned 3D latent space before decoding it to geometry and appearance. A mesh stores an explicit surface, while neural radiance fields (NeRFs) and 3D Gaussian splats represent a scene for rendering new views. Mesh generators such as Hunyuan3D and TRELLIS.2 can also produce physically based rendering (PBR) material parameters that let a renderer change the lighting.
Timeline (9)
2016
Paper2022
PaperPoole and colleagues introduced DreamFusion. Score distillation sampling uses a pretrained text-to-image diffusion model to score renders from different camera angles and update one shared 3D radiance field. This let the team use a 2D image model for text-to-3D, at the cost of an optimization loop for every prompt.
- Paper
Deitke and colleagues presented Objaverse, a collection of more than 800,000 annotated 3D assets. A model can be rendered from many camera angles to create training examples that pair images with known 3D shapes.
2023
PaperOpenAI introduced Shap-E, which trains a diffusion model to generate the parameters of a function representing a 3D asset. Shap-E can use those parameters to extract a textured mesh or render the asset as a neural radiance field.
- Paper
Hong and colleagues introduced the Large Reconstruction Model (LRM). A transformer trained on multiview data predicts a neural radiance field from a single image in one forward pass, so it can synthesize new views without separately optimizing a radiance field for each object.
2024
PaperTRELLIS anchored its learned shape and appearance features to a sparse 3D grid. A flow-matching transformer could generate one set of 3D latents from a prompt and decode it to a mesh, a radiance field, or 3D Gaussians.
2025
ReleaseTencent released Hunyuan3D 2.1 with downloadable weights and training code. Its paint stage produces material maps for a mesh, so a metallic surface or a rough surface can respond differently as the scene lighting changes.
- Paper
The TRELLIS.2 paper introduced O-Voxel, a sparse 3D grid that encodes mesh surfaces and material properties together. Its geometry encoding can represent open surfaces such as sheets and leaves without requiring a closed volume.
2026
PaperLi and colleagues introduced Pixal3D, which projects features from a reference image into a 3D grid using the camera model. This ties visible details to 3D locations and lets the system combine evidence from multiple viewpoints when they are available.
See also (8)
Open full graph3D Generation