Google presented MusicLM, which generates musical audio through a hierarchy of semantic and acoustic tokens. The conditioning lets the model follow a text description while retaining a reference melody.
Music Generation Models
Also known as: Text-to-music · AI music generation · MusicLM · MusicGen · Lyria · Stable Audio
Plain English
Music generation models make audio from prompts, melodies, or other references. The result can be an instrumental clip, a full song with vocals, or audio that a producer edits into a separate project.
Technical Overview
Some models predict sequences from audio codecs, while others denoise compressed waveforms with latent diffusion. Conditioning can specify melody, instrumentation, lyrics, structure, or duration. Model comparison should distinguish audio fidelity from musical coherence over longer passages and account for each release’s weight and output terms.
Timeline (4)
2023
Paper- Paper
Meta described MusicGen, which interleaves multiple audio-codec streams in a single Transformer. This makes text and melody conditioning possible without a separate autoregressive model for each stage.
2026
ReleaseGoogle Cloud made Lyria 3 and Lyria 3 Pro endpoints available in public preview. The short model targets 30-second clips, while Pro can generate a roughly three-minute song.
- Release
Stability AI released Stable Audio 3.0 in downloadable and hosted variants. The downloadable Medium model can generate variable-length music of up to six minutes and twenty seconds.