Riffusion

Riffusion takes a unique approach to AI music generation: rather than working directly with audio waveforms, it uses Stable Diffusion to generate visual spectrograms (images of sound) and then converts those images back to audio. This means any image diffusion technique -- text prompts, image-to-image, inpainting -- can be applied to music generation. Users can blend musical styles, create smooth transitions, and explore novel genre combinations. Experimental musicians, AI researchers, and creative technologists use Riffusion to explore the intersection of image and audio generation. The approach produces distinctive-sounding music that often has a more psychedelic or experimental quality than other AI music tools. Users experiment with it for electronic music production, background sound design, and creating audio textures that are difficult to describe in traditional music terminology. Riffusion started as an open-source research project that went viral in 2022 and has since developed into a more polished product. Its technical approach remains unique in the music AI space and continues to attract interest from researchers and experimenters. The spectrogram visualization makes the generation process more transparent than black-box music AI systems.