ElevenLabs debuts AI-powered tool to generate sound effects

Voice cloning startup ElevenLabs introduced a new tool for users to generate sound effects through prompts today. The company first announced the project back in February.

The tool, available to all users starting today, allows users to type in prompts like “waves crashing,” “metal clanging,” “birds chirping” and “racing car engine” to generate snippets of sounds.

The sound effects tool can also generate instrumental musical clips of up to 22 seconds with prompts such as guitar loops, jazz saxophone solos and music techno loops.

Free users get 10,000 character generations per month — a sound byte generation takes around 150 characters per request. Essentially, free-tier users can generate nearly 60 sound effects per month. They also have to attribute the sound to “elevenlabs.io” in the title while publishing any content containing the sound clip.

ElevenLabs said that it used Shutterstock’s audio library containing licensed tracks as a tool to train its model. The company added that users that tried out the tool during the alpha testing phase included video game developers, film producers, social media content creators and marketers.

The startup noted that the tool doesn’t allow sound generation through prompts that violate its Prohibited Content and Uses Policy, which includes topics like self-harm, threats to child safety and fraud.

While there are only a few companies and startups working on AI-powered sound generation, ElevenLabs might find the music generation space crowded. Stability AI-backed Harmonai has released Dance Diffusion; Google has worked on MusicLM; OpenAI has Jukebox; and Meta has its AudioCraft model. TikTok and Adobe have also experimented with generative AI-based music creation tools of their own.

We’re launching an AI newsletter! Sign up here to start receiving it in your inboxes on June 5.