Note: This article is based on a workshop by Christopher Wieduwilt and Danny Brown.
Watch the full recording
From Thomas Edison’s phonograph to the present day, technological innovation has long helped audio professionals accomplish things previously considered impossible. Early wax recordings allowed for the creation of the first record labels. Innovations in MP3 compression made the iPod revolution possible. Affordable DAWs empowered artists to record award-winning albums from their living rooms. Through each of these breakthroughs, the talented people who embraced the latest technology led their industries, while those on the sidelines were forced to catch up.
Today, the major breakthrough is artificial intelligence. According to a report by market.us, the AI Audio Processing market is projected to grow from $3.8 billion in 2023 to $18 billion by 2033. And many professionals are already seeing the benefits. In this article, we’ll focus on four key categories where AI technology can help professionals process audio more effectively.
Stem Separation
With AI models like those from Music AI, audio professionals can input any existing audio, and the software will split it into its separate parts while keeping a very high signal to distortion ratio (SDR). This allows you to modify each of them separately or potentially remove some parts altogether. The use cases for this technology are endless. For example, a record label could remove vocals from songs in their library and license them to advertisers or movie studios, professional sports teams could edit out copyrighted music from game clips so they can share them on social media, and artists could lift a vocal performance from an existing track to reimagine the recording with new instrumentals.
Learn how to do it yourself with our how-to demo
Metadata & Classification
Many audio professionals can relate to spending hours looking for just the right music or sound effect in a large audio library. Sometimes, you never even find what you’re looking for. It’s a common problem, because organizing audio is extremely difficult. Different people hear sounds differently, which can lead to a lot of inconsistency in how things are labeled. Additionally, even if organization were done perfectly, manually tagging a large amount of audio can take a massive amount of time.
There are serious consequences to these issues. Not only do producers lose valuable time, but artists may lose royalties because their work is improperly labeled, and listeners might get lower quality recommendations from streaming services because the platform mistakenly mislabeled its content.
AI models help solve these problems by quickly analyzing large amounts of audio, then tagging and labelling it across numerous potential categories, including genre, mood, BPM, instruments used, and so on. Music AI partners with some of the top audio technology solutions in the industry, such as Audible Magic, Source Audio, and Cyanite to help deliver this type of fast, consistent classification.
See how to do it yourself
Try the Identify Music From a Video Workflow Template. Try the Remove Music from a Video Workflow Template.
Transcription & Alignment
Manual transcription can take countless hours of going through videos or audio tracks, often second by second. Then, even after all that work, the transcription usually still isn’t perfectly accurate.
Models like those from Music AI can go through vast amounts of content and quickly provide transcription and alignment with an extremely low error rate. This not only frees up time for audio professionals to focus on other tasks, but it also allows artists and businesses to produce content that simply wouldn’t be feasible otherwise. For example, making lyrics videos for a large collection of music suddenly becomes fast and easy with these tools.
Translation & Localization
Reaching a global audience has historically been almost unattainable for many smaller creators because of the costs of translation and localization. In the past, content creators would almost have to remake content from scratch for new markets, which meant things like hiring all new voice actors, and potentially working with localization consultants. This unfortunately often meant that only the biggest firms had the resources to go truly global.
AI has the power to take large amounts of audio content and translate it into more than 80 languages, then produce vocal recordings in the chosen language while still maintaining the basic characteristics of the original voice recording. Imagine using one voice actor to deliver a quality performance in dozens of languages. This incredible technology can help level the playing field for smaller firms, while also helping larger companies move more quickly.
Learn how to do it yourself with our how-to demo
Try the Voice Localization and Transcription Workflow Template.
AI on the Edge: A Glimpse into the Future
As incredible as these use cases are, they are still only the beginning of how AI will ultimately impact the way we process sound. There are so many exciting developments just around the corner.
To take just one example of an upcoming trend, let’s look at edge AI. With the development of specialized AI hardware such as the latest generation of Snapdragon chips from Qualcomm, it’s only recently become feasible to start running many of these models directly on consumer devices. This is already starting to allow users to enjoy many of the features we’ve discussed here without the latency that comes from needing an internet connection.
Just think of all the exciting things you could do with this type of technology in real time. Imagine watching a TV show and finally being able to raise the volume of the dialog without also turning up all of the background sounds, or using stem separation to identify and eliminate the sound of feedback before an audience can hear it during a live performance. It’s a very exciting time to be working in audio.
Check out this video to learn more
The workshop this article is based on was made in collaboration with Christopher Wieduwilt, The AI Musicpreneur. He's a former broke musician who now helps music professionals master AI for music production and promotion. Join his newsletter for weekly AI tools, prompts, and strategies.
