No products in the basket.
Artificial intelligence (AI) has been a debated topic for decades, often taking an antagonistic role in numerous science fiction films, but before our eyes its capabilities and appearance in the present day continue to grow.
So, is AI really the enemy, or a tool to take us further?
Back in 2016, an AI programme named Benjamin scripted a short science fiction movie entitled Sunspring. If you haven’t seen it, we highly recommend you check it out (here). While the plot is far from perfect, it is a notable step forward for AI and one that caused a tidal wave of developmental possibilities.
Nearly a month after the film’s premier, deep learning was unveiled by MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL). Deep learning is an AI programme that attempts to replicate how humans process and recognise sounds in their neocortex. In simpler terms, rather than seeing a video of wind blowing through grass and producing a wind blowing through grass noise, the software infers what humans would expect to hear from something long and flowy moving in wind and play that back; the fact that it is grass is irrelevant. This is because the programme is focusing on the sound waves associated with specific materials rather than the subject items themselves by utilising a parametric synthesis model. While sounding foreign, the process is not too different from foley artists frying bacon for rainfall or snapping vegetables for breaking bones. It’s all to match was a neocortex would expect to hear, and after watching films all our lives we expect foley to sound a certain way.
To achieve this result, the AI programme was fed 46,000 raw audio/visual signals featuring various objects being hit, scraped, or poked by a drumstick in order for it to ‘learn’ which type of sound should be associated with each visual material. Then, the AI programme was given videos without sound and expected to synthesize a new composite audio based on pulling samples from its database of impact sounds that were generated during the learning process. The result? Sound design that fooled the human participants who chose the AI-generated sound design examples nearly twice as often as the original audio.
Taking AI a step further, Google released NSynth, a music making software that’s creating sounds never heard before by combining audio from instruments. At its simplest, NSynth’s neural network is analysing thousands of sounds from nearly 1000 different instruments, analysing how each note would be played, and then creating a composite vector from each of these so that it can accurately mimic how an entirely new instrument would sound.
What is the future of AI in the audio industry?
While NSynth may not be putting all of the world’s orchestras out of a job, this topic is important to discuss. With various algorithmic systems used to create musical scores in academia for decades, and interest in dynamically created scores for open-world games, interest in this area is certainly growing. Similar to how procedural audio has allowed for smaller memory resources and greater flexibility, AI’s entry into the audio realm should be seen as something that could radically change the sound design process for the better.
Although the creators of the AI programmes admit they need further development, their use in future industry projects might not be that far away. These impressive tools have the ability to work alongside sound designers and take on the laborious aspects of a job and, more importantly, provide a baseline of suggestions to get the designer started. Then, it’s up to art, craft, and expertise of composers and sound designers to take these AI directions and weave them into the final project. While these AI tools are undoubtedly powerful, teaching them the cultural and aesthetic nuances that we expect, and demand from modern entertainment content may be the greatest challenge yet. We’re confident that the invaluable, and subtle touches of a skilled audio team will be valued for many years to come, and that AI will become yet another tool in a skilled designer’s kit.