Imagine being on a rollercoaster. Every time you ride the rollercoaster, the experience is the same. It has been designed to provide a captive audience with a carefully planned and designed experience. This is an example of ‘ordinary media’, like films or TV. The director plans everything out for you and each time you watch it, you experience the same things in the same order along the same timeline.
Interactive media is different and so cannot be planned out by a director in the same way. Video games can loosely fall into two categories. Semi-linear and completely free. Semi-linear games are analogous to a maze; where the player makes choices at pre-determined junctions and is therefore relatively controlled by the game. The player has a limited amount of control and choice. Many adventure games, or narrative driven games fall into this category in broad terms. Each level begins and ends at the same place with the same cutscene, no matter how you choose to play the game. A completely free game, or ‘sandbox’ game, like the GTA series, offer different challenges to the sound designer. You can’t have a soundtrack for a level, or a prescribed length of time for a piece of music to play – the player may well not cooperate with your direction.
One answer to this issue is procedural sound design.
Imagine there are four Ace playing cards face down in front of you. You have to guess which suite the card belongs to before it’s turned over. After 6 or 7 goes, you are going to know the order of the cards. That game will become very boring very quickly. Now imagine you add a King, Queen and Jack into the game. Not only do you have to name the suite of the cards, but the order in which they come AND the order I mixed up at random. The guessing game now becomes impossible. There are a limited number of possibilities, but the number is so great that there is no discernible pattern. This is what we mean by procedural sound design.
Instead of playing cards, imagine instead we’ve chopped up five sounds of explosions falling into the sections of ‘crack’ ‘boom’ ‘tail’ and ‘thump’. If we stitch these back together in a completely random order, perhaps even randomising some filters, modulation and volume, then from the listeners point of view, we have an endless cycle of explosions without two full explosions ever repeating themselves. This has the benefit of sounding natural, yet we’ve managed to do it only a small amount of memory and space being used. To quote the Genie in Aladdin “phenomenal cosmic power in an iddy-biddy little space” (Disney 1992).
Here’s an example of it:
According to the Oxford Handbook of Interactive Audio,
In the visual and tangible realms, the apprehension of conditions and process is required, but it is usually the results of the process, not the process itself, that are of interest. (Andy Farnell, Edited by Karen Collins, Bill Kapralos, and Holly Tessler, 2014)
So, basically, how it works is of little interest to most people, but the results are important. When looking at a game that doesn’t use procedural sound design, the effect is obvious.
It can be argued that in a game like Minecraft, a more realistic or natural sound design policy wouldn’t fit with the aesthetics of the game, the point still stands that the same explosion sounds repeated does not sound realistic at all.
However, in games where procedural sound design does operate, the effect is a very natural result.
Gunfire, footsteps, shouts, explosions and most sounds feel natural and don’t instantly repeat on themselves.
In practice, however, it’s not necessarily easy to get right. If one is chopping up a sound that would normally be continuous, there are the issues around stitching sounds together that need to have similar enough timbres that they don’t jar on the ear. This is my first attempt and it suffers from not sounding properly ‘joined up’. This is obviously a skill tooth’s I need to learn.
So after a lot more work and (only a few) tears, I think I’m happy with this version. It feels more natural to me.