Thursday, February 18, 2016

More Space Warrants More Content, Especially in Audio

Something's been bugging me for a while, and I finally realized that there's something that makes it less of an issue in video than audio.

Map size. Yes, mainstream games have been getting bigger and bigger, in accordance with Moore's Law or something. Audio Games seem to be aiming this way, too, but it somehow feels more annoying than exciting. I wonder why?

Because the audio equivalent to vast open spaces is listening to your own footsteps for an extra 30 seconds on every mission.

It's true: staring at a vast, empty landscape for an extra 30 seconds can hurt, too. But do you know what vast, empty landscapes have that a series of effectively stationary footsteps do not? A sense of progress! Visually, it's possible to see that one is going somewhere. Without a ton of decorative sounds, footsteps are basically the "You're moving, trust me" sound. (There's something wonderful about the sonar in Bokurano Daibouken 3, and that is that you get an extra 7-15 seconds of hearing yourself get closer to things.)

But how in the world is this to work in audio? I mean, visually, the solution to this is just to have patterns, and occasionally throw in something extra, or even use some randomization (sand dunes, for example). You might not always be able to tell exactly where you are, but you can tell that you're going somewhere.

The audio equivalent to tiles would be just tiling the soundscape with more sounds. But that's inefficient and annoying. On the other hand, all the subtle little details of textures and patterns in the background aren't quite so meaningful in audio, so maybe just tossing in a couple extras every now and then would be sufficient? Especially if they're the left and right channels of a stereo sound and you can fake some kinda spatial sound source or something, but that's a bit beyond the scope of this ramblement.

You know, I'm not sure if it's more common to literally fill up a map with instances of a repeating pattern, or if most games, new and old, just say "Use this pattern here", but I have this strong suspicion that it's the latter. I mean, back in the 16 and 8 bit days, ram was expensive enough that everything had to be arranged in tiles, and further hardware constraints meant lots of other data compression techniques were the norm. Even these days, though, visual editors and graphics libraries include textures and patterns and such, along with the option to try and wrap a whole image around an object.

That's the thing: you don't get repeating patterns in audio. Audio is woefully inefficient and hardware acceleration was destroyed by a combination of surround sound and XBox long ago. Do you really want to try and mix and play an extra dozen, hundred, or thousand sound sources, just to have something approximating tiling patterns? Sure, most of them will be out of earshot, but the more sources you have, the more you have to check on all these conditions (Yay graphs and tree structures, boo me failing to use them in any way approximating reasonably).

Probably not. I mean, most audio games probably wouldn't be big enough or complex enough to care, and since audio is nowhere near as quick or parallelizable as vision, information overload and drowning out useful data with noise are serious concerns. But let's say you want to keep your sound sources to a minimum anyway. Can we create cycling audio patterns? Do we have an audio renderer that looks anything like a screen, with all its pixels and such?

What if multiple copies of the same sound are in earshot at the same time?

So confusing, yet so many opportunities. I fully expect most people will just shrug and go back to whatever they were doing, but eh.


 (Aprone went back and added wind to random places in the more open areas of Swamp. I keep finding myself wishing there were crickets or something in some areas, too.)