More Space Warrants More Content, Especially in Audio

Something's been bugging me for a while, and I finally realized that there's something that makes it less of an issue in video than audio.

Map size. Yes, mainstream games have been getting bigger and bigger, in accordance with Moore's Law or something. Audio Games seem to be aiming this way, too, but it somehow feels more annoying than exciting. I wonder why?

Because the audio equivalent to vast open spaces is listening to your own footsteps for an extra 30 seconds on every mission.

It's true: staring at a vast, empty landscape for an extra 30 seconds can hurt, too. But do you know what vast, empty landscapes have that a series of effectively stationary footsteps do not? A sense of progress! Visually, it's possible to see that one is going somewhere. Without a ton of decorative sounds, footsteps are basically the "You're moving, trust me" sound. (There's something wonderful about the sonar in Bokurano Daibouken 3, and that is that you get an extra 7-15 seconds of hearing yourself get closer to things.)

But how in the world is this to work in audio? I mean, visually, the solution to this is just to have patterns, and occasionally throw in something extra, or even use some randomization (sand dunes, for example). You might not always be able to tell exactly where you are, but you can tell that you're going somewhere.

The audio equivalent to tiles would be just tiling the soundscape with more sounds. But that's inefficient and annoying. On the other hand, all the subtle little details of textures and patterns in the background aren't quite so meaningful in audio, so maybe just tossing in a couple extras every now and then would be sufficient? Especially if they're the left and right channels of a stereo sound and you can fake some kinda spatial sound source or something, but that's a bit beyond the scope of this ramblement.

You know, I'm not sure if it's more common to literally fill up a map with instances of a repeating pattern, or if most games, new and old, just say "Use this pattern here", but I have this strong suspicion that it's the latter. I mean, back in the 16 and 8 bit days, ram was expensive enough that everything had to be arranged in tiles, and further hardware constraints meant lots of other data compression techniques were the norm. Even these days, though, visual editors and graphics libraries include textures and patterns and such, along with the option to try and wrap a whole image around an object.

That's the thing: you don't get repeating patterns in audio. Audio is woefully inefficient and hardware acceleration was destroyed by a combination of surround sound and XBox long ago. Do you really want to try and mix and play an extra dozen, hundred, or thousand sound sources, just to have something approximating tiling patterns? Sure, most of them will be out of earshot, but the more sources you have, the more you have to check on all these conditions (Yay graphs and tree structures, boo me failing to use them in any way approximating reasonably).

Probably not. I mean, most audio games probably wouldn't be big enough or complex enough to care, and since audio is nowhere near as quick or parallelizable as vision, information overload and drowning out useful data with noise are serious concerns. But let's say you want to keep your sound sources to a minimum anyway. Can we create cycling audio patterns? Do we have an audio renderer that looks anything like a screen, with all its pixels and such?

What if multiple copies of the same sound are in earshot at the same time?

So confusing, yet so many opportunities. I fully expect most people will just shrug and go back to whatever they were doing, but eh.

 

 (Aprone went back and added wind to random places in the more open areas of Swamp. I keep finding myself wishing there were crickets or something in some areas, too.)

Spontaneous Ponderings: Accessible Gaming Convention

I have not put much thought into this idea. It's not even an hour old in my head at the time of this writing. But I can already tell it falls dangerously close to the category of Awesome, but Impractical. Mostly, I was trying to think of something awesome I could do in the horrible down time between now and August.

I would probably go to an accessible gaming convention, provided not horribly inconvenient/pathetic/etc. The problem that comes to mind immediately is this: would anyone else? And the answer strikes me as a resounding "Meh, probably not".

The community is not that large

There are plenty of gamers and would-be gamers out there who have needs or interests that tie in to accessibility issues, but that's when you consider it world-wide. AudioGames.net has a very international community, with every continent save Antarctica represented (heck, for all I know, Tierra Del Fuego or Ross Island have some members that I didn't notice). That's a great quality for an online community, but it doesn't translate well into meetspace (or meatspace, I never have checked the spelling of that term).

And, well, my focus is the accessibility for the blind element. Most people interested in that are already hanging out at AudioGames.net, or Game-Accessibility, or Audyssey, etc. What fraction of those people would be willing, interested, and able to attend an in-person event like this? Keep in mind that most of them are legally blind, and pretty spread out, even inside single countries. The only conventions I know about that successfully gather blind people on mass are those held by the National Federation of the Blind, and community opinions on the NFB are quite mixed.

What about Corporate Participants?

Something like this certainly could get the attention of related companies: game publishers in the most optimistic case, narrower companies like Freedom Scientific, Senseg, or maybe even Disney. But they need a promise of an audience. While involvement from companies like these would be quite the boost, what incentives can someone like me provide to get some manner of commitment from such comparative giants?

What about bringing in New people?

I do like the idea of using this to get the attention of people who had no idea that blind people playing games was a thing. The more awareness, the stronger the market can become. At the same time, what I'm imagining is running an ad on the local radio station, and maybe five people coming in for the novelty, and that being the end of that.

It's not horribly hopeless, though; there are lots of aspiring game developers out there, and I imagine a decent minority of those people in the general area might find themselves interested enough to make the trip. Still, when you run the numbers, it'll take more marketing skill than I have to pull off something like this.

Reverse Conjunctive Fallacy! You can't just multiply the probabilities of each category and declare the idea doomed! Marketing doesn't work that way!

Exactly why I'm writing this instead of tossing it onto my pile of "kinda cool ideas to get to eventually". Besides, if push comes to shove, I could sit around with the two people that showed up playing Mortal Kombat 9 all weekend.

I might have reason to wind up in the vacinity of the NFB national convention this year, too, which makes for one heck of an advertising opportunity. Sure, I don't have a booth or anything, there (note to self: add "inquire about getting a booth at the NFB National Convention" to today's todo list).

But mostly, the real problem comes down to the fact that I'm thinking about me doing this. The biggest project I've managed to announce and finish with any degree of quality is the JF IM Adventure, which manages to confuse everyone but me, and like one fellow from Indonesia, to whom I gave many tips, especially regarding the early puzzles. While this would be one fish of a victory (... don't stare at the fish!), does it really sound the least bit realistic for someone whose mutant power is to spread confusion wherever he goes?

Now, if the idea sounds worthwhile, independent of the fact that the loser behind this blog is involved, then that means you might could contribute. And that suddenly boosts the potential success of such an endeavor by an order of magnitude.

Conclusion

So, when it comes down to it, there's potential in a new game accessibility convention. Companies, gamers with disabilities, and random people who thought it sounded kinda neat could conceivably carry it to fruition. The more important question is: can I, should I, and what enormous steps must I take beforehand?

On Platforming Accessibility

(This cross posted at the AudioGames.net forum.)

My biggest problems with audio games are these: They are too verbose, and most accessibility features make no sense in any other context, both of which help keep us isolated from the considerably more exciting mainstream.

How audio platforming generally works:

• They are tile-based.
• You get a list of all objects that are not tiles, with their position mentioned explicitly. Sometimes this is relative (Perilous Hearts), most of the time it's absolute (Mota, Mario, Bokurano Daibouken). BK is notable for including chimes when viewing the list, to make it easier to find them.
• Walls and platforms make sound, but in curious ways. In Mario, you hear them whenever you take a step. In Bokurano Daibouken 3, you hear them on a loop if you have sonar turned on (and this feature is not available early on in the game).
• Pits make sound with every step you take within a certain distance. We've moved away from having your footsteps sound different within two steps of a drop, but it's still countable.
• The Accessible Camera involves examining the map tile-by-tile to learn what's there. In the case of Mario, there's a hear column feature with it to speed things up. You're basically "feeling" of the map, but with audible feedback.

I am totally guilty of doing exactly the above in Mario, and I even applied it to my 3D action/adventure/whatever you call it that no one can make sense of.

I've sense realized that these features only help to further separate accessible platforming from the mainstream. You sure as taxes can't apply the same features to, say, Sonic the Hedgehog (except the object lists. *May be speaking from experience*).

About this time last year, it occurred to me to try making better use of more mainstream features for accessibility's sake. For example, I don't think there's a single audio game (other than my experiment) that uses camera panning the way the mainstream has for a good 20+ years.

We actually had a little discussion on the use of cameras in audio games a while back, and basically, none of the blind-from-birth people understood the concept, and no one could explain it satisfactorily. Which is annoying, because early 90s-style camera-panning, possibly with one or two more degrees of freedom, seems like a low-hanging solution to the problem of determining the shape/size of objects in audio. That was the entire point of my experimental platformer (that relies on the sounds for Swamp).

Now, while I think part of why the Swamp Platformer failed is because people are having a hard time understanding how this works, I also built it with what I had lying around. The sounds in there are not optimized for platform indicators; they're ambiences and events. Even with the game zoomed in considerably to keep clutter to a minimum and detail discernible, sounds still play over each other and make a huge mess. (Also, ledge detection: it is implemented here, but very crudely. You only hear the edge of the exact object you're standing on, and the looping wind sound that is used for this is often drowned out by other sounds.)

My most recent thoughts are that it's best to improve on the camera-and-repeated-sound system, but I recognize that people aren't going to give up tile-by-tile groping that easily (even when the concept of tiles is replaced with arbitrary shapes).

So, I'm thinking that, for future titles, I'll let people feel around the map, but only in realistic ways. For example, if they're carrying around something that can function similar to a cane, they can use it to examine anything in its reach. For a serious range penalty, I might even let them use the character's hands. And these feeling implements will not be protected from traps, without some sort of special protection. So, stick your wooden stick in fire--no more wooden stick. Stick your hand in fire--hope you have gloves. From a mainstream perspective, poking things with a stick is a good way to uncover hidden traps without stumbling into them, or to activate switches from afar, etc (there's a reason sighted people have been known to carry staves in the wilderness). You get to keep the accessibility tool, but it makes sense in context instead of being something only blind people would use.

In the past 24 hours, though, I've run up on a different problem.

Audio Platformers are flat. Mainstream platformers from the early 1990s are not, at least not always. And uneven terrain throws up a whole new accessibility hurtle. Sure, there's braille output (Audio Mario has this), or something to trace the path (Tails in Audio Sonic), but the former is difficult and requires braille displays, and the latter is very difficult to include as anything but an accessibility gimmick.

For now, I'm thinking I'll include some kind of path-tracer in my next game, just to introduce people to the concept, then try to improve it away, somehow. Of course, I could always make it have consequences; for example, it could work similar to the cane trick, and set off traps at a distance... and those traps might need to be set off later. (Mainstream platformers force the player to learn a lot from trial and error. I do not object to Audio Games doing the same.)

Other things I've thought about:

• screwing with DSP effects on footsteps (pitch, volume, echo, reverb, high/low-pass filters) to indicate changes in the amount of vertical space. Example, your steps might have a different pitch when walking under a platform.
• I was not originally opposed to just using echo for everything, but this actually gets a little confusing if not used sparingly. I'm currently thinking it belongs best with platforms that are above the player, though I don't mind the idea of using it for walls and such, too. (BK3 uses echo to indicate drops, which sounds much cleaner than what Mario does.)
• In-character descriptions aren't too terrible, assuming they can be written well enough that a sighted player might find them interesting even if they aren't needed. (Example: if Airik the Cleric's Scout occasionally interjected commentary when reporting. It'd be ridiculous to do this all the time, the way Scout works, but eh.).
• Braille or other tactile output is nice. Too bad it's horribly impractical: braille displays are expensive and most people don't even have them attached to their gaming devices, and any other tactile displays aren't likely to become anything resembling ubiquitous unless someone goes back in time and tosses ESense or Wave-bending at 2007 Apple's feet. That said, Mario's window output mode can display braille through NVDA or Jaws, at least (no word on Window Eyes/System Access/Supernova), and I intend to attempt some sort of braille support in future games. I can only hope that braille tech stops sucking at some point in the near future, so it will be more benefitial than gimmicky.

You will notice that I did suggest that all objects make sound, even though I also said that this could get messy. It's important to choose these sounds well, so that they don't drown one another out. I am also unconvinced that just playing most sounds on a loop is necessarily the best way to go; while it slows down information reception for audio-only players compared to graphical players, it's much easier to follow sounds like these if they play on a rhythm.

I also encourage getting these sounds to make as much sense in context as possible (and in the case of an accessible game with graphics, make the graphics cooperate). For example, if you're using birds to indicate a platform, maybe have the birds fly away when the player lands, and come back when they jump away. Or, if you're having something make a lot of creeking sounds, you might consider giving it a slight animation to show this (could be a lurching motion, or could just be a frame or two of the object appearing offset by a few pixels).

Conclusion

Audio Platforming is all by itself and still in the 1980s, at best. We can do better. As much as I want to get all the attention for something super awesome, someone better at making games that don't confuse people to death could probably apply these ideas FOR GREAT JUSTICE.