Civic Creative Base Tokyo

Closed today

JA

Home / Audio Game Center Caption

Audio Game Center Caption

The Three Sections of This Exhibition

  1. Section I: Audio Game Exhibits: The Fun of Playing Games through Sound
    • In addition to playing the racing games, horror games and action games, visitors can also play on the Appcreate audio games using their bodies.
      1. Super Audio Racing! (Racing Game)
      2. Screaming Strike neo (Rhythm and Action Game)
      3. Where the Ghost Dwells (Horror Game)
      4. Audio AR Game Maker (Audio Game Design App)
  2. Section II: Hackathon: Collaborate on Making Games from Sound
    • Exhibited here the 7 prototypes of the games created during the Hackathon of the opening event.
  3. Section III:Audio Game Archive+: Play the World of Audio Games
    • Play the audio games developed by creatorsfrom around the world! We’ve also prepared accessibility reviews of the games explaining how they can be enjoyed.

Foreword by Civic Creatie Base Tokyo [CCBT]

Civic Creative Base Tokyo [CCBT]’s feature as a creative hub and lab to introduce artistic and technological expressions as well as their processes and technical backdrop is fully introduced in the space’s Showcase, one of its core programs. For Audio Game Center + CCBT, the Showcase’s third iteration, we welcome DDD (Disability Driven Design) Project—consisting of programmers, curators, and disabled collaborators—as our partner.
It goes without saying, audio games have been released online by developers across the globe and played by a wide range of gamers and, since 2017, DDD have been developing their own audio games and organizing exhibits and workshops to widen their reach of players. It can be said that DDD’s core exploration is in the potential broadening of the human perception as they develop and play games with those who are visually impaired. The universe of audio games are created with sound alone and, furthermore, allows players to experience movement, speed, and the game’s portrayal through only auditory senses. As such, these experiences evoke a rich layer of information that is inherent in sound, and stimulate our imaginations in ways that we easily dismiss in our daily lives.

During Audio Game Center + CCBT, we will exhibit works developed by DDD thus far and organize several programs where CCBT will open its space as a laboratory for experimenting and thinking about the means and meaning of “creating with sound” and “playing with sound.” In the hackathon “Creating Games with Sound,” the public will work together with blind facilitators, engineers, and artists to create games. These prototypes will then be presented in the “Audio Game Archive+” section of the Showcase that exhibits audio games collected from around the world. Furthermore, we will organize meet-ups where participants can experience games and discuss, with developers, the many issues surrounding accessibility and inclusion in art and technology.
We hope that you will enjoy experiencing these auditory expressions created by the merging of technology and human perception and, moreover, we welcome you to join our endeavor in proposing and designing novel ideas in the making of audio games.

Note ”What is Audio Game Center + CCBT?”

DDD Project (Disability Driven Design Project)

Games connect people; they are means to communicate. As such, “disabilities” are challenges that players must overcome together, and, therefore, the criteria for who can and cannot participate must not exist.
Audio games, which are created from and played with sound, are a form of entertainment that has primarily been enjoyed by people with visual impairment. Our initiative for the “Audio Game Center” began in 2017 and emerged from the idea of creating a space that is centerd and focused on those with visual impairment but still welcomes many others, regardless of their visual perception. This differs somewhat from the conventional audio games where they were primarily enjoyed by people with visual impairment. We believe this is a more inclusive project that is opposite to the usual thinking: sighted people are helped by those with visual impairment to a doorway that leads into a universe constructed by sound.
Sighted people, who are usually accustomed to processing a lot of information visually, often complement audio information with visual information. For example, once they see a car pass by, few people continue to focus on the sound of the car until it fades away. Therefore, the “audio games” created by sighted people often end up being “visual games” where players take cues from the visual elements on screen. To put it another way, because they are unaccustomed to playing with sound, it is, in fact, those who depend on sight who are at a disadvantage when playing audio games.
We have always tried to provide opportunities for people to join and play the games we have developed until now. However, for Audio Game Center+CCBT, we have been fortunate to expand our reach to showcasing audio games created by the pioneers in the field and also organize hackathons and workshops. We hope each of these will allow us to share many exciting moments as we “create and play with sound” and help grow the community of creators interested in the field. Beyond this, we believe, lies the collaborative foundation for which we can work together in overcoming the many “disabilities” we each may have.

Section I: Audio Game Exhibits: The Fun of Playing Games through Sound

These exhibits feature audio games that can be played with others. The games are made through expertise in art, technology, and design as well as the ideas and sensations generated by the conditions of a disability. In addition to racing, horror, and action games, visitors can try an app that enables them to create audio games using their body.

Super Audio Racing! (Racing Game)

  1. Outline
    • In this racing game, you drive a car by relying on a melody that tells you which direction to go. Turn the steering wheel depending on whether you hear the melody in the left or right headphone. You have to steer faster as the race speeds up if you want beat the other players! Sense the space from the sounds, imagine the speed you’re going, and then roar across the finish line.
    • Supervision: Kakehi Yasuaki
    • Game Design & Development: Audio Racing Team
  2. Technical Note by Kakehi Yasuaki
    • Guided by the signal sounds from the headphones, the player navigates through the space of this racing game, turning left and right until they reach the goal. The signal sounds are presented through the sound image localization distributed from the stereo panning from both sides, prompting the players to steer the wheel as they make their way through the course. In addition, we have added live commentaries, rapping, and sounds effects from the objects placed on the course to provide multiple auditory cues to help the players better understand the game’s dynamics for less restricted movement in their race towards the goal.
  3. Technical keywords
    1. Real Time
      • If a character moves on the screen and a sound is heard the instant we press a button, it creates an immersive experience for the player as if they are the character. This might seem like a simple process, but the game has to receive the input from the button, play the sound, and then send that signal to the speaker, all timed to the millisecond and while displaying the screen and processing other things.
    2. Convolution
      • If we record the reverberation of short pulses in a space, and then integrate another sound recorded separately from that data (a process called convolution), it is possible to add the reverberation that would result if that sound was made in the space. Much like recorded sound, reverberation can be sampled.
    3. Speech Synthesis
      • There is a long history of attempts to artificially reproduce the human voice, from Wolfgang von Kempelen’s speaking machine that used bellows to Bell Labs’ vocoder that synthesized the human voice through an analog electrical circuit by pressing a keyboard. With the development of machine learning, recent examples of speech synthesis are almost indistinguishable from an actual human voice. The day may soon come when character dialogue is not recorded but entirely generated in real time.

Screaming Strike neo (Rhythm and Action Game)

  1. Outline
    • There’s just one rule: hit where you hear something! This audio game offers that universal feeling of release and satisfaction you get from hitting something. To play the zombie version of the game, listen to the sounds to tell which direction the enemies are coming from and knock down the ones coming at you. In the “flying tiles” version, punch and smash the boards and tiles you hear flying through the air.
    • Development: Nozawa Yukio
    • Narration: Kitamura Naoya
  2. Director’s Note
    • This is a simple game where the player relies on his/her auditory senses to hit enemies that slowly close in. Wearing headphones—the volumes from the left and right ear are each adjusted so that the enemies seem to approach from the left, right, or in front—players carefully listen and try to hit them. So the key to achieving a high score is to correctly identify the direction from which they draw near.
      This evolved from a game I personally created in 2013, and, since the early versions, I had selected and recorded over 100 different “strange screams” for it. The idea was less about being a violent game, and more about making players laugh at how absurd the screams sound when playing.
      This game was chosen as one of the first pieces to be included when the Audio Game Center project began because its system was very straightforward. Now, after several updates, we proudly present the current “neo” version.
      Having players actually try to hit the enemies adds an immersiveness to the game where they are made to use both their auditory and somatic senses. Furthermore, in addition to the abundant use of screams, as in the original mode, we have added an exhilarating new mode where players smash tiles to a lively rhythm. With the exciting narration that accompanies the game, we do hope you enjoy playing it.
  3. Technical keywords
    1. Listening to Algorithms
      • In the early days of computers, speakers were output devices that were cheaper and easier to debug than a display. Programmers would connect a speaker to an electrical circuit to directly listen to a signal, and check if their calculations were correct. Taking advantage of this, people then began to write programs that produced pulses at particular rhythms through a speaker, which marked the start of using computers to make music.
    2. Object-based Audio
      • In a game, recorded sound effects and music do not simply play in time with the on-screen action. In combination with the sound source or metadata (the player and an enemy or item’s location or type), the tone and orientation is processed in real time. In first-person shooter games, for instance, this provides important clues about the position or direction of a player or opponent.
    3. Auditory Localization
      • Sounds to our left reach our left ear slightly faster. Based on this lag and difference in volume, we are able to sense the location of the source of a sound to the left or right of us. We perceive differences between up and down or front and behind by shifts in sound quality caused by sound bouncing off our ears, head, or the walls of a room. If we play recorded sound through earphones, for instance, we can reproduce its auditory localization, and digitize sound quality change patterns to hear the sound from any location. Exhibits include Rainbow Six Siege and Cyberpunk 2077.

Audio AR Game Maker (Audio Game Design App)

  1. Outline
    • Audio AR Game Maker lets you create your own audio game using your phone or other device to insert various sounds in a physical space through augmented reality technology. The app is designed for both blind and sighted users. Brainstorm rules with your collaborators and then make your own game!
    • Game Development: Urata Taiga
    • Sound Creation: Nozawa Yukio
    • Technical Direction: Kakehi Yasuaki
    • Production: Tanaka Miyuki
    • Technical Advice: Inukai Hiroshi, Kato Hideyuki
    • Cooperation: Misawa Tsuyoshi
  2. Technical Note by Kakehi Yasuaki
    • With sound sources placed in the physical exhibit space, this Augmented Reality app allows users to develop and experience an impromptu audio game. The app has a total of 4 modes: settings, sound placement, sound editing, and play mode, where users can simply input commands by swiping the screen. The command and its feedback are provided via audio, so users can continue to control the modes and play without checking the screen. Furthermore, the location sensor embedded in devices such as smartphone cameras allows the app to approximate the users’ positions, allowing multiple users to play simultaneously by simply reading each others’ markers.

Where the Ghost Dwells (Storytelling Horror Game)

  1. Outline
    • In this narrative-driven horror game, you’re guided by sounds and a female ghost. Why does this friendly ghost appear in front of you? Follow the various sounds you hear (created with stereophonic sound technology) to solve the mystery as you proceed through the different parts of the story.
    • Game Design & Text: Fujiwara Kana
    • Game Design & Concept: Tanaka Miyuki
    • Music: Kakudo Manami
    • Technical Development: Akagawa Tomohiro (A-KAK Inc.), Kakehi Yasuaki  3D Sound
    • Production Technical Support: Sony PCL Inc.
  2. Technical Note by Kakehi Yasuaki
    • Sony’s 360 Reality Audio technology offers an immersive sound field created by a group of speakers positioned within the exhibit space. Information on the player’s position and their orientation is gathered in real-time through the handheld controller and is interactively reflected into the sounds the players hear and experience throughout the game. By providing several layers of information—including acoustic feedback through open-ear headphones and haptic feedback via the controller—multiple players can each experience worlds and encounters that are unique to their selections within the game.

Section II: Hackathon: Collaborate on Making Games from Sound

  1. Outline
    • To commemorate the opening of Audio Game Center + CCBT, we held a three-day hackathon where participants brainstormed and developed their own audio games. A total of 22 participants, selected by open-call, joined our engineers, artists, and blind-facilitators to create a total of 7 games. Exhibited here are documentations and also the prototypes of their games made during the hackathon.
    • Dates: July 13 – July 15 [3 days]
    • General Facilitator: Inukai Hiroshi (e-Sports Producer, Game Director, Sports-play Expert)
    • Facilitators: Ishida Hayato, Egashira Misato (esports player / ePARA), Omachi Yoshiki, Kato Hideyuki (systems engineer, musician), Shirai Takaaki (violinist / Director, Invisible Gaming Lab game accessibility research team), suzueri (Suzuki Elico) (sound artist), Tsuji Katsutoshi (accessibility consultant / SmartHR Inc.), Nagashima Chihiro (engieer, researcher), Nozawa Yukio (programmer), Mashiro (graduate student, Rikkyo University), Matsuura Tomoya (sound-maker), Miura Daiki (engineer / CCBT technical staff)
  2. Hackathon Report by Inukai Hiroshi (General Facilitator)
    1. Day 1: July 13 (Sat)
      The hackathon began with a bang with a total of 34 members: 22 open-call participants, 6 “blind-facilitators” who have visual impairment, and 6 engineer/artist-facilitators. The first 40 minutes was given to a talk on the definition(s) of audio games, the differences between audio games and accessibility in other games, the relevant technologies, and the nature of games–how they are played and enjoyed.
      We designed the hackathon to be an environment where the participants—whose nationalities, ages, gender, impairment, and tastes and hobbies all differ—can make good use of the technologies currently available to us. Without pens or paper, they came up with many different ideas for new audio games using the text-to-audio functions and Google Docs on computers and devices. We then categorized the participants’ ideas and interests both manually and via ChatGPT which resulted in grouping them into 7 teams, from Team A to Team G.
      The hackathon then entered the phase of developing the games. Each team continued to brainstorm and finesse their games and, during the progress presentations, each team introduced their “Game Proposal” and “Tasks Required” to realize their ideas. 
    2. Day 2: July 14 (Sun)
      The participants dedicated their entire time—-from 9 in the morning to 9 in the evening—-to further developing their games.
      Two types of facilitators—engineers and artists—were appointed to each team to help with the developing process and played the important role of providing technical support and creative advice, sharing their expertise and ideas on audio engineering, how they could improve their game designs, and methods of art-making. Our blind-facilitators also gave feedback on the experiential qualities and levels of accessibility from the perspective of players who have visual impairment. A total of 12 facilitators—an attentiveness that was characteristic of this hackathon—played crucial roles in providing constructive responses that elevated the playfulness of audio games and level of accessibility of the teams’ ideas.
      While participants enjoyed the game-making aspect of the hackathon, they also faced communication difficulties, time constraints, and technical challenges, occasionally experiencing tension and conflict among them.
      For the progress presentation in the evening, each team prepared their games so that they could be played. Upon receiving suggestions from other teams, the Programs Directors, and staff, the teams were able to revise their games objectively and identify any improvements that could be made.
    3. Day 3: July 15 (Monday)
      The hackathon’s last day began at 9AM sharp and, until 1PM, the teams continued to refine their games from different approaches: those that were incorporating as much of their ideas as possible, others that were simplifying their ideas, those that were debating how best to explain their game, and also those who were refining their sound recordings. Visitors to Audio Game Center + CCBT were also coming in and the excitement at CCBT was building.
      At 1PM, the game-making time came to a close. Each team displayed their games in a simple manner so that others and the visitors could play them. Initial proposals, blueprints, devices that ended up being left out, and woodwork that were still mid-way were left in disarray in the room, leaving CCBT’s space to look like an arcade. Watching the games’ creators positioning their games for display was like watching a “build-and-play game show.” The participants played each others’ games, shared their thoughts, and commended each others’ creativity.
      Although the three-day hackathon had ended, the participants continued to chat online and had already begun to discuss further improvements and plans for exhibiting their prototypes.
  3. In Exhibiting the Prototypes
    • During the three-day hackathon, the participants communicated, in teams, the Who, What, When, and How of materializing their audio games. For the participants, we hope the exhibit will be an opportunity to reflect—both on personal and societal levels— the reasons behind developing and creating their particular games. For visitors, on the other hand, we hope they will experience both the coincidental and inevitable aspects that emerged as solutions to the participants’ creative thinking. By organizing these occasions where developing and playing go back and forth and sharing them, we hope to foster the culture of “developlaying (developing-and-playing)” nature and society.
    • General Facilitator: Inukai Hiroshi (e-Sports Producer, Game Director, Sports-play Expert)

Results exhibition  Hackathon: Collaborate on Making Games from Sound

  1. sound sommelier
    • Developer: Kihara Tomo, Kusano Kota, Yoshihara Miwa
    • Outline
      • A game where you try to distinguish between two very similar sounds, like the sound of pouring cold water versus hot water. On the left of the screen is Sound 1, and on the right is Sound 2. Tap to switch between the two sounds, and press long to select the sound. Playing this game may make you more sensitive to the subtle differences in sound that you wouldn’t normally notice.
    • You can play this game with multiple people, in which case use a majority vote to determine your answer.
    • Japanese (Visuals and Audio)
    • English (Visuals only)
    • Review by Inukai Hiroshi
      • The simple idea of “distinguishing sounds” in this game so eloquently captures the essence of audio games. While we played with identifying musical scales in electronic games, games that required us to distinguish subtle, everyday sounds only appeared after the Hi-Fi- audio environment of the 21st century became available to us. This game points out that fact in the history of games and will, I believe, be a reference point for many creators to come.
  2. Auditory Pairs
    • Developer: Iwatani Mahiro, Omachi Yoshiki, Tsuji Katsutoshi, Fukano Shota, Fukuhara Toshitaro      
    • Outline
      • In this game, you are trying to make pairs of sounds–an auditory version of the classic matching game in cards. There are 25 buttons, each producing different sounds when pressed and you can find and eliminate a pair of sounds by pressing two buttons that sound the same. You have 10 minutes to find all the pairs and eliminate all the sounds. But be careful of the bomb! Pressing it can reduce your remaining time.
    • Language: Japanese
    • Review by Kakehi Yasuaki
      • The simple rule of replying only on sounds to play the matching game is excellently elevated with the addition of a time limit and bombs, creating an more intense experience for players. We can tell the team worked hard on the sound design too which adds to the intensity and also enjoyment in playing the game. The eagerness of the hackathon participants to play this game in their daily lives is telling.
  3. Ordinary Unfriendly Elevator
    • Developer: Iwatani Mahiro, Omachi Yoshiki, Tsuji Katsutoshi, Fukano Shota, Fukuhara Toshitaro
    • Outline
      • This is an escape game from an elevator where you reach for the first floor by relying solely on sound by stopping at various floors. As you make your way down, listen to the sounds through your headphones and, when you think you’ve arrived at the first floor, press the ↑ key on your keyboard. If you think it’s another floor, press the ↓ key, and make your way safely to the first floor. This game was inspired by real-life experiences of a person with a visual impairment who had difficulties getting off at the desired floor.
    • Language: Japanese
    • Inspiration for this game
      • When discussing what kind of game to create, we decided on trying an “auditory escape game.” We started brainstorming where the game would take place and what kind of places we could escape from when Egashira-san, our blind-facilitator, mentioned, “I actually have trouble with elevators.” So all tried riding the elevator with our eyes closed and realized that, with no announcements, it was extremely difficult: we couldn’t tell which floor we were on nor whether the elevator was going up or down. This inspired us to create a game that can share the challenges people with visual impairment face every day. 
        We recorded the sound of building’s floors and elevator on the field using binaural microphones and shotgun microphones for realism.
    • Review by Kakehi Yasuaki
      • A game about going down floors only with the help of sound: a situation that people with visual impairment frequently experience. It offers players a unique sense of focus and tension, together with the fun of imagining the “sceneries” of each floor based on sounds we usually may not pay much attention to. During the hackathon, participants worked on creating a device that simulates an actual elevator and its buttons. It is exciting to see how this project will continue to develop.
  4. OL, a magical girl from the office
    • Developer: Suzuki Ryota, Nozawa Yukio, Miyamura Azusa
    • Outline
      • This is an action game where OL, an office-lady turned magical girl, battles and saves the world from NeruMon a monster who puts everyone to sleep and causes mischief!
    • How to play
      1. The monster NeruMon is attacking! Hold the wand vertically and shake it side to side to transform yourself from an office lady to a magical girl!
      2. Once you’ve transformed, it’s time to attach NeruMon! Shake your wand left-to-right to charge your power and thrust your wand forward to attack!
      3. Give two attacks to defeat NeruMon and save the world!
        Note: During your attacks, NeruMon could release his sleeping power! If you get hit by it three times, you’ll be too sleepy to fight so be careful. If you do sleep, wake up and try again!
    •  Language: Japanese
    • Inspiration for this game
      • We wanted to create something flashy AF!
        We wanted a grand theme, like saving the world!
        So, we decided to become our long-time aspirations, a magical girl!
    • Review by Tanaka Miyuki
      • This game makes us realize the surprising fact that there are very few heroines in audio games; at least, the audio games that I have experienced so far have been developed from the male perspective. In this world of sound where visuals are unnecessary and the constraints of appearances of gender is removed, any player can immerse themselves in enjoying being the heroine.
  5. Birdatch
    • Developer: Ishida Hayato, Kai Kasei, sion, Shirai Takaaki, Nakayama Sogen, Nambu Ryuichi, Mashiro, Li Isei
    • Outline
      • In this game,  you have to find and feed  Audion, the invisible little bird. Walk around the space while carrying your smartphone and look for him. As you get closer, his chirps and feathers fluffing will become louder anddouble-tap to throw food for him. If you are successful, you will hear it eating. If you feed him five times, Audion will be full and you’ve cleared the game. Play with Audion as he flies around the room!
    • Language: Japanese
    • Review by Nozawa Yukio
      • Games often require players to defeat enemies and opponents which can sometimes become tiring. Audion, the bird in this game, is designed carefully to its details and, despite him being purely an auditory element in the game, he seems to be truly alive. The game offers a calming experience, allowing players to momentarily detach from the harsh, competitive reality and relax with Audion.
  6. Touchstones
    • Developer: Carmen Papalia, Suzueri (Suzuki Elico), Michael Smith-Welch, Matsuura Tomoya, Yamasaki Ami
    • Outline
      • Touchstones is an observational game that deepens the player’s curiosity and appreciation for the sounds and textures of the built environment. In Touchstones the self-described non-visual artist, Carmen Papalia, invites you into the non-visual world and navigates you through Shibuya Tobu Hotel and the surrounding areas while explaining the sounds that are important to him. Instead of beating the clock or destroying a monster with a magical sword, you are invited to use Carmen’s detection cane to activate his descriptions and the sounds of the objects that he encountered on his walk.
        Stand in place with his cane and move the way someone using a cane would to navigate unfamiliar spaces and detect obstacles. A series of four gestures will activate four corresponding sound descriptions where Carmen will explain the relationship between the hidden sounds and the objects that made them. Then, using the cane as a tactile sensor, tap the floor panels (switches) and discover the sound that Carmen found on his explorations. 
        We hope that Touchstones will inspire each player to begin their own explorations into the beauties and nuances of the non-visual world!
    • How to play
      • Use the detection cane to activate the sounds of the objects Carmen encountered on his walks. You can hear 4 different voices (sound descriptions) by moving the cane in four different ways.
      • Four Ways to Move the Cane:
        1. Tap the floor more than 4 times in the same spot
        2. Swing the cane side by side for more than 5 seconds
        3.  Move the cane back and forth for more than 5 seconds
        4. Tap the floor more than 5 times while shaking the cane side to side.
      • After listening to Carmen’s descriptions, tap the 4 floor panels with the cane and find the sounds being described.
      • Note: Think about the relationships between the sounds you hear and their descriptions.
    • Language: English / Japanese
    • Review by Tanaka Miyuki
      • This ia an experimental game resulting from a collaboration of skilled artists exploring a game that is not about winning or losing. Although there is no final goal or narrative, we can consider it a form of “game” or “play” as players borrow Carmen’s body and senses, a person who is blind, to interact with the game field.
  7. VibClap
    • Developer: Kato Hideyuki, Takeuchi Shin
    • Outline
      • We all know what it is like to have mosquitoes buzzing around in the middle of the night when you’re trying to sleep, and not being able to see them and only hearing them approaching can be annoying. In this game, Fairy and Clappy can help us in that situation. Fairy will vibrate to indicate to you the direction of the mosquito and Clappy will snap to swat the mosquitoes away.
    •  Language: Japanese
    • Review by Inukai Hiroshi
      • Pinch the cute “Fairy” with your left hand and it will pull you towards the direction of the enemies. This device uses our sensory illusions of how we think we are being pulled when we feel specific vibrations. Players can enjoy the unique combination of sound from left and right and the intriguing sensations of touch. Do try it yourself; you won’t understand until you do.

Section III:Audio Game Archive+: Play the World of Audio Games

Visitors here experience arcade, puzzle, and action games made mainly by blind developers using sound. Attune your senses of hearing and touch to anticipate a space through how sounds resonate, perceive the presence of people and objects, and play games with others.

  1. Zone for Playing with Sound
    1. Atodashi Janken (2009)
      • Developer: Nozawa Yukio　© 2009 Nyanchan Games
      • Keep winning with delayed rock-paper-scissors to the rhythm of the music!
      • Level: Easy
      • Genre: Rhythm game
      • Video Availability: Audio-Only
    2. Minesweeper (2023)
      • Developer: Morokuma　©2023 Morokuma
      • Avoid hidden mines and rely on sound to remove all of them!
      • Level: Easy
      • Genre: Puzzle Game
      • Video Availability: Audio-Only
    3. Audio Strike (2019)
      • Developer: Morokuma　©2019 Morokuma
      • Use sound to aim and shoot at targets!
      • Level: Normal
      • Genre: Shooting Game
      • Video Availability: Audio-Only
    4.  Din（2007）
      • DeveloperTeam_Bill, created during Global Game Jam 2007
      • Hear the instructions through the noise… Listen carefully and complete the mission
      • Level: Easy
      • Genre: Listening Game
      • Video Availability: Audio with Video
    5.  Audio Game Hub (2016-)
      • Developer: Jarek Beksa (New Zealand)
      • Audio Game Hub is a smartphone app that allows you to play multiple audio games including BLOCKS.
      • Level: Hard
      • Genre: Tetris
      • Video Availability: Audio with Video
  2.  Zone for Enjoying Sound with Visuals
    1. Accessibility Review of Rhythm Heaven Fever
      • A rhythm game where you play using the A and B buttons of the Wii Remote to the flow of the music.Since the release of the first “Rhythm Heaven” in 2006, the series has become so popular that four titles have been created.
    2. Accessibility Review of Blind Drive
      • Play as Donnie, trying to make a quick buck in a scientific study but quickly finding himself in over his head, cuffed to the wheel and driving blindfolded against traffic. Plus he’s late for dinner with Grandma. (Quote from the official site)
    3. Blind Drive (2021)
      • Developer: Lo-Fi People
      • Level: Easy
      • Genre: Driving game
      • Video Availability: Audio with Video
    4. Accessibility Review of The Last of Us: Part I
      • An award-winning action RPG that depicts two protagonists traveling across America. With a wide range of  accessibility options, it is the game that can not only be played but also completed regardless of vision.
  3. Technical keywords
    1. Music Games
      • Games premised on sounds and music frequently emphasize auditory localization and the combined use of visual and auditory senses, such as matching a rhythm to what appears on the screen. But what would a game be like in which making sounds itself was the goal? Would it be a kind of musical instrument? Exhibits include PaRappa the Rapper, beatmania, Rez, Tetris Effect, and Rhythm Heaven.
    2. Voice-User Interface
      • In addition to the sounds heard in a game, sound can function as a way to play a game, such as through the player’s spoken commands. Due to recent improvements in accuracy ushered in by machine learning, games have appeared that are played entirely by speech recognition without using a controller at all. Exhibits include Hey You, Pikachu!, SEAMAN, and The 3% Challenge.
    3. Screen Reader
      • A screen reader is a form of assistive technology that reads out information on a screen, and examples have existed ever since the days of computer operating systems that relied on inputted text (command-lines). As on-screen interaction has become more complex, so too has the technology required for reading it out, but free, open-source software like NonVisual Desktop Access is now widely available, used especially by visually impaired programmers.

Note “Audio Games, Until Now”

Nozawa Yukio (Program Director of “AUDIO GAME CENTER+CCBT” / Programmer)

  1. What we now call “audio games” were originally created by those who had impairment but wanted to play digital games. While there is no specific definition of what constitutes an “audio game,” from previous developments, it can be considered a genre that encompasses games played solely with sound and without needing to view a screen.The oldest audio game we can confirm is a bowling game created in 1988. The game utilized a buzzer by prompting players to press a button the moment they heard the sound of the bowling pins, thereby knocking them down. In this way, earlier audio games primarily relied on text information conveyed through text-to-speech and sound effects were limited to beeping sounds.
    Into the 21st century, development platforms began to shift to the use of Windows PCs. On it, the OS environment supported playing back the recorded sound effects allowing stereo sound reproduction and the volume to be changed with far greater flexibility. The method of information presentation devised during this period continues to be widely used today and is, in my opinion, a cornerstone of composing audio games.
    In 2001, GMA Games succeeded in developing the world’s first audio FPS (First-Person Shooter) game. Every element of the game—the players’ action, surrounding objects, and approaching monsters—was assigned a corresponding sound and, as players wore headphones for stereo-field sound positioning, real-time sound synthesis that is centered around the player became possible.
    Furthermore, in 2002, Audiogames.net—the world’s largest website that collects information on audio games—was launched. Since, many developers began creating games on the Windows PC and, by 2010, nearly every genre of games known to us were created, the number of games amounting to several hundred.
    The development of audio games has also rapidly accelerated in Japan since 2006; although the number of developers is estimated to be fewer than 10. Ever since, although development has remained small in scale, the competition in audio game development has intensified and has led to the evolution in the quality and quantity of audio games that continues to this day.
    *FPS: Abbreviation of First-Person Shooter. A shooting game where players attack enemies from their own viewpoint .
  2. Technical keywords
    1. Crossmodal
      • The way in which we perceive things changes due to combinations of sensory modalities, such as though a sound comes out of only a single speaker, it seems to move when played simultaneously with an object shown moving across a screen. In addition to sight, examples of crossmodal perception include changes in food texture if we eat potato chips while hearing the amplified sounds of the food crunching in our mouth.
    2. Paracusia (Auditory Hallucination)
      • Much like with sight, hearing is also subject to hallucinations, such as auditory masking, when we are unable to perceive one sound because of the presence of another, or the missing fundamental phenomenon, when we hear a sound that is not actually there. If we use an auditory hallucination (also known as paracusia) to our advantage, it can result in a kind of data compression akin to an MP3 that discards information we cannot perceive.
    3. Sonification
      • Similar to the effect of visualizing a message or other information as icons instead of text, sonification (using non-speech audio to convey information) can be a more intuitive way to perceive. Much like how a pictogram simplifies a shape to optimize communication, we can control the degree of abstraction in the message we want to convey by using recorded sound effects and melodies by musical instruments in different ways. Imagine what sound might best convey that a user has successfully completed saving.

Note “Technical Challenges and Opportunities of Audio Games”

Kakehi Yasuaki (Program Director of “AUDIO GAME CENTER+CCBT” / Researcher, Artist / Professor at the University of Tokyo)

Below are the technical challenges and possibilities for deepening our understanding of audio games and expanding the player and developer communities.

  • Feeling Sound
    • By designing audio with stereophonic sound and head-related transfer functions in mind, we can achieve a more realistic and three-dimensional audio presentation, enabling players to experience the game more spatially. Furthermore, delivering tactile sensations through vibration actuators—capable of presenting subtle patterns—further enhances realism in the game and provides better information accessibility.
  • Controlling Sound
    • In addition to keyboards, we introduced new control methods—spray can-shaped and handle-shaped tangible interfaces that utilize physical movements for example—and an Augmented Reality (AR) where users walk around a given space to enhance the playability of the game. We also believe that combining tactile feedback and voice commands is effective in achieving a manner of control that is independent of visual information.
  • Sharing Sound
    • To have multiple players and also deliver the game to the surrounding spectators and audiences, a wide range of technological approaches exist that can provide shared experiences of immersion in and enjoyment of sound in the game. Some of the potential methods include selective and localized audio presentation using directional speakers, low-latency remote synchronization, and sound visualization for individuals with hearing impairments.
  • Making Sound
    • Advancements in audio-generation technologies, such as real-time voice synthesis and spatial audio simulation, enable the development of diverse and realistic games. While the environment for developing and easily handling spatial audio such as Unity has emerged, the challenge remains as these still rely heavily on visual information. As such, there is a need to  further expand these developmental environments and interfaces that are compatible with screen readers.

Needless to say, differing technologies are required depending on the purpose and objective of their use: whether it is to broaden the reach of games, create  the best possible experiences, or build on the senses and skills. Along with improving the usability and accessibility in game, we look forward to seeing technologies that open up new forms of “enjoyment” that contribute to the playability of the game itself.

Note “Audio Games and Sound Accessibility. What is the Difference?”

Tanaka Miyuki (Program Director of “AUDIO GAME CENTER+CCBT” / Curator, Producer

The difference between audio games and sound accessibility lies in the role of sound: in [1] the former, sound is the sole constituent of the game rules whereas, in the latter, it follows the rules constructed by visuals and merely complements them.

In video games, it can be said that sound has been used primarily to enhance the player’s visual experience and, indeed, games utilizing spatial audio technologies have been created to pursue realism of sound. However, discrepancies between the actual sounds players with visual impairments hear and see from their movement in the games—still hearing footsteps despite bumping into a wall, for instance—it creates a confusion for them.

Moreover, realism in sound alone does not make a game accessible for visually impaired players: a game can only be considered accessible if players can launch it, navigate menus, and select characters or the settings easily and as needed.

As mentioned earlier, the majority of “audio games created by sighted individuals often are “visual games” where players rely on the visual cues on screen to synchronize with the rhythm. Also, conventional sound middleware—systems and mechanisms that play and manage the sounds of game engines—cannot operate without using vision. This means that, in reality, the environment for “creating from sound and playing with sound” is lacking adequate support for both creators and players of audio games.

In this “Audio Game Archive+,” we are introducing three games involving visual elements together with audio games: The Last of Us: Part I, Rhythm Heaven, and Blind Drive. The Last of Us: Part I serves as an example of how to make video games accessible using a range of accessibility features such as text-to-speech, sound cues, and navigation assistance.  As Rhythm Heaven is regarded as an audio game, it may appear as an accessible game. However, players need to memorize the menus and rules as it does not provide text-to-speech of the elements on the screen. Finally, although Blind Drive is an audio game, it incorporates graphical elements for accessibility to sighted players.

Looking at the differences among these games, we’d like to encourage players and audiences to reflect on how to create and approach games where sound plays a primary role.