AR for your ears
For the past years development within AR has focused on building technology utilizing our eyesight, but now our ears are coming up as a fierce contestant. This as computing and sensor technologies are becoming smaller and more sophisticated. Which also means that new software applications are being developed that cleverly use audio to transport users to another space.
PairPlay is an iOS app that guides partners, friends, or kids through imagined scenarios within their own homes. It utilizes Apples Airpods, and the fact that they can be shared, two persons having one pod each. To play PairPlay you and your playmate are given a setting and two different versions of a scenario, one to each earpiece. There are several different settings, in one of them one participant is turned into a robot, whilst another scenario simulates a zombie apocalypse, urging players to race around the house, close the windows, and find hiding spots, all the while not knowing if the other person has been “infected.” The entrepreneur Jonathan Wegener is the brain behind PairPlay, and he sees his product as an alternative to AR glasses, and that PairPlays offer is “Screen-free AR”. In an interview he explained the offering by saying “It’s a much more compelling experience than holding a screen in front of your eyes and pretending there’s a monster on your floor. We want to work within that constraint of, ‘OK, put your phone away and play with your partner and use your imagination.”
PairPlay can be seen as an example of “immersive” sound experience, as it is a two-channel system being delivered to two separate ears. This in relation to spatial audio that is a core part of authentic-sounding virtual reality experiences. This kind of technology is now seen in smaller personal devices too. Apple is currently a company in the forefront of this trend. Across its product line, Apple is rolling out new spatial audio features, which create immersive, multidimensional sound that mimics surround sound. In October 2021 Apple announced that its new AirPods 3 and MacBook Pro laptops will be equipped for spatial audio. Apple spatial audio takes 5.1, 7.1 and Dolby Atmos signals and applies directional audio filters, adjusting the frequencies that each ear hears so that sounds can be placed virtually anywhere in 3D space. Sounds will appear to be coming from in front of you, from the sides, the rear and even above. The idea is to recreate the audio experience of a cinema.
Connected to this is also the emerging field of “earable computing”. In 2020 CSL’s Systems and Networking Research Group (SyNRG) started to define a new sub-area of mobile technology that they call “earable computing”. Romit Roy Choudhury, professor in electrical and computer engineering describes this emerging technology as “The leap from today’s earphones to ‘earables’ would mimic the transformation that we had seen from basic phones to smartphones,” He and his colleagues think that tomorrow’s earphones not only will run acoustic augmented reality, but also continuously sense human behavior, track user motion and health, as well as having Alexa and Siri whisper just-in-time information.
References
Augmented reality already arrived in our ears
Earable computing — why it matters to you
The future 100 2022
What is Apple spatial audio
