Multisensory design
We are very used to looking at our screens, and to hear sounds from our devices. But not very often we might smell or taste what we are seeing on the screen. That might however be changing as the tech world has started experimenting with engaging more senses as part of the user experience. This could maybe be seen in relation to the development of the term “multisensory design” — an empathetic approach which extends beyond the prevailing focus on visual design with features that also appeal to other senses, such as smell or touch. It recognizes the many ways in which people experience and react to a product or service, environment or experience — by engaging all of our senses — vision, touch, sound, smell and taste. Let’s look at some interesting examples.
In June 2019, Alibaba Group released a video capturing the possibility of a multi-sensory shopping experience of the future. To create a sensory experience their technology called “Refinity” employs auto-stereoscopic objects driven by ultrasound haptic displays. Very little has been written about the project since its video release, so it seems still to be at an early research stage. As Alibaba has focused more on touch, professor Homei Miyashita at Meiji University has instead focused on taste and developed a lickable TV screen prototype. The Taste the TV, or TTTV imitates the flavors of food by spraying a combination of 10 different flavors to get the desired effect. This will solve the problem for all of us wishing we could taste what the chefs cook at the morning shows on TV…
In 2021 Professor Advaith Siddharthan and his team from the Open University received a share of £8 million from the Engineering and Physical Sciences Research Council (EPSRC), part of UK Research and Innovation, to carry out an interdisciplinary research project with the aim to develop new technology which will allow children to experience nature through their smartphones. This by ‘feeling’ textures such as feathers, fur and tree bark. The technology will be based on a haptic adapter that is to be fitted to pre-existing smartphones and tablets. The adaptors work by modulating the friction of the touchscreen, either through electrostatic methods or by vibrating the surface at ultrasound frequencies to create an air layer beneath the finger. When the surface friction is controlled as a function of image pixels under the finger, this gives the sensation of texture.
Now these examples might be considered a bit fun, and maybe even gimmicky, but there are also some very real benefits of multisensory design. Because digital experiences that are designed to be “multisensorial” in nature, makes it possible for the users to receive information through multiple senses (eye, ears, touch). Meaning that the information flow becomes more diversified, offering more alternatives, which in turn benefits those that depend on other senses other than vision. Which in practice makes it more inclusive. But if multisensory design is something we will see more of in the future is yet to be seen (or tasted, smelled, felt, or heard).
References
Fjord trends 2022
A lickable screen that can imitate flavours is in development
How multi-sensory design can help you create memorable experiences
What is multisensorial design and why is it important now
Haptic tech turns phones into multi-sensory nature learning tools
