Taste and Smell Internet : A Multisensory (Media) Communication Breakthrough

by Adrian David Cheok

Mixed Reality Lab, Graduate School of Media Design, Keio University, Japan

Introduction: 

The Mixed Reality Lab (MXR) (www.mixedrealitylab.org) at Keio University aims to push the boundaries of research into interactive new media technologies through the combination of technology, art, and creativity.

The key objectives of the Mixed Reality Lab are to create a world centre of excellence for interactive media and entertainment technology, to provide multi-disciplinary project-based learning environment for students, to modify creative media technology to promote economic development and to open new doors for creativity and artistic students.

The lab is interested in multi-sensory communication research involving digital transmission of taste and smell over the internet. We want to allow people to share experiences not only with video, pictures, audio, or tactile feedback but also with smell and taste. We

are committed to discover a new way to elicit these senses using a novel digital method that is both non-chemical and non-evasive.

With the continuous advancements in computing and media, technology has widened to include multi-sensory experiences in remote interactions. Using electrical, thermal, and magnetic stimulation technologies we are currently experimenting on reproducing smell and taste sensations digitally. As a fundamental aim of this research, we will develop novel user interfaces that empower people’s life with digitized taste and smell communication capabilities. This research will generate important avenues for further research areas using smell and taste based interactions and new media. As the ultimate goal of this project we will develop devices which could actuate taste and smell sensations digitally through the Internet.

 We will need to develop new protocols to codify these sensations as well as how to transmit them over the internet. New interfaces for how to send and receive these kinds of sensations will have to be designed. We hope that this will open the doors to new paradigms in human-computer interface design and new fields of research in academia.

 Why do we need to send tastes and smells over the internet and what kind of products could be developed?

A single taste and smell can influence people’s moods and affect their work performance. It also can bring back a flood of memories thought to be long forgotten. Our brain forges a link between the smell, taste, emotional feeling and memory, associating the individual and the interaction. Moreover, when you encounter the smell or taste again, the link is already there ready to elicit a memory or a mood. The usage applications to use this smell and taste interaction remotely can be used in many different kind of online services and it can be communicating with the user’s smartphone and the ideas of the applications that possibly can be utilized with many different kinds with smell-taste services using the mobile phone and Internet communication. In other words, sending tastes and smells over Internet has a huge potential market in online marketing and communications.

For example, online shopping is the process whereby consumers directly buy goods or services from a seller in real-time, without an intermediary service, over the Internet. Online shopping is an easy and comfortable way of shopping for a large range of products. There are innumerable advantages of online shopping. You save a lot of time which you normally need to personally go to the retail shop and buy the products which you want. Initially, the customer base for online shopping activities was the rich class with a lot of purchasing capacity even foods and perfumery items. However, it is impossible to know the fresh and good taste from online and TV shopping in current technology. Our system can contribute in people being able to smell and taste goods through the internet.

One type of consumer is the virtual reality enthusiast or gamer who wants a more emersive experience than just using visual, audio, and tactile stimuli. Also this can make serious games better teaching tools for teaching officials like police and fire-fighters. Another consumer group are people who have some form of smell or taste disability, or even a mental illness. Almost anyone who uses a social network to communicate with family and friends could benefit. We want to allow these people to share smell and tastes along with videos and pictures of their everyday lives.

How the technology works:

For taste stimulation we use a combination of a thermal pad with electrodes to produce a heating sensation and electric taste on the tip of the tongue. Various combinations of these two parameters at varying degrees and amplitude produces different taste sensations. For smell simulation, we are proposing a new application of non-evasive electro-magnetic brain stimulation applied to the olfactory region of the brain.

The digitize tasting actuation case, it is possible to induce electrical charge inside the nerves using the polarization of the nerves cells. By stimulation with different frequencies and different strengths we are developing a wearable clip kind of gadget which stimulates smell receptors using magnetic stimulation to regenerate smell sensations. Transcranial Magnetic Stimulation (TMS) is the current technology that we are experimenting to develop this smell interface gadget. This gadget enables smell communication through the Internet and regeneration of smell in a remote location.

In electrical stimulation subsystem, we use a digital potentiometer with a constant current source to provide a constant current to all the participants. The electrical stimulation module provides square wave pulses to the silver electrode with diverse current from 0µA to 200µA and frequency from 50Hz to 1000Hz. In thermal stimulation subsystem, we use a Peltier semiconductor module (Peltier Junctions) to change the temperature of the tongue, both cooling and heating within 20C – 35C. The initial experimental results suggested that sourness (electrical stimulation) and saltiness (electrical stimulation) are the main sensations that could be evoked besides several evidences of sweet (thermal stimulation) and bitter (electrical stimulation) sensations. Furthermore, several subjects reported that they felt the minty taste, refreshing taste (when cooling down from 35C to 20C), and also slight spiciness (when heating up from 20C to 35C). 

The digitize smell actuators using chemical smells case, the current smell actuation system is designed for embedding to a wearable accessory. It consists of a tiny smell actuator and its control system in mobile phone and the smell is subtly emitted from the small smell actuator. The current configuration allows a user to set his/ her preference smell identity using his/her mobile phone, and during a conversation with a partner, it seamlessly actuates a personalized choice of smell on their partner’s accessories. 

The accessory communicates with the user’s smartphone via Bluetooth. The user can set his/her Bluetooth MAC address in his/her mobile phone app. The app helps him/her to connect his/her smell accessory to mobile phone automatically. The user’s Bluetooth on his/her mobile phone detects his/her partner’s a mobile phone, data containing the user’s name, contact number and his unique smell signature ID are sent to the partner through wireless interaction. In response, the recipient’s phone communicates with his or her accessory, which in turn emits the user’s signature smell ID after saving encounter logs. 

 The tiny smell actuator module contains eight kinds of solid state aroma. Once a selected piece is heated to 46°C (115°F) with a wire, the right smell is released and the small speaker below of perfume heater helps pulsate the aroma from the smell actuator module. The current design describes a small perfume actuator that can be embedded into wearable accessories with multi-channel smell and sound expressions controlled wirelessly from a mobile phone application and the current system does so in a form factor that can be used in a variety of applications. 

 Applications in the future:

The smell interface gadget is enabling customers to smell online items which he/ she plans to buy. This gadget will be developed by combining our experiment results obtained from the magnetic stimulation research and chemical stimulation techniques. As users browse through the products (ex. scents, flowers, foods) the particular fragrance will be emitted by the smell generating gadget. 

The digital taste interface is attached through a wireless interface to a computer or a television, which is used for online shopping. Thus, the digital taste interface is connected to the web-store or the online store. As a user selects an edible product such as chocolates, cakes, or a sweet, the corresponding taste will be generated by the digital taste interface. Users will be able to verify the taste of the selected product by attaching the tongue to the digital taste interface as shown in the scenario. The main benefit of this technology is the generating different taste sensations digitally. This device uses electrical and thermal stimulation on the tongue to generate different taste sensations. The device will appear as a digital lollipop, where many users are comfortable to use in their everyday lives. On one hand, this technology enables a new paradigm for designers and sellers to improve the online shopping experiences. On the other hand, this will clearly introduce an additional modality (i.e. taste) for consumers when shop online. 

For example, Internet and mobile phone services can be integrated with the smell-taste accessory to provide smell and taste information for their digital contents. For example, when a restaurant introduces a new menu, they can conduct a special promotion through SMS using the smell-taste accessory. They can send online coupons to their customers containing smell-taste information for new dish and when the customer begins to read this message, he or she can smell the scent or taste to the sound of the new menu from their accessory.

Of course, this scenario has many potential possibilities and we can work with on-line shopping malls, restaurants, food companies, and commercial agencies that can use the smell and taste integrated with their products to attract people. Customers are able to check the restaurant menu before visiting through the smell-taste accessory, by logging into the diner’s homepage.

Watching a recipe video gives us two sources of information – visual and auditory. In the near future we could see audio visual systems that are expanded to utilise smell and taste. You can check your own dish in progress by comparing the taste to the celebrity chef’s dish. Similarly, we could add new dimensions to food entertainment media, when the lid comes off a steaming pot on your television screen; you experience the same warm aroma and sensation.

Profile:

Adrian David Cheok (www.adriancheok.info) is a Full Professor at Keio University, Graduate School of Media Design. He is Founder and Director of the Mixed Reality Lab, Singapore. He was previously Associate Professor in the Department of Electrical and Computer Engineering, National University of Singapore. He has previously worked in real-time systems, soft computing, and embedded computing in Mitsubishi Electric Research Labs, Japan. 

Amongst numerous high quality academic journals, keynotes, exhibitions, he was featured in worldwide broadcasts on his research such as CNN and Discovery Channel. In addition he was the recipient of numerous prestigious awards and prizes such as the A-STAR Young Scientist of the Year, Hitachi Fellowship, SCS Singapore Young Professional of the Year, Microsoft Research Award in Gaming and Graphics, C4C Children Competition Prize, Integrated Art Competition Prize, Creativity in Action Award, Mind trek Award, the Keio University Japan Gijyuju-sho award, and was awarded Young Global Leader 2008 by the World Economic Forum.

::NOTES::: Possible links :