HomeTechnologyVirtual RealityXR’s wide range of interface technologies: Achieving total immersiveness

XR’s wide range of interface technologies: Achieving total immersiveness

While consumers have become very familiar with speech recognition, current discussions around the metaverse and extended reality (XR) largely ignore the relevance that speech and sound can play in virtual or augmented environments. After all, most people interact with their environment almost constantly when making and identifying sounds.

This is a major omission. As the push for an immersive metaverse experience continues, the goal of true immersivity will only be achieved when virtual landscapes or elements can provide a portfolio of all sensations that allow users to become one with these environments.

Siri has become one of the best-known voice assistants since Apple introduced the feature in 2011. The technology goes back to a 2003 SRI International project that the research institute spun off as an independent company in 2007. Apple acquired this company in 2010 and since then Amazon.com introduced Alexa and Google created its Google Assistant.

Microsoft has its own speech recognition and acquired it in March 2022 Nuance Communications, a provider of speech recognition and artificial intelligence (AI) technologies. Coincidentally, Nuance partially harks back to another SRI International spin-off that was acquired in 2005 before you become part of Microsoft.

Sound can become the story

But sound can be more, and movie and game developers have shown that sound can become the story – whether it’s the ticking of a watch or an almost subliminal bass echoing the beating of a heart, just to name a few. silent about explosions and the like.

Flexible has developed a technology that can be embedded in theater or gaming chairs, but also in cushions or even furniture. The system makes it possible to create personal sound atmospheres and allows users to feel the sound with their body. In the meantime, Sennheiser has developed immersive sound which Netflix now uses in some of its movies and selected scenes. Sennheiser has also created AMBEO, which provides spatial sound for a wide variety of applications, including cinematic virtual reality.

Also, many companies already offer services to design meaningful sound environments for specialized applications, even if their current focus is not on XR or metaverse-related applications. These efforts range from fairly comprehensive services to well-targeted applications.

Spatial Incfor example, develops soundscapes for shops, catering environments, office spaces and museums. Sen soundmeanwhile focuses on designing more pleasant, less stressful sounds for hospitals and healthcare facilities. It is easy to see how such design understanding will find its way into creating more meaningful and immersive augmented reality (AR) and virtual reality (VR) applications.

The telling touch of haptics

Another set of interface technologies already at play in high-end gaming will help XR applications become more immersive and, in many cases, more inclusive and relevant. Haptics can enable use cases that are currently difficult to translate from real to virtual environments.

Haptics divide into tactile and kinesthetic sensations. Tactile experiences mainly relate to the skin, such as the perception of texture, touch, pressure or vibration. Kinesthetic experiences relate to muscles, tendons and joints and the perception of weight and stretch, as well as the movement of body parts. Several interfaces already exist that will be developed to address this type of sensation.

Perhaps most beneficial is the feeling of haptics in the hands. Holding a tool, squeezing an object or feeling a surface in virtual environments will enhance the authenticity of the experience if haptic feedback is added. At its most basic level, haptic feedback enables the use of virtual presentations of standard interfaces, such as buttons that can be pressed or rotary knobs that provide a sense of click sensation.

Holding a tool, squeezing an object or feeling a surface in virtual environments will enhance the authenticity of the experience if haptic feedback is added.

A number of companies offer gloves as interfaces. HaptX provides both types of feedback and includes motion tracking for applications in VR or for telerobotics operations. Sense Glove provides an advanced solution that allows users to get an idea of ​​the size, density and resistance of objects in virtuality. Sensations in fingers and hands to manipulate and explore objects in VR offer clear advantages.

So it’s no surprise that Facebook’s Reality Lab is experiment with haptic gloves to understand their capabilities. Other companies are trying to achieve similar sensations without having to put on a glove. Ultra jump is one of them and uses ultrasound to project haptic sensations onto the hands.

These different approaches can be used in different environments and for different use cases. Ultrasound can be more easily used in public spaces and for AR, where the need to put on gloves can add friction to the experience. Meanwhile, physical gloves can create more diverse sensations and represent certain objects more accurately.

To create a complete sense of immersion, entire suits can provide enveloping sensations. To become completely at one with virtual environments for entertainment, training, diagnostic and therapeutic applications, systems that can provide haptic sensations to the upper body or the whole body are a welcome addition. One of which is bHaptics’ series of so-called TactSuit products, essentially vests with haptic feedback spread across the body. The company also offers a haptic glove.

Tesla suit, meanwhile, offers a set of haptic garments: the Teslaglove and Teslasuit, which consists of a jacket and trousers. Here the haptic sensations are based on electrostimulation.

More immersive virtual environments

Researchers are also experimenting with different approaches to creating haptic sensations. A group of scientists from the University of Chicago say they have “identified five chemicals that can provide lasting haptic sensations: tingling (sanshool), sedative (lidocaine), tingling (cinnamic aldehyde), warming (capsaicin) and cooling (menthol)”. These haptic devices include a sheath that surrounds part of the forearm and a strip that can be applied under the visual headset on the user’s cheeks.

Perhaps attachable accessories for mass-market headsets—similar to the way the University of Chicago researchers use their strip in conjunction with headsets—could become a market niche to provide users with certain sensations. Feel realthat started as one Kickstarter campaign, is a device that looks like a shield and attaches to the lower part of visual headsets. The device can provide the sensation from a cool breeze, warmth, splashes of water and a wide variety of scents.

In the meantime, OVR technology offers a device that connects to VR headsets to provide a wide variety of scents. The company cites use cases such as meditation and response training for this device. Harnessing VR users’ sense of smell should come naturally to developers looking to create more immersive virtual environments. OVR CEO Aaron Wisniewski says: “The metaverse without smell would be like a life in black and white.”

Such accessories can provide a wide range of interaction possibilities. A research team from the Salzburg University of Applied Sciences has developed AirRes, an add-on for the Meta Quest 2 headset. The device looks like a gas mask that the researchers tout as a breathing interface. It uses a wearer’s breathing as input information through a resistance valve to turn on interact with virtual wind instrumentsblowing out a birthday cake or using a blowpipe to propel projectiles, for example.

The resistance device can also restrict a user’s breathing, simulating, for example, entering a smoke-filled room.

Applications in gaming or emergency training are obvious. Some of the applications the researchers present point to the device’s potential to enhance virtual environments with new levels of authenticity. Exhaling light at mirrors to fog them can reveal hidden numbers left behind, similar to the way children leave messages for each other in mystery games.

Such a fogged surface of glass or metal can create a sense of realistic interaction with objects in environments and offers the possibility to leave messages for other players on virtual windows or metal surfaces, for example for gaming purposes.

In the meantime, H2L technologies has developed a wristband that can inflict pain through small electric shocks. The company’s CEO, Emi Tamaki, says, “Feeling pain allows us to transform the metaverse world into a real world, with heightened feelings of presence and immersion.” Although the device can give a feeling of pain, its main purpose is to create a feeling of resistance and weight when interact with objects in VR.

Finally, brain interfaces could be the ultimate in connectivity to virtual worlds and augmented landscapes. Such connections to XR environments are currently speculative, but as time passes and neuroscience advances, fairly simple applications will become conceivable and companies are already exploring related technologies.

Meta Platforms’ Reality Labs is exploring the use of a brain-computer interface (BCI) for AR glasses, specifically for BCI use in communication applications. Others also see benefits in the combination of AR glasses and brain interfaces. In March 2022, snap acquired NextMind, a developer of BCIs. Presumably, Snap is exploring the use of such an interface with its AR smart glasses, Snap Spectacles. Potential uses of the interface can include: gaming or control devices, for example.

Meanwhile, more broadly, Elon Musk’s Neuralink is working to develop interfaces that “a direct link between the brain and everyday technology”. In 2021, the company released information about a macaque monkey ability to move cursors on a screen and play the video game Pong through brain activity alone.

Advanced XR

Currently, many developments in XR and the metaverse are experimental, and interface technologies are part of such experimentation. Truly immersive environments require sophisticated interfaces that can replicate authentic real-world representations. But unwieldy and hard-to-use interfaces can actually deter users from immersing themselves in VR as well.

Cost is another consideration and safety will play an increasingly important role. But VR applications will drive the search for interfaces that allow more extensive interactions with virtual objects and worlds – and advanced interfaces will enable increasingly sophisticated XR applications.

Martin Schwirn is the author of Small Data, Big Disruptions: Recognizing Signals of Change and Managing Uncertainty (ISBN 9781632651921). He is also a senior advisor, strategic foresight at Business Finland, where he helps start-ups and established companies find their footing in tomorrow’s market.

Must Read
Big Eyes Coin is Here to Develop and Expand, Just like Solana and Chiliz


Please enter your comment!
Please enter your name here

Most Popular

Recent Comments