How Do Virtual Reality Headsets Work?

Artificial intelligence (AI)

Artificial intelligence is a potentially world-changing technology. It could help cure cancers, control autonomous cars, and augment human intelligence. Or it could lead to a robot apocalypse and the downfall of humanity. It depends on who you ask.

Artificial intelligence or AI simply means software used by computers to mimic aspects of human intelligence. For example, a program that recommends what you should read based on books you’ve bought or a robot vacuum that has a basic grasp of the world around it.

So why all the fuss? In the last decade a particular flavour of AI, called machine learning, has become extremely powerful. The technique is behind everything from DeepMind’s world champion Go playing AIs to Google translate, and face recognition algorithms to digital assistants, such as Amazon Alexa.

Rather than programmers giving machine learning AIs a definitive list of instructions on how to complete a task, the AIs have to learn how to do the task themselves. There are many ways to attempt this, but the most popular approach involves software called a neural network that is trained by example.

Advertisement

Neural networks

A neural network is a large web of connections, inspired by the way neurons connect in the brain. Inputs work their way through the network, guided by the strength of the connections, to find the appropriate output.

In the case of a robot vacuum, the inputs could be all of the various measurements from its sensors, and the output could be how it decides to move. To train the vacuum, it could be shown thousands of examples of humans vacuuming rooms along with the relevant sensor inputs. By strengthening the relevant connections, a neural network vacuum would then eventually learn which inputs correspond to which actions so that it can clean the room by itself.

Neural networks have been around since the 1940s and 1950s, but only recently have they started to have much success. The change of fortunes is due to the huge rise in both the amount of data we produce and the amount of computer power available.

The AIs require anywhere between thousands to millions of examples to learn how to do something. But now millions of videos, audio clips, articles, photos and more are uploaded to the internet every minute, making it much easier to get hold of suitable data sets – especially if you are a researcher at one of the large technology companies holding people’s files.

Processing these data sets and training AIs with them is a power-hungry task, but processing power has roughly doubled every two years since the 1970s meaning modern supercomputers are up to the task.

Artificial general intelligence

However, neural networks can’t do everything. A neural network trained to do one thing is next to useless at doing something else. For example, ask Libratus, an AI that is the world’s best heads up no-limit Texas hold ’em poker player, which country Las Vegas is in and it wouldn’t even be able to process the question.

There are ambitions to create AIs with a broader range of abilities, known as artificial general intelligence (AGI), which could perform any task that the human brain can. But we’re a long way from AGI at the moment.

Because of the limitations of neural networks, some argue a completely different approach may be needed to reach AGI. That could be giving neural networks more underlying structure to begin with to help them learn things in a better way, moving to programs that learn by evolution, or trying to better mimic animal brains.

Once we have AGIs, there are worries that could spell the end for humanity. AGIs could become so smart that they iteratively improve their own intelligence to far surpass human intelligence. These superintelligences could have motivations of their own, and keeping humans around may not be one of them.

AI problems

There’s no need to worry just yet though. We’re still a long way from reaching AGI and there are some pressing concerns regarding current AIs.

AI is already involved in making important decisions, such as someone’s eligibility for parole in parts of the US. It is also increasingly assessing people’s suitability for a job, loan, or insurance product.

Yet AI is often biased, meaning its recommendations can be too. Time after time, researchers have found that neural networks pick up biases from the data sets they are trained on. For example, face recognition algorithms have much lower accuracy for anyone who is not a white man – a hallmark of the lack of diversity in the data sets.

AI decisions are also opaque. How neural networks come to conclusions is very hard to analyse, meaning if they make a crucial mistake, such as missing cancer in an image, it’s very difficult to find out why they made the mistake or to hold them accountable. This could slow the progress of AI in applications where trust and safety are particularly important. Timothy Revell

What Is Artificial Intelligence (AI)?

Data Preparation

Taking raw data and making it useful for an accurate, efficient, and meaningful model is a critical step. In fact, it represents most of your AI effort.

Data preparation requires domain expertise, such as experience in speech and audio signals, navigation and sensor fusion, image and video processing, and radar and lidar. Engineers in these fields are best suited to determine what the critical features of the data are, which are unimportant, and what rare events to consider.

AI also involves prodigious amounts of data. Yet labeling data and images is tedious and time-consuming. Sometimes, you don’t have enough data, especially for safety-critical systems. Generating accurate synthetic data can improve your data sets. In both cases, automation is critical to meeting deadlines.

AI Modeling

Key factors for success in modeling AI systems are to:

Start with a complete set of algorithms and prebuilt models for machine learning, deep learning, reinforcement learning, and other AI techniques

for machine learning, deep learning, reinforcement learning, and other AI techniques Use apps for productive design and analysis

for productive design and analysis Work in an open ecosystem where AI tools like MATLAB ® , PyTorch, and TensorFlow™ can be used together

where AI tools like MATLAB , PyTorch, and TensorFlow™ can be used together Manage compute complexity with GPU acceleration and scaling to parallel and cloud servers and on-premise data centers

System Design

AI models exist within a complete system. In automated driving systems, AI for perception must integrate with algorithms for localization and path planning and controls for braking, acceleration, and turning.

How Do Virtual Reality Headsets Work?

Virtual reality (VR) is now the fastest-growing content segment in the world.

Research by PwC found that VR content will grow at a compound annual rate of 30 percent between 2021 and 2025, outstripping over-the-top (OTT) video, video games, and even traditional cinema.

VR headsets can allow users to consume VR content by providing them with an immersive, three-dimensional experience.

What Is a Virtual Reality Headset?

A VR headset is a head-mounted device that includes a display screen, stereo sound, sensors, and compatible controllers to deliver an immersive and interactive audiovisual experience.

When a user puts on a VR headset, they can no longer see the world around them, but instead only see VR content projected on the display screen such as 360-degree videos and VR games, workspaces, or meeting rooms for other activities.

Unlike augmented reality (AR) headsets or mixed reality (MR) headsets, VR headsets do not allow users to see any element of the external physical world.

Along with the headset itself, the user will rely on a set of VR controllers to navigate the experience. As mentioned, the device offers an interactive experience, requiring a controller to point to objects, select, drag, and drop, scroll up or down, navigate between different VR spaces, demarcate boundaries, and other functions.

Most VR headsets that are available in the market use handheld controllers that function similarly to joysticks. More futuristic models may provide haptic gloves, where users can navigate through the virtual world using their fingers, gestures, touch, and other naturalized movements.

All VR headsets consist of the following four components:

Basic Components of VR Headsets

An array of sensors

Unlike 2D video, virtual reality is not a passive experience. Users interact with virtual worlds, which adapts according to the user’s continuous inputs.

To achieve this, VR headsets come with a number of sensors, and some devices even have a six degrees of freedom (6DoF) system for head tracking.

Using gyroscopes, accelerometers, and other sensors, a 6DoF system tracks head movements and repositions the display accordingly. Some headsets also have eye-tracking sensors that can understand when eyes focus on a VR object or location.

Lenses and screens

The lenses and screen setup makes up the bulk of the VR headset’s hardware. There are stereoscopic lenses positioned between the screen and your eyes that distort the image into appearing three-dimensional.

Two images are passed through the lens, one for each eye, similar to how our eyes perceive and process visuals in the real world. Additionally, images in VR headsets appear to move side-to-side to recreate a 360-degree experience and is achieved by subtly moving the display content in response to head tracking data.

Immersive audio

A stereo audio feed comes from two directions or one for each ear, but in the real world, users have a much more layered experience of sound where audio is directly linked to our perception of distance and space.

VR headsets mimic this experience using 360-degree or immersive audio technology. Binaural audio is one such technology, and the new spatial audio pioneered by companies like Apple marks another milestone in VR audio innovation.

Controllers

Finally, VR headset controllers are your bridge between the real and the virtual worlds. Interestingly, there are a variety of controllers you can use, apart from the usual set of two handheld controllers that come with most headsets.

For instance, Samsung offers a single hand motion controller for its Gear VR kit, and HTC VIVE also has single hand joystick-like controllers that come with a base station to dock them.

Meta reportedly has developed a set of haptic-based controllers in the works that could enable pressure-sensitive touch and navigation. Also, Valve Index has a unique take on controllers that incorporates a fist gripping design.

Understanding How a VR Headset Functions

All of these components, together with sophisticated VR software, allow the headset to function properly. Once the headset boots up, users are greeted by a realistic virtual environment that acts as a lobby and is equivalent to a computer’s homepage. While there, users can choose different apps, hang out with other virtual people, change settings, update devices, and other features from this space.

Meanwhile, images are fed through a video source such as a smartphone, desktop, or more likely the cloud in modern headsets. The lens will split the video image into two and calibrate them into a stereoscopic 3D image, which is what you see on the screen. Thanks to built-in sensors, the environment changes subtly, as you look around, shift the focus of your eye, or raise your hands.

Apart from this basic functionality, VR headsets are extremely powerful. For instance, there are productivity apps that let you create product designs in VR and save your designs as 3D files to the cloud. Sophisticated VR headsets have a very high screen refresh rate to render and update content instantaneously.

What Makes a Good VR Headset?

There are a few key features that characterize a good VR headset, such as:

● Light form factor – The screen and sensors can add to the headset’s bulk, and anything heavier than 500-600 grams will be difficult to use on a regular basis. This is why Apple’s upcoming mixed reality (MR) headset’s current 150-gram weight is such a breakthrough.

● Easy to use controllers – The controllers will inevitably have numerous buttons, wheels, and sticks to help navigate in VR. They must be ergonomically designed and provide a seamless user experience.

● Onboard storage – While most VR headsets rely on the Internet and the cloud, it is good to have at least 32GB of onboard storage to install applications, ensure timely updates, and store a few files without slowing down the system.

After topping 9.36 million shipments in 2021, VR headset shipments are expected to reach 13.59 million worldwide by 2022 as per IDC’s December 2021 report. As demand grows, we can expect new innovations – built on these existing core functionalities – to create VR experiences that are more enriching, seamless, and accessible.

Leave a Reply

Favorite articles

Most popular articles

Latest articles