The core idea
Virtual reality tries to make you feel as if you are somewhere else. That feeling is often called presence. A headset creates it by controlling what each eye sees and by updating that view as you move your head. Three parts do most of the work: small screens, lenses and tracking. None works alone, and weakness in any one can break the effect.
Two images, one scene
People see depth partly because each eye views the world from a slightly different position. A headset reproduces this by showing a separate image to each eye. The two images are rendered from viewpoints about as far apart as human eyes, and the brain merges them into a single scene with depth.
Some headsets use one screen divided between the eyes, while others use two separate small screens. Either way, the screens are only a few centimeters from your face. At that distance your eyes cannot focus on them directly, which is where the lenses come in.
What the lenses do
A lens sits between each eye and its screen. It does two jobs:
- It lets your eye focus on a screen that is very close, by making the light from the screen appear to come from farther away.
- It magnifies the image so that it fills a large part of your view.
Magnification brings side effects. A simple lens bends the picture, making straight lines look curved, and it can split colors at the edges. To correct this, the software pre-distorts the picture in the opposite direction, so that the lens undoes the distortion and the picture looks straight. Some color fringing is also corrected in software. That is why headset images can look slightly strange if you view the raw output on a flat monitor.
Lens design involves trade-offs between how wide a view you get, how sharp the image is across the lens, how heavy the headset is, and how forgiving the fit is. The sharpest area is typically at the center, and sharpness may fall off toward the edge. Some headsets use thinner lens designs to reduce bulk, but those bring their own challenges.
The screen’s job
The screens in a headset have unusual requirements. They must be small, high in resolution, fast and reasonably bright, and they must not use much power or give off much heat. Their sharpness is judged by how many pixels cover each degree of view, not just the total count, which is why headset resolution is a topic of its own. The gaps between pixels can show up as the screen-door effect once magnified.
Speed matters because your head moves constantly. If pixels change slowly, moving objects can smear. Many headsets therefore show each frame only briefly before going dark, a technique sometimes called low persistence, which reduces blur at the cost of some brightness. The refresh rate also matters, and a higher rate typically makes motion feel smoother, though the system has to render more frames to keep up.
Tracking closes the loop
Presence relies on the picture responding to you. Tracking systems measure the position and orientation of the headset, and often the hands and controllers, using combinations of motion sensors and cameras. The software uses that data to update the viewpoint.
Timing is critical. The delay between moving your head and seeing the matching change in the image is called latency. If it is too long, the world seems to lag behind you, and many people find this uncomfortable or nauseating. Systems try to reduce latency in many ways, including predicting where your head will be by the time the frame appears. The exact tolerance varies between people, so a setup that feels fine to one person may bother another.
When the pieces disagree
The senses usually agree about the real world. In virtual reality, they sometimes do not:
- Your eyes see motion, but your inner ear reports that you are sitting still, as when a game moves you through a virtual landscape. This mismatch is often linked to discomfort, and designers use techniques such as limiting speed or narrowing the view during movement.
- Your eyes aim at a virtual object that seems close, yet they focus at the lens’s fixed distance. This mismatch can contribute to eye fatigue in some people.
- Latency or low frame rate causes the image to lag your motion.
Each mismatch reduces the sense of presence or causes discomfort. How sensitive someone is varies, and breaks are sensible. People with medical conditions or concerns should consult a qualified professional before extended use.
Beyond a single design
Headsets differ in what they put in front of the eyes. Some use opaque screens to replace the world. Others use cameras to show the surroundings and blend digital objects into them, and others use see-through optics, a field discussed under augmented reality. The goal in each case is to merge the real and the virtual convincingly, and the display and optics define what is possible.
A balance of constraints
Improving one part of a headset often costs another. Wider views need bigger optics. Higher resolution needs more processing power. Brighter screens use more energy and produce more heat on the face. This is why progress tends to come in steady steps rather than leaps, and why the best headset for one person or purpose may not be the best for another.