Douglas Lanman has been with Meta for eight years and is the head of the department that develops display systems. In June, his team unveiled new VR glasses prototypes.
The prototypes address various technical challenges such as resolution, brightness, and form factor. The researchers’ goal is to develop a VR display that can visually reproduce physical reality while fitting into a small headset.
Lanman presented on the top ten challenges remaining in developing a near-perfect VR headset at this year’s Siggraph conference. Following is a list of those ten challenges, along with a brief explanation. The video of Lanman’s talk is available at the end of the article.
High resolution
In terms of resolution, current VR headsets do not come close to human vision. The resolution of VR displays must be significantly increased in order for virtual worlds to appear as real and sharp as if they were physical, and for texts to be read well even at medium distances.
Meta’s preliminary target is 8K per eye and a pixel density of 60 PPD. In comparison, the Meta Quest 2 (review) manages only 2K per eye and 20 PPD.
Meta created the Butterscotch prototype, an experimental headset that allows researchers to experience “retina resolution” and evaluate its immersive effect. The most difficult issue is not the development and production of high-resolution displays. Rather, the issue is where to house the computing power required to power such high-resolution displays. Although focalized rendering and cloud streaming may be useful, they present significant technical challenges in and of themselves.
Larger field of view
Lanman believes that much work needs to be done in terms of field of view. The horizontal field of view of a human is approximately 200 degrees wide. Commercially available virtual reality headsets typically have a horizontal field of view of 100 degrees. However, there is still room for advancement in the vertical field of view.
A wider field of view presents significant challenges for lens technology, resulting in image distortions at the field’s edges. The issue of computing power arises here as well. The more pixels the VR headset has to display, the wider the field of view. As a result, power requirements increase and waste heat increases.
Ergonomics VR headsets are still large and cumbersome. Meta Quest 2, for example, weighs over a pound and sticks out nearly three inches from the face. VR headsets should ideally be comfortable to wear for extended periods of time, as well as much narrower and lighter in design.
Pancake lenses and holographic lenses may be useful. Meta’s fully functional Holocake 2 prototype demonstrates the form factor’s potential. The issue is that Holocake 2 relies on custom-built lasers as a light source, which are not yet mass-producible.
Vision correction display
The ideal VR headset should be capable of detecting and compensating for users’ vision deficiencies, so that traditional glasses or contact lenses are not required to see well in virtual reality. Will prescription glasses fit under the virtual reality headset without scratching the lenses or pressing down on the wearer’s face? In the future, consumers should not have to deal with such issues.
The problem could be solved with special attachments or, even better, with a lens that can be adjusted to the user’s own visual acuity.
Variable focus
The human eye cannot naturally focus in VR environments, which is particularly annoying at close range and can lead to eye fatigue and headaches over time. Technically, this is known as vergence-accommodation conflict.
To address this issue, Meta’s researchers created a display that supports “progressive vision.” The display simulates different focal planes as well as blurring, allowing the eye to see the virtual world as if it were created in nature. Meta’s varifocal prototypes are dubbed “Half-Dome.”
The problem with eye tracking is that it doesn’t work equally well for all people and occasionally has dropouts. A reliable solution with broad demographic coverage is needed. Otherwise, the technology frustrates rather than helps and is rejected by consumers.
Distortion correction
Lenses inherently introduce image distortion that must be corrected by software. The slightest movement of the pupil causes fine but noticable distortions. They impair visual realism, especially in combination with other technologies such as progressive displays.
