Virtual Reality

Summary

The applied science of virtual reality (VR) engages in the design and engineering of and research related to special immersive interactive computer systems. These virtual reality systems synthesize environments, or worlds, which are simulations of reality that are usually rendered using three-dimensional computer images, sounds, and force feedbacks. Virtual reality applications are used for pilot and astronaut training, entertainment, communication, teleoperation, manufacturing, medical and surgery training, experimental psychology, psychotherapy, education, science, architecture, and the arts. This technology, which submerges humans into altered environments and processes, intensifies experience and imagination, thereby augmenting research and education. Virtual reality training systems can simplify and improve manufacturing and maintenance while simultaneously reducing risk exposure.

Definition and Basic Principles

A virtual reality system is an interactive technology setup—software, hardware, peripheral devices, and other items—that acts as a human-to-computer interface, immersing users in a computer-generated three-dimensional environment. Virtual reality is the environment or world the user experiences while using such a system. Although the term virtual implies that this simulated world does not exist, the term reality refers to the user's experience of the simulated environment as being real. The more the system compellingly involves the senses, the more genuine the perceived experience will be and the more intense the imagination. Most virtual reality systems stimulate sight, hearing, touch, and other tactile-kinesthetic sense perceptions, such as equilibrioception, torque, and even temperature. Less often, they include smell, and some have experimented with taste. Virtual reality must be almost indistinguishable from reality in some applications, such as pilot training, but it can often differ significantly from the real world in, for example, games.

Virtual reality in a narrow sense (a computer-generated simulation that exists virtually but not materially) is not the same as augmented reality (enhanced reality) or telepresence (in the sense of teleconferencing). Augmented reality technology improves the perception of and supplements the knowledge about existing entities or processes (highlighting data of interest while abstracting the less important information). Telepresence (as teleconferencing) refers to a remote virtual recreation of a real situation (for example, to enable audio and visual interactions between people at diverse places). However, a wider notion of virtual reality includes notions of both augmented reality and telepresence.

Background and History

The idea of simulated reality is often traced back to the ancient Greek philosopher Plato's allegory of the cave in Politeia (fourth century BCE; Republic, 1701). In the allegory, spectators observe images (shadows) of objects on a cave wall that they take for real objects. The term “virtual” is derived from the Latin word "virtue" (which means goodness or manliness). “Virtual” then means existing in effect or in essence but not in actuality. The notion of virtual reality can be traced back to the French theater director, actor, playwright, and illustrator Antonin Artaud, who described theater as la réalite virtuelle in his influential Théâtre et son double (1938; The Theatre and Its Double, 1958). Computer scientist and artist Myron W. Krueger coined the technical term artificial reality in his book of the same title, published in 1983. Computer scientist and artist Jaron Lanier popularized the notion of virtual reality as a technical term in the 1980s. Many artists, science-fiction authors, and directors incorporated computer-generated, simulated, or augmented reality concepts in their creations. One of the most prominent examples is the holodeck, an advanced form of virtual reality featured on the television program Star Trek: The Next Generation (1987–1994).

Technology. In the late 1960s, computer scientist Ivan Sutherland created the first head-mounted device, which was capable of tracking the user's viewing direction. In the 1970s, Sutherland and David Evans developed a computer graphic scene generator. In the 1970s and 1980s, force feedback was incorporated into tactile input devices such as gloves and interactive wands. Lanier and Thomas Zimmerman developed sensing gloves, which recognized finger and hand movements.

This kind of virtual reality technology was intended to improve flight simulators and applications for astronaut training. In 1981, the National Aeronautics and Space Administration (NASA) combined commercially available Sony liquid crystal display (LCD) portable television displays with special optics for a prototype stereo-vision head-mounted device called the Virtual Visual Environment Display (VIVED). NASA then created the first virtual reality system, which combined a host computer, a graphics computer, a noncontact user position-tracking system, and VIVED.

Scott Fisher and Elizabeth Wenzel developed hardware for three-dimensional virtual sound sources in 1988. Also, in the late 1980s, Lanier built a virtual reality system for two simultaneous users, which he named RB2 (Reality Built for Two). Fisher incorporated sound systems, head-mounted device technology, and sensor gloves into one system called the Virtual Interactive Environment Workstation (VIEW), also used by NASA. The first conference on virtual reality, “Interface for Real and Virtual Worlds,” was held in Montpellier, France, in March 1992. Later that year, scientists, engineers, and medical practitioners assembled in San Diego, California, for the “Medicine Meets Virtual Reality Conference.” In September 1993, the Institute for Electrical and Electronics Engineers (IEEE) organized its first virtual reality conference in Seattle.

The first commercial virtual reality video games debuted in the 1990s, but attempts by both SEGA and Nintendo to make the technology profitable failed. In 2012, the company Oculus Rift brought virtual-reality gamers to PC gamers. The company proved successful and was later bought by Facebook for USD$2 billion. The technology exploded in popularity in the late 2010s, with Oculus and HTC Vive leading the way. By 2021, Facebook’s Oculus Quest 2 was the leading VR headset on the market. Following Facebook, Inc.'s 2021 rebranding to Meta Platforms, Inc., the company's Meta Quest 2 VR headset became popular. In October 2023, Meta released the Meta Quest 3 VR headset, which was also widely accepted among consumers. Other models, such as Sony's PlayStation VR2 and Valve's Index, were rated highly by critics.

In 2023, American tech company Apple announced the Apple Vision Pro. This headset utilizes both virtual and augmented reality to project apps, programs, and other forms of media onto the user's surroundings. Termed "revolutionary" by the company, the Apple Vision Pro was described by Apple as offering improved functionality and performance compared to its peers. However, it was also reported to be much more expensive, retailing at USD$3,499 compared to the Meta Quest 3's price of USD$500.

How It Works

To create a realistic computer-generated world, several high-end technologies must be integrated into a single virtual reality system. This kind of high-end system is used at universities, military training, governmental organizations, and private research laboratories. Adequate computing speed, power, fast image and data processors, broad bandwidth, and sophisticated software are essential. Other requirements include high-tech input-output devices or effectors (such as head-mounted devices), three-dimensional screens, surround-sound systems, and tactile devices (such as wired gloves and suits, tracking systems, and force feedback devices, including motion chairs and multidirectional treadmills).

Input. The input devices used for virtual reality systems—sensing gloves, trackballs, joysticks, wands, treadmills, motion sensors, position trackers, voice recognizers, and biosensors—are typically more complex than those used for personal computers. Biosensors recognize eye movement, muscle activity, body temperature, pulse, and blood pressure, all vital to surgical applications. Position trackers and motion sensors identify and monitor the user's position and movements. The tracking systems used in virtual reality systems are mechanical, optical, ultrasonic, or magnetic devices. Steering wheels, joysticks, or wands are used for pilot training and games. Sophisticated devices for research and experiments (such as those for molecular modeling) offer six degrees of freedom input. Such input devices allow the computer to adjust the virtual environment according to the data received from the user. When motion sensors and position trackers detect the user's movement, the data are processed by the computer in real-time, and the display, also in real-time, has to accurately render the image (such as the interior of a building). If slow data processing creates a time lag, the user may experience simulator sickness or motion sickness (nausea or dizziness), especially if the user's senses register conflicting data. In a virtual reality parachute training session, for example, equilibrioception might conflict with visual perception if movement is represented faster by the display than by the force feedback system.

Output. Output devices are intended to stimulate as many senses as possible for a high degree of immersion. The important visual output devices are head-mounted devices, LCDs, and projectors and screens. Developers compete to make these displays the most immersive. All displays must be able to render three-dimensional images. Other output devices are for sound and touch. Sound systems can assist in conveying the impression of three-dimensional space. Force feedback systems give the user the sense of physical resistance (essential in surgical virtual reality systems), torque, tilt, and vibration (appreciated for games and essential for pilot training).

The CAVE (Cave Automatic Virtual Environment) is a surround-sound, surround-screen, projection-based room-sized virtual reality system developed by the Electronic Visualization Laboratory at the University of Illinois at Chicago (the name is trademarked by the University of Illinois Board of Regents). Users put on lightweight stereo glasses and walk around inside the room, interacting with virtual objects. One user is an active viewer, controlling the projection, and the others are passive viewers, but all users can communicate while in the CAVE. The system was designed to help with the visualization of scientific concepts.

Hardware and Software. Standard personal computers have limited memory capacity and limited performance capability for running professional virtual reality applications. Therefore, high-end hardware and software have been developed for specific purposes, such as games, research programs, and training in aviation, combat, and medicine. Computer languages such as C++, C#, Java, JavaScript, and Python are used to program virtual reality software. Virtual reality computers handle tasks like data input and output and the interaction, integration, and recomposition of all the data required in virtual environment management. Because a virtual reality system needs high computing, processing, and display speeds, the virtual reality computer architecture can use several computers or multiple processors.

Applications and Products

Aerospace and Military. One of the early applications of virtual reality was in pilot training. Modern flight simulators are convincingly close to real flight experiences, although the simulation of acceleration and zero gravity are still challenges. In the military, virtual reality applications are used not only by pilots, paratroopers, and tank drivers but also by battle strategists and combat tacticians to enhance safety training and analyze battle maneuvers and positions. These military applications make targeting more precise, thereby reducing human casualties and collateral damage. Virtual reality systems are also used to evaluate new weapons systems. In space exploration, virtual reality systems help astronauts prepare for zero-gravity activities such as the repair of solar panels on the outside of a spaceship. Virtual reality training systems give the user the opportunity to review and evaluate specific sequences in a training session or the entire session.

Entertainment and Games. The military took commercial games and adapted them to create flight simulators and other professional virtual reality applications, which, in turn, were adapted to use as games. By the beginning of the 2020s, increased interest in virtual reality game applications and the virtual games industry meant that some arcades dedicated to virtual reality were popping up in countries worldwide, and they were also making significant headway into the home and the mobile entertainment market. By that time, several companies had become involved in the virtual reality gaming market, including Meta, which had acquired the manufacturer Oculus, and such companies had produced more advanced virtual reality systems and headsets. More work was being done on creating the right games to complete the best virtual gaming experience.

Some companies lease virtual reality game equipment (such as training applications for golf, racing, and shooting) to customers for entertainment or to corporations for internal team-bonding experiences. Some virtual reality applications enable users to journey through fantastic and futuristic worlds. The common equipment for virtual reality games—depending on the quality and level of sophistication of the games—has included head-mounted devices, multiple screens, tracking systems, omnidirectional treadmills for walking, force feedback or rumbling seats (or platforms) for flight and race simulations, batons for tennis, and guns, wands, or sticks for shooting. The more sensory feedback provided, the more immersive the virtual reality game application.

Education. The more senses involved in the learning and training process, the more a student can become engaged in the subject matter and the better the educational impact, especially when learning skills or practical content. The employment of a virtual reality system enables data and images to become interactive, colorful, three-dimensional, and accompanied by sound. For example, a visitor to a virtual museum can virtually touch the artifacts, taking them from their shelves, turning them around, and viewing them from different perspectives. In an immersive experience, a user can witness, seemingly first-hand, historical events such as famous battles as they take place. Virtual reality applications exist in almost every area of education, including sports (such as golfing), sciences (such as astronomy, physics, biology, and chemistry), the humanities (such as history), and vocational training (such as medical procedures and mechanical engineering techniques).

Art. Although classic artworks can be immersive, few of them are interactive. People are not allowed to touch exhibits in most art galleries and museums. In 1993, the Solomon R. Guggenheim Museum became the first major museum to dedicate an entire exhibition to virtual reality art. The exhibition featured virtual reality installations from Jenny Holzer, Thomas Dolby, and Maxus Systems International. Virtual reality technology enables artists to blur or combine genres (such as music, graphic arts, and video) and involves the viewer in creating art. Every viewer or user of the artwork perceives the work differently depending on his or her input into the virtual reality work of art. A virtual reality art exhibit can be programmed to produce sound or visual feedback according to, for example, a visitor's footsteps or voice input from an audience. Artworks thus become interactive, and viewers become cocreators of the art. VR technology is utilized by multimedia artists and animators to create content and make three-dimensional storyboards.

Science, Engineering, and Design. Virtual reality systems can advance the creative processes involved in science, engineering, and design. Results can be tested, evaluated, shared, and discussed with other virtual reality users. Chemists at the University of North Carolina, Chapel Hill, used virtual reality systems for modeling molecules, and similar systems have been used to “observe” atoms. Buildings, automobiles, and mechanical parts can be designed on computers and evaluated using three-dimensional modeling tools and visualization techniques in a virtual reality environment. An architect can take a client on a virtual walk through a building before it is constructed, or the aerodynamics of a new automobile design can be evaluated before manufacturing a model or prototype. Virtual reality applications also can simulate crash tests. The use of virtual reality applications reduces cost, waste, and risk.

Business. Business applications include online stores featuring virtual showrooms and 360-degree, three-dimensional views of products. This 360-degree view capability was also increasingly used in the real estate industry in the late 2010s and 2020s. Another critical application is teleconferencing. Unlike traditional conference calls, virtual reality applications may permit multisensory evaluations of new products. Virtual reality systems also help network, combine, and display data from diverse sources to analyze financial markets and stock exchanges. In these situations, virtual reality applications serve as decision support systems.

Medicine, Therapy, and Rehabilitation. Experienced surgeons, as well as physicians in training, use virtual reality systems to practice surgery. The images for such training programs are taken from X-rays, computed tomography (CT) scans, and magnetic resonance imaging (MRI). Virtual operations can be recorded and repeated as many times as desired, in contrast to practice operations on animals or human corpses, which cannot be repeated. Force feedback gloves give practitioners the realistic feel and touch needed, for example, to determine how much force is required for certain incisions.

Virtual reality environments are being developed for use in training patients and doctors. Virtual human limbs are being prototyped and studied for use in training patients to use and become comfortable with prosthetic limbs and for patient rehabilitation. Some of the virtual prosthetics in development appear as avatars in a virtual reality environment and can accept both kinematic and neural control inputs from the patient.

In psychotherapy, virtual reality applications may be used as an alternative therapy to help treat clients with phobias, anxiety, and post-traumatic stress disorder (PTSD). Clients often undergo desensitization treatment, which can be especially useful in treating phobias and PTSD. Virtual reality allows clients who fear enclosed spaces (claustrophobia), dirt and germs (mysophobia), or snakes (ophidiophobia) to gradually confront their fears without being exposed to the actual condition or object. Virtual reality applications can also help clinicians better understand specific psychological problems by enabling them to experience what the client or the patient experiences. For example, a psychiatrist, psychologist, or counselor may take a virtual bus ride in the role of a client experiencing schizophrenia, during which they experience some simulated symptoms of this disorder, such as distorted images viewed through the windows of the bus or strange voices that seem to appear from nowhere.

Careers and Course Work

Because the development of virtual reality systems and applications involves many areas of applied science, students pursuing careers related to virtual reality systems should be versatile. A bachelor of science in a field such as information technology, mechanical engineering, or electrical engineering is an appropriate foundation for a career in virtual reality. Still, most positions in the field require multidisciplinary skills and experience with designing and developing three-dimensional software. Therefore, the best approach is to take additional courses in other fields.

For high-level positions or to work as a researcher in industry or academia, a master of science degree or a doctorate is recommended. Work can also be found with governmental entities, like NASA or the military, and manufacturers of virtual reality systems and products, such as head-mounted devices, screens, force feedback systems, and tracking systems. Virtual reality software and hardware producers need skilled professionals like electrical and mechanical engineers, roboticists, software developers, and information technologists. Job opportunities related to virtual reality technology and research remain relatively limited because of the field's highly technical and specialized nature. It is generally believed that product managers, business development, sales representatives, programmers, and engineers will be needed as the VR industry grows.

Social Context and Future Prospects

Some experts believe virtual reality, like the computer and the internet, will become commonplace and indispensable in everyday life. Still, others do not see the potential for such a wide implementation. Most agree, however, that once computing speed and power, broad bandwidth, and peripheral systems become more affordable for average consumers, virtual reality will be more widely used. Beyond science, education, and other professional applications, the entertainment industry is thought most likely to want affordable virtual reality innovations. Critics of virtual reality applications point to personal and societal risks such as isolation, desocialization, and alienation, but advocates emphasize the technology's proven potential for augmenting people's lives. Both critics and advocates agree that experiences in virtual reality can alter people's perceptions of and responses to the real world. These changes are often intentional and welcome, but sometimes, they take place in an unintended and potentially dangerous manner. Airplane pilots have reportedly made mistakes that could be linked to the limitations of training with a flight simulator, which, for example, is incapable of realistically simulating acceleration. However, virtual reality applications are valuable in highly technical and precise medicine, industry, business, and research areas. They are likely to gain more users despite the potential risks and expenses.

By the 2020s, the amount of wide-scale investment that had been made by several companies in the development of virtual reality technology had resulted in greater accessibility to lower-cost systems. After the coronavirus (COVID-19) pandemic, analysts noted the versatility of the technology had become even more prominent as people remained at home and reduced physical interactions to slow the spread of the virus. While businesses such as museums turned more to virtual, online tours during lockdowns, the application of virtual reality in mental health was especially highlighted, as the technology was viewed as a beneficial alternative to in-person therapy.

Bibliography

Al-Jumaily, A., and R. A. Olivares. "Bio-Driven System-Based Virtual Reality for Prosthetic and Rehabilitation Systems." Signal, Image and Video Processing, vol. 6, no. 1, 2012, pp. 71–84. doi.org/10.1007/s11760-010-0180-x.

Burdea, Grigore C., and Philippe Coiffet. Virtual Reality Technology. 2nd ed., Wiley-Interscience, 2003.

Craig, Alan B., William R. Sherman, and Jeffrey D. Will. Developing Virtual Reality Applications. Foundations of Effective Design. Morgan Kaufmann, 2009.

De Luna, Alvin Albuero. Introduction to Virtual Reality. Arcler Press, 2022.

Greenwald, Will. "The Best VR Headsets for 2023." PC Gamer, 15 Apr. 2024, www.pcmag.com/picks/the-best-vr-headsets. Accessed 15 May 2024.

Hamad, Ayah, and Bochen Jia. “How Virtual Reality Technology Has Changed Our Lives: An Overview of the Current and Potential Applications and Limitations.” International Journal of Environmental Research and Public Health, vol. 19, no. 18, 8 Sep. 2022. doi:10.3390/ijerph191811278.

Levin, Mindy F., Emily A. Keshner, and Patrice L. (Tamar) Weiss. Virtual Reality for Physical and Motor Rehabilitation. Springer, 2016.

LaValle, Steven Michael. Virtual Reality. Cambridge UP, 2023.

Loeffler, John. "The History and Science of Virtual Reality Headsets," Interesting Engineering, 30 Sept. 2021, interestingengineering.com/the-history-and-science-of-virtual-reality-headsets. Accessed 15 May 2024.

Paris, Francesca. "A Stange New World: Has Virtual Reality Gaming Lived Up to Its Promise?" WBUR, 24 Feb. 2020, www.wbur.org/hereandnow/2020/02/24/virtual-reality-orion-13-games. Accessed 22 Mar. 2021.

Ruzive, Von N., and Peter Jeun Ho Tsang. Fashion Tech Applied: Exploring Augmented Reality, Artificial Intelligence, Virtual Reality, NFTs, Body Scanning, 3D Digital Design, and More. Apress, 2023.

Rathburn, Betsy. Virtual Reality. Bellwether Media, Inc., 2021.

Rizzo, Albert, and Stéphane Bouchard. Virtual Reality for Psychological and Neurocognitive Interventions. Springer, 2019.

Robertson, Adi. "Apple Vision Pro Is Apple's New $3,499 AR Headset." The Verge, 5 June 2023, www.theverge.com/2023/6/5/23738968/apple-vision-pro-ar-headset-features-specs-price-release-date-wwdc-2023. Accessed 3 Aug. 2023.

Sherman, William R., and Alan B. Craig. Understanding Virtual Reality: Interface, Application and Design. 2nd ed., Morgan Kaufmann, 2019.

Spiegel, Brennan. "Virtual Reality and the COVID Mental Health Crisis." Scientific American, 15 Nov. 2020, www.scientificamerican.com/article/virtual-reality-and-the-covid-mental-health-crisis. Accessed 22 Mar. 2021.

"US VR and AR Users, 2019–2023." eMarketer, 22 Feb. 2021, www.emarketer.com/chart/244851/us-vr-ar-users-2019-2023-millions. Accessed 22 Mar. 2021.

"US VR and AR Users, 2019–2027." eMarketer, 13 Mar. 2023, www.emarketer.com/chart/261863/us-ar-vr-users-2019-2027-millions. Accessed 20 May 2024.