EMERGING EXPONENTIAL TECHNOLOGIES 20MBA301

Module -5 Augmented Reality & Virtual Reality

Introduction to AR, Virtual reality (VR), Augmented Reality (AR) VS mixed reality (MR),
Architecture of AR systems. Application of AR systems (education, medical, assistance,
entertainment) workshop oriented hands demo.

Introduction to AR (https://www.youtube.com/watch?v=oH_LfXnklRw)

In layman's terms, Augmented Reality is a technology that enhances the real world by affixing
layers of digital elements onto it. These elements include computer-generated graphics, sound or
video effects, haptic feedback, or sensory projects.
The term Augmented Reality was coined back in 1990 by Boeing researcher Tom Codell and
one of the first commercial uses of this technology was in television and the military. As the
world shifted towards becoming more technology-driven, AR became increasingly prominent in
multiple fields, rolling out its second wave and drawing its connection towards the interactive
concept.

So, an augmented reality is combination of a real scene viewed by a user and a virtual scene
generated by a computer that augments the scene with additional information. The process of
superimposing digitally rendered images onto our real-world surroundings, giving a sense of an
illusion.

Augmented Reality & Virtual Reality 1

EMERGING EXPONENTIAL TECHNOLOGIES 20MBA301

Augmented View

Simulation

A simulation is the imitation of the operation of a real-world process or system over time.
Simulations requires the use of models; the model represents the key characteristics or behaviors
of the selected system or process, whereas the simulation represents the evolution of the model
over time. AR is a technology that lets people superimpose digital content (images, sounds, text)
over a real-world environment. AR got a lot of attention in 2016 when the game Pokémon Go
made it possible to interact with Pokémon superimposed on the world via a smartphone screen.

Augmented Reality & Virtual Reality 2

EMERGING EXPONENTIAL TECHNOLOGIES 20MBA301

Augmented reality is what it sounds like: reality, enhanced with interactive digital components.
The most commonly used AR applications these days rely on smartphones to showcase the
digitally augmented world: users can activate a smartphone’s camera, view the real world around
them on the screen, and rely on an AR application to enhance that world in any number of ways
via digital overlays:

 Superimposing images, digital information and/or 3D models

 Adding real-time directions
 Inserting labels
 Changing colors
 Altering the user or their environment's appearance via "filters" on Instagram, Snapchat,
and other apps

Various devices can display AR, and the list is only growing: screens, glasses, handheld and
mobile devices, and head-mounted displays.

In understanding what AR is, it’s also important to understand what it is not.AR is not a fully
immersive experience like virtual reality (VR). While virtual reality requires users to don a
special headset and pulls them into a completely digital world, AR lets them continue interacting
with the physical world around them

Architecture of AR systems

Augmented Reality & Virtual Reality 3

EMERGING EXPONENTIAL TECHNOLOGIES 20MBA301

1) Capturing the real world: The background of AR applications is the real world, done by
cameras on mobile devices.
2) The construction of the virtual world: on the basis of the real world, many virtual object
models can be added to form a complex virtual world.
3) Rendering: Renders the captured real world as the background of the AR application;
renders the virtual world into the captured real world.
4) World Tracking: When the real world changes, such as moving the camera, it is necessary
to track the position and angle change information of the current camera relative to the
initial time in order to render the position and angle of the virtual world relative to the
real world in real time
5) Scene analysis: In order to make the virtual world and the real world more realistic, it needs
a certain degree of understanding of the real world, such as plane recognition.
6) Interacting with the virtual world: interacting with the object model of the virtual world to
increase the interest of the AR application.

In the Web-based mobile AR application, WebRTC is usually used to open the camera to
realize real-world capture and rendering; use Three.js to realize the construction of virtual
world, the rendering of virtual world and the interaction with virtual world. The WebRTC-
based AR application uses WebTRC technology to open the camera, capture the real-world
video stream, and render it to the screen. When a real-world image is needed, the image is
taken from the video stream.

In doing so, the calculation delay is very large. After the image is obtained, the image is
processed, and the processed result is reflected into the virtual world. There is a large delay in
these steps, which may cause the virtual world to be out of sync with the real world. In order
to alleviate the pressure on the mobile side and ensure the unification of the virtual world and
the real world, a hybrid rendering concept is proposed, which encapsulates the capture and
rendering of the real world into the browser, using the native capabilities of the mobile
terminal, and the virtual world-related parts. The rendering is still on the Web front end.

The world tracking and scene analysis part requires a lot of calculations, and the computing
ability of the Web side limits the implementation of the Web-based mobile AR application in
this respect. Therefore, only a few simple application modes can be implemented. This part is
also the biggest limitation of current Web-based AR applications. Based on the native mobile
application, there is no such trouble. The powerful computing power of the mobile terminal
ensures the realization of the world tracking and scene analysis. Therefore, this part of the
ability is packaged into the browser, using the mobile end native ability to achieve, to make
up for the lack of Web-based AR applications in this regard.

Augmented Reality & Virtual Reality 4

EMERGING EXPONENTIAL TECHNOLOGIES 20MBA301

AR System Architecture Overview

The AR System is built on a multi-tier, client-server architecture. Tiers are characterized by

the type of processing that takes place within them, or in the way one tier communicates with
another. The AR System consists of three tiers:

• Client Tier

• Mid-Tier

• Server Tier

Client tier—Contains AR System clients. Most clients present information to application users
and receive input from them, but the tools for migration and application development are also
clients.

Mid-tier—Contains components and add-in services that run on a web server, enabling users
to view applications on the web.

Server tier—Contains the AR System server, which controls workflow processes and access
to databases and other data sources in the data tier. This tier also contains server-side
applications (such as Approval Server, Email Engine, and the Flashboards server) and the C
and Oracle Java plug-in servers with plug-ins.

Data-tier—Contains database servers and other data sources that can be accessed by the AR
System server. The database server acts as the data storage and retrieval engine.

Augmented Reality & Virtual Reality 5

EMERGING EXPONENTIAL TECHNOLOGIES 20MBA301

Applications of AR in various business sectors

1. Manufacturing

When we talk about the manufacturing industry we always keep in mind that even the smallest
mistakes can cause interruptions of the whole process. The complex tasks that this industry has,
require precise and accurate decisions in a very fast time. Thanks to the Augmented Reality
technology, empowerment of the workforce with remote support is possible. This enhanced
remote support makes it possible in real-time for the most capable experts to give instructions to
site workers who encounter many problems during production processes, mainly with the many
machines used in this industry. Through AR remote assistance it is possible to reduce machine
downtime through the efficient solutions found making it possible to save thousands of dollars
that could be lost.

2. Mining

As mining is a very delicate industry, the inspection of machinery and the working environment
is an important process that follows many subsequent procedures. Augmented Reality makes it
possible for the center experts to inspect from a distance virtually all the machines as well as the
surrounding environment. In addition, through the smart glasses and the zoom-in / out feature,
every detail is displayed, including the dangerous areas, but always maintaining the safety of the
workers. So, thanks to this technology, many critical situations that can arise are avoided, thus
facilitating the work of the site workers.

Augmented Reality & Virtual Reality 6

EMERGING EXPONENTIAL TECHNOLOGIES 20MBA301

3. Maritime

Augmented Reality is showing great opportunities to help the maritime industry to overcome the
challenges it is facing especially in maintenance, remote assistance, inspections, and workforce
training. We can say that the training of employees operating offshore is extremely important.
Through AR remote assistance, center experts can provide real-time training to current
inexperienced employees on more complex issues. Although located offshore, employees
through this distance training become better prepared for unforeseen situations that may arise in
the future.

Augmented Reality & Virtual Reality 7

EMERGING EXPONENTIAL TECHNOLOGIES 20MBA301

4. Insurance

The insurance sector is mainly concerned with complex processes, often critical and unexpected
situations occur and in such cases, immediate customer support is required. Insurance companies
that normally operate and also deal with property or vehicle insurance definitely need to send an
insurance agent to inspect and assess the damage. Now through AR remote assistance all the
inspections can be done in real-time and reducing the time and cost that sending the agent to the
clients may require. Through this technology, the whole process of inspection and damage
assessment is done in a faster and simpler way. This whole process can even be documented for
later purposes. At the same time, this technology makes agents virtually closer to customers,
significantly improving customer service and their satisfaction.

5. Education

AR technology is making a difference in the field of education by engaging more the education
staff but at the same time students as well. In addition to the fun sphere making the educational
environment more attractive, at the same time, it gives teachers the opportunity to present real
situations through 3D modeling. In this way, the information passed on to the students is easier,
perceptible, and understandable. This kind of explanation of different topics, in addition to
making students more interested in lessons, replaces the traditional way of learning. This gives
the opportunity to create a more informed and knowledgeable generation for the future. Even

Augmented Reality & Virtual Reality 8

EMERGING EXPONENTIAL TECHNOLOGIES 20MBA301

empowered games through this technology make students more engaged, especially in the
theoretical hours of physical education, thus promoting an active and healthy life.

6. Healthcare

The healthcare industry is embracing and interacting with AR as a tool that helps to improve the
education not only of the medical staff but also of the patient. Initially AR helps medical students
who need to learn a great deal of information about anatomy and how the body works. They are
able to see different 3D models thus being able to learn every function of the body in detail as
well as be taught to find solutions to problems given through stimulus tests. At the same time this
AR technology also helps specialized doctors by facilitating those numerous work processes.
Usually, surgeons are using AR as a tool that enables the modeling of three-dimensional
representations of patient anatomy. From this application, we have an easier process of
visualizing the area where the surgery will be performed and impacting the accuracy for a
satisfactory result for the patient.

Augmented Reality & Virtual Reality 9

EMERGING EXPONENTIAL TECHNOLOGIES 20MBA301

7. Travel

Another area where this innovative technology is being used is travel. Making the tourist
experiences even more special, AR technology is being widely embraced in many countries.
Thanks to this technology, tourists now have access to tourist information and guides not only in
the written version but also in the form of audio, making it even easier to explore tourist spots.
Another special feature is the 360-degree tours that are easily accessible, thus making it possible
to virtually explore a place before visiting reality. This technology is not only used by travel
agencies but now also by restaurants and hotels that offer virtual tours of their facilities, giving
guests a virtual experience of what to expect.

8. Supply Chain

Inventory management is a significant factor in the supply chain. Augmented Reality by adding a
virtual experience on top of inventory data helps businesses and provides field workers
instructions for step-by-step container loading and tracking different procedures. In addition,
smart glasses integrated with AR remote assistance provide hands-free task performance. Users
can work with their heads up and they are less likely to be distracted or injured. Improved safety
increases and training cost and downtime are reduced at the same time.

What we mentioned above are just some of the industrial areas where Augmented Reality
technology is contributing extensively to facilitate all industrial processes and is adding value to
businesses operating in different sectors. We can say that this technology is establishing a new

Augmented Reality & Virtual Reality 10

EMERGING EXPONENTIAL TECHNOLOGIES 20MBA301

order in the business world by improving the way companies interact with customers and at the
same time being the right tool to increase employee engagement effectively. As this technology
continues to revolutionize the world, all businesses are encouraged to adopt it as a tool that offers
many long-term benefits.

Virtual Reality

Virtual Reality (VR) is a computer-generated environment with scenes and objects that appear to
be real, making the user feel they are immersed in their surroundings. This environment is
perceived through a device known as a Virtual Reality headset or helmet. VR allows us to
immerse ourselves in video games as if we were one of the characters, learn how to perform
heart surgery or improve the quality of sports training to maximise performance.

Although this may seem extremely futuristic, its origins are not as recent as we might think. In
fact, many people consider that one of the first Virtual Reality devices was called Sensorama, a
machine with a built-in seat that played 3D movies, gave off odours and generated vibrations to
make the experience as vivid as possible. The invention dates back as far as the mid-
1950s. Subsequent technological and software developments over the following years brought
with them a progressive evolution both in devices and in interface design.

Augmented Reality & Virtual Reality 11

EMERGING EXPONENTIAL TECHNOLOGIES 20MBA301

Augmented Reality vs. Virtual Reality

Augmented Reality Virtual Reality

• System augments the real world scene • Totally immersive environment

• User maintains a sense of presence in real • Senses are under control of system
world

• Needs a mechanism to combine virtual and • Need a mechanism to feed virtual world to
real worlds user

• Hard to register real and virtual • Hard to make VR world interesting

Augmented Reality & Virtual Reality 12

EMERGING EXPONENTIAL TECHNOLOGIES 20MBA301

Milgram’s Reality-Virtuality Continuum

The RV (reality-virtuality) continuum, as initially proposed by Milgram and Kishino, is shown in

Figure 1. They anchor one end with a purely real environment, “consisting solely of real
objects,” and the other, with a purely virtual environment, “consisting solely of virtual objects”
(Milgram and Kishino, 1994). They consider any environment which consists of a blending of
real and virtual objects to be mixed reality (MR). Mixed reality environments where the real
world is augmented with virtual content are called augmented reality (AR), while those where
most of the content is virtual but there is some awareness or inclusion of real world objects are
called augmented virtuality (AV). Of note is that this original version of the continuum was
explicitly concerned only with visual displays.

While the original version of the continuum has undoubtedly served the field well, we have
identified limitations. One is that, as mentioned above, Milgram and Kishino were explicitly
concerned with visual displays, and primarily with display hardware. A second is that nowhere in
this continuum is seen the notion of an observer or a user with senses other than visual and prior
life experiences. Finally, content was described only in relation to realism (e.g., wireframes vs.
3D renderings), with no concern for the coherence of the overall experience. We will soon argue
that the notion of an environment without an experiencing being—the aforementioned
observer—is incomplete. That is, the mediating technology, content conveyed, and resulting user
experience must be considered together to adequately describe MR experiences.

AR vs VR vs MR

 Virtual reality (VR) immerses users in a fully artificial digital environment.

 Augmented reality (AR) overlays virtual objects on the real-world environment.

 Mixed reality (MR) not just overlays but anchors virtual objects to the real
world.

Augmented Reality & Virtual Reality 13

EMERGING EXPONENTIAL TECHNOLOGIES 20MBA301

Difference between AR, VR, and MR

Augmented reality (AR) adds digital elements to a live view often by using the camera on a
smartphone. Examples of augmented reality experiences include Snapchat lenses and the game
Pokemon Go.
Virtual reality (VR) implies a complete immersion experience that shuts out the physical world.
Using VR devices such as HTC Vive, Oculus Rift or Google Cardboard, users can be transported
into a number of real-world and imagined environments such as the middle of a squawking
penguin colony or even the back of a dragon.
In a Mixed Reality (MR) experience, which combines elements of both AR and VR, real-world
and digital objects interact. Mixed reality technology is just now starting to take off with
Microsoft’s HoloLens one of the most notable early mixed reality apparatuses.

Augmented Reality & Virtual Reality 14

EMERGING EXPONENTIAL TECHNOLOGIES 20MBA301

Mixed Reality in the Real World

The possibilities and implications of Mixed Reality's continuous growth demonstrate that the
future of MR is both realistic and limitless. The use of MR in industries and other fields of
Science and Management is astounding. Mixed Reality is effectively used in the following
sectors:

(a) Healthcare and medicine: Mixed Reality simulations are already helping medical students
understand their patients better. It is a boon, particularly for those who have hearing and visual
problems. Through MR technology, how these patients experience the real world is simulated,
leading to greater empathy and understanding from medical practitioners. Doctors and surgeons
can be trained in complicated surgeries and develop cutting-edge procedures. The surgeons can
operate in a Virtual Reality mode and access the outcomes of such operations before taking the
patient to the Operation theatre. The medical students can also better prepare for work scenarios
safely and without risk. making them better prepared for real-life situations.

(b) Education: Experiential education is one of the most effective learning and teaching tools.
Through Mixed Reality, students are able to interact with what they are learning be it a theory or
an experiment. It should not be mistaken about using auditory and visual aids for teaching, but
actual experiences that promote deeper, immersive learning'. Mixed Reality may also be used in
the future to expand on Virtual Reality's work with immersing learners in different industry and
work scenarios. Learners will be able to learn about equipments and processes as if it existed in
real life. MR can also be used to make people get an immersive experience about other cultures,
social, political and economic concerns.

(c) Gaming: Mixed Reality is widely used in garing, unlocking an experience that combines the
intense and impressive worlds in video games into the actual environment. A mixed reality game
(or hybrid reality game) is a game that includes and operates in both reality and virtual reality

Augmented Reality & Virtual Reality 15

EMERGING EXPONENTIAL TECHNOLOGIES 20MBA301

simultaneously. Some examples of mixed reality games on Android smartphones and the iPhone
are Spectrek, Ingress and Gbanga. With digital elements being embedded in the actual
environment, people will be motivated to physically move to access digital content and
experience them hands-free. This is better than simply looking at screens of mobiles or other
handheld devices. Some of the Mixed Reality games as in 2021 are:

Compatible VR Games
 Audica (Quest & Rift)
 Beat Saber (Quest & Rift)
 Dead & Buried 2 (Quest & Rift)
 Gadgeteer (Quest & Rift)
 Kingspray (Quest)
 Racket Fury (Quest)
 Real VR Fishing (Quest & Richie's Plank Experience (Quest)

(d) Retail and business: Many possibilities await businesses when they make use of Mixed
Reality technologies. Companies are now maximizing VR and AR applications in attracting
consumers and retaining them. Some applications provide augmented reality maps to customers
for better understanding and access to their stores. With MR, stores can give customers unique
and detailed information of the sections they are through and also the details of the products on
display. A walk through the aisles, empowers customers to make informed and confident
purchases without the need for human intervention. The customers can also experience a product
or a service before purchasing. Mixed Reality will unlock a whole new way for customers to
experience the world of market and stores.
Very soon, people will be able to experience MR in some of the industries like cloud
kitchens, sports, art and culture, traditional and non-traditional therapies, counselling
cinematography, space, scientific experimentations, agriculture, engineering, management
science and transportation.

Important Questions:

1. What is Augmented Reality (AR)?

2. What is Virtual reality (VR)?
3. Explain the architecture of AR system.
4. Explain the applications of AR in different business sectors.
5. Explain 3 differentiation point of AR vs VR.
6. Explain Milgram’s Reality-Virtuality Continuum with a diagram.
7. Explain the various uses of mixed reality in different sectors.

Augmented Reality & Virtual Reality 16