Moderator: Steve Mann, "The father of the wearable computer" [IEEE ISSCC 2000], University of Toronto, MIT ('97), P. Eng (On)
Panelists:
Original registration page: https://engagestandards.ieee.org/Metaverse-Congress-5.html
Paper summary: https://doi.org/10.48550/arXiv.2212.07960
Abstract (link).
6-minute introduction here:
XV is a proper superset of the metaverse, just as XR is a proper superset of VR.
XR includes VR + AR + MR, i.e. XR includes any mixture of reality and virtuality, or any computer-generated addition to reality, but, more importantly, XR also eXtends beyond reality and virtuality. Examples of XR as extensions to reality are HDR vision (e.g. the HDR welding helmet that allows a person to see a dynamic range of more than 100 million to one), and SWIM (Sequential Wave Imprinting Machine) which allows people to see ultraviolet, infrared, or electromagnetic radio waves, sound waves, brainwaves, etc., in vacuum, air, water, or solid matter, without the need for any eyewear. In this way XR eXtends human senses even underwater (XR includes WaterHCI), beyond what can be provided by VR + AR + MR.
Additionally, XR serves as a broad "umbrella" term that covers all "The Realities", so that those who work in the liminal space between AR and VR, for example, can describe their work and research as existing in the field of XR. In this sense "X" can be interpreted as an all-encompassing reality variable, as in X ∈ ℝ, ∀ of the "Realities", ℝ (X is anything in the set of all "Realities").
We propose XV as a persistent social (shared) immersive XR environment, as illustrated in this sculpture:
Caption: The building blocks of XV: Whereas PR (Physical Reality) is along the Reality ("Atoms") axis and VR (Virtual Reality) is along the Virtuality ("Bits") axis, XR eXtends our perception and intelligence beyond the 100% point on both axes. Consider a third axis, Sociality ("Genes"). Metaverse extends out from VR along this third axis. XV extends out from XR along this third axis.
XV (eXtended meta/uni/Verse) combines XR (eXtended Reality, perceived with senses or peripherals including brain-interface technology), XI (eXtended Intelligence, including HI = H. Int. = Humanistic Intelligence = human-in-the-loop Artificial Intelligence), XB (eXtended Being, including Digital Twin), XE (eXtended Economy), and XS (eXtended Society) into a vision that covers the generally agreed scope of metaverse while extending the impacts and emphasizing the implications on our consciousness and humanity.
Timeline leading up to XV:
Here are 3 practical examples of XV that are not typically associated with Metaverse but are core elements of XV as eXtended meta/uni/Verse beyond the metaverse:
Caption: Showing electromagnetic currents flowing in an electric motor to Ford Motor Company and the Minister of Economic, Development, Job Creation and Trade
Caption: A robotic mechanism plays a violin to display the sound waves coming from it, while a collaborative design process is completed.
Caption: Sequence of photographs of intersecting metaveillance flux lines: collaboration in studying the interplay between a surveillance camera mounted to a toy gun as an XV video game, combined with a metavision wand to make visible the camera's veillance flux lines. Members of the audience can see the veillance flux even when not wearing any eyewear or looking through any device!
In summary, XV is to XR as the metaverse is to VR.
In this panel we will address the following:
Announcement: Please check out our CfP (Call for Papers) for IEEE Consumer Electronics.
PDF CfP (Call for Papers): OpenXV.org/XV_CfP.pdf.
IMPORTANT DATES: • ARTICLE SUBMISSION DUE: May 15, 2023 • FINAL ACCEPTANCE NOTIFICATION: September 30, 2023 • APPROX. PUBLICATION DATE IN PRINT: Q2 2024 GUEST EDITORS: • Abdulmotaleb El Saddik, University of Ottawa, Canada • Fabrizio Lamberti, Politecnico di Torino, Italy • Steve Mann, EyeTap, Canada • Filippo G. Pratticò, Politecnico di Torino, Italy • Ruck Thawonmas, Ritsumeikan University, Japan • Yu Yuan, VerseMaker, China
We propose XV (eXtended meta/uni/Verse), as a concept and taxonomy at the nexus of XR (eXtended Reality), XI (eXtended Intelligence), XB (eXtended Being, including Digital Twin), XE (eXtended Economy), and XS (eXtended Society). XV inclues and extends the metaverse (shared virtual reality) with the real world (universe), akin to the way that XR is an extension beyond just VR.
The concept of eXtended Reality (XR) is well-known as is "All Reality" as proposed by Mann, Yuan, Furness, and others [Mann and Wyckoff 1991] [Mann, Furness, Yuan, Iorio, and Wang 2018][Mann, Havens, Iorio, Yuan, and Furness 2018].
Consider the simple taxonomy of realities and their technologies,
real on the across axis
and virtual on the up-down axis, as shown below:
This simple taxonomy divides the plane into four quadrants:
Reality (i.e. Physical Reality),
Augmented Reality, Virtual Realty, and Diminished Reality (e.g.
where reality is deliberately diminished, attenuated, or reduced in an
isolation tank also known as a "float tank").
The across-axis denotes the real universe that is made of atoms, and the word "atom" is a Greek word that begins with the letter alpha (the first letter of the Greek alphabet). So we label this axis "alpha". The up-axis denotes the virtual world that is made of "bits", so we label this axis "beta" (the second letter of the Greek alphabet).
At the origin (zero on both axes), we have the example of an isolation tank in which a person can receive zero stimulus from the real world (no sound or light or anything from reality), and zero stimulus from the virtual world (no cellphone, no internet, etc.). This leaves the person in a hallucinogenic dreamlike state, akin to lucid-dreaming in the absense of any real or virtual stimulus. (Note that our taxonomy pertains to the absence of stimulus, which does not in any way imply an absence of perception. For example, hallucination is perception which can occur even with zero external stimulus. Hallucinations can be perceived as real, i.e. they can be perceived as if they were external true reality. Thus it is important to understand that our taxonomy is a simplification that helps us categorize technologies separate from perception, i.e. by the ``hard sciences'' like physics rather than by the ``soft sciences'' like psychology.)
At the lower right we have maximum stimulus from the real world and zero stimulus from the virtual world. A good example of a fully immersive real-world experience is "bare body" (no wetsuit) icewater swimming. It is a fully immersive extreme sport that requires 100% focus on the real world. No time to check email or "surf" the web. Alpha (atoms) = 100%. Beta (bits) = 0.
At the upper left we have maximum stimulus from the virtual world and zero stimulus from the real world. A good example of a fully immersive virtual experience is the VR (Virtual Reality) float tank [Mann 2003][Mann, Hao, Werner, 2018], i.e. wearing a fully immersive VR headset inside an isolation tank. Zero input from the real world and 100% input from the virtual world. Alpha = 0. Beta = 100%.
We've put such sensory-attenuation technologies to useful applications like the creation of a lucid dream machine [Mann, Furness, Yuan, Iorio, and Wang 2018].
A good example of an immersive experience that is both real and virtual is
a swim with the Mersivity underwater VR eyeglass. Alpha = 100%. Beta = 100%,
as illusrated in the upper right below:
At least that is the goal, whereas in practice we might lose some attention to
reality when introducing virtuality and vice versa.
Ideally we aim to develop technologies that extend our reality, i.e.
technologies that use virtuality to actually increase, rather than decrease,
or perception of reality.
For example, icewater swimming while wearing a head-up display, might gives us perhaps 150% immersive reality and maybe 50% virtuality, in the sense that the display heightens, rather than diminishes, our capacity to engage in icewater swimming. This all depends, of course, on what applications are running, as well as the general mix of hardware, software, and human factors and human conditions (e.g. willingness to pay attention to one or the other), and can vary with mood swings, diet, sleep, etc..
This state of XR (eXtended Reality)
is possible because an awareness of our physiological conditions
helps us be even more "in the moment" while calming our breathing,
reducing our heart rate using biofeedback, and enhancing our awareness of our
surroundings using overlays.
The figure below shows an example of where a typical situation
could fit in the XV continuum:
The monocular eyeglass display
takes nothing away from the full impact of the icewater reality,
while it also adds useful safety information like an awareness of one's heart
activity, water temperature, compass heading, map overlays, and location of
potential hazards such as rocks and larger ice formations.
Analogous to the four quadrants of XR (PR, AR, VR, and DR), we now have an additional four quadrants for social XR, i.e. this is XV:
Analogously the virtuality axis goes from "Bits", through "Little data" (distributed), to "Big data" (centralized) and the Metaverse.
The social axis extends out of the page, from individual, to couple, family, community, etc., as we move from the Autonoverse (the autonomous verse) to bigger and bigger layers of governance, municipal government, state government, federal government, world government, and ultimately to the laws of outer space.
Here is a simplified version of the above diagram:
Each of these three axes has a "little" end and a "big end", thus defining
a bifurcation, e.g. on the physical scale there is a taxonomy of
Invironment (us ourselves) versus Environment (that which surrounds us).
The environment might be the classroom environment, the urban environment,
or the natural environment -- whatever it is that is around us -- our
surroundings.
There is often a clear distinction between that which we regard as part of us
and that which we do not. We typically regard our clothes as part of us,
for example, or our cars or boats as part of us, such as when we hear a
boater or motorist say "You hit me" when vessels or vehicles collide
(rather than saying "Your boat hit my boat").
If zero is at the origin, we can think of a logarithmic axis which extends
in both directions, thus splitting each axis in two, as shown below:
Here we choose a particular scale at which to place the origin, as, say,
unity, and values less than unity (i.e. negative on a logarithmic axis)
are one the negative side of the origin.
The negative sides of each of the three axes are
Body, Control, and Ownership, i.e. the individual human body,
the control of something by an individual, or the ownership of something by
an individual.
At the other extreme is the Universe, the control of something by large entities,
and the ownership of something by large entities.
In this way, we can sort (taxonomize) technologies into 8 octants. Technologies can be said to be invironmental (e.g. wearables), or environmental (e.g. non-cyborg technologies that are not really physically regarded as part of our bodies). Here 0 represents a small value and 1 represents a large value:
α β γ 0 0 0 InterInfraAutonoVerse: Example: sousveillant wearable NFT 0 0 1 InterInfraPoliVerse: Example: surveillant wearable NFT 0 1 0 InterMetaAutonoVerse: 0 1 1 etc.. 1 0 0 1 0 1 1 1 0 1 1 1 UniMetaPoliVerse: Example: big data smart city surveillance
Typically we choose our own clothing, and we own it also, and this situation fits into the octant at the origin, (0, 0, 0). Ordinarily, those technologies physically closer to us are generally our own, whereas the technologies that surround us (environment) are often someone else's. For example we might wear or carry our own camera in our pocket, whereas the cameras mounted on the wall or ceiling or lamp post might belong to a larger (remote or distant) entity.
Technologies can both help and hinder us. As technologies become closer to the body, there is a growing need to understand harm versus benefit analysis of these "cyborg" technologies. Technologies can be like handcuffs that are close to the body physically, yet their locus of control is distant or remote. They exist in the uncanny off-diagonal space (refer to the diagonal line from the origin out equally along the axes).
For example, whether at a prison, or at certain workplaces and certain schools, we might be required to wear a uniform that we also don't own, which fits at (0, 1, 1). We might be required to purchase a work or school uniform, so we own it, but we don't really have control over its color, style, etc., this this technology would exist in the (0, 1, 0) octant. When we rent clothes, or a bicycle, canoe, or car, for example, we don't own it, but we have control of it. As cyborg extensions of our mind and body, rentals falls into octant (0, 0, 1).
As we build the XV, as a persistent shared physical and virtual and social reality, we must be aware of the importance of maintaining a congruence of space, and as such, we call for free open source, and a code of ethics on human augmentation (see Pages 9-13 in the VRTO 2016 Proceedings) [Mann, Leonard, Brin, Ingle, Nickerson, Braverman, Ali, Moffat, Serrano, Fisher, Matthews, Janzen, Yang, Nerkar, Sanchez, Harris, Damiani, Button 2016].