logo资料库

Understanding Virtual Reality: Interface, Application, and Design, 2nd Edition.pdf

第1页 / 共916页
第2页 / 共916页
第3页 / 共916页
第4页 / 共916页
第5页 / 共916页
第6页 / 共916页
第7页 / 共916页
第8页 / 共916页
资料共916页,剩余部分请下载后查看
Cover
Understanding Virtual Reality: INTERFACE, APPLICATION, AND DESIGN
Copyright
Dedication
About the Authors
Foreword
Preface
What This Book Is
What This Book Is Not!
Who This Book Is For
How to Use This Book
How to Help Us Improve This Book
Quirks of Authorship
Threads
Acknowledgments From the First Edition
Acknowledgments for the Second Edition
Part I: What is Virtual Reality?
Introduction
1 - Introduction to Virtual Reality
Defining Virtual Reality
Five Key Elements of the Virtual Reality Experience
Key Elements 1 and 2: The Participants and the Creators
Key Element 3: Virtual World
Key Element 4: Immersion
Physical and Mental Immersion
Key Element 5: Interactivity
Computationally Simulated Realities
Collaborative Environments (Interacting With Other People)
Combining the Elements
Virtual Reality Paradigms
Head Based
Stationary
Hand Based
Virtual Reality, Telepresence, Augmented Reality, and Cyberspace
Artificial Reality
Virtual
Virtual World and Virtual Environment
Cyberspace
Augmented Reality
Telepresence
A History of Virtual Reality: Where Did Virtual Reality Technology Come From
Chapter Summary
Terminology: Tip of the Iceberg
2 - VR: The Medium
Communicating Through a Medium
A Medium’s Content: A Virtual World
Suitability of Virtual Worlds for a Particular Medium
Communication: Conveyance of Ideas
Common Issues of Human Communication Media
The Interface to the Virtual World
Language
Why Discuss Language
The Idiom of Virtual Reality
Language Elements of Time and Space
Representation of Self
Authorship (Versus Creatorship)
What Is Special About Virtual Reality
The Study of the Medium of Virtual Reality
Narrative: Immotive Versus Interactive
Interactivity, Participation, and Multipresence
Narrative Flexibility: Directed and Undirected Narratives
Form and Genre
Experience Versus Information
The Role of Artists and Technologists
Virtual Reality: One of the Media of Attraction
Capturing Virtual Reality Experiences
Mixed Reality Videos and Trompe-l’oeil
Chapter Summary
Part II: Virtual Reality Systems
Introduction
3 - The Human in the Loop
Connecting the Human to the Simulation
Matching the Display to the Needs of the Human
Human–Computer Interface
Affordances
Affordances in Virtual Reality
False Affordances in Virtual Reality
Reinforcing Perceived Affordances—Feedback
The Human Perceptual System
Sensation: The Physiological Side of Perception
Hierarchical Processing and Selectivity
Plasticity and Adaptability
The Science: Measuring Perception/Sensation
Perception of Change
Cross-Modal Perception
Visual Perception
Human Physiology of Vision
Light Receptors
The Retina
Luminosity (Dynamic Range)
Temporal Sensitivity
High-Level Processing
Visual Illusions
Visual Depth Cues—Distances
Monoscopic Image Depth Cues
Stereoscopic Image Depth Cue (Stereopsis)
Motion Depth Cues
Physiological Depth Cues
Distance Misperception (Compression)
Visual Depth Cues Summary
Aural Perception
Human Physiology for Audition
The Outer Ear (Pinna)
The Middle Ear
The Inner Ear
The Cochlear Nerve
The Limits of Human Hearing
Aural Illusions
Aural Localization Cues
Aural Localization Summary
Haptic Perception
Human Physiology for Somatosensation (“Haption”)
Haptic Sensor Types
Haptic Spatial and Temporal Resolution
Haptic Illusions
Haptic Localization Perception
Vestibular Perception
Human Physiology for Vestibulation
Vestibular Illusions
Vestibular Localization Perception
Olfactory Perception
Gustation Perception
Cross-Sensory Effects and Virtual Reality
Sensorial Precedence
Benefits of Cross-Modal Effects
Negatives of Cross-Modal Effects
Making Virtual Reality Design Choices Based on Cross-Modal Effects
Presence and Embodiment: Self-Perception Within the Virtual World
The Concept of Presence
What Is Gained by Studying Presence
Evolving Ideas Regarding Presence
Deconstructing Presence: The Elements of Presence
Presence in Other Media
The Determinants and Responses of Presence
Perceptual and Visceral Elements
Behavioral Responses
Measuring Presence
Embodiment
Inducing the Body Ownership Illusion
The Influence on Participants Through Agency and Body Ownership
Increasing the “Reality” of the Virtual World
Transference of Object Permanence
Breaking Presence: Things That Make the World Seem Less Real
Chapter Summary
4 - Input: Interfacing the Participant(s) With the Virtual World
Input Technologies
Active Versus Passive Inputs
Continuous Versus Discrete Inputs
Bandwidth: Amount/Type of Information
Absolute Versus Relative Inputs (Frame of Reference)
Frame of Reference
Physical Versus Virtual Inputs
Input Classifications
Position-Tracking Technologies
Mechanical Tracking
Ultrasonic Tracking
Electromagnetic Tracking
Optical Tracking
Videometric (Optical) Tracking
Beam-Scan Tracking (Lighthouse Tracking)
Inertial Tracking and Other Microelectromechanical System Technologies
Tracking Through Range-Finding Technologies
Muscular/Neural Tracking
Improving Position Tracking
Using Inputs Within a Virtual Reality System
Position Tracking the Body
The Head
The Hand and Fingers
The Eyes
The Torso
The Feet
Biological and Medical Sensor Technologies
Physical Input Devices
Physical Controls
Props
Platforms
Body Posture and Gesture Recognition
Speech Recognition (Audio Input)
Chapter Summary
5 - Output: Interfacing the Virtual World With the Participant(s)
Visual Displays
Properties of Visual Displays
Visual Presentation Properties of Visual Displays
Aural Displays
Properties of Aural Displays
Aural Presentation Properties
Logistic Qualities
Aural Display Paradigms
Stationary Aural Displays—Speakers
Head-Based Aural Displays—Headphones
Hand-Based Aural Displays
Combining Aural Display Systems
Summary of Aural Display Paradigms
Haptic Displays
Properties of Haptic Displays
Haptic Presentation Properties
Logistic Properties
Haptic Display Paradigms
Tactile Haptic Displays
Components of Tactile Displays
Features of Tactile Displays
Interface Issues of Tactile Displays
Summary of Tactile Displays
End-Effector Displays
Components of End-Effector Displays
Features of End-Effector Displays
Interface Issues of End-Effector Displays
Summary of End-Effector Displays
Robotically Operated Shape Displays
Components of Robotically Operated Shape Displays
Features of Robotically Operated Shape Displays
Interface Issues of Robotically Operated Shape Displays
Summary of Robotically Operated Shape Displays
Passive Haptic Displays
Mixed Displays
3D Hardcopy
Summary of Haptic Displays
Vestibular and Other Sensory Displays
Vestibular Display
Motion Bases (Platforms)
Moving Platforms
Other Vestibular Options
Olfaction, Gustation, and Other Senses
Chapter Summary
6 - Presenting the Virtual World
Representation of the Virtual World
Verisimilitude
The Realism Axis
Human Comprehension
Generalization
Semiotics
The Representation of Ideas
Form: The Gestalt Approach to Perception
Choosing a Mapping
Quantitative and Qualitative Representations
Representation Issues Related to VR
Sensory Substitution
Visual Representation in VR
How Visuals Are Used in a VR Experience
Incorporation of the Real World
Aural Representation in VR
Features of Sound
How Sound Is Used in a VR Experience
Verisimilar Sounds
Sampled Sounds. The easiest way to produce verisimilar sounds is to record samples of sounds from the real world. Analogous to t...
Simulated Sounds. Typically, it is necessary to have sounds that maintain realism even when the parameters of the environment ch...
Data Sonification
Ambient Sounds
Markers
Index Sounds
Vocal Messages
Haptic Representation in VR
Features of Haptics
How Haptic Information Is Used in a VR Experience
Force Representations (Kinesthetic Cues)
Tactile Representations (Cutaneous Cues)
Passive Haptic Representations
Representation of Other Senses
Representing Vestibulation
Representing Olfaction and Gustation
Representation Summary
Rendering Systems
Rendering System Hardware (Computation)
Where Is the Scene Rendered
How the Scene Is Delivered to the Display
Visual Rendering Systems
Visual Rendering Methods
Object-Based Rendering Versus Pixel-Based Rendering
Rendering of Different Geometrical Representations
Nongeometric Rendering Systems
Rendering Complex Visual Scenes
The Rasterization Rendering Pipeline (Object-Based)
Shading
Reducing Polygon Count
Advanced Texture Mapping Techniques
Regionally Targeted Rendering
Rendering to a 360-Degree Spherical View
Visual Rendering Latency
Multiplexed Rendering (Decomposition)
Time Warping
Stereoscopic Camera
Temporary Scene Reduction
Visual Rendering Process
Graphics Engines (aka GPUs)
Visual Asset Encodings (Internal Computer Representation)
The Role of Visual Rendering Software
Coordinate System Transformations
Sonic Rendering Systems
Sonic Rendering Methods
Sound Generation
Sound Propagation Algorithms
Rendering Complex Sounds
Frequency Modulation (Spectral)
Additive and Subtractive Techniques (Spectral)
Granular Synthesis
Chorusing
Modal Analysis
Propagation and Environmental Effects
Adaptive Rectangular Decomposition
Localization and Spatialization
Combining Effects
Sonic Rendering Process
Special-Purpose Hardware for Sound
Sonic Asset Encodings (Internal Computer Representation)
VR Sonic Rendering Software
Linking the Virtual to the Real
Haptic Rendering Systems
Haptic Rendering Methods
Vibrotactile (Cutaneous)
Skin Pressure (Cutaneous)
Surface Texture (Cutaneous)
Thermal Rendering (Cutaneous)
Environmental (4D) Effects (Cutaneous)
Force Rendering (Kinesthetic)
Simplified Shape Rendering Models With Forces
Shape Rendering With Robotically Operated Shape Display (Kinesthetic)
Inertial and Resistant Effects (Kinesthetic)
Error Correction (Kinesthetic)
Physical Object Rendering (3D Hardcopy) (Cutaneous and Kinesthetic)
Rendering Complex Haptic Scenes With Force Displays
Constraint of Movement
Trading Force for Torque
5-DOF From Two Points of Contact (Pinching)
Multiple Points of Contact (Grasping)
Haptic Rendering Process
Haptic Rendering Hardware
Haptic Asset Encodings (Internal Computer Representation)
VR Haptic Rendering Software
Linking the Virtual to the Real
Rendering of Other Senses
Rendering Vestibulation
Rendering Olfaction and Gustation
Chapter Summary
7 - Interacting With the Virtual World
Interaction Design Basics
User Interface Metaphors
Key Interactions: Manipulation, Navigation, and Communication
Manipulating a Virtual World
Manipulation Methods
Direct User Control
Physical Control
Virtual Controls
Agent Controls
Properties of Manipulation
Activation Mechanism
Feedback
Ratcheting
Constraints
Distance
Pointer Beam Scope
Hysteresis
Frame of Reference
Bimanual Interface
Control Location
Control Visibility
Movement Formula (Control Order and Gain)
Selection
Direction Selection
Pointer-Directed Selection
Gaze-Directed Selection
Reticle-Directed Selection
Torso-Directed Selection
Valuator-Directed Selection
Coordinate-Directed Selection
Landmark-Directed Selection
Item Selection
Contact-Select
Point to Select
3D-Cursor-Select
Aperture-Select
Name to Select
Menu-Select
Select in Miniworld
Alphanumeric Value Selection
Manipulation Operations
Positioning and Sizing Objects
Exerting Force on a Virtual Object
Modifying Object Attributes
Modifying Global Attributes
Altering the State of Virtual Controls
Controlling Travel
Manipulation Summary
Navigating in a Virtual World
Wayfinding
Creating a Mental Model
Wayfinding Aids
Path Following
Maps
Landmarks
Memorable Placenames
Bread Crumbs (Leaving a Trail)
Compass
Instrument Guidance
Exocentric View
Coordinate Display and Grid Structure
Constrained Travel
Travel
Properties of Travel Interfaces
Manipulation Method
Constraints
Frame of Reference
Movement Formula (Control Order and Gain)
Classes of Travel Methods
Physical Locomotion
Ride Along
Towrope
Fly-Through (and Walkthrough)
Pilot-Through
Move-the-World
Scale-the-World
Put-Me-There
Orbital Viewing
Travel Through Time
Navigation Summary
Interacting With Others
Shared Experience
What Can Be Shared
Ways of Sharing
Why Should I Share
Summary of Shared Experience
Collaborative Interaction
Communication Among People
Synchronous and Asynchronous Communication
Annotation
Floor Control
World Congruity
Interacting With the VR System (Metacommands)
Chapter Summary
Part III: Applied Virtual Reality
Introduction
8 - Bringing the Virtual World to Life
Immersion
Physical/Sensory Immersion
Mental Immersion
The Role of Realism in Immersion
How Real to Make It
Components of Immersion
Levels of Immersion
Fooling the User
Redirected Walking
Redirected Touching
Change Blindness Redirection
Providing the Context
Point of View
First-Person Point of View
Second-Person Point of View
Third-Person Point of View
Inside-Out Versus Outside-In
Venue
How the Venue Shapes the VR Experience
How the Venue Shapes the Participatory Experience
The Virtual World
Substance of the Virtual World
World Geography
Artifacts
Agents
User Interface Elements
Object Modeling and World Layout
Object Behavior
Static
Rigged
Dynamic
Trigger
Object Forms (Models and Mathematics)
World Layout
The Real World as Part of the Virtual World
Motivations for Live World Capture
Motivations for Offline World Capture
World Capture Technologies
Rules of the Virtual World: Physics
Types of Virtual World Physics
The Static World
Cartoon Physics
Newtonian Physics
Aristotelian Physics
Choreographed Physics
Other World Physics
Extent of the World Physics
Simulation/Mathematical Model
Object Cointeraction
World Persistence
Implementing Persistent Virtual Worlds
Inconsistencies Between the Physics of the Real and Virtual Worlds
Software to Manifest the VR Experience
VR Software Integration
Game Engines
Using Game Engines in VR Development
Web-Delivered Virtual Reality
The Experience Creation Process
Building Your First VR Application
Chapter Summary
9 - Experience Conception and Design: Applying VR to a Problem
Can VR Meet Your Goals
Is VR the Appropriate Medium
What Makes an Application a Good Candidate for VR
Conceiving a New VR Application
Adapting From Other Media
Drawing Inspiration From or Adapting From an Existing VR Experience
Creating a New VR Experience From Scratch
A Cornucopia of Application Areas
Exemplary VR Experiences
Designing a VR Experience
Design Deliberately
Prototype
Design With the System in Mind
Design With the Venue in Mind
Design With the Audience in Mind
Design to Engage the Audience
Consider Social Interactions
Consider Design Tradeoffs
Design the User Objective
Design the End of the Experience
User Testing
Document, Deploy, and Evaluate the Experience
The Past and Future of VR Design
Chapter Summary
10 - Virtual Reality: Past, Present, Future
The State of VR
Technology Trigger
Peak of Inflated Expectations
Trough of Disillusionment
Slope of Enlightenment
Plateau of Productivity
The Maturation of VR
VR in the Laboratory
VR in the Classroom
Transition: Media of Attraction to Common Use
Trends
Increased Use of Augmented Reality
VR in the Home
VR-Ready Machines
Less Encumbrance
Higher Experience Fidelity
Software Availability
New Drivers/Disruptive Technologies
Technology Futures and Past Predictions
Display Technologies
Visual Displays—Looking Back
Visual Displays—Looking Forward
Audio Displays—Looking Back
Audio Displays—Looking Forward
Haptic Displays
Haptic Displays—Looking Back
Haptic Displays—Looking Forward
Olfactory and Gustatory Displays
Olfactory Displays—Looking Back
Olfactory and Gustative Displays—Looking Forward
Input Technologies
Unencumbered Input—Looking Back
Unencumbered Input—Looking Forward
New Input Devices—Looking Back
New Input Devices—Looking Forward
Direct Neural Connections—Looking Back
Direct Neural Connections—Looking Forward
Software
World Creation Software—Looking Back
World Creation Software—Looking Forward
Hardware Interface Software—Looking Back
Hardware Interface Software—Looking Forward
Rendering Software—Looking Back
Rendering Software—Looking Forward
Application Development Software—Looking Back
Application Development Software—Looking Forward
Application Futures
Application Software—Looking Back
Application Software—Looking Forward
Chapter Summary: The Future is Happening Now
References
Subject Index
Index of Media Experiences
Back Cover
The Morgan Kaufmann Series in Computer Graphics Understanding Virtual Reality I N T E R F A C E , A P P L I C A T I O N , A N D D E S I G N Second Edition William R. Sherman Indiana University Alan B. Craig University of Illinois
Morgan Kaufmann is an imprint of Elsevier 50 Hampshire Street, 5th Floor, Cambridge, MA 02139, United States Copyright © 2019 Elsevier Inc. All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage and retrieval system, without permission in writing from the publisher. Details on how to seek permission, further information about the Publisher’s permissions policies and our arrangements with organizations such as the Copyright Clearance Center and the Copyright Licensing Agency, can be found at our website: www.elsevier.com/permissions. This book and the individual contributions contained in it are protected under copyright by the Publisher (other than as may be noted herein). Notices Knowledge and best practice in this field are constantly changing. As new research and experience broaden our understanding, changes in research methods, professional practices, or medical treatment may become necessary. Practitioners and researchers must always rely on their own experience and knowledge in evaluating and using any information, methods, compounds, or experiments described herein. In using such information or methods they should be mindful of their own safety and the safety of others, including parties for whom they have a professional responsibility. To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume any liability for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions, or ideas contained in the material herein. Library of Congress Cataloging-in-Publication Data A catalog record for this book is available from the Library of Congress British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library ISBN: 978-0-12-818399-1 For information on all Morgan Kaufmann publications visit our website at https://www.elsevier.com//books-and-journals Publisher: Katey Birtcher Acquisition Editor: Steve Merken Editorial Project Manager: Ana Claudia A. Garcia Production Project Manager: Punithavathy Govindaradjane Designer: Christian J. Bilbow Typeset by TNQ Technologies
Dedicated to Sheryl and all those who followed with love Dedicated to my mother Mary Craig, the grand supporter of all my endeavors
xv About the Authors William R. Sherman Sr. Technical Advisor for the Indiana University Advanced Visualization Lab Division of Research Technologies William R. Sherman is a member of the Indiana University Advanced Visualization Lab, where he leads efforts in scientific visualization and virtual reality. He also teaches undergraduate and graduate courses on VR and visualization, which he has done for two decades, including at the University of Nevada, Reno (UVR), and at the University of Illinois at Urbana–Champaign (UIUC). Previously he founded the Center for Advanced Visualization, Computation and Modeling (CAVCaM) at the Desert Research Institute (DRI), where he led the VR and visualization efforts, including overseeing the installation of a FLEX CAVE- style VR system as well as a six-sided CAVE system. Prior to DRI, he led the virtual reality effort at the National Center for Supercomputing Applications (NCSA) at UIUC, working with the Electronic Visualization Lab to install and operate the sec- ond CAVE VR system. He has authored several book chapters and papers on the topics of sci- entific visualization and virtual reality, and has organized and led “bootcamps” on immersive visualization in collaboration with the Idaho National Lab and Kitware Inc. Sherman is the architect of the FreeVR VR integration library. He has attended every single IEEE Virtual Reality Conference since 1995, and was chair of the 2008 Conference.
xvi | About the Autho RS Alan b. Craig, Ph.D. Independent Consultant, Inventor, Musician, Author, Scientist, Teacher Dr. Alan B. Craig is an independent consultant in virtual reality, augmented real- ity, visualization, and high-performance computing. Prior to this role, he contrib- uted much to these fields during his 30-year career at the University of Illinois at Urbana–Champaign (UIUC) as a Research Scientist at the National Center for Supercomputing Applications (NCSA) and as Senior Associate Director for Human– Computer Interaction at the Institute for Computing in Humanities, Arts, and Social Science (I-CHASS). Among his other consulting roles, he is currently engaged with the Extreme Science and Engineering Discovery Environment (XSEDE). Dr. Craig has been called upon to speak as an expert in VR and AR at count- less worldwide events and continues to speak at various venues. He has taught classes related to VR and AR online as well as on-site at universities, companies, and high school campuses. Dr. Craig has worked with government and industry entities regarding VR and AR applications. He has been interviewed by numerous publications, television, and news outlets. In addition to Understanding Virtual Reality (with William R. Sherman) he also authored Developing Virtual Reality Applications (with William R. Sherman and Jeffrey D. Will) and Understanding Augmented Reality. Additionally, he has written multiple book chapters and articles. He has developed many virtual reality and augmented reality applications in content areas ranging from archeology to zoology. He also teaches and advises on related topics. His primary focus has been on the use of virtual reality and augmented reality in educational applications and his work centers on the continuum between the physical and the digital. He holds three patents.
xvii Foreword Our lives are spent in a mixed reality that integrates the real and the virtual—a reality in which much of our day is filled with synthesized experiences, to the point that “real” reality may be experienced less than “virtual” reality. We are often more engaged with our phones than with our physical environment. I believe that mixed reality has the potential to reinvent our economy, tread lightly on the environment, and enable more satisfying creation, collaboration, and communication. With people texting each other across the dinner table and children confused when “pinch-to-zoom” does not work on the pages of a paper magazine, that may seem a foolhardy belief. But only because we have not opened ourselves up to the potential of a medium that frees us from the constraints of the physical world yet simultaneously keeps us perceptually grounded in our bodies. We are in the middle of a transition from the industrial to the informa- tional. Our mind-numbing life of email and web forms is a vestige of bygone think- ing. We now must explore the perceptual, cognitive, and emotional power of the experiences that are possible in this new world, and study how the heart, mind, and body can joyfully inhabit the virtual realm. In 1992 (the same year I met Alan and Bill), I wrote “the designer must abandon the physical environment of everyday perception and the characteristics of other media in order to completely embrace the nature and character of vir- tual environments.” I find this to be ever more important as the field progresses: Surrealism is essential in virtual environment design because of its counterintui- tive ability to enhance the feeling of presence while transcending reality. The psychologist J.J. Gibson posited that humans make sense of an envi- ronment by first modeling what is there, and then what it affords and enables us to do. That is how I think of this book. It goes beyond typical cursory descriptions of virtual reality—those that simply describe the surface of the medium—to consider structure and context. It dives below the surface to look at the topic from multiple
xviii | FOREWORD angles: historical, perceptual, technical, artistic. In doing so, it gives the reader a feeling not just for what the medium is, but what can be done with it. That is a wise approach, because we are not at a point yet where anyone can truly describe what virtual reality is. It is far too powerful to know the course it will take—and that is what makes it so exciting to be a creator in the medium. That said, we are beginning to discover some of its boundaries, and I appreciate that Bill and Alan emphasize that virtual reality experiences are inherently co-created between developer and participant. It enables the finding of story more than the telling. Similarly, the book does not attempt to serve as a textbook with neat answers in the back. Rather, it enables readers by providing a framework upon which to base their own explorations and experiments. I am happy to introduce this guidebook on our travels together to discover the nature of a new reality. Mark Bolas Bellevue, Washington
分享到:
收藏