KEMBAR78
COMP 4010 Lecture9 AR Displays | PDF
LECTURE 9: AR
TECHNOLOGY: DISPLAYS
AND TRACKING
COMP 4010 – Virtual Reality
Semester 5 – 2016
Bruce Thomas, Mark Billinghurst
University of South Australia
October 11th 2016
Augmented Reality Definition
•  Defining Characteristics [Azuma 97]
• Combines Real andVirtual Images
• Both can be seen at the same time
• Interactive in real-time
• The virtual content can be interacted with
• Registered in 3D
• Virtual objects appear fixed in space
Azuma, R. T. (1997). A survey of augmented reality. Presence, 6(4), 355-385.
Augmented RealityTechnology
! Combining Real and Virtual Images
•  Display technologies
! Interactive in Real-Time
•  Input and interactive technologies
! Registered in 3D
•  Viewpoint tracking technologies
Display
Processing
Input Tracking
AR DISPLAYS
Display Technologies
! Types (Bimber/Raskar 2003)
! Head attached
•  Head mounted display/projector
! Body attached
•  Handheld display/projector
! Spatial
•  Spatially aligned projector/monitor
DisplayTaxonomy
HEAD MOUNTED DISPLAYS
Head Mounted Displays (HMD)
•  Display and Optics mounted on Head
•  May or may not fully occlude real world
•  Provide full-color images
•  Considerations
•  Cumbersome to wear
•  Brightness
•  Low power consumption
•  Resolution limited
•  Cost is high?
Types of Head Mounted Displays
Occluded
See-thru
Multiplexed
ImmersiveVRArchitecture
Head!
Tracker
Host!
Processor
Data Base!
Model
Rendering!
Engine
Frame!
Buffer
head position/orientation
to network
Display!
Driver
Non see-
thru!
Image
source &
optics
virtual
object
Virtual
World
See-thruARArchitecture
Head!
Tracker
Host!
Processor
Data Base!
Model
Rendering!
Engine
Frame!
Buffer
head position/orientation
to network
Display!
Driver
see-thru!
combiner
Virtual
Image
superimpos
ed!
over real
world object
real
world
Image
source
Optical see-through head-mounted display
Virtual images
from monitors
Real
World
Optical
Combiners
Optical See-Through HMD
Epson Moverio BT-200
▪ Stereo see-through display ($700)
▪  960 x 540 pixels, 23 degree FOV, 60Hz, 88g
▪  Android Powered, separate controller
▪  VGA camera, GPS, gyro, accelerometer
ViewThrough Optical See-Through HMD
Strengths of optical see-throughAR
• Simpler (cheaper)
• Direct view of real world
• Full resolution, no time delay (for real world)
• Safety
• Lower distortion
• No eye displacement
• but some video see-through displays avoid this
VideoARArchitecture
Head!
Tracker
Host!
Processor
Graphics!
renderer
Digital!
Mixer
Frame!
Buffer
head position/orientation
to network
Display!
Driver
Non see-
thru!
Image
source &
optics
Head-
mounted
camera
aligned to
display optics
Video!
Processor
Video image
of real world
Virtual image
inset into
video of real
world
Video see-through HMD
Video
cameras
Monitors
Graphics
Combiner
Video
Video See-Through HMD
Vuzix Wrap 1200DXAR
▪ Stereo video see-through display ($1500)
■ Twin 852 x 480 LCD displays, 35 deg. FOV
■ StereoVGA cameras
■ 3 DOF head tracking
ViewThrough aVideo See-Through HMD
Strengths ofVideo See-ThroughAR
• True occlusion
• Kiyokawa optical display that supports occlusion
• Digitized image of real world
• Flexibility in composition
• Matchable time delays
• More registration, calibration strategies
• Wide FOV is easier to support
Optical vs.VideoAR Summary
• Both have proponents
• Video is more popular today?
• Likely because lack of available optical products
• Depends on application?
• Manufacturing: optical is cheaper
• Medical: video for calibration strategies
Eye multiplexedARArchitecture
Head!
Tracker
Host!
Processor
Data Base!
Model
Rendering!
Engine
Frame!
Buffer
head position/orientation
to network
Display!
Driver
Virtual
Image inset
into!
real world
scene
real
world
Opaque!
Image
source
Virtual Image‘inset’ into real
Google Glass
ViewThrough Google Glass
Vuzix M-100
▪ Monocular multiplexed display ($1000)
■ 852 x 480 LCD display, 15 deg. FOV
■ 5 MP camera, HD video
■ GPS, gyro, accelerometer
DisplayTechnology
•  Curved Mirror
•  off-axis projection
•  curved mirrors in front of eye
•  high distortion, small eye-box
•  Waveguide
•  use internal reflection
•  unobstructed view of world
•  large eye-box
See-through thin displays
•  Waveguide techniques for thin see-through displays
•  Wider FOV, enable AR applications
•  Social acceptability
Opinvent Ora
Lumus DK40
SPATIAL AUGMENTED REALITY
SpatialAugmented Reality
• Project onto irregular surfaces
•  Geometric Registration
•  Projector blending, High dynamic range
• Book: Bimber, Rasker “Spatial Augmented Reality”
Projector-basedAR
Examples:
Raskar, MIT Media Lab
Inami, Tachi Lab, U. Tokyo
Projector
Real objects
with retroreflective
covering
User (possibly
head-tracked)
Example of projector-basedAR
Ramesh Raskar, UNC, MERL
Example of projector-basedAR
Ramesh Raskar, UNC Chapel Hill
Head Mounted Projector
•  NVIS P-50 HMPD
•  1280x1024/eye
•  Stereoscopic
•  50 degree FOV
•  www.nvis.com
HMD vs.HMPD
Head Mounted Display Head Mounted Projected Display
CastAR - http://technicalillusions.com/
• Stereo head worn projectors
• Interactive wand
• Rollable retro-reflective sheet
• Designed for shared interaction
Demo: CastAR
https://www.youtube.com/watch?v=AOI5UW9khoQ#t=47
Pico Projectors
• Microvision - www.mvis.com
• 3M, Samsung, Philips, etc
Demo: Pico Projector
https://www.youtube.com/watch?v=vtdH3CLKEuY
MIT Sixth Sense
• Body worn camera and projector
• http://www.pranavmistry.com/projects/sixthsense/
Demo: Sixth Sense
https://www.youtube.com/watch?v=Q4Z9sOtiWUY
OTHER AR DISPLAYS
Video MonitorAR
Video
cameras Monitor
Graphics Combiner
Video
Stereo
glasses
Examples
Virtual Showcase
• Mirrors on a projection table
•  Head tracked stereo
•  Up to 4 users
•  Merges graphic and real objects
•  Exhibit/museum applications
• Fraunhofer Institute (2001)
•  Bimber, Frohlich
Demo: Virtual Showcase
https://www.youtube.com/watch?v=iXl4FolFUzc
Augmented Paleontology
Bimber et. al. IEEE Computer Sept. 2002
Alternate Displays
LCD Panel Laptop PDA
Handheld Displays
• Mobile Phones
• Camera
• Display
• Input
OtherTypes ofAR Display
• Audio
• spatial sound
• ambient audio
• Tactile
• physical sensation
• Haptic
• virtual touch
Haptic Input
•  AR Haptic Workbench
•  CSIRO 2003 – Adcock et. al.
Phantom
• Sensable Technologies (www.sensable.com)
• 6 DOF Force Feedback Device
AR Haptic Interface
•  Phantom,ARToolKit, Magellan
www.empathiccomputing.org
@marknb00
mark.billinghurst@unisa.edu.au

COMP 4010 Lecture9 AR Displays