Modeling the Virtual World

1
A VR system architecture

Modeling the
Virtual World

Geometry
Physics
Haptics

VR Toolkits

[Burdea]

2
VR object modeling

◼ Geometric modeling
◼ Kinematics modeling
◼ Physical modeling
◼ Model behavior
◼ Model management

3
VR object modeling

[Burdea]

4
Geometric modeling for VR

◼ Surface shape
❑ Triangle meshes
❑ Different LODs
❑ Spline curves and surfaces

◼ Visual appearance
❑ Lighting and shading
❑ Texture mapping
❑ Multi-texturing

5
3D models for VR and AR

◼ Virtual / Augmented reality

6
Shape

◼ Define from scratch using VRML / X3D, OpenGL,

VTK, …
❑ Tedious; requires skill

◼ Obtain from CAD files or model databases

❑ Convert to compatible VR formats
❑ Use of existing models in manufacturing applications

◼ Create using a 3D digitizer or a 3D scanner

❑ 3D digitizer : stylus
❑ 3D scanner : tracker, cameras and laser

7
Shape

[Burdea]

8
Visual appearance
◼ Scene illumination – Local methods
❑ Handle one polygon / triangle at a time
❑ Flat / Gouraud / Phong shading

◼ Scene illumination – Global methods

❑ How is the appearance of a given object influenced by
other objects ?
◼ Inter-reflections + Shadows
❑ More realistic results
❑ Mode demanding !

◼ Texture mapping
9
Flat shading vs Phong shading

[Wikipedia]

10
Realistic lighting

Vertex lighting of low polygon Vertex lighting of high polygon

count surface count surface
- lights are diffuse - lights have realistic appearance
- tessellated - high computation load

(from NVIDIA technical brief) [Burdea]

11
Ray-Tracing example

http://radsite.lbl.gov/radiance/book/img/plate10.jpg

12
Another Ray-Tracing example

http://www.tjhsst.edu/~dhyatt/superap/samplex.jpg

13
Ray-Tracing – Diamond ring

[http://www.okino.com]

14
Ray-Tracing – Chemist’s flask

[http://www.okino.com]

15
Radiosity

Without radiosity With radiosity

[Burdea]

16
Texture mapping
◼ Map texture space coordinates to triangle
vertices

◼ Then map to pixel coordinates

◼ Advantages ?
❑ Increased scene realism
❑ Better 3D spatial cues
❑ Reduced number of triangles
◼ Increased frame rate

17
Texture mapping

Smooth shading Environment mapping Bump mapping

[Angel]

18
Textures – Simulating Ray-Tracing
[http://www.okino.com]

◼ Increased realism !!
❑ 11 light sources + 25 texture maps

19
Textures
◼ Define from scratch using interactive paint
software
❑ Create bitmaps

◼ Custom textures from scanned or digital

photos
❑ Convert to appropriate formats

◼ Use texture libraries

❑ Construction materials, cars, people, …

20
Texture mapping

[Burdea]

21
Texture mapping

◼ It glues an image to a polygon / surface

◼ Texture size has restrictions

❑ Texels

◼ Magnification vs minification
❑ Bilinear interpolation to assign colors to pixels

22
Tree represented as a texture

Higher resolution model Texture

45,992 polygons 1 polygon, 1246 x 1280 pixels
[Burdea]
(www.imagecels.com)

23
 Textures
Normal
texture

Background
texture
Reflectivity
texture

Transparency
texture

[Burdea]
Bump maps

24
Multi-Texturing
◼ Several texels can be overlaid on one pixel
❑ Texture blending cascade

◼ Application : Bump-Mapping
❑ Simulate lighting effects caused by surface
irregularities
❑ Surface irregularities are coded as a texture
❑ No changes in model geometry
◼ No added computations !!

25
Bump-mapping

Normal Multi-texture Bump

[Burdea] texture texture
26
Light maps
◼ Realistic looking lighting can be done with 2D
light maps
❑ Realistic and low polygon count

Standard lighting map Light map texture overlaid

2-D texture on top of wall texture
[Burdea] (from NVIDIA technical brief)

27
VR kinematics modeling

◼ Modeling and viewing transformations

◼ Object position
◼ Object hierarchies

28
Modeling and viewing transformations

◼ Homogeneous coordinates
◼ Matrix concatenation

◼ Create various model instances – How ?

❑ Position + Orientation + Size

◼ Change the position and aim of the virtual

camera – How ?
❑ Fly-by, …

29
Model instantiation

◼ Static objects
❑ Modeling matrix

◼ Moving objects
❑ Modeling matrix varies along the time

30
Tracking a virtual hand

[Burdea]

31
Tracking a virtual hand
◼ Transformation matrices are concatenated to
obtain the resulting motion

◼ If the object is grasped, its position does not

change versus the hand
◼ The movement of the grasped object

[Burdea]

32
VR kinematics modeling

[Burdea]

33
Object hierarchies
◼ Models are structured as a hierarchy of parts
❑ Dynamics
❑ Different behavior for different model parts

◼ Parent-children relationship
❑ Higher-level vs lower-level
❑ The motion of a parent part is replicated by its
children
❑ But not the other way around !!

34
Object hierarchies

[Burdea]

35
Virtual hand

◼ Concatenate transformation matrices to

obtain the motion of a fingertip
Given by the glove tracker

[Burdea]
Given by the glove sensors

36
Virtual hand

[Burdea]

37
VR physical modeling
◼ Add increased realism to the virtual world

◼ Integrate the physical characteristics of each

object
❑ Weight
❑ Inertia
❑ Surface roughness
❑ Compliance : hard / soft
❑ Deformation mode : elastic / plastic
❑ …
38
VR physical modeling

◼ Assign tasks to the haptics rendering pipeline

❑ What sequence ?
❑ Computational load ?

39
VR physical modeling

[Burdea]

40
The rendering pipelines

[Burdea]

41
The haptics rendering pipeline

[Burdea]

42
Collision detection
◼ How to check for collisions between objects ?
❑ It has to be done fast !!

◼ Exact collision detection

❑ Handle polygons / polyhedra
❑ Computational load !!
◼ Approximate collision detection – First step !
❑ Use bounding boxes !!
❑ Easy to check if objects do not collide or might
collide

43
Bounding boxes
◼ Bounding boxes for faster response times
❑ Variable size / orientation
❑ Fixed size / orientation – Faster, but less precise !
❑ Axis-aligned bounding boxes – AABB

[Burdea]

44
Testing for possible collisions
◼ Do bounding boxes
ovelap ?

◼ Yes – What to do ?

◼ No – OK !

◼ Test object pairs

◼ For every frame !?! [Burdea]

45
Undetected collision
[Burdea]

◼ If the ball travels fast enough, the collision is

not detected
❑ Increase rendering speed ?!?
❑ Take into account scene coeherence
46
Two-stage collision detection
[Burdea]

◼ Approximate detection – Bounding boxes

❑ Identify / remove BBs that do not intersect
◼ Followed by exact detection
❑ Slower – Various methods
47
The haptics rendering pipeline

[Burdea]

48
Surface deformation – Meshes

◼ Deforming a polygonal mesh

❑ Direct vertex manipulation

◼ Use the topological information

◼ Preserve mesh connectivity ?

◼ Deformation propagation laws

49
Surface cutting – Meshes
◼ Extreme case !!
❑ Contact force exceeds a given threshold
❑ Topological changes

◼ Co-located twin vertices separate and the cut

enlarges
❑ Spring / damper laws
❑ Process propagates along the cutting path

50
Surface cutting – Meshes

[Burdea]

51
Force computation
◼ User interacts with 3D object surfaces
❑ Deformable or not

◼ User should feel reaction forces

◼ Reaction forces have to be

❑ Computed by the haptics pipeline
❑ Sent to the haptics display

52
Force computation
◼ Take into account
❑ Object physical and kinematics features
❑ Type of surface contact
❑ Kind of surface deformation

◼ Single-point contact
❑ Simplest case !

◼ Multiple contacts with a grasped object

❑ Haptic gloves
53
Force computation
◼ Immobile rigid objects
❑ Walls

◼ Mobile rigid objects

◼ Mobile elastic objecs

❑ Rubber balls

◼ Plastically deformable objects

❑ Soda cans

54
Single-point contact
[Burdea]

55
Elastic deformation
[Burdea]

56
Non-uniform hardness
[Burdea]

57
Virtual push-button

[Burdea]

58
Plastic deformation

[Burdea]

59
Virtual Walls

[Burdea]

60
The haptics rendering pipeline

[Burdea]

61
Force smoothing
◼ Assumptions up to now:
❑ Single interaction point
❑ Frictionless contact

◼ The direction of the resistive force is the

direction of the surface normal at the point of
contact
❑ Unwanted discontinuities for polygonal meshes
approximating curved surfaces
❑ Due to abrupt changes in normal direction

62
Force smoothing
◼ Objects feel faceted / tesselated
❑ Even if the graphics image does look smooth !!

◼ IDEA
❑ Extend the graphics shading approach to haptics

◼ Force “shading” changes the direction of the

feedback force
❑ Polygonal surface models
❑ Simulate contact with smooth, curved surfaces

63
Force smoothing

[Burdea]

64
Haptic meshes
◼ Fingertip-object contact
❑ A single haptic interface point (HIP) does not
suffice !!
◼ More realistic modeling of the curvature of
the fingertip and object deformation
❑ Haptic meshes
[Burdea]

65
Haptic meshes

[Burdea]

66
Haptic meshes

[Burdea]

67
The haptics rendering pipeline

[Burdea]

68
Force mapping

[Burdea]

69
The haptics rendering pipeline

[Burdea]

70
Haptics texturing
◼ Enhance the realism of the physical model of
an object’s surface
❑ Haptics texturing
❑ Similar to graphics textures adding realism to an
object’s visual appearance

◼ Add additional surface information

❑ Cold / warm
❑ Smooth
❑ Slippery
❑ …
71
Haptics texturing

[Burdea]

72
Haptics texturing

◼ TIME Tech – October 09, 2013

❑ Disney Just Figured Out Virtual Textures, Letting

You ‘Feel’ Bumps and Ridges on a Touchscreen

❑ YouTube: Tactile Rendering of 3D Features on

Touch Surfaces

73
Behavior modeling
◼ Up to now we have been modeling
❑ Shape
❑ Visual appearance
❑ Kinematics
❑ Physical properties

◼ And assuming that each object is controlled

by the user

◼ How to model behavior that is independent of

a user’s actions ?
74
Behavior modeling

◼ Critical in large simulation environments

where users cannot control all interactions
taking place

◼ Interactive objects

◼ Agents

75
 Behavior modeling
◼ Thalmann et al. (2000) distinguish three
autonomy levels
❑ Guided / Programmed / Autonomous

[Burdea]

76
Object behavior
◼ Model object behavior by accessing external
sensors

◼ Such virtual objects are independent from the

user’s actions

◼ Example – Virtual office

❑ Clock – Access VR engine system time
❑ Thermometer – Updated through a temp. sensor
❑ …

77
Virtual human behavior

◼ Behavior model composed of

❑ Perception
❑ Emotions
❑ Behavior
❑ Actions

◼ Perception makes the agent aware of his /

her surroundings

78
Agent behavior

[Burdea]

79
Reflex behavior

[Burdea]

80
Emotional behavior

[Burdea]

81
Crowds behavior

◼ Emphasize group (rather than individual)

actions

◼ Guided crowd
❑ User needs to specify path points, for instance

◼ Autonomous crowd
❑ Group perceives environment
❑ Group decides how to reach goal

82
Model management
◼ Rendering complex models / scenes /
environments
❑ Maintain constant frame rates and simulation
quality
❑ Allow interactivity

◼ Heavy computational load !!!

◼ How to address ?

83
Addressing computational load
◼ LODs
❑ Close vs far objects
❑ Adaptive LOD

◼ Cell segmentation
❑ Architectural walk-throughs

◼ Lighting and bump mapping

84
Reference

◼ G. C. Burdea and P. Coiffet, Virtual Reality Technology,

2nd Ed., Wiley-IEEE Press, 2003

85