Prerequisite:	ECE 273, 371 or equivalent
Textbooks	Virtual Reality Technology	Grigore Burdea, Philippe Coiffet Wiley InterScience, 1994
	Virtual Reality in Engineering	Kevin Warwick, John Gray, David Roberts IEE Computing Series, London, United Kingdom

Topics

Introduction to virtual simulation	1 class
Virtual simulation tools	1 class
Computing architectures	1 class
Geometric modeling	2 classes
Kinematic modeling	2 classes
Physical modeling	1 class
Programming in virtual simulation	2 classes
Human factors in virtual simulation	1 class
Case study in engineering application	2 classes

Grading

Category	Comments		Dates	Points
Midterm Exam	1.5 hours			150
Final exam	2.0 hours		April 30 7:30-9:30pm	200
term project	demo/presentation report		April 16	200
Total points				550

ECE 577 Engineering in Virtual World

Credits 3.

Catalog Data:

An in-depth study of selected topics in design and development of virtual systems in industrial environment. Topics include cyberspaces, techniques for generating virtual world in enginering application, building blocks of virtual environments including hardware and software, and case study.

Prerequisites: ECE 273, 371 or equivalent

Textbook: ìVirtual Reality Technology,î Grigore Burdea, Philippe Coiffet, Wiley InterScience, 1994

ìVirtual Reality in Engineering,î Edited by Kevin Warwick, John Gray and David Roberts

IEE Computing Series, London, United Kingdom,

Coordinator: Assist. Prof. Yi Lu Murphey, Dept. of ECE

Goals:

Introduce fundamentals in the construction of a virtual world, address design methodologies and critical research and engineering issues, and potential engineering applications

Prerequisites by Topic:

1. Representations of signals and systems

2. Linear algebra

3. Knowledge of one high level programming language

Topics:

1. Introduction to the virtual world (2 class)

2. Mathematical foundations of cyberspace (4 classes)

3. Geometric modeling (12 classes)

4. Position tracking, path planning and motion control(7 classes)

5. 3D object representation and rendering (8 classes)

6. Case study: Engineering applications of virtual world (8 class)

Computer Usage: SGI workstations

Design projects:

Students will be given a term project assignment which involves both modeling and rendering virtual environment. Every student will be asked to produce a short three dimensional animation sequence on a topic chosen by the student, and the animation will be recorded on video tape.

Estimated ABET Category Content:

Engineering Design 1 credits 34%

Engineering Science: 2 credits or 66%

Prepared by: Assist. Prof. Yi Lu Murphey

Let us try to characterize what we saw in the video and the SGI demo.

ï What are the applications using VR technology?

video tape review.

What is Virtual Reality

ï A simulation in which computer graphics is used to create a realistic looking world,

ï the synthesized world is dynamic and provides real time interaction, i.e. it responds to user inputs such as gestures, verbal commands, etc.

ï VR make the simulated world more realistic by using all human sensorial channels.

visual scene,

sound device,

interface tools:

ï Three ìiîs

-- immersive,

-- interactive

-- imagination

[

ï Three distinct aspects of VR:

-- virtual past,

-- virtual analysis, and

-- virtual design.

]

ï VR incorporates the the 2D or 3D objects, acoustic, color and atmospheric impressions.

Virtual simulation in engineering

What is the difference between virtual engineering and VR

-- confine to the engineering field

-- concentrate on the simulation of certain engineering procedure using VR technology:

try to make the necessary simulation as real as we can, however we may trade reality with other functions if we have too. We are not limit to head mount devices, we can use a mouse to interact with the system.

ï Virtual engineering simulation is an engineering simulation uses VR technology

-- a model of reality whose representation is available in the computer and

-- can be experienced by user directly and interactively.

ï What makes VS different from traditional CAD design?

[ In the other types of design simulation, ]

-- Has anyone worked in CAD design?

* In advanced CAD application, the designer is able to study specific aspects of a design by having separate programs perform calculations on certain properties.

* the computer is mainly used as a means to externalize the design ideas and

* a designer does not have a picture of the environment in which the product is used:

he has to hold the representation of an entire design for a new vehicle or building or battle field with more or less his or her imagination.

[

ï A technician attempting to repair a machine may look at a photo in which overlaid graphics make apparent otherwise occluded components.

]

-- In a virtual design environment,

the computerized information,

* rather than being represented as a separate, two- or three-dimensional object for observation,

* surrounds the designer who finds himself directly inside the data space.

-- In a virtual simulation environment, the designer is entirely ìinsideî the design and influences it from within.

* Analysis thus becomes a more integral part of design.

-- Virtual simulation provides the means for the interactive exploration of a data model based on various types of simulation and thus allows for a multitude of combined experiences.

* example

In battle field simulation, each weapon system needs a visual model to represent it in the computer generated world and most weapon systems are already currently modeled using Computer Aided Design (CAD) systems. It is cost effective to automatically transfer CAD modeling data to a virtual simulator since many CAD modeling data exist.

ï Virtual engineering will be the focused topic for this new course.

-- VR provides the physical and technical means to continue a well developed direction in industrial design: futuristic scenarios.

-- therefore, we need to study all the technologies used in VR

-- concentrate on the simulation of certain engineering procedure using VR technology:

try to make the necessary simulation as real as we can, however we may trade reality with other functions if we have too. We are not limit to head mount devices, we can use a mouse to interact with the system.

3. Design procedures:

What are the events to be simulated

Scene design

sound design

others.

ï What are the key technical components involved?

-- build the visual world

ï Building virtual worlds is a time consuming task due to obstacles on different levels.

ï Different methods for building the virtual world:

-- computer graphics

-- from 3D software packages

-- CAD drawing (the same as above?)

-- video

[

ï The combination of multimedia and virtual worlds is undertaken for several reasons

-- multimedia sources can be used to enrich the appearance of virtual worlds,

* making virtual worlds more real

* more interesting

* more appealing

-- in some cases the use of multimedia techniques simplifies the development of virtual worlds since parts of relatively complex worlds can be replaced by video.

-- both fields have much in common - especially the eminet role of computer graphics

ï A key concern is that components in the virtual world should be capable of operating under a variety of synchronization requirements.

]

-- physics, force, kinematics

-- mechanic engineering:

-- computer software programming

-- human computer interaction

determine the frame rate for the visual scene, and other factors

-- Synchronization in Virtual worlds

There are different synchronization issues pertaining to virtual world applications.

(1) Synchronization in component networks

-- synchronized display:

* display of 3D graphics on two separate displays

-- allow more than one remotely located users to ìwalk aroundî in the virtual world together, make comments and exchange information about the artifacts.

-- Data stream synchronization

* keep time shifts in the presentation of the pointing and audio data streams within some interval in order to preserve the consitency of the information conveyed by these data streams.

-- Synchronizing user interaction -- coupled viewing

multiple users at different locations interact with the world.

-- synbchronization of user motion with rendered graphics and sound output

so give user an impression of reality.

(2) Synchronization issues in virtual worlds

-- Time-based media synchronization:

Intrastream synchronization

-- User-level synchronization

single or multi-user application

-- Asyncrhonous operation of system components:

-- distributed processing:

ï Component Synchronization

The wide range of options for designing virtual worlds leads to many issues such as coordinating input and output devices, synchronization of system activities, combining different data gypes, etc.

ï Tracking and Interaction Devices:

-- Tracking systems monitor the location and orientation of various body parts

* eg. hands or the head

-- Five types of tracking systems:

magnetic, acoustic, inertial, mechanical, and optical.

ï Different Media Devices

Computer-controlled audio/video equipment adds another dimension to a virtual world system.

-- Media input devices include

* video cameras,

* videodisc players, MIDI(Musical Instrument Digital Interface)-devices

These devices are often used for interaction

* microphones, CD-players

-- Media output device:

* recorders

*display devices

-- Different types of data

the essential type is 3D model data that defines the virtual universe, the scene and its objects.

* objects and their positions may be changed by a suer,

* some objects may be animated by an associated process.

* Images should be available to be used as, for example, textures to improve the impression of reality

* sound and video (analog or digital), have to be stored and played back when appropriate.

-- Different degrees of reality

* wire frame

* shading

* texture

-- Different styles of interaction

ï The scientific and commercial aspects of VR:

ï VR infrastructure

ï Translation of data for use in VR and generation of virtual worlds

ï Application of special VR software for high quality presentations

ï Execution of innovative VR applications

ï Consulting for VR use in new application areas

ï Presentation and testing of new VR equipment and techniques

ï Education and training in VR systems

ï Intelligent objects.

-- These are the building blocks out of which the virtual design models are constructed.

-- Intelligent objects have physical properties like building elements or furniture as well as functional properties such as steering wheel in vehicle design.

-- Ultimately, the system will allow the simulation of intelligent objects containing information about energy, cost, structure and other properties, which are normally not visible.

-- Interfaces. Designs not created with intelligent objects may be imported from other modelers such as CAD and other data.

ï Modeling of physical attributes and behavior.

Examples include sound, 3D interaction, object attributes and object behavior, will be represented in the VR environment.

ï Interaction.

Walk-throughs based on different user and maintenance requirements.

ï Tutorials for the use of trackers, large stereo screen projections, and stereo sound for simulation will be provided for teaching purposes.

ï Extraction, description and representation of 3D objects have been our on-going research topic

ï computer graphics part including

-- the Phong illumination model, incremental shading techniques for polygons, ray tracing, texture mapping, and radiosity

ï Object modelling techniques:

-- polygons, curved surfaces, fractals, particle systems, modelling hierarchical objects, aliasing and antialiasing, and color theory.

ï Students will be given two programming assignments which involve both modelling and rendering. One project involves fractals, particularly the modelling and rendering of fractal landscapes, and the other is to implement a simple ray tracer.

ï principles of computer animation.

-- key frame systems, kinematic specification of motion control for objects and cameras, scripting systems, and physically based modelling

-- [Bar89] AH Barr ìTopics in Physically Based Modelling,î Addison-Wesley, New York, 1989

ï Final project, each student produces a short three dimensional animation sequence of up to 30 seconds duration on a topic chosen by the student.

-- Based on the Irisís animation abilities, we can generate real time smooth animation. Popular topics have been particle systems and the motion of simple polygonal and wire-frame objects.

-- the calculation time per frame should be short enough that the animations are not slowed down.

-- The animations will be recorded on video tape to provide a permanent record fo the studentís work, and allow them to be reviewed for assessment purpose.

-- [Bar89] AH Barr ìTopics in Physically Based Modelling,î Addison-Wesley, New York, 1989

[

Synchronization in Virtual Worlds

Michael Papthomas, Christian Breiteneder, Simon Gibbs, Vicki de Mey

Virutal Worlds and Multimedia

]

ï focus on virtual world generation.

-- geometric modeling

* shape of virtual objects

use MutliGen

* specify texture, surface reflection coefficients, color

Texture mapping

use MultiGen

(I consider Multi Gen is higher level than CAD tools)

Texture can be generated in the following ways:

(1) use a ìpaintî program to interactively create and store texture bitmaps

(2) use a photograph of the desired texture and electronically scan it.

(3) use texture databased such as ImageCELs

-- Kinematic Modeling

object position, collisions, grasping, scaling, surface deformations, flying

Modsaf?

--

ï long term projects:

-- build the virtual world using video images

* people have used 3-D digitizer to construct objects that are not part of existing database.

(HyperSpace Model 100)

-- build intelligent agents.

-- Synchronization issues

multiple user interaction

-- Open system environment

ï Definitions

Cyberspace: electronically mediated experience

Virtual reality: broad bandwidth first-person participation in cyberspace

Artificial reality: third-person virtual reality

Virtual Worlds:

virtual reality configured and presented for natural perception

Virtual Body/Virtual Environment:

The coupled subjective/objective components of virtual worlds

ï We will expand and justify these definitions, focusing on the proposed fundamental concepts of participant, inclusion, and information

-- participant: environmentally interactive sentience

-- inclusion: subjective experience of environmental closure

-- Information:

comprehendible symbolic structure.

-- redefine Cyberspace:

Cyberspace is electronic information which mediates by inclusion the experience of partipants, it is being inside symbolic structure.

ï We need physical tools to allow user to interact with the computer system

-- these tools capture the input from user in a variety of ways:

-- body motion: is tracked by 3D motion sensors

-- hand gestures are digitized by sensing gloves

-- visual feedback from the computer is sent to stereo displays

-- virtual sound is computed by 3-D sound generators

-- scene perspective and orientation is changed with track-balls, joysticks, mouse, etc.

3-D Position Sensors:

to measure the body position in real-time

ï applications: robotics, biomechanics, architecture, CAD, education

ï to capture the position of a moving object:

-- three Cartesian coordinates (translations)

-- three rotations:

ï It should not be intrusive:

the technique does not limit or hinder the freedom of the moving body

ï types of sensors: magnetic and ultrasound sensors

-- Four sensor parameters:

* update rate: number measures/sec

typically ranges from 30 data sets to 120 data sets

* latency

time delay between action and result

(in the case of the 3D tracker, latency is the time between the change in object position/orientation and the time the sensor detects this change.)

This is an important parameter. large latencies can generate serious negative impacts on the simulation.

* accuracy: the difference between actual and measured positions.

* resolution: granularity or the minimum change in position that the sensor can detect.

-- sensor modes;

* streaming mode:

sensor data is sent to the host computer continuously,

applicable to the situation in which there are fast moving objects or a quick response to a change in the moving objectís position.

requires large volumns of data to transmit:

(1) If the VR world need to be recomputed at 30 frames/sec,

How often does it require to draw a new frame?

every 33 msec a frame needs to be drawn

time to draw a new frame = data transmission time + computational time + graphics display time.

(2) If a 9600-baud serial communication line is used, how long doe it take to transmit a data set of six numbers, each consisting of 16-bit words, and considering a start of transmission character:

16*6 + 16 = 112

9.6 bits per ms, 112/9.6 = approximately 12 msec

Therefore, communication alone takes at least 36% of the time available: 12/33

* polling or one-shot

only one data set of six numbers is sent whenever the host requests it.

If the sensor is polled once every second, then, on average, communication will only account for about 1%(12/1000) of the available time.

ï 3-D Magnetic Sensors

-- employes alternating low-frequency fields to determine the moving objectís position and orientation.

-- transmitter, that generates the low-frequency field, is an assembly of three stationary orthogonal antennas.

-- receiver contains

* a second set of orthogonal antennas and

* algorithms to determine the position/orientation of the receiver in relation to the transmitter.

-- receiver is attached to the moving object so the motion of the object can be determined.

* e.g. in head-mounted device

to measure the motion of userís head

used in immersive VR

In this case, the receiver is placed on the userís head, when the posture of the head changes, so does the position of the receiver

Figure 2.2 3-D magnetic sensor in a HMD

--receiver transmits the measured data to the host computer

-- the computer uses the tracker data to calculate a new user viewing of the scene

* the scene is converted to video signals displayed by the two LCD screens.

-- dataglove( Isotrack]

* Figure 2.3 3-D magnetic sensor used by the DataGlove

receiver on the glove

transmitter is on a device

the device and glove are connected with optical fibers.

* maximum sampling rate was 60 measurements/sec, though the recommended one was 30 positions/sec.

Latency is quite large.

* The sensor range is small and signal noise was large.

* Figure 2.4 illustrates the noise problem.

Figure 2.4 DataGlove 3-D magnetic sensor noise

It is a plot of sampled data from a stationary reciever of a DataGlove

180 data points were sampled within 11 seconds

The sensor did not move, but we can see that

(1) the roll and yaw angle varied about 0.5

(2) When sampled by the host computer this noise results in a ìtremorî of the virtual hand on the screen. The tremor can blur the image and is visually tiring for the user.

* Fastrack Figure 2.5 Fastrack block diagram, which uses DSP(Digital Signal Processing) architecture

skip the figure

[

(1) signal induced in the receiver coils ....

]

This results in an improved signal-to-noise ratio and noise shape, and doubled sampling rate. of the older isotrack.

ï Work envelop of the sensor:

region of space in which accuracies satisfy a given threshold:

For fastrack, the work envelop is about 75cm from the transmitter.

Beyond this distance, the accuracy degrades, but the measurements can still be made up to 3 m from the transmitter.

Surfaces of equal accuracy are hemispheres centered on the transimtter: see Figure 2.7

* Insidetrack(1993)

smaller: electronics interface was compressed to fit on a PC-card that plugs inside 386 or 486 compatible computers.

The price is $2,300 which is half of the fastrack glove

However it is noiser than Fastrack according to to experiment conducted by Burdea and Coiffet, the authors of our text book.

-- 3D magnetic sensors have the advantage of larger work envelopes than ultrasound-based trackers

-- drawback is accuracy degradation owing to a large metallic objects.

* if a ferromagnetic object or another strong magnetic field is located within sensor range, the transmitterís low-frequency fields are distorted. See Figure 2.7(b)

* Therefore, large metallic objects should be placed close to the trasmitter or receiver, otherwise very complicated calibration nd compensation measures are needed.

-- For large number of receivers without a drop in the update rate, Flock of Birds sensors made by Ascension(1993)

* Each receiver has an update rate of up to 144 data sets/sec, and this stays the same even with up to 30 receivers connected to one transmitter.

* Figure 2.8 Flock of Birds Block Diagram

Three electronic units named Bird 1,2 and 3

* Each bird has a transmitter XMTR and receivers RCVR

* Communication between the various electronic units and the host computer is done over a fast serial bus, ìFast Bird Busî

* a transmitter,

a set of three ultrasonic speakers mounted about 30 cm from eacher on a rigid and fixed triangular frame

ï Ultrasound 3-D Sensors

-- Three components

* a receiver,

a set of three microphones mounted on a similar rigid frame which is placed at the top fo the head mounted display

* an electronic unit.

-- Figure 2.9 Ultrasound head tracker

-- a direct line of sight is required between the speakers andthe microphones

[

Alternatively, the microphones may be part of 3-D mice, stereo glasses or other interface tools.

]

-- How does it work

* the speed of sound in air depends on room temperature based on the law:

where c is the velocity of sound in m/sec and T_k is the air temperature in degrees Kelvin

* Therefore for a given temperature, the speed of sound is known and can be used to measure distances based on ìtime of flight.î

* In the triangulation arrangement,

(1) each speaker is activated in cycle

(2) the three distances from the speaker to the three microphones are calculated

(3) A total of nine distances are measured in order to determine the position and orientation of the plane that contains the three microphones

-- The control unit CPU samples the microphones, converts their readings into position and orientation, based on calibration constants,, then transmits the data to the host computer for scene update on the head mounted display.

-- The update rate is 50 data sets/sec, which is less than that of modern magnetic position trackers.

-- Drawbacks:

* line-of-sight requirement,

if some object obstructs the line of sight, or the head is truned away, then the signal is lost.

* signal can be corrupted by background noise in the room, as well as other ultrasound sources (such as the ones used in building security).

-- is cheaper than magnetic sensors.

Trackballs

ï absolute sensors:

-- e.g. 3D position sensors, magneticc and ultrasound

-- they return the absolute position and orientation of a moving object with respect to a fixed system of coordinates.

-- the position of the VR objects controlled on the screen is directly mapped to the postion of the absolute sensor in the transmitter fixed system of coordinates.

-- absolute position data is never a zero set (even if the sensor is at rest)

ï relative(differential) sensors:

tracking ball

Sometimes called space ball. it is different from the trackball we use on regular computer, such as portable compuater key pad.

-- allows for incremental position control,

* returns the position relative to the last measure

-- therefore, a relative position sensor will always return zeros if not acted upon.

-- provides six degrees of freedom

* the position of the VR object is changed by six signed quantities at every simulation cycle.

-- Strain gauges measure the amount of pressure applied to the space ball to provide input for spatial positioning and orientation as the ball is pushed or pulled in various directions

-- A differential sensor example is the ìDimension6î trackball shown in Figure 2.11

: a tracking ball that measures:

* three forces and

* three torques

* The central part of the ball is fixed and has six light emitting diodes (LEDs)

* six photo sensors are placed on a moving external ball

* when the user applies forces or torques on the moving ball, the photo sensors output is used to measure three forces and three torques.

* these forces and torques are sent to a host computer over a serial line.

* the computer then computes the the gains and returns a differential change in VR object position and orientation.

* larger gains will result larger speeds for the VR object, but its motion will not be smooth if the host cannot refresh the screen fast enough.

[

-- VR objects can also be controled by the force applied to the tracking ball

]

-- track ball can also be used to ìflybyî in the simulation.

* in this case the sensor affects the velocity and orientation of a Virtual Camera looking at the simulated world

-- Several pus buttons are provided on the ball:

* off/on button

* other buttons can be programmed to, e.g.

one can be used to increase the simulation gains, one can be used to decrease the simulation gains

or

one be used to increase the VR object speed, one be used to decrease the VR object speed

couple or decouple the ball with the VR object it controls

start or stop the simulation

reset the simulation to a default start when the user becomes disoriented.

-- disadvantage:

* suffer from sensor coupling(since rotation and translation are coupled):

user may wish to have the VR object translate, but not rotate, the VR object may do both translations and rotations anyway.

This is due to non-zero sensed torques when pure forces were applied by the user fingers

solution:

These unwanted motions can be suppressed with software or hardwareìfiltersî which read forces only and not torques. Hardware filters can be pushbottons on the track ball to allow user to select ìtranslation-only,î ìrotation-only,î or ìdominant motionî input.

[

* open loop mode: the user may control forces between the virtual object

and its environment, but he cannot feel these forces in the hand.

an improved device Global Devices 3D controller does provide such sensing ability.

]

2.4 Sensing Gloves

ï Track ball is compact and simple, however it does not capture the motion of the userís hand.

ï Sensing glove provides large gesture-based I/O interaction with VR simulation

ï Sensing gloves are designed to capture the following human thumb and fingerís motion:

-- flexion-extension

-- lateral abduction-adduction

-- thumb has an anteposition-retroposition motion which brings it in opposition to the palm(see Figure 2.13(d))

ï commercial models are:

-- VPL DataGlove

-- Vertex CyberGlove

-- Mattel powerGlove

They have sensors that measure some or all of the finger joint angles

Some have 3D sensors to track the userís wrist motion.

-- Figure 2.13 Terminology of hand and finger motions

ï The DataGlove

-- invented by Thomas Zimmerman was funded by NASA

Figure 2.15

* Each finger has a fiber loop for each of the joints whose positions are to be measured.

The figure only showed two sensors are attached to the back of each finger to measure the two major joints

additional sensors for minor joints as well as abduction-adduction anteposition and retroposition are available as an option

* the optical fibers are joined and connected to an opto-electronic interface.

* one end of each fiber loop is connect to a LED, while light returning to the other end is sensed by a phototransistor.

when the fiber is straight, there is no attenuation in the transmitted light.

* raw sensing readings: the finger joint angle is measured by the intensity of the return light

[

* At the location of the finger joints, the fibler walls are treated to change their index of refraction such that the light will escape upon flexion of the userís finger. In this way it is possible to measure the joint angle indirectly based on the intensity of the returned light.

]

-- glove calibration:

transforming raw sensor data into finger joint angles

* calibration is required every time the DataGlove is put on at the start of a new simulation session

Since different users may put on glove in a different tight fit on the fingers

[

calibaration method differs between flexion sensors, and abduction/adduction ones

]

* least square given bh Hong and Tan

where r = raw sensor reading, (r) = flex angles of the finger, a, b, c are calibration constants

where a, b, and c are determined by storing finger angles (r_i) and the corresponding raw data r_i for a number of test points m.

Note an error in the book, the first element of the matrix at the right side was r_i

* the calibration of the adduction/abduction sensor is more involved than that of the flexion sensor (see book for one method).

2.5 Stereo Viewing Devices

ï These devices are designed to simulate Human vision. Human vision has the most powerful sensorial channel and has an extremely large processing bandwidth.

-- The equivalent of multi-million polygon-based computer images are processed rapidly by the brain without conscious effort

* Every 3D object in a scene can be represented by multiple polygons.

ï Typical viewing devices are:

-- head-mounted displays( the most common device)

-- booms

-- stereo active glasses and monitors

-- large-screen stereo projectors

ï We will first discuss human vision system in particular in

-- Depth perception combined with side field of view, stereo imaging and interactivity enhance the userís feeling of immersion

2.5.1 Human Eye Stereo viewing

One eye perception :(Monocular Imaging)

ï Perception generation:

Light reflected from objects in the scene is focused by the lens inside the eyes onto the retina, the file at the back of the eye, to from images. These images are then transmitted to the brain, generating mental representations of the scenes - our perceptions

-- same image can generate several perceptions.

ï Image formed on the retina(film) is upside down and reversed left to right, just as is the image that appears on photographic film.

ï Point Project is te fundamental model for the transformation wrought by our eye.

ï pinhole project: projecting scene points through a single point onto an image plane:

Figure 2.1 (B&B) A geometric camera model

* image is behind the point of projection

* the image is reversed

-- In order to make it more intuitive, we make the project(artificial model)

as in Figure 2.2 (B&B)

* the view point is behind the image plane

* the object is straight

* focal length, the distance between the viewpoint to the image plane.

-- the projected height of an object yí is determined by its real height, its position z, and the focal length f: note image plane is at z=0.

similarly,

perspective transform is usually thought to distort the z component just as it does the x and y , so perspective projection is

(xí, yí, zí) =

* Orthographic projection can be considered as a special case when the viewpoint is at infinit in the z direction, i.e. f -->

(xí, yí, zí) = (x,y,z)

ï depth perception

depth is perceived based on cues inherent in the image, such as:

* linear perspective

* shadows

* occlusions (distant objects being blocked from view by closer ones)

* texture and

* object detail

* motion parallax:

closer objects seem to move further than distant ones when the head is moved

-- Stereopsis:

we have

(f-z)xí = (x-d) f, (f-z)xî = (x+d)f

we get

z = f -

both eyes register an image and the brain uses the horizontal shift in image position registered by the two eyes to measure depth

* Figure 2.23: Physiological model of stereoscopic vision

* field of view: the total viewing angle

about 180 degree horizontally

about 150 degree vertically

* Within this field of view, eyes can register the objects surrounding the viewer: such as

object A is in front of object B. (note the figure 2.23 has marked object A as B, and B as A).

* convergence angle:

the angle between the visual axis and the line to the fixation point F, which is the feature of A on which the eyes concentrate.

This angle also depends on the IPD(Inter-pupillary distnace): The larger the IPD the larger the convergence angle.

IPD: the distance between the right and left pupils

IPD varies among male and female adults within a range of 53 to 73 mm.

The IPD is the ìbaseline from which a person interprets distances to objects in the real world.

* parallex

Feature point F appears shifted horizontally between the right and left eyes because of its different position in relation to the two eyes.

* VR display device should simulate or replicate this parallax in order to help the brain interpret depth in the simulated world.

This is not a simple task.

2.5.2 Head Mounted Displays

ï displays scene very close to eyes

-- LEEP: special lenses(optics) needed to

* allow the eyes to focus at such short distances without tiring easily

* to magnify the screen image to fill as much as poissble of the eyesí field of view.

-- LEEP is used in recognition of the first company that introduced such optics in 1989.

* wide optics is used to accommodate all user intrapupillary distances

* show Fig. 2.24

* LCD: Liquid Crystal Display

* interaxial spacing of the lenses is chosen to be slightly smaller than the average adult IPD so that the two eyes will converge when looking at the two screens (rather than diverge, which is tiring)

* eliminate the double vision effects:

plastic fresnel lenses are used on the lenses to shift the two images closer to each other and merge them.

* diffuser: used to blur the edges between pixels to overcome the low resolution problem of LCD.

Typically 360x240 pixels which is 28% of VGA monitor in monochrome and 9% in color.

* Image distortion: straight lines displayed on LCD can be perceived by user as curves.

Use methods to model eye optics: Figure 2.25 Optics model of the eye

(a) Single-eye

(b) stereoscopic model

For stereo viewing, there will be two virtual points b₁ and b₂ which are converged into a stereo point c perceived by the user.

Linear graphics primitives are written to a ìvirtual screenî and then all pixels are shifted according to a predistortion function and are sent to the screen image buffer for display.

The distortion function is modeled as a third order polynomial approximation:

Let (x_vn, y_vn) be the pixel position on the virtual screen, (x_sn, y_sn) be the corresponding pixel location seen by the eye, then the relationship between the two pixels are represented by a distortion function

where

d= [1 + k_vs ( + )]

coefficient k_vs is used to express the amount of distortion present. Since the distance from the eye to the virtual image plane is a constant, z_v is same before and after the distortion.

the inverse of d, d^-1 is the required pre-distortion function

d^-1 = [1 + k_sv ( + )]

where k_sv = - k_vs

note d and d^-1 are third-order polynomial approximations, they will not be exact inverse of each other.

* Table 2.3 Parameter calculation for the optics model

ï Important criteria for selecting LCD-based HMDs

-- resolution

-- field of view

-- weight

important, can increase user fatigue

-- comfort

-- cost

Device name	resolution	field of view	weight	cost($)
Eyephone	360x240 pixels	100o (H), 60o (V)	2.4 kg	11,000(90)
Flight Helmet			2kg	6000
Cyberface II	479x234	140o (H), 60o (V) largest availabe to date	1kg
Tier 1		112o (H), 60o (V)	680g	3800
HMSI 1000 the smallest, lightest(1993)	479x234 color pixels	65o (H), 46o (V)	0.23kg	5000(1993) -- 2000(1994)

CRT-based HMDs

CRT(Cathode Ray Tubes)

ï workstation can dypically have a 1000x1000 pixel resolution

-- only 11% of this can be displayed on the best-resolution LCD-based HMD

ï use miniature CRTs instead of LCDs for display.

-- special care is required in the electrical circuitry because of the high voltage of the CRTs, which are close to the userís head.

-- Figure 2.28 CRT-based Hds

(a) HMD-131 from Virtual Reality Inc. (b) Datavision 9c from n-Vision Inc.

device name	resolution	cost ($)
HMD-131	1280x1024 at 30Hz 640x480 at 60Hz	70,000
datavision 9c	1280x960 at 30Hz 640x512 at 60Hz
other smaller lower cost devices	640x480	1000

ï On going research :

replacing the CRT algorther and moving the display element onto a chip, :digital micromirror device(DMD)

Texas Instrument: Figure 2.29

Binocular Omni-Orientation Monitors (Booms)

ï Both LCD and CRT-based HMDs are adversely affected by head-tracking latency.

-- 3d position sensors have latencies of at least 12 msec or more

-- transmission delay

-- graphics rendering

-- conversion delays,

-- the combined dealy, which is the visual delay: is a serial problem:

when a user moves his head, he expects the displayed image to move in the opposite direction.

* if this delay is too large, simulation sickness may occur

* we need to have almost instantaneous response to the userís head motion.

-- in order to alleviate these problems, in addition to fast grphics rendering, we need a different kind of tracking.

ï CCSV workstation (NASA)

Counter-balanced CRT-based Stereoscopic Viewer

Figure 2.30 The CCSV workstation data flow

-- a mechanical arm to support the CRT so that high resolution CRT is possible.

-- has position sensors at six joints, which produced a global 3-D tracker for the head motion

* raw analog data from the arm sensors was first converted into floating point angles (based on internal calibration constants.)

* then using known direct-kinematics equations (much like modern robots do), it was possible to obtain the position and orientation of the end of the arm that supports the viewer.

* these parameters were then sent to the graphics workstation, which rendered the images for the two eyes.

ï Boom (Fakespace Inc)

-- each joint uses relative encoders with 4000 counts/revolution or a resolution of 0.09^o (360/4000 = 0.09).

-- resolution in the high end model such as Boom2c is 1280x1024 pixels, which is higher than in any other display.

-- field of view

produced by LEEP optics is the largest: 140 ^o horizontally and 90 ^o vertically

-- cost: high end, $88,000, lower end, ìBoom2î monochrome version costs, $42,000.

-- major advantage: low latency:

* tracking update rate is 60Hz with essentially no latency or noise

* tracking is not adversely affected by either magnetic fields or ultrasound background noise

* increased functionality over other MHDs

Boom allows inermittent use of computer keyboards without having to repeatedly take off and then put back on a HMD helmet.

-- Disadvantage:

less ìfreedom of motionî due to a ìdead zoneî created by the supporting pole in the center of the work space.

-- boom work envelope:

1.5x0.75 m

which has an exclusion core of 0.5m

The ìCyberscopeî

ï low-cost I/O tools designed to caputer the widespread PC market.

ï Cyberscope (Simsalabim Systems Inc.)

-- $200.

-- tatally passive (no power supply)

-- attaches mechanically to the computer screen (see Figure 2.31 (a))

* Figure 2.31: The cyberscope

(a) System configuration

(b) split image display

-- two images are displayed simultaneously on the computer

* the left and right-eye images are rotated 90 ^o counterclockwise and clockwise, respectively

* there is a small separation zone tween the two images in order to avoid interference int he merged stereo scene.

-- the optics inside the Cyberscope rotate the two images and display them to the user on two small lenses.

* the resultant focl distance is a comfortable 30cm, with a field of view of 47 by 35 degrees

-- its resolution and brightness are limited by the monitor it is attached to.

* For proper functioning the monitor diagonal needs to be tween 14 to 17 inches, and the display area must have a height of not more than 9.3 inches.

-- the scope assumes a default pupillary separation of 6.1 cm, but allows adjustments through variables set in the display software.

ï Table 2.4 summarizes the parameters of commercial head-mounted displays

2.5.3 Stereo Glasses

ï For a team of specialists, or users who need to see the same stereo image, it is too expensive for each of them to have a HMD

ï use time-sequential images:

the same screen pixels display left and right images in a time sequential way.

ï stereo ìactiveî glasses in connection with stereo=ready monitors.

ï Figure 2.32:Active stereo glasses: (a) standard confguration

(b) VR configuration

-- a ultrasonic head tracker(the triangular frame) is placed on top of the monitor near the infrared controller

-- the active glasses are modified to incorporate three ultrasonic micropones that detect the ultrasound of the transmitter.

-- the tracker controller converts sound phase difference into head 3-D information and trnsmits it at up to 50 data sets/sec to the graphics computer

-- software uses the 3-D data to re-calculate the virtual scene and the simulation loop is closed

ï additional I/O tools such as sensing gloves or track balls can be added to the system in order to increase the VR simulation interactivity.

-- Figure 2.33 Communication diagram for a VR system using active glasses

ï the direct line-of-sight requirement of ultrasound trackers is easily satisfied as the user looks at the screen during the simulation

ï In a multi-user environment only one person has control over the virtual scene perspective, as only one head tracker exists.

*-- the person controlling the simulation has to be within the tracker active area of about 0.2 m³ , while other participants can look from further away.

name	company	weight	cost($)
CrystalEyes-VR (including a Logitech ultrasonic tracker and one pair of active glasses)	StereoGraphics Co.	85 g	3,900 each additional glass costs 1,100 stereo ready monitor: 4,800
CrystalEyes-PC (including a Logitech ultrasonic tracker and one pair of active glasses)	StereoGraphics Co.		980.
Sega, for video games	Sega		30

2.5.4 Stereo Projection Screens

present left and right eye information through spatial multiplexing

ï Figure 2.35: Spatial multiplexed image

-- left eye virtual image is sent to a modulation step which darkens half of the screen in a checkered configuration

-- the right eye virtual image is sent to a complemental modulator

-- then the two are multiplexed (merged).

-- SMI: spatially multiplexed image has both left and right eye pixel data

-- then we need a way to de-modulate the image so that the user sees it in stereo

ï Figure 3.36:SMI de-modulation with passive glasses:

(a) Micro-pole array

(b) use with passive glasses

-- a ìmicro-poleî array

* consists of small checkered polarization rectangles

* the alternating polarizations P1 and P2 are rotated 90^o from each other

* the dimensions of the micro-poles, as well as their spatial distribution, matches the SMI pixel organization

-- a micro-pole film is overlaid on tp of the SMI such that P1 poles cover all R pixels and P2 all L pixels.

-- the user wears a pair of ìpassiveî polarizing glasses

* the right lens is polarized with orientation P1 and will block all pixels with orientation P2, therefore all left pixels

* the right lens is polarized with orientation P2 and will block all pixels with orientation P1, therefore all right pixels

-- the brain then integrates the two images for stereo effect.

ï advantages:

-- only one image is used, so no special monitors are needed

-- low cost:

passive glasses cost only about $1. it is suitable for large groups of people looking at a stereo image than using ìactiveî glasses

-- can be projected on large screens with micro-ple array

-- the same SMI technique can be replicated by output to printers or photographic media

* it is thus possible to have cheap 3-D sereo images printed in a book, for documentation of VR graphics. The reader then puts on the polarizing glasses provided with the book.

ï Vrex Inc. has the product costs $6,500.

stereo display using micro-poles including the screen, projector and glasses, but not the head tracker

Autostereoscopic 3-D Displays

it produces a stereo effect without using glasses

The tools and systems can be seen as enabling technologies for the VR simulation. Another important aspect in the simulation is the modeling of hte virtual world.

Steps of tasks:

ï obligatory task analysis

defines the underlying processes mathematically and allocates existing hardware resources.

ï object databases and modeling optimization.

modeling object shape, appearnce, behavior, constraints and mapping corresponding I/O tools to the simulated world.

NETworking Aid for Engineers

A new facility, the Edinburgh Engineering Virtual Library

(EEVL), which aims to make the Internet more

user-friendly for engineers has been introduced in the

UK. Enabling engineers to locate the information they need from among

the vast array of available resources, the (EEVL) links directly to over

1300 Internet resources in engineering.

A user-friendly interface which will allow anyone with access to the

Internet World Wide Web to find the latest engineering information by

keyword, subject, or title. The free service is primarily intended for

engineers, both academics, students, and the commercial sectors.

The 200,000-pound project, which has taken a year to get up and

running, is managed and updated by a team of engineering information

specialists, headquartered at Heriot-Watt University Library, in

Edinburgh, Scotland.

Resources in the EEVL database include Web sites for engineering

e-journals and electronic newsletters, engineering companies,

professional societies and institutions, engineering departments within

higher education institutions, government sources, engineering email

lists, resource guides and directories, research centers, recruitment

services, and more.

EEVL Project Manager Roddy MacLeod, Senior Faculty Librarian at

Heriot-Watt, said "This new virtual Engineering Library is a powerful

tool, providing immediate, one-stop access to the information needed by

those working in the rapidly developing world of engineering...and all

without having to step away from their desks.

"Its benefits will be immediately apparent to academics and engineering

students, and I believe the commercial sector will soon come to

appreciate the opportunity for relevant, up-to-date information and

personal contacts which EEVL offers them. We are expecting a very high

hit rate."

As well as providing access to resources, EEVL encourages the creation

and provision of new engineering resources, and gives assistance to

non-networked engineering organizations for this purpose.

EEVL is funded through the UK Electronic Library Programme (eLib).

The project lead sites are Heriot-Watt University Library and the

Institute for Computer Based Learning, and partner sites are the

University of Edinburgh, Napier University, Cambridge University,

Imperial College of Science, Technology and Medicine, the Nottingham

Trent University, and the Institution of Electrical Engineers (IEE).

The service can be found at:

http://www.eevl.ac.uk/.