come a three of the and

i hope we're or you had good sleep

after is to is for circular

so back to this go i i i hope you enjoy it

and do you your the the music there

so why won't read or the names some the links a to repaid use but in case you want to

have

with brave guys and are also uh invited to your garden party

there will be doing so the on the uh on the conference web page

i was especially a it is harm core just band but

that was playing in one of the rooms that was re reexamine

people from uh

ordinary secondary school mode even music second can very group

so uh a good and you can you can visit the

their rip sorry

or just type harm core prague to do ruin

you will get it

uh there is one a

warm

mixed white please

oh yes

okay so one one practical remark remote the restaurant

we had the sum compliance like to you in the plumbing you

uh a short clues from you use the not be type of food i won't

so uh the the the solution is to ask or or do not a a uh uh the information desk

and she will be happy to a device you a rest around in be vicinity of these place

or or or or or or the there

no uh uh this at all

or the practical in four

and we will have for an invitation a two us very here you are a conference

number you are most probably only is from a some you know of pixels

we morning

this morning i would like doing that are used to you

that two thousand and two L international conference on merging signal processing applications

or use by twenty two well

my name is by most problem in is and i am the general so obvious park where B to out

use but where and good where all these marked it works wrong

it is a full quorum forums organise by the signal processing society

it is expected to be the third major problem for some of the some side view

but together with i ask for nice here

but because it used

you is not dressed primarily to practising engineers

of course everybody is welcome wall

i was uh going to be a course in practice problem

something else that makes use by be my when i see here

use that to do will be held every year here at the same place and time

you "'cause" been plan one for one with a consumer electronics show

so give her beam last very guns

in early january

when we say practising and you here

my the and some more who has a master's degree

in general signal processing

i i is or the presentations and expected to be applications already it

and more can thing as a very clear to

to first a good participation from mean dusting use by we also include but as patients without that paper

of course papers first can be some you two

and that these papers would be are in the R you i drip we explore

but we're one from is mark expect to be a if you're at the girl

in another major component of the conference

is that the most rooms

what we call show and they are here by cast

and in have you sure we will have to house

plan i to present creations

and by discussions

you can find more maybe from in the call for papers that we have a the information of both here

by cash

or blues and see me

so here are all invited you to submit papers for is is where well

one more thing

the corn firms "'cause" not go about to the local yeah

we can you to submit your idea for a ago

and that there will be a three hundred the world

for the log which or

so i hope to see you all next gen was very guess it would be fine

thank you

thank you nose

and it's time

i

i'm i'm to start the

brisbane is primary

or speaker will be a the channel

and the like colleague your

and chambers

technical program chair

we introduce or speaker

from a

good morning uh light is in uh uh and from then i'd like to uh gain by repeating what ones

as Z and for that was of you who attended the uh check the evening uh yes to the evening

it was a about very special

that a page in and the uh music uh uh well as a a special uh experience that with

thank you very much to the local organisers work i in that

a bit by great applied to to uh

introduce a a a it it

doubt this morning

just a battery quick look at is uh uh uh C B

or by a will show you that uh

it's a sort of battered simple

whose who or but uh

some of the leading

academic

and industrial

unit

in the U

he V C he's degree in the nineteen eighty two

from

print then

but really

and than food

following that C work

at

at and T bell lab

multi he'll you to see

the about

how out so

with that so and uh in california

and since nine

ninety i

he's work that uh the microsoft

with a

and uh in redmond washington U N

where he heads the communications and collaborative systems systems research

philip has being a wheel

so than for the ieee signal processing society

most recently each yeah the multimedia technical committee

and old so it's a so on the signal processing society by committee

each distinguish is a a with it B and twelve

a side please

piper what in ninety ninety three

and more recently in two thousand and seven

even a the best paper award from the ieee transactions on multimedia

so thank you again guy in to fit it for agreeing to uh make this presentation

i i and i'm sure the whole O D and is looking forward to is

to all

in that it communication

it

you very much for the introduction it's a great honour to be on the stage especially with fred jelinek over

here he was an inspiration to me early in the career

and thanks all of you for coming uh to the plan area to at and after

have to the banquet i don't think i ever be in

at M plan a right after the bank so this a new experience for me i'm kind of looking for

to seeing what these plan is actually like

so i'll be talking about immersive of communication

um

there won't be any equations

in my slides

no signal processing people sometimes don't really understand things and less there

this math behind it so i'll give you some references

uh

signal processing magazine had the special issue in numbers of communication that came that in january you can look at

that

and just last week a bunch of us submitted a paper to the ieee proceedings as part of their hundred

anniversary um both of these are numbers of communication

so the last

second

paper here um the fines of communication to be

exchanging natural social signals with remote people

as in a face to face meeting in a weighted suspense

disbelief

uh in being there

this is uh

not a new idea by any means the

telephone was invented a hundred and thirty five years ago and that was the first grade breakthrough an immersive communication

it wasn't long after the telephone was invented a here's a cartoon that came out oh

three years later

uh that people were starting to look at much more immersive scenarios

so this is a

by george them are yeah a

um

the

caption of the bottom

read something like uh

uh as as these uh

parents are here in london in

uh uh uh watching their daughter in ceylon

play badminton

and

father says speech

come over here i want to whisper

she comes over

that's yes

pop here

uh

who who's that charming

uh lady

playing by charlie side

uses a

um she's just come over from london um i'll introduce you after the game

so we don't really have this kind of immersive of communication system even today but perhaps something that comes fairly

close

are tell a present systems

tell presence uh as defined by

the industry conference

newsletter wayne house review is

a video conferencing experience that creates the illusion that the remote participants are in the same room with you

so probably the quickest way to get a sense of what that means today is to take a look at

a thirty seconds just go

commercial

yeah

hmmm hmmm

hmmm

ooh

i

i

i

i

uh

so so one can be uh

so here you seen

high definition video conferencing so compelling that one participant has forgotten that his counterpart is

is remote

so this just go tell the present system uh

uh and others

uh like them H B halo another others

um off offer

high definition audio and video lots of bandwidth

um

to try to create this solution of being in the same room

um

is this a breakthrough in a of communication or is just a lot of uh

high definition televisions and a lot of bandwidth well

in a sense they're both

okay so

i contend that these are

bridge

uh to the future and we're about to see um

uh a rapid progress in this area of for the next few years

so put that in the context as take a look at

um

a brief history of television

television

was invented in its current form and

nineteen

twenty

six

uh

as a counterpart to the telephone

so here you see one of the first television set

eighteen T uh bell labs

uh

next to a telephone

because the television is meant to communicate

as a visual part of the telephone

so here it is

eighteen T president walter gifford

uh

in a you are K at bell labs talking to herbert hoover then secretary of commerce and washington D C

in nineteen twenty seven

so the first

distance television call

uh and shortly thereafter the television became

the broadcast medium that we know today whereas

video telephony

um took another forty years to become the eighteen T picture phone

uh

the picture phone was a

a stunning

technical success but also a stunning

commercial failure

so by nine teen seventy nine when i was

in intern at bell labs

down the hall from me on the desk of my lab director bob lucky was the

last remaining

working

picture phone in the world

but would often complain that nobody ever called them on it

did a forty years after that

uh experience is actually very similar

uh

except that now these little rectangular images are on general purpose computers

um

or or on your phone now

um but pretty much the experience the same there small little rectangular images of video

video conferencing

similarly

um has been

the same

uh

the for the last forty years in some sense here's the bell labs video conferencing system in nineteen sixty seven

you can see the round tape around

tables and multiple monitors cameras on top of

each monitor or the

data monitor

um which looks very similar to the

you know

the recent tell presents as

today

so

why do why contend that

there's about to be a series of rapid breakthroughs in immersive communication when

the last few years not much as happened in visual communication well

several reasons

the first is the internet

okay so the internet

has caused a divorce between

the format

of the content and then medium over which it is scary

so when the past telephone calls were carried over telephone networks

television over television networks are over

radio and so forth

today day all those are carried over the internet so there's a big possibility now

of

inventing arbitrary formats and they'll all be carried over in

uh the second is of course the cost of computation band with an resolution has a in dropping exponentially for

so long

and that there essentially free now compared to what they work twenty years ago

and the third reason is the technology

evolves faster than biology

okay so

they will become

bill be a threshold at which

the number of bits per second that were able to capture transmit and render

uh sir is what we can actually pass through the neural cut set around our bodies okay so at that

point

um

we should have freedom to do whatever

we like

and

the question is

what do we want

the future of you min

communication to look like at that

a

so to the extent that hollywood is the

uh keeper of our collective dreams

you know

be answer that hollywood

would give is that they want

communication to be immersive whether it's like

uh

the uh a hole attack in star trek or the jedi council meetings and star wars or

the matrix in the matrix or

um

in in avatar

um

communication should be mercer

however

not all communication will be numbers

okay we will continue to send S M S messages

uh and the reasons are that there you know there are some reason such as privacy that we we we

don't want to send everything

about about so this is anticipated you can buy

the jetsons and nineteen sixty two

so here's

change S and putting on her vanity filter

before she makes an early morning video call to her friend gloria

unfortunately flores

on vanity filter has an embarrassing

mel function

then there's one of my famous my favourite cartoons

uh

it's this

uh

from pete steiner in the new york in nineteen ninety three

i'm the internet

nobody knows you're a dog

no know that's

still pretty much true today there reasons why uh we don't want to review reveal everything about ourselves

and yet for the more complex

human interactions

a more creative things that we do with each other

um we need more margin

because we're social animals we have of all to work with each other best face to face

so how we position ourselves relative to each other

no where we sit

what how we just your

are are gaze

um awareness of all of those things they're all very important

and these have been studied um

but social psychologists and

and others engineers who build system

for example uh the work by no i and and can

and them the berkeley multi view project it build a system that

uh preserves i gaze

teleconferencing and they've shown that

trust improves

uh with correct i gaze

so

i gaze is bin

uh

a subject a video conferencing for

a long time there many solutions

have meters cameras behind screens

you interpolation and and many others here's an example of view interpolation where there's an upper camera

you when a lower camera view one they can be interpolated to get

um

better i gave

not only I gaze it's important it's uh i

reference and gesture

so here some examples from the early nineties

tongue and min "'em" and showed it chi ninety one

a a system that

had had a shadow of the remote collaborator behind the the collaboration surface

and a year later

you she had a same is clear board

um where the actual video of the

of the remote collaborator

was projected onto the

on the for the shared sir

here's an example uh that's

um

more recent P H P connect board

which

uh

is an update on it she's clear board

in that

perfect i gaze is established

by tracking

the position of the

observers had as well as the eyes of the

uh a of the

the uh a person in the video

so that

um um

the eyes of the on the video are always on the um

path

between the observer and the camera which is located time

the screen

at there many other

cues the uh days and gesture

um

both auditory and visual starting perhaps with

peripheral awareness of what's going on in the room

and consistency between the local space in the remote's space

so if you look back to at

the tell present systems today

you'll see that

if these preserve those first two characteristic

so

per full awareness is provided by very large

tell it

okay so we know what's going on and that remote room

um

consistency between the local room and the remote room as

is preserved by

uh making the

the desks look the same painting the walls to be a the same colour and all the different room so

uh

no difference between the back

of of of one room and the back of another

um

so

if you talk to the H P

guys

um they'll tell you that the halo system which was one of the first

which was probably be first um system in this category to come out

that was the collaboration between H P and

the film the hollywood studio dreamworks

and

you know how we what is is set you know be know how to

they know how to um

uh

use "'em" uh you um use illusion just suspend spend one's disbelief in actually being there

but there are many other

immersive of cues uh spatial cues

um for example uh distance

uh in audio is is uh indicated by

a combination of relative loudness

uh direct reflected energy and direct reverberant energy ratios

as we can see and in these uh these clip

oh yeah it's

right and i seven is does two yeah

so that one's

presumably further away than this one

oh yeah face

a in the next set is

does a to that was is probably adjusting the volume

but you can

uh i'm

and of course direction is is given by in true inter aural temporal role and interaural intensity differences between you

two years

um

and on the visual side

um there are many many is

cues on give you some examples of those

relative size and perspective combined to give you a sense of

distance

and um also absolute size

this of the same cues it are used in the famous

aims room illusion

uh occlusion lighting and shadow are also very important first giving a sense of

at distance

um

in augmented reality it's very important to be able to paint you objects into a scenes such the

occlude the background and yet uh do not occlude the foreground

uh lighting in shadow were very important not only for real is some but also

for um actual depth perception so if you look at this picture of the foot prints in the and

um

lighting and shot oh

give you an impression of one

uh a print being impressed into the the and the other one

uh

being above above the sand

so

a show of hands how many of you

perceive the top foot prints to be the one that's impressed into the same

into into the same

okay

and how many the bottom

okay uh

maybe one percent

think the bottom a ninety nine the top

um now keep your eyes fixed on that

"'kay" don't blink because if you blink you probably change your mine

okay so

even even shot all uh a has something to do with depth perception

uh focus as well so use a very large

city D china

uh but if we change the a focus

the foreground background it suddenly shrinks down to look something like like a little model that might be

uh

next year

model trains set

of course there the the uh

very strong cues of skates star got be and motion parallax

so some of you may be able to cross your eyes and fuse these images

and you get a very strong sense

uh of of dat

um

and all show an example of motion parallax later on

but

so are these cues uh important for communication

that's the question and i contend that the R

um clearly are current communication systems do not convey this information

and clearly they are not

satisfactory are are are not uh

um

no the can't fulfil every need for communication that we have we continue to meet in person and whenever we

are able

we continue to travel to work

we continue to

attend parties and

weightings in person we continue to come to meetings like this you know why don't we just

email all are talks

our slides to each other

and just a stay at home well because there's something

valuable about we feel about being here in person

um and our current communication systems don't convey that

but i

believe that um the tools of signal processing will help us

um bridge that yeah

so

if we're of a physicist you might take a different approach

case of this is is uh i've read have uh actually succeeded in transporting matter

uh a small molecule

um over us

significant distance

um and this is how it would work if you were to use it for uh

uh transporting you know somebody

so you me too

a different part the world i would step into a matter transporter

i would be frozen down apps the zero

i be turned into a little break of

bowes einstein compensate inside

in the process of schooling down i would it be light

light would be transmitted to another place

would shine on another little break

of bowes einstein kind and set

and then

my original state would be reproduced

of course if i

don't

plead lee cool down to zero i may leave some state behind hind uh with unforeseen consequences

it's also sobering to think about

uh what quantisation and data compression would do on um on the way

rate distortion optimized mpeg four thousand

um

but at microsoft research for mostly computer scientists and signal processing people so

we take a different approach which is

we do the

the um transportation uh uh matter virtually rather than physically

um and we achieve that through court a transformation

so if you imagine

a person in a real space over here

um

he's got some court

system in as local space and we just do a coordinate transformation

uh of that over into to some

no map that coordinate system into the coordinate system of some other states

an if we do that for several people then we get them into the same

the same

so depending on whether the space that there being transported into

is a virtual or real

um it leads to two different approaches

uh i'll

um

talked about both of these you might sort of a

identify the first one with

the movie the matrix if you're familiar with that

everybody goes into a a virtual world

um

the second approach could be identified with the movie avatar in which you send

a physical representative of yourself into are real physical

space

so i'll talk about

um

each of these

starting with the first one

so the virtual or

the matrix movie approach

um can be applied to what we call fully distributed meetings

so we have people who are

and completely separate places around the world maybe your

you when your colleagues are writing a paper together and you're sitting in your office is

different universities

and you wanna talk to each other and the ideas to make it seem as if you're sitting around a

see the around the same

table

um talking to each other

so the way we are doing that is by

capturing geometry of each person

doing a court transformation to put them all into a common scene

a life size scene

and then you give each person a window on that's scene it's basically there display

and uh each person gets a

you know i

own personal view of that scene

um

and uh people interact through the

in this way preserving i gaze gesture direction and so forth

but doesn't matter what that the projection is

on on uh

a flat

surface or

a two a multiple monitor situation or maybe even a curve

screen so

well we've done is build a a

a

wide field of view curved screen

um and from a particular point of view make it appear is if

uh uh vocal task is extended

uh in space

and then when we capture

people's geometry using a depth can where that situated on top of the

the screen

then we can position those people around in space as if they're having

a meeting so if you've never seen

um

what images from a that's camera look like

this will give you

you give you sense use up

here's the still frame from a

adept camera

see you can

render or uh render person from an arbitrary view

okay so basically that's what we do capture

geometry and texture

of um

of each participant

place them in there's

virtual world

and then um

what them collaborate so here's

here's an example of what you might see if you were telling one of these meetings

your

collaborator sitting across the table from you

uh

maybe a on the wall on the right there some artwork it's glowing to reflect the mood of the conversation

in the room

on the back wall perhaps as a large active computer display that can be used for the

uh for the meeting

um a on the left wall here might be a window out on to

the uh

skyline of singapore in this case or wherever L C may wanna hold your meeting

as other participants join the meeting

um the people in the meeting can move around the table to make room

people can bring in their own data

other people may join the meeting and they come in from ordinary web cams or so so they don't actually

have their dept associate with them but they can be represented as

as a virtual

displays here

um

data such as this what's showing in the centre of the table here can

float above the

the table and be manipulated and so forth so

these are some of the things you can do

um

uh uh in this uh

this scenario

so that

image of course um was

i i didn't show was taken from one particular static point of view but as people move around

uh

the image is move so

um here's a video that shows

uh this the fact of motion parallax

so by tracking the position of the camera in this case

um you can change what is poor

presented on and ordinary

um

display

so that the peers as if there's some that's find the display

oh this is just using motion parallax to give that sense

depth

the same can be done with a audio actually

okay so you can

uh

do you head tracking

by

you know visually

find out where where persons head is

point

and then um

a table look up to find out

the relationship between that person's head and uh

and this

and uh uh a virtual source

get the H R T S for that and find out what's the signals

that you want to each year

similarly you know what the relationship of the head is to each of these loudspeakers

and then you can

or to

transfer matrix invert inverted

do some crosstalk cancellation

and

produce at these loudspeakers

signals that depend on

where your head is

at

uh so that the

the perceived

location of the virtual source stays put

so it doesn't drift into one of the that the lots and the one of the speakers as you move

towards one speaker

apple

so we've

done experiments on different form factors as well

this is an example

of the

of a wall

where uh we've projected onto a whole a graphics screen the same kind of thing you might see in a

high tech show

you know that store

window

um

and

we try to make its

a appear

through motion parallax and stereo stereo parallax

um that the person is standing in the same room with you

so

i want this

take a

five minute interlude here to talk about that the cameras because that's cameras at had a recent breakthrough

in

um

in cost and performance

so there are new sensor that's really of air are available uh very red to everyone here just like a

uh a a webcam cam might be

so let me tell you about them

um

you might have heard about the can act for X box three sixty

it's a device

like this

um

that makes

use the game player

uh the controller

and what it was introduced in october

last year

it's sold

eight million units in the next sixty days making it the

uh a fast so link consumer electronics device in history

reading it some position and the guinness book

world records

a

oh

hmmm

hmmm

and

a so that's enough for that

uh

basically it's a

it's a the camera

okay

so at every

pixel it produces a their distance from the camera to something in the scene so it enables you to

uh

with do uh

some skeleton analysis for example

of where what people are doing enable you and labeling them to control

the games

so under the covers

it's uh

it's this kind of device we have

um a regular

rgb camera

we have an infrared camera

uh

that's pretty much just like the rgb camera except that has a filtered front of it

ooh allowing to see only infrared

there's a infrared projector that projects

and they'll pattern on the scene

and by correlating that known pattern with the observed infrared image um

you get a disparity at each pixel in the data

also in this unit is a a

uh design by even to even on you have who's

possibly here

uh

and

uh there's a motor or dsp unit U is P

uh

use P port so the motor tips the

the unit up and down

and all of this comes for a hundred fifty dollars

so

um

it's not surprising that within two weeks after it was a

it was released

it was hacked into

and people it up to their P C's

and did all sorts of crazy things

with it

um there's a web here

devoted to connect tax

and

by

wow one month after release

uh

uh there were about ninety projects

um um on that site

uh

in three months there work twenty four pages of

the projects uh yeah as of yesterday there were forty seven pages the projects and these

these are all crazy things like a um

um for

robot but

uh navigation to um

to seeing for the blind

two

greeting

storefront windows that react to to people wandering in front of them uh and so forth and so on so

if you're interested um

in using this for some of your signal processing i invite you to jump in

uh

uh there is an open source uh driver or but

it doesn't have any of the scale to tracking error or or eve ons uh

uh

mike array processing

uh if you want that

you can wait if few

weeks uh

i believe will be having a

um non commercial

uh S T K available for all of you to use uh

free of charge

so

i talked about

uh

talked about how we were using depth cameras

uh for

burst of communication here's a here's an application that's very closely related

and it will be actually released two

um X box live gold subscribers

um

uh sometime this spring

so i thought i'd show that to you

it's called avatar connect

uh rather than showing you an advertisement i've recorded my own video here

uh this is corey she's talking to me on her

right and to sash on her left

her her images

the image of her avatars

uh in this picture in picture thing

i

i

no who can see here avatars a her emotions

so in addition to

uh

in addition to um still to tracking of arms we have to do facial expression tracking so

you know who's uh

uh uh when people are talking

you have to figure out where they're looking so you can at intimate their avatars and so forth

and you can imagine that um

uh

E there uh many other

uh backgrounds you can use uh for this

for this uh

avatar can

thing

so you can do a uh

a uh

talk show for example with your friends uploaded to you to and so forth

so you can imagine that as

uh the avatars get more and more

um

sophisticated

eventually we may move into any era where inverse of communication is something like a mass of multiplier video game

where

you know you enter you enter into this world

um

and you have terms are controlled by your actual motions your expressions

and you're

oh case alone let me

um

return to the second

approach

um this is this

real or so called avatar like

avatar movie approach

um where you send a physical representative of yourself

two a real space

so this works well for so called and satellite meetings where there

several people around you a real space like a real conference room

and there are some remote attendees

um called satellites lights and they sense some

proxy in the me into the meeting

so we call these embodied social proxies

here's an example of an early

uh

early device

uh

a regular L C D used for the

the face

uh

fisheye cameras and pan tilt zoom cameras used for eyes

phone for mouth

and years

all of this is put on a cart

is a computer

wifi a battery backup and so forth so they can that end

meetings for you

and these are

uh our current meeting

we have four of these cards in this particular room

uh one here one here one here and one out of sight

we have collaborators

um

around the world john it's and silicon valley and over here's and cambridge

england

the you know the meeting room itself is in is in redmond washington near seattle

um

some of these are

um on

on cards that have to be wheeled in

this one is actually robotic it walked through the door on its own

the view from these carts of the room

look like this

okay so you get a wide field of view

peripheral awareness of what's in the room

and if you want to see it some detail like what's a a on the

screen

um

you click on the screen the pan tilt zoom camera goes over there and gives you a high resolution version

so we use these not only yeah for our own meetings but we've done studies on them across real

groups

that are

using them so we deployed then in for different microsoft product development teams where one of the

members of the team was remote

we the plate them for an eight week period

uh

conducted interviews and surveys at the beginning and at the end to determined

um

whether people use these things

uh

and by

asking questions on the surveys that are based on the seven point likert scale

yeah um

we determine meeting effectiveness awareness of social

aspects by asking questions such as

i think

X has a good sense

of my reactions

or

for awareness

i'm where

i'm aware of what X is currently working on and what is important to X work social aspects that as

i have the sense of closest to

and so by comparing this surveys that the beginning and at the end

we can determine

you know uh

whether they've improved or not

and

um

uh quite dramatically we see a lot of improvement uh a across

all the four different groups that we put

we put them into one also different uh

category

these are verified by user comments

um

when user for example

so that um

it actually succeeds in creating a workable allusion

but the remote T member is in the room

that overcomes the barrier

physical distance

and direct observation also showed that

in contrast to

the uh

the audio conferencing that they get been using before

is now allows rapid turn taking in conversation

um

following of what's going on the whiteboard and brainstorming

resolving issues uh at the meeting instead of waiting for the next time the person

uh a of is it

um

saving some trips

and also assisting not native

english speakers to understand what's have

uh and all of all of the

uh proxies were named

and given hats

and so they became

part of the T

these are not a new ideas um

in the early nineties

uh bill buxton then that be cell and a

at their hydra system

uh more recently paul of that the bed back um introduce the really cool uh

volumetric display

uh

a few ks

uh you and C

it's been doing some work on

and am at run X

uh and

there are tell a presence robots

and they've even a an to the

a boy to attend school even though he has some immune deficiency

that was reported by C N and back in january

and ieee spectrum and than your times last september both had cover articles

on these tell a presence robot

user comments in both of those articles feel pretty much the same things that we get also already observed so

eric oh itself from

ieee spectrum

says that

you know you participating is a row by you feel you get people's attention

and there's a better sense of being there

and

mike bells there

so you get the same kind of or interpersonal connection that you'd have

um

as if you were at the meeting

a

interestingly

even in the one you know and all those cases

in the new york times in and spectrum

uh

the device is for given names and they were given had

are many issues

uh

save these one i went down to visit any but

robotics in uh

uh in silicon valley and found that they had one of their humanoid robots

lashed to wall

after had punched a big hole

uh which are still visible in the wall

a next to it

so

um

there some safety issues with robots

wifi coverage you know what happens when robots from

walk out of

at of range or run out of power

um can make keep up with people who are walking

you know how do they open doors or get on elevator

you know you don't want people's hacking into your robot button taking it over

um and their social issues like well do you want them to

C better than humans here better than humans

um what height should they be at is socially acceptable to

to touch them and move them you know so there are many

issues um including um

how close should they come to looking

human

uh and maybe you've seen in ieee spectrum some of you should grows robots

these are quite fascinating

uh

and the not just wax models they actually move

i

i

oh

i

and so this is a gemini T K here is a

is professor have wreck sharp

at all berg university

and apparently according to ieee spectrum has wife says that

she prefers body number one

but she's suggest that we should always send body number two to the conferences and stuff

so don't be surprised if it future icassp cast you see you know some

some robots wandering around

so

so far i've talked

pretty much about

two ends of a spectrum and one approach we've had

uh these virtual

you know everybody

transports into to virtual space and on the other approach

you uh transport into a real space through your proxy

um um

in some sense the first is about virtual reality which was

uh

promulgated by are and land here and the early eighties

it's about embedding people into the computers world

um

the second one is about

ubiquitous computing promulgated by mark wise or in the late eighties

and so about embedding computation computational devices are computers

in are world

and so there's really um

uh uh these are really at two extremes and i think what's really interesting is is what could be in

the middle and the first time might became

aware of that space

was attending

uh a a talk at stanford in that in the early nineties

on um

uh V V are male virtual reality

markup language um it just been invented in the

the inventors there were

were discussing

the virtues of it in showing as um

how they had

um

made a model of the entire

university and they were showing is how they could

how we could um

navigate their way

through it

and so they wander down the halls of the university and then came to a door

of the conference room where we were sitting

and

i expected that they would just kind of open the door N

inside we would

you know we would see all of us sitting inside the room

um

and so i

got a nervous feeling about oh maybe behind me through the door will come some giant eyeball ball and then

there all be all these people behind a kind of looking at this

so i realise that there's is big gap between

uh

the virtual reality part and and and the physical reality

um and so that's an area that

uh

at scenario i think it'd be

uh need a lot more exploration

i'll just say briefly that we've been doing some some work in that area

um

we've called a parallel worlds that one point it's not overlaying real and virtual worlds on top of each other

and the people to sort of cross back and forth between one and the other so how might

people walk

through plaza for example

or tent

no go to a museum or real these are real places how mike the attendees these remotely

and

uh experienced um remotely and have also have the people in these real places experience the remote people as if

they were there

uh

i you know in the real space

so

we you know we'd had applications to attending weddings for example

or trade shows and conferences and going to poster sessions how might you have

a virtual person come to your poster and you could converse with that person as if they were there

so the a lot of interesting signal processing is use how do you instrument the space to capture the

the audit

tori pardon the visual part

um so you can represent them remotely

and more

a more difficult thing is actually how you

rent their remote people into a real space

so i

want to include

here

um i've time talked

uh

mostly from a technical point of view about a of communication

um here all just

talk about

uh this is societal need so

climate

energy and environment are are L

the big words of the day

uh i flew here

uh from seattle

it's five thousand miles

why other five thousand miles getting back and the amount of C O two i've released into the atmosphere

uh

is equivalent to all my local transportation

you know i've done this uh

just for this week i've used up my years quota of local transportation

uh

uh probably all of the you know many of you of also flown from overseas

um

collectively i would say we probably released into the atmosphere coming to this conference

about

a thousand times

of C O two

so

i don't think it's something we can afford really into the future

um in terms of productivity

you know i spent

three days getting here for for a meeting so that

ratio is not really good

um but even on a day to day

uh basis in at least in the us

people spend close to fifty minutes per day on average

uh

uh commuting

perhaps about that ten percent of their work

their work

um

information workers though spend about fifty six percent of their time in communicating with other people

um

sixty percent of their time feel that they could do there

job duties these just as well it's some remote location for example home so there

there are things we can do about that

and

uh

thomas friedman so the world is flat but it actually needs more flattening in today's economy we need to bring

people to jobs and jobs to people

in this is physical security

i i guess the volcano one iceland lent started to rub thing again a but a year ago

uh a little over your goal for period of three weeks

it's uh

cost the cancellation of a hundred thousand flights

stranded it eight million people across europe

uh many of you may have been among that group

uh

and uh cost the airline industry to one a half a billion euro

so uh the economic damage is real

um

you remember

uh i two thousand three was cancelled because of sars so

if you know if there a viral out breaks

yeah um they can shut down whole economies

um

uh uh not i mean is N

earthquakes

uh

you know these things can also have a big

uh

a big um

a a fact

um and terrorist threats right

to more traditional reasons for having uh tell a presence and numbers of

um communication have to do with

uh bringing families close together

and then um

pretty health care and education

uh around the world

so

um

i'll just uh

mentioned that

you know many of these

national academy of engineering grand challenges

um

from

uh

having to do with a the global climate

as well as um health care

security is are all addressed by and by

like tell a press

so in conclusion um

although

visual communication has not really changed much in the last eight years

the conditions are right do the convergence of

affordable technologies and high social need for

human communication to break through to new levels of them are given

and uh the signal processing community in a unique position to be able to address

uh the needed advances so

i think we should embrace the opportune

thank you

thank you and that's in the a uh it it so that uh an exciting and uh informative a talk

i I and the uh feel that we have deal

then it faded it can having you do they bring that to okay about new have a yeah yeah

how can you tell

yeah yeah yeah yeah

she

anyway we have uh an opportunity or some uh a question in the floor

are uh

oh we have one at the back if you would got the mike lee

well i'm and you think of this end so

got

yeah order

you immersive it

in case

to touch and what else

i in greater converter

um feeling

so probably very important but uh it's the least well understood

there's some uh articles in the uh verse of communication issue about have tex

um

uh

you know i i would say it's least well understood at this point O

there's a lot of work to be done

so uh

basically when i was a a a a it be lot asking the little guys so

oh

looking at a them was station i ask them one question

what about the band

i mean

you need a lot of they got to so and so one

and so

there are saying there are using a special line

that's right

so

but everybody can these a special lines so

i mean

oh do you see that like a like that that's like this amount model of a got

or the word you don't on so you're still basically but still

you don't so a large amount of data

and you need some basic the but well um

bandwidth with capacities going to vary very over a very wide range uh

and

so

you can do various

uh i think to address lower band with

so the avatar can act as an example where you just controlling

um

you're parametric avatar sending a series of parameters over

over the line so it's much more like a

uh

you know

a again

um

and you still get you still can get someone out of a margin and those

situation so i think

uh

the amount of realism we'll just depend on how much bandwidth with you have available to you

um and their strategies for

are going down to lower band

that's sufficient

yeah well

how much bandwidth with you need for different amounts of perception is

uh is an area first

a have that one should be taken this but more of line we have to for one last question

oh okay okay i dean man

one can not really right i

you human you mean

would you be able to you come and that share you

um

so no matter what technology we use

there will be things that we just cannot not

uh replace

with human to human communication is that where you ask

these

so

you know i i i suppose supposed just probably one of the good examples that's that difficult to do right

now

i

i really can't say uh

you do the future what

whether smell and

other things

you know

i mean i suppose eventually you just the jack to the matrix and put something the back of your brain

and then

uh

replace everything

uh you become a a brain in of that

at that point

uh but

um

what you lose

philosophers last say we're all or or all ready brains

sorry that's point two

with

hence

okay we should uh i think uh it again and used to call thing to do

X

really