come a three of the and
i hope we're or you had good sleep
after is to is for circular
so back to this go i i i hope you enjoy it
and do you your the the music there
so why won't read or the names some the links a to repaid use but in case you want to
have
with brave guys and are also uh invited to your garden party
there will be doing so the on the uh on the conference web page
i was especially a it is harm core just band but
that was playing in one of the rooms that was re reexamine
people from uh
ordinary secondary school mode even music second can very group
so uh a good and you can you can visit the
their rip sorry
or just type harm core prague to do ruin
you will get it
uh there is one a
warm
mixed white please
oh yes
okay so one one practical remark remote the restaurant
we had the sum compliance like to you in the plumbing you
uh a short clues from you use the not be type of food i won't
so uh the the the solution is to ask or or do not a a uh uh the information desk
and she will be happy to a device you a rest around in be vicinity of these place
or or or or or or the there
no uh uh this at all
or the practical in four
and we will have for an invitation a two us very here you are a conference
number you are most probably only is from a some you know of pixels
we morning
this morning i would like doing that are used to you
that two thousand and two L international conference on merging signal processing applications
or use by twenty two well
my name is by most problem in is and i am the general so obvious park where B to out
use but where and good where all these marked it works wrong
it is a full quorum forums organise by the signal processing society
it is expected to be the third major problem for some of the some side view
but together with i ask for nice here
but because it used
you is not dressed primarily to practising engineers
of course everybody is welcome wall
i was uh going to be a course in practice problem
something else that makes use by be my when i see here
use that to do will be held every year here at the same place and time
you "'cause" been plan one for one with a consumer electronics show
so give her beam last very guns
in early january
when we say practising and you here
my the and some more who has a master's degree
in general signal processing
i i is or the presentations and expected to be applications already it
and more can thing as a very clear to
to first a good participation from mean dusting use by we also include but as patients without that paper
of course papers first can be some you two
and that these papers would be are in the R you i drip we explore
but we're one from is mark expect to be a if you're at the girl
in another major component of the conference
is that the most rooms
what we call show and they are here by cast
and in have you sure we will have to house
plan i to present creations
and by discussions
you can find more maybe from in the call for papers that we have a the information of both here
by cash
or blues and see me
so here are all invited you to submit papers for is is where well
one more thing
the corn firms "'cause" not go about to the local yeah
we can you to submit your idea for a ago
and that there will be a three hundred the world
for the log which or
so i hope to see you all next gen was very guess it would be fine
thank you
thank you nose
and it's time
i
i'm i'm to start the
brisbane is primary
or speaker will be a the channel
and the like colleague your
and chambers
technical program chair
we introduce or speaker
from a
good morning uh light is in uh uh and from then i'd like to uh gain by repeating what ones
as Z and for that was of you who attended the uh check the evening uh yes to the evening
it was a about very special
that a page in and the uh music uh uh well as a a special uh experience that with
thank you very much to the local organisers work i in that
a bit by great applied to to uh
introduce a a a it it
doubt this morning
just a battery quick look at is uh uh uh C B
or by a will show you that uh
it's a sort of battered simple
whose who or but uh
some of the leading
academic
and industrial
unit
in the U
he V C he's degree in the nineteen eighty two
from
print then
but really
and than food
following that C work
at
at and T bell lab
multi he'll you to see
the about
how out so
with that so and uh in california
and since nine
ninety i
he's work that uh the microsoft
with a
and uh in redmond washington U N
where he heads the communications and collaborative systems systems research
philip has being a wheel
so than for the ieee signal processing society
most recently each yeah the multimedia technical committee
and old so it's a so on the signal processing society by committee
each distinguish is a a with it B and twelve
a side please
piper what in ninety ninety three
and more recently in two thousand and seven
even a the best paper award from the ieee transactions on multimedia
so thank you again guy in to fit it for agreeing to uh make this presentation
i i and i'm sure the whole O D and is looking forward to is
to all
in that it communication
it
you very much for the introduction it's a great honour to be on the stage especially with fred jelinek over
here he was an inspiration to me early in the career
and thanks all of you for coming uh to the plan area to at and after
have to the banquet i don't think i ever be in
at M plan a right after the bank so this a new experience for me i'm kind of looking for
to seeing what these plan is actually like
so i'll be talking about immersive of communication
um
there won't be any equations
in my slides
no signal processing people sometimes don't really understand things and less there
this math behind it so i'll give you some references
uh
signal processing magazine had the special issue in numbers of communication that came that in january you can look at
that
and just last week a bunch of us submitted a paper to the ieee proceedings as part of their hundred
anniversary um both of these are numbers of communication
so the last
second
paper here um the fines of communication to be
exchanging natural social signals with remote people
as in a face to face meeting in a weighted suspense
disbelief
uh in being there
this is uh
not a new idea by any means the
telephone was invented a hundred and thirty five years ago and that was the first grade breakthrough an immersive communication
it wasn't long after the telephone was invented a here's a cartoon that came out oh
three years later
uh that people were starting to look at much more immersive scenarios
so this is a
by george them are yeah a
um
the
caption of the bottom
read something like uh
uh as as these uh
parents are here in london in
uh uh uh watching their daughter in ceylon
play badminton
and
father says speech
come over here i want to whisper
she comes over
that's yes
pop here
uh
who who's that charming
uh lady
playing by charlie side
uses a
um she's just come over from london um i'll introduce you after the game
so we don't really have this kind of immersive of communication system even today but perhaps something that comes fairly
close
are tell a present systems
tell presence uh as defined by
the industry conference
newsletter wayne house review is
a video conferencing experience that creates the illusion that the remote participants are in the same room with you
so probably the quickest way to get a sense of what that means today is to take a look at
a thirty seconds just go
commercial
yeah
hmmm hmmm
hmmm
ooh
i
i
i
i
uh
so so one can be uh
so here you seen
high definition video conferencing so compelling that one participant has forgotten that his counterpart is
is remote
so this just go tell the present system uh
uh and others
uh like them H B halo another others
um off offer
high definition audio and video lots of bandwidth
um
to try to create this solution of being in the same room
um
is this a breakthrough in a of communication or is just a lot of uh
high definition televisions and a lot of bandwidth well
in a sense they're both
okay so
i contend that these are
bridge
uh to the future and we're about to see um
uh a rapid progress in this area of for the next few years
so put that in the context as take a look at
um
a brief history of television
television
was invented in its current form and
nineteen
twenty
six
uh
as a counterpart to the telephone
so here you see one of the first television set
eighteen T uh bell labs
uh
next to a telephone
because the television is meant to communicate
as a visual part of the telephone
so here it is
eighteen T president walter gifford
uh
in a you are K at bell labs talking to herbert hoover then secretary of commerce and washington D C
in nineteen twenty seven
so the first
distance television call
uh and shortly thereafter the television became
the broadcast medium that we know today whereas
video telephony
um took another forty years to become the eighteen T picture phone
uh
the picture phone was a
a stunning
technical success but also a stunning
commercial failure
so by nine teen seventy nine when i was
in intern at bell labs
down the hall from me on the desk of my lab director bob lucky was the
last remaining
working
picture phone in the world
but would often complain that nobody ever called them on it
did a forty years after that
uh experience is actually very similar
uh
except that now these little rectangular images are on general purpose computers
um
or or on your phone now
um but pretty much the experience the same there small little rectangular images of video
video conferencing
similarly
um has been
the same
uh
the for the last forty years in some sense here's the bell labs video conferencing system in nineteen sixty seven
you can see the round tape around
tables and multiple monitors cameras on top of
each monitor or the
data monitor
um which looks very similar to the
you know
the recent tell presents as
today
so
why do why contend that
there's about to be a series of rapid breakthroughs in immersive communication when
the last few years not much as happened in visual communication well
several reasons
the first is the internet
okay so the internet
has caused a divorce between
the format
of the content and then medium over which it is scary
so when the past telephone calls were carried over telephone networks
television over television networks are over
radio and so forth
today day all those are carried over the internet so there's a big possibility now
of
inventing arbitrary formats and they'll all be carried over in
uh the second is of course the cost of computation band with an resolution has a in dropping exponentially for
so long
and that there essentially free now compared to what they work twenty years ago
and the third reason is the technology
evolves faster than biology
okay so
they will become
bill be a threshold at which
the number of bits per second that were able to capture transmit and render
uh sir is what we can actually pass through the neural cut set around our bodies okay so at that
point
um
we should have freedom to do whatever
we like
and
the question is
what do we want
the future of you min
communication to look like at that
a
so to the extent that hollywood is the
uh keeper of our collective dreams
you know
be answer that hollywood
would give is that they want
communication to be immersive whether it's like
uh
the uh a hole attack in star trek or the jedi council meetings and star wars or
the matrix in the matrix or
um
in in avatar
um
communication should be mercer
however
not all communication will be numbers
okay we will continue to send S M S messages
uh and the reasons are that there you know there are some reason such as privacy that we we we
don't want to send everything
about about so this is anticipated you can buy
the jetsons and nineteen sixty two
so here's
change S and putting on her vanity filter
before she makes an early morning video call to her friend gloria
unfortunately flores
on vanity filter has an embarrassing
mel function
then there's one of my famous my favourite cartoons
uh
it's this
uh
from pete steiner in the new york in nineteen ninety three
i'm the internet
nobody knows you're a dog
no know that's
still pretty much true today there reasons why uh we don't want to review reveal everything about ourselves
and yet for the more complex
human interactions
a more creative things that we do with each other
um we need more margin
because we're social animals we have of all to work with each other best face to face
so how we position ourselves relative to each other
no where we sit
what how we just your
are are gaze
um awareness of all of those things they're all very important
and these have been studied um
but social psychologists and
and others engineers who build system
for example uh the work by no i and and can
and them the berkeley multi view project it build a system that
uh preserves i gaze
teleconferencing and they've shown that
trust improves
uh with correct i gaze
so
i gaze is bin
uh
a subject a video conferencing for
a long time there many solutions
have meters cameras behind screens
you interpolation and and many others here's an example of view interpolation where there's an upper camera
you when a lower camera view one they can be interpolated to get
um
better i gave
not only I gaze it's important it's uh i
reference and gesture
so here some examples from the early nineties
tongue and min "'em" and showed it chi ninety one
a a system that
had had a shadow of the remote collaborator behind the the collaboration surface
and a year later
you she had a same is clear board
um where the actual video of the
of the remote collaborator
was projected onto the
on the for the shared sir
here's an example uh that's
um
more recent P H P connect board
which
uh
is an update on it she's clear board
in that
perfect i gaze is established
by tracking
the position of the
observers had as well as the eyes of the
uh a of the
the uh a person in the video
so that
um um
the eyes of the on the video are always on the um
path
between the observer and the camera which is located time
the screen
at there many other
cues the uh days and gesture
um
both auditory and visual starting perhaps with
peripheral awareness of what's going on in the room
and consistency between the local space in the remote's space
so if you look back to at
the tell present systems today
you'll see that
if these preserve those first two characteristic
so
per full awareness is provided by very large
tell it
okay so we know what's going on and that remote room
um
consistency between the local room and the remote room as
is preserved by
uh making the
the desks look the same painting the walls to be a the same colour and all the different room so
uh
no difference between the back
of of of one room and the back of another
um
so
if you talk to the H P
guys
um they'll tell you that the halo system which was one of the first
which was probably be first um system in this category to come out
that was the collaboration between H P and
the film the hollywood studio dreamworks
and
you know how we what is is set you know be know how to
they know how to um
uh
use "'em" uh you um use illusion just suspend spend one's disbelief in actually being there
but there are many other
immersive of cues uh spatial cues
um for example uh distance
uh in audio is is uh indicated by
a combination of relative loudness
uh direct reflected energy and direct reverberant energy ratios
as we can see and in these uh these clip
oh yeah it's
right and i seven is does two yeah
so that one's
presumably further away than this one
oh yeah face
a in the next set is
does a to that was is probably adjusting the volume
but you can
uh i'm
and of course direction is is given by in true inter aural temporal role and interaural intensity differences between you
two years
um
and on the visual side
um there are many many is
cues on give you some examples of those
relative size and perspective combined to give you a sense of
distance
and um also absolute size
this of the same cues it are used in the famous
aims room illusion
uh occlusion lighting and shadow are also very important first giving a sense of
at distance
um
in augmented reality it's very important to be able to paint you objects into a scenes such the
occlude the background and yet uh do not occlude the foreground
uh lighting in shadow were very important not only for real is some but also
for um actual depth perception so if you look at this picture of the foot prints in the and
um
lighting and shot oh
give you an impression of one
uh a print being impressed into the the and the other one
uh
being above above the sand
so
a show of hands how many of you
perceive the top foot prints to be the one that's impressed into the same
into into the same
okay
and how many the bottom
okay uh
maybe one percent
think the bottom a ninety nine the top
um now keep your eyes fixed on that
"'kay" don't blink because if you blink you probably change your mine
okay so
even even shot all uh a has something to do with depth perception
uh focus as well so use a very large
city D china
uh but if we change the a focus
the foreground background it suddenly shrinks down to look something like like a little model that might be
uh
next year
model trains set
of course there the the uh
very strong cues of skates star got be and motion parallax
so some of you may be able to cross your eyes and fuse these images
and you get a very strong sense
uh of of dat
um
and all show an example of motion parallax later on
but
so are these cues uh important for communication
that's the question and i contend that the R
um clearly are current communication systems do not convey this information
and clearly they are not
satisfactory are are are not uh
um
no the can't fulfil every need for communication that we have we continue to meet in person and whenever we
are able
we continue to travel to work
we continue to
attend parties and
weightings in person we continue to come to meetings like this you know why don't we just
email all are talks
our slides to each other
and just a stay at home well because there's something
valuable about we feel about being here in person
um and our current communication systems don't convey that
but i
believe that um the tools of signal processing will help us
um bridge that yeah
so
if we're of a physicist you might take a different approach
case of this is is uh i've read have uh actually succeeded in transporting matter
uh a small molecule
um over us
significant distance
um and this is how it would work if you were to use it for uh
uh transporting you know somebody
so you me too
a different part the world i would step into a matter transporter
i would be frozen down apps the zero
i be turned into a little break of
bowes einstein compensate inside
in the process of schooling down i would it be light
light would be transmitted to another place
would shine on another little break
of bowes einstein kind and set
and then
my original state would be reproduced
of course if i
don't
plead lee cool down to zero i may leave some state behind hind uh with unforeseen consequences
it's also sobering to think about
uh what quantisation and data compression would do on um on the way
rate distortion optimized mpeg four thousand
um
but at microsoft research for mostly computer scientists and signal processing people so
we take a different approach which is
we do the
the um transportation uh uh matter virtually rather than physically
um and we achieve that through court a transformation
so if you imagine
a person in a real space over here
um
he's got some court
system in as local space and we just do a coordinate transformation
uh of that over into to some
no map that coordinate system into the coordinate system of some other states
an if we do that for several people then we get them into the same
the same
so depending on whether the space that there being transported into
is a virtual or real
um it leads to two different approaches
uh i'll
um
talked about both of these you might sort of a
identify the first one with
the movie the matrix if you're familiar with that
everybody goes into a a virtual world
um
the second approach could be identified with the movie avatar in which you send
a physical representative of yourself into are real physical
space
so i'll talk about
um
each of these
starting with the first one
so the virtual or
the matrix movie approach
um can be applied to what we call fully distributed meetings
so we have people who are
and completely separate places around the world maybe your
you when your colleagues are writing a paper together and you're sitting in your office is
different universities
and you wanna talk to each other and the ideas to make it seem as if you're sitting around a
see the around the same
table
um talking to each other
so the way we are doing that is by
capturing geometry of each person
doing a court transformation to put them all into a common scene
a life size scene
and then you give each person a window on that's scene it's basically there display
and uh each person gets a
you know i
own personal view of that scene
um
and uh people interact through the
in this way preserving i gaze gesture direction and so forth
but doesn't matter what that the projection is
on on uh
a flat
surface or
a two a multiple monitor situation or maybe even a curve
screen so
well we've done is build a a
a
wide field of view curved screen
um and from a particular point of view make it appear is if
uh uh vocal task is extended
uh in space
and then when we capture
people's geometry using a depth can where that situated on top of the
the screen
then we can position those people around in space as if they're having
a meeting so if you've never seen
um
what images from a that's camera look like
this will give you
you give you sense use up
here's the still frame from a
adept camera
see you can
render or uh render person from an arbitrary view
okay so basically that's what we do capture
geometry and texture
of um
of each participant
place them in there's
virtual world
and then um
what them collaborate so here's
here's an example of what you might see if you were telling one of these meetings
your
collaborator sitting across the table from you
uh
maybe a on the wall on the right there some artwork it's glowing to reflect the mood of the conversation
in the room
on the back wall perhaps as a large active computer display that can be used for the
uh for the meeting
um a on the left wall here might be a window out on to
the uh
skyline of singapore in this case or wherever L C may wanna hold your meeting
as other participants join the meeting
um the people in the meeting can move around the table to make room
people can bring in their own data
other people may join the meeting and they come in from ordinary web cams or so so they don't actually
have their dept associate with them but they can be represented as
as a virtual
displays here
um
data such as this what's showing in the centre of the table here can
float above the
the table and be manipulated and so forth so
these are some of the things you can do
um
uh uh in this uh
this scenario
so that
image of course um was
i i didn't show was taken from one particular static point of view but as people move around
uh
the image is move so
um here's a video that shows
uh this the fact of motion parallax
so by tracking the position of the camera in this case
um you can change what is poor
presented on and ordinary
um
display
so that the peers as if there's some that's find the display
oh this is just using motion parallax to give that sense
depth
the same can be done with a audio actually
okay so you can
uh
do you head tracking
by
you know visually
find out where where persons head is
point
and then um
a table look up to find out
the relationship between that person's head and uh
and this
and uh uh a virtual source
get the H R T S for that and find out what's the signals
that you want to each year
similarly you know what the relationship of the head is to each of these loudspeakers
and then you can
or to
transfer matrix invert inverted
do some crosstalk cancellation
and
produce at these loudspeakers
signals that depend on
where your head is
at
uh so that the
the perceived
location of the virtual source stays put
so it doesn't drift into one of the that the lots and the one of the speakers as you move
towards one speaker
apple
so we've
done experiments on different form factors as well
this is an example
of the
of a wall
where uh we've projected onto a whole a graphics screen the same kind of thing you might see in a
high tech show
you know that store
window
um
and
we try to make its
a appear
through motion parallax and stereo stereo parallax
um that the person is standing in the same room with you
so
i want this
take a
five minute interlude here to talk about that the cameras because that's cameras at had a recent breakthrough
in
um
in cost and performance
so there are new sensor that's really of air are available uh very red to everyone here just like a
uh a a webcam cam might be
so let me tell you about them
um
you might have heard about the can act for X box three sixty
it's a device
like this
um
that makes
use the game player
uh the controller
and what it was introduced in october
last year
it's sold
eight million units in the next sixty days making it the
uh a fast so link consumer electronics device in history
reading it some position and the guinness book
world records
a
oh
hmmm
hmmm
and
a so that's enough for that
uh
basically it's a
it's a the camera
okay
so at every
pixel it produces a their distance from the camera to something in the scene so it enables you to
uh
with do uh
some skeleton analysis for example
of where what people are doing enable you and labeling them to control
the games
so under the covers
it's uh
it's this kind of device we have
um a regular
rgb camera
we have an infrared camera
uh
that's pretty much just like the rgb camera except that has a filtered front of it
ooh allowing to see only infrared
there's a infrared projector that projects
and they'll pattern on the scene
and by correlating that known pattern with the observed infrared image um
you get a disparity at each pixel in the data
also in this unit is a a
uh design by even to even on you have who's
possibly here
uh
and
uh there's a motor or dsp unit U is P
uh
use P port so the motor tips the
the unit up and down
and all of this comes for a hundred fifty dollars
so
um
it's not surprising that within two weeks after it was a
it was released
it was hacked into
and people it up to their P C's
and did all sorts of crazy things
with it
um there's a web here
devoted to connect tax
and
by
wow one month after release
uh
uh there were about ninety projects
um um on that site
uh
in three months there work twenty four pages of
the projects uh yeah as of yesterday there were forty seven pages the projects and these
these are all crazy things like a um
um for
robot but
uh navigation to um
to seeing for the blind
two
greeting
storefront windows that react to to people wandering in front of them uh and so forth and so on so
if you're interested um
in using this for some of your signal processing i invite you to jump in
uh
uh there is an open source uh driver or but
it doesn't have any of the scale to tracking error or or eve ons uh
uh
mike array processing
uh if you want that
you can wait if few
weeks uh
i believe will be having a
um non commercial
uh S T K available for all of you to use uh
free of charge
so
i talked about
uh
talked about how we were using depth cameras
uh for
burst of communication here's a here's an application that's very closely related
and it will be actually released two
um X box live gold subscribers
um
uh sometime this spring
so i thought i'd show that to you
it's called avatar connect
uh rather than showing you an advertisement i've recorded my own video here
uh this is corey she's talking to me on her
right and to sash on her left
her her images
the image of her avatars
uh in this picture in picture thing
i
i
no who can see here avatars a her emotions
so in addition to
uh
in addition to um still to tracking of arms we have to do facial expression tracking so
you know who's uh
uh uh when people are talking
you have to figure out where they're looking so you can at intimate their avatars and so forth
and you can imagine that um
uh
E there uh many other
uh backgrounds you can use uh for this
for this uh
avatar can
thing
so you can do a uh
a uh
talk show for example with your friends uploaded to you to and so forth
so you can imagine that as
uh the avatars get more and more
um
sophisticated
eventually we may move into any era where inverse of communication is something like a mass of multiplier video game
where
you know you enter you enter into this world
um
and you have terms are controlled by your actual motions your expressions
and you're
oh case alone let me
um
return to the second
approach
um this is this
real or so called avatar like
avatar movie approach
um where you send a physical representative of yourself
two a real space
so this works well for so called and satellite meetings where there
several people around you a real space like a real conference room
and there are some remote attendees
um called satellites lights and they sense some
proxy in the me into the meeting
so we call these embodied social proxies
here's an example of an early
uh
early device
uh
a regular L C D used for the
the face
uh
fisheye cameras and pan tilt zoom cameras used for eyes
phone for mouth
and years
all of this is put on a cart
is a computer
wifi a battery backup and so forth so they can that end
meetings for you
and these are
uh our current meeting
we have four of these cards in this particular room
uh one here one here one here and one out of sight
we have collaborators
um
around the world john it's and silicon valley and over here's and cambridge
england
the you know the meeting room itself is in is in redmond washington near seattle
um
some of these are
um on
on cards that have to be wheeled in
this one is actually robotic it walked through the door on its own
the view from these carts of the room
look like this
okay so you get a wide field of view
peripheral awareness of what's in the room
and if you want to see it some detail like what's a a on the
screen
um
you click on the screen the pan tilt zoom camera goes over there and gives you a high resolution version
so we use these not only yeah for our own meetings but we've done studies on them across real
groups
that are
using them so we deployed then in for different microsoft product development teams where one of the
members of the team was remote
we the plate them for an eight week period
uh
conducted interviews and surveys at the beginning and at the end to determined
um
whether people use these things
uh
and by
asking questions on the surveys that are based on the seven point likert scale
yeah um
we determine meeting effectiveness awareness of social
aspects by asking questions such as
i think
X has a good sense
of my reactions
or
for awareness
i'm where
i'm aware of what X is currently working on and what is important to X work social aspects that as
i have the sense of closest to
and so by comparing this surveys that the beginning and at the end
we can determine
you know uh
whether they've improved or not
and
um
uh quite dramatically we see a lot of improvement uh a across
all the four different groups that we put
we put them into one also different uh
category
these are verified by user comments
um
when user for example
so that um
it actually succeeds in creating a workable allusion
but the remote T member is in the room
that overcomes the barrier
physical distance
and direct observation also showed that
in contrast to
the uh
the audio conferencing that they get been using before
is now allows rapid turn taking in conversation
um
following of what's going on the whiteboard and brainstorming
resolving issues uh at the meeting instead of waiting for the next time the person
uh a of is it
um
saving some trips
and also assisting not native
english speakers to understand what's have
uh and all of all of the
uh proxies were named
and given hats
and so they became
part of the T
these are not a new ideas um
in the early nineties
uh bill buxton then that be cell and a
at their hydra system
uh more recently paul of that the bed back um introduce the really cool uh
volumetric display
uh
a few ks
uh you and C
it's been doing some work on
and am at run X
uh and
there are tell a presence robots
and they've even a an to the
a boy to attend school even though he has some immune deficiency
that was reported by C N and back in january
and ieee spectrum and than your times last september both had cover articles
on these tell a presence robot
user comments in both of those articles feel pretty much the same things that we get also already observed so
eric oh itself from
ieee spectrum
says that
you know you participating is a row by you feel you get people's attention
and there's a better sense of being there
and
mike bells there
so you get the same kind of or interpersonal connection that you'd have
um
as if you were at the meeting
a
interestingly
even in the one you know and all those cases
in the new york times in and spectrum
uh
the device is for given names and they were given had
are many issues
uh
save these one i went down to visit any but
robotics in uh
uh in silicon valley and found that they had one of their humanoid robots
lashed to wall
after had punched a big hole
uh which are still visible in the wall
a next to it
so
um
there some safety issues with robots
wifi coverage you know what happens when robots from
walk out of
at of range or run out of power
um can make keep up with people who are walking
you know how do they open doors or get on elevator
you know you don't want people's hacking into your robot button taking it over
um and their social issues like well do you want them to
C better than humans here better than humans
um what height should they be at is socially acceptable to
to touch them and move them you know so there are many
issues um including um
how close should they come to looking
human
uh and maybe you've seen in ieee spectrum some of you should grows robots
these are quite fascinating
uh
and the not just wax models they actually move
i
i
oh
i
and so this is a gemini T K here is a
is professor have wreck sharp
at all berg university
and apparently according to ieee spectrum has wife says that
she prefers body number one
but she's suggest that we should always send body number two to the conferences and stuff
so don't be surprised if it future icassp cast you see you know some
some robots wandering around
so
so far i've talked
pretty much about
two ends of a spectrum and one approach we've had
uh these virtual
you know everybody
transports into to virtual space and on the other approach
you uh transport into a real space through your proxy
um um
in some sense the first is about virtual reality which was
uh
promulgated by are and land here and the early eighties
it's about embedding people into the computers world
um
the second one is about
ubiquitous computing promulgated by mark wise or in the late eighties
and so about embedding computation computational devices are computers
in are world
and so there's really um
uh uh these are really at two extremes and i think what's really interesting is is what could be in
the middle and the first time might became
aware of that space
was attending
uh a a talk at stanford in that in the early nineties
on um
uh V V are male virtual reality
markup language um it just been invented in the
the inventors there were
were discussing
the virtues of it in showing as um
how they had
um
made a model of the entire
university and they were showing is how they could
how we could um
navigate their way
through it
and so they wander down the halls of the university and then came to a door
of the conference room where we were sitting
and
i expected that they would just kind of open the door N
inside we would
you know we would see all of us sitting inside the room
um
and so i
got a nervous feeling about oh maybe behind me through the door will come some giant eyeball ball and then
there all be all these people behind a kind of looking at this
so i realise that there's is big gap between
uh
the virtual reality part and and and the physical reality
um and so that's an area that
uh
at scenario i think it'd be
uh need a lot more exploration
i'll just say briefly that we've been doing some some work in that area
um
we've called a parallel worlds that one point it's not overlaying real and virtual worlds on top of each other
and the people to sort of cross back and forth between one and the other so how might
people walk
through plaza for example
or tent
no go to a museum or real these are real places how mike the attendees these remotely
and
uh experienced um remotely and have also have the people in these real places experience the remote people as if
they were there
uh
i you know in the real space
so
we you know we'd had applications to attending weddings for example
or trade shows and conferences and going to poster sessions how might you have
a virtual person come to your poster and you could converse with that person as if they were there
so the a lot of interesting signal processing is use how do you instrument the space to capture the
the audit
tori pardon the visual part
um so you can represent them remotely
and more
a more difficult thing is actually how you
rent their remote people into a real space
so i
want to include
here
um i've time talked
uh
mostly from a technical point of view about a of communication
um here all just
talk about
uh this is societal need so
climate
energy and environment are are L
the big words of the day
uh i flew here
uh from seattle
it's five thousand miles
why other five thousand miles getting back and the amount of C O two i've released into the atmosphere
uh
is equivalent to all my local transportation
you know i've done this uh
just for this week i've used up my years quota of local transportation
uh
uh probably all of the you know many of you of also flown from overseas
um
collectively i would say we probably released into the atmosphere coming to this conference
about
a thousand times
of C O two
so
i don't think it's something we can afford really into the future
um in terms of productivity
you know i spent
three days getting here for for a meeting so that
ratio is not really good
um but even on a day to day
uh basis in at least in the us
people spend close to fifty minutes per day on average
uh
uh commuting
perhaps about that ten percent of their work
their work
um
information workers though spend about fifty six percent of their time in communicating with other people
um
sixty percent of their time feel that they could do there
job duties these just as well it's some remote location for example home so there
there are things we can do about that
and
uh
thomas friedman so the world is flat but it actually needs more flattening in today's economy we need to bring
people to jobs and jobs to people
in this is physical security
i i guess the volcano one iceland lent started to rub thing again a but a year ago
uh a little over your goal for period of three weeks
it's uh
cost the cancellation of a hundred thousand flights
stranded it eight million people across europe
uh many of you may have been among that group
uh
and uh cost the airline industry to one a half a billion euro
so uh the economic damage is real
um
you remember
uh i two thousand three was cancelled because of sars so
if you know if there a viral out breaks
yeah um they can shut down whole economies
um
uh uh not i mean is N
earthquakes
uh
you know these things can also have a big
uh
a big um
a a fact
um and terrorist threats right
to more traditional reasons for having uh tell a presence and numbers of
um communication have to do with
uh bringing families close together
and then um
pretty health care and education
uh around the world
so
um
i'll just uh
mentioned that
you know many of these
national academy of engineering grand challenges
um
from
uh
having to do with a the global climate
as well as um health care
security is are all addressed by and by
like tell a press
so in conclusion um
although
visual communication has not really changed much in the last eight years
the conditions are right do the convergence of
affordable technologies and high social need for
human communication to break through to new levels of them are given
and uh the signal processing community in a unique position to be able to address
uh the needed advances so
i think we should embrace the opportune
thank you
thank you and that's in the a uh it it so that uh an exciting and uh informative a talk
i I and the uh feel that we have deal
then it faded it can having you do they bring that to okay about new have a yeah yeah
how can you tell
yeah yeah yeah yeah
she
anyway we have uh an opportunity or some uh a question in the floor
are uh
oh we have one at the back if you would got the mike lee
well i'm and you think of this end so
got
yeah order
you immersive it
in case
to touch and what else
i in greater converter
um feeling
so probably very important but uh it's the least well understood
there's some uh articles in the uh verse of communication issue about have tex
um
uh
you know i i would say it's least well understood at this point O
there's a lot of work to be done
so uh
basically when i was a a a a it be lot asking the little guys so
oh
looking at a them was station i ask them one question
what about the band
i mean
you need a lot of they got to so and so one
and so
there are saying there are using a special line
that's right
so
but everybody can these a special lines so
i mean
oh do you see that like a like that that's like this amount model of a got
or the word you don't on so you're still basically but still
you don't so a large amount of data
and you need some basic the but well um
bandwidth with capacities going to vary very over a very wide range uh
and
so
you can do various
uh i think to address lower band with
so the avatar can act as an example where you just controlling
um
you're parametric avatar sending a series of parameters over
over the line so it's much more like a
uh
you know
a again
um
and you still get you still can get someone out of a margin and those
situation so i think
uh
the amount of realism we'll just depend on how much bandwidth with you have available to you
um and their strategies for
are going down to lower band
that's sufficient
yeah well
how much bandwidth with you need for different amounts of perception is
uh is an area first
a have that one should be taken this but more of line we have to for one last question
oh okay okay i dean man
one can not really right i
you human you mean
would you be able to you come and that share you
um
so no matter what technology we use
there will be things that we just cannot not
uh replace
with human to human communication is that where you ask
these
so
you know i i i suppose supposed just probably one of the good examples that's that difficult to do right
now
i
i really can't say uh
you do the future what
whether smell and
other things
you know
i mean i suppose eventually you just the jack to the matrix and put something the back of your brain
and then
uh
replace everything
uh you become a a brain in of that
at that point
uh but
um
what you lose
philosophers last say we're all or or all ready brains
sorry that's point two
with
hence
okay we should uh i think uh it again and used to call thing to do
X
really