Přepis řeči - Directions in Immersive Communication

0:00:12	come a three of the and
0:00:16	i hope we're or you had good sleep
0:00:18	after is to is for circular
0:00:20	so back to this go i i i hope you enjoy it
0:00:24	and do you your the the music there
0:00:27	so why won't read or the names some the links a to repaid use but in case you want to
0:00:32	have
0:00:33	with brave guys and are also uh invited to your garden party
0:00:37	there will be doing so the on the uh on the conference web page
0:00:42	i was especially a it is harm core just band but
0:00:45	that was playing in one of the rooms that was re reexamine
0:00:48	people from uh
0:00:49	ordinary secondary school mode even music second can very group
0:00:53	so uh a good and you can you can visit the
0:00:56	their rip sorry
0:00:57	or just type harm core prague to do ruin
0:01:01	you will get it
0:01:02	uh there is one a
0:01:05	warm
0:01:07	mixed white please
0:01:13	oh yes
0:01:14	okay so one one practical remark remote the restaurant
0:01:18	we had the sum compliance like to you in the plumbing you
0:01:22	uh a short clues from you use the not be type of food i won't
0:01:26	so uh the the the solution is to ask or or do not a a uh uh the information desk
0:01:32	and she will be happy to a device you a rest around in be vicinity of these place
0:01:37	or or or or or or the there
0:01:40	no uh uh this at all
0:01:42	or the practical in four
0:01:44	and we will have for an invitation a two us very here you are a conference
0:01:49	number you are most probably only is from a some you know of pixels
0:01:58	we morning
0:02:00	this morning i would like doing that are used to you
0:02:02	that two thousand and two L international conference on merging signal processing applications
0:02:09	or use by twenty two well
0:02:12	my name is by most problem in is and i am the general so obvious park where B to out
0:02:18	use but where and good where all these marked it works wrong
0:02:21	it is a full quorum forums organise by the signal processing society
0:02:26	it is expected to be the third major problem for some of the some side view
0:02:30	but together with i ask for nice here
0:02:33	but because it used
0:02:35	you is not dressed primarily to practising engineers
0:02:39	of course everybody is welcome wall
0:02:42	i was uh going to be a course in practice problem
0:02:45	something else that makes use by be my when i see here
0:02:49	use that to do will be held every year here at the same place and time
0:02:53	you "'cause" been plan one for one with a consumer electronics show
0:02:58	so give her beam last very guns
0:03:00	in early january
0:03:03	when we say practising and you here
0:03:06	my the and some more who has a master's degree
0:03:09	in general signal processing
0:03:11	i i is or the presentations and expected to be applications already it
0:03:16	and more can thing as a very clear to
0:03:19	to first a good participation from mean dusting use by we also include but as patients without that paper
0:03:27	of course papers first can be some you two
0:03:29	and that these papers would be are in the R you i drip we explore
0:03:33	but we're one from is mark expect to be a if you're at the girl
0:03:38	in another major component of the conference
0:03:40	is that the most rooms
0:03:42	what we call show and they are here by cast
0:03:45	and in have you sure we will have to house
0:03:48	plan i to present creations
0:03:50	and by discussions
0:03:52	you can find more maybe from in the call for papers that we have a the information of both here
0:03:57	by cash
0:03:58	or blues and see me
0:04:02	so here are all invited you to submit papers for is is where well
0:04:06	one more thing
0:04:07	the corn firms "'cause" not go about to the local yeah
0:04:10	we can you to submit your idea for a ago
0:04:13	and that there will be a three hundred the world
0:04:16	for the log which or
0:04:17	so i hope to see you all next gen was very guess it would be fine
0:04:22	thank you
0:04:28	thank you nose
0:04:29	and it's time
0:04:32	i
0:04:33	i'm i'm to start the
0:04:35	brisbane is primary
0:04:37	or speaker will be a the channel
0:04:39	and the like colleague your
0:04:42	and chambers
0:04:44	technical program chair
0:04:45	we introduce or speaker
0:04:47	from a
0:04:57	good morning uh light is in uh uh and from then i'd like to uh gain by repeating what ones
0:05:03	as Z and for that was of you who attended the uh check the evening uh yes to the evening
0:05:09	it was a about very special
0:05:11	that a page in and the uh music uh uh well as a a special uh experience that with
0:05:17	thank you very much to the local organisers work i in that
0:05:24	a bit by great applied to to uh
0:05:27	introduce a a a it it
0:05:29	doubt this morning
0:05:31	just a battery quick look at is uh uh uh C B
0:05:36	or by a will show you that uh
0:05:39	it's a sort of battered simple
0:05:41	whose who or but uh
0:05:44	some of the leading
0:05:45	academic
0:05:46	and industrial
0:05:49	unit
0:05:50	in the U
0:05:51	he V C he's degree in the nineteen eighty two
0:05:56	from
0:05:56	print then
0:05:58	but really
0:05:59	and than food
0:06:02	following that C work
0:06:03	at
0:06:04	at and T bell lab
0:06:06	multi he'll you to see
0:06:09	the about
0:06:10	how out so
0:06:11	with that so and uh in california
0:06:14	and since nine
0:06:17	ninety i
0:06:18	he's work that uh the microsoft
0:06:21	with a
0:06:22	and uh in redmond washington U N
0:06:25	where he heads the communications and collaborative systems systems research
0:06:32	philip has being a wheel
0:06:34	so than for the ieee signal processing society
0:06:39	most recently each yeah the multimedia technical committee
0:06:45	and old so it's a so on the signal processing society by committee
0:06:51	each distinguish is a a with it B and twelve
0:06:55	a side please
0:06:56	piper what in ninety ninety three
0:06:59	and more recently in two thousand and seven
0:07:03	even a the best paper award from the ieee transactions on multimedia
0:07:10	so thank you again guy in to fit it for agreeing to uh make this presentation
0:07:16	i i and i'm sure the whole O D and is looking forward to is
0:07:20	to all
0:07:21	in that it communication
0:07:23	it
0:07:31	you very much for the introduction it's a great honour to be on the stage especially with fred jelinek over
0:07:36	here he was an inspiration to me early in the career
0:07:39	and thanks all of you for coming uh to the plan area to at and after
0:07:43	have to the banquet i don't think i ever be in
0:07:46	at M plan a right after the bank so this a new experience for me i'm kind of looking for
0:07:50	to seeing what these plan is actually like
0:07:54	so i'll be talking about immersive of communication
0:07:56	um
0:07:57	there won't be any equations
0:07:59	in my slides
0:08:00	no signal processing people sometimes don't really understand things and less there
0:08:04	this math behind it so i'll give you some references
0:08:07	uh
0:08:09	signal processing magazine had the special issue in numbers of communication that came that in january you can look at
0:08:14	that
0:08:15	and just last week a bunch of us submitted a paper to the ieee proceedings as part of their hundred
0:08:20	anniversary um both of these are numbers of communication
0:08:24	so the last
0:08:25	second
0:08:26	paper here um the fines of communication to be
0:08:30	exchanging natural social signals with remote people
0:08:33	as in a face to face meeting in a weighted suspense
0:08:36	disbelief
0:08:38	uh in being there
0:08:41	this is uh
0:08:42	not a new idea by any means the
0:08:44	telephone was invented a hundred and thirty five years ago and that was the first grade breakthrough an immersive communication
0:08:51	it wasn't long after the telephone was invented a here's a cartoon that came out oh
0:08:55	three years later
0:08:57	uh that people were starting to look at much more immersive scenarios
0:09:01	so this is a
0:09:02	by george them are yeah a
0:09:03	um
0:09:05	the
0:09:06	caption of the bottom
0:09:08	read something like uh
0:09:10	uh as as these uh
0:09:12	parents are here in london in
0:09:14	uh uh uh watching their daughter in ceylon
0:09:18	play badminton
0:09:20	and
0:09:21	father says speech
0:09:23	come over here i want to whisper
0:09:25	she comes over
0:09:26	that's yes
0:09:27	pop here
0:09:28	uh
0:09:29	who who's that charming
0:09:30	uh lady
0:09:32	playing by charlie side
0:09:34	uses a
0:09:35	um she's just come over from london um i'll introduce you after the game
0:09:39	so we don't really have this kind of immersive of communication system even today but perhaps something that comes fairly
0:09:44	close
0:09:45	are tell a present systems
0:09:47	tell presence uh as defined by
0:09:50	the industry conference
0:09:52	newsletter wayne house review is
0:09:55	a video conferencing experience that creates the illusion that the remote participants are in the same room with you
0:10:01	so probably the quickest way to get a sense of what that means today is to take a look at
0:10:07	a thirty seconds just go
0:10:09	commercial
0:10:13	yeah
0:10:14	hmmm hmmm
0:10:17	hmmm
0:10:22	ooh
0:10:26	i
0:10:27	i
0:10:28	i
0:10:31	i
0:10:37	uh
0:10:39	so so one can be uh
0:10:44	so here you seen
0:10:45	high definition video conferencing so compelling that one participant has forgotten that his counterpart is
0:10:53	is remote
0:10:55	so this just go tell the present system uh
0:10:58	uh and others
0:10:59	uh like them H B halo another others
0:11:02	um off offer
0:11:04	high definition audio and video lots of bandwidth
0:11:07	um
0:11:08	to try to create this solution of being in the same room
0:11:12	um
0:11:13	is this a breakthrough in a of communication or is just a lot of uh
0:11:19	high definition televisions and a lot of bandwidth well
0:11:21	in a sense they're both
0:11:23	okay so
0:11:24	i contend that these are
0:11:26	bridge
0:11:26	uh to the future and we're about to see um
0:11:30	uh a rapid progress in this area of for the next few years
0:11:34	so put that in the context as take a look at
0:11:37	um
0:11:38	a brief history of television
0:11:40	television
0:11:41	was invented in its current form and
0:11:43	nineteen
0:11:44	twenty
0:11:45	six
0:11:46	uh
0:11:47	as a counterpart to the telephone
0:11:49	so here you see one of the first television set
0:11:52	eighteen T uh bell labs
0:11:55	uh
0:11:56	next to a telephone
0:11:58	because the television is meant to communicate
0:12:00	as a visual part of the telephone
0:12:02	so here it is
0:12:05	eighteen T president walter gifford
0:12:07	uh
0:12:08	in a you are K at bell labs talking to herbert hoover then secretary of commerce and washington D C
0:12:14	in nineteen twenty seven
0:12:16	so the first
0:12:17	distance television call
0:12:20	uh and shortly thereafter the television became
0:12:24	the broadcast medium that we know today whereas
0:12:27	video telephony
0:12:28	um took another forty years to become the eighteen T picture phone
0:12:33	uh
0:12:34	the picture phone was a
0:12:35	a stunning
0:12:37	technical success but also a stunning
0:12:39	commercial failure
0:12:41	so by nine teen seventy nine when i was
0:12:45	in intern at bell labs
0:12:47	down the hall from me on the desk of my lab director bob lucky was the
0:12:52	last remaining
0:12:53	working
0:12:55	picture phone in the world
0:12:57	but would often complain that nobody ever called them on it
0:13:04	did a forty years after that
0:13:06	uh experience is actually very similar
0:13:09	uh
0:13:10	except that now these little rectangular images are on general purpose computers
0:13:15	um
0:13:16	or or on your phone now
0:13:18	um but pretty much the experience the same there small little rectangular images of video
0:13:24	video conferencing
0:13:25	similarly
0:13:26	um has been
0:13:28	the same
0:13:29	uh
0:13:29	the for the last forty years in some sense here's the bell labs video conferencing system in nineteen sixty seven
0:13:35	you can see the round tape around
0:13:38	tables and multiple monitors cameras on top of
0:13:41	each monitor or the
0:13:43	data monitor
0:13:44	um which looks very similar to the
0:13:46	you know
0:13:47	the recent tell presents as
0:13:48	today
0:13:50	so
0:13:52	why do why contend that
0:13:55	there's about to be a series of rapid breakthroughs in immersive communication when
0:13:59	the last few years not much as happened in visual communication well
0:14:03	several reasons
0:14:05	the first is the internet
0:14:06	okay so the internet
0:14:08	has caused a divorce between
0:14:11	the format
0:14:12	of the content and then medium over which it is scary
0:14:16	so when the past telephone calls were carried over telephone networks
0:14:20	television over television networks are over
0:14:22	radio and so forth
0:14:24	today day all those are carried over the internet so there's a big possibility now
0:14:29	of
0:14:30	inventing arbitrary formats and they'll all be carried over in
0:14:34	uh the second is of course the cost of computation band with an resolution has a in dropping exponentially for
0:14:42	so long
0:14:43	and that there essentially free now compared to what they work twenty years ago
0:14:48	and the third reason is the technology
0:14:51	evolves faster than biology
0:14:53	okay so
0:14:54	they will become
0:14:56	bill be a threshold at which
0:14:59	the number of bits per second that were able to capture transmit and render
0:15:04	uh sir is what we can actually pass through the neural cut set around our bodies okay so at that
0:15:10	point
0:15:11	um
0:15:12	we should have freedom to do whatever
0:15:14	we like
0:15:15	and
0:15:16	the question is
0:15:17	what do we want
0:15:19	the future of you min
0:15:21	communication to look like at that
0:15:24	a
0:15:24	so to the extent that hollywood is the
0:15:27	uh keeper of our collective dreams
0:15:30	you know
0:15:30	be answer that hollywood
0:15:32	would give is that they want
0:15:34	communication to be immersive whether it's like
0:15:37	uh
0:15:38	the uh a hole attack in star trek or the jedi council meetings and star wars or
0:15:43	the matrix in the matrix or
0:15:45	um
0:15:46	in in avatar
0:15:47	um
0:15:48	communication should be mercer
0:15:53	however
0:15:53	not all communication will be numbers
0:15:56	okay we will continue to send S M S messages
0:16:00	uh and the reasons are that there you know there are some reason such as privacy that we we we
0:16:05	don't want to send everything
0:16:07	about about so this is anticipated you can buy
0:16:10	the jetsons and nineteen sixty two
0:16:13	so here's
0:16:13	change S and putting on her vanity filter
0:16:16	before she makes an early morning video call to her friend gloria
0:16:20	unfortunately flores
0:16:22	on vanity filter has an embarrassing
0:16:24	mel function
0:16:26	then there's one of my famous my favourite cartoons
0:16:29	uh
0:16:30	it's this
0:16:31	uh
0:16:31	from pete steiner in the new york in nineteen ninety three
0:16:34	i'm the internet
0:16:36	nobody knows you're a dog
0:16:38	no know that's
0:16:38	still pretty much true today there reasons why uh we don't want to review reveal everything about ourselves
0:16:45	and yet for the more complex
0:16:47	human interactions
0:16:50	a more creative things that we do with each other
0:16:52	um we need more margin
0:16:54	because we're social animals we have of all to work with each other best face to face
0:16:59	so how we position ourselves relative to each other
0:17:02	no where we sit
0:17:03	what how we just your
0:17:05	are are gaze
0:17:06	um awareness of all of those things they're all very important
0:17:11	and these have been studied um
0:17:13	but social psychologists and
0:17:15	and others engineers who build system
0:17:18	for example uh the work by no i and and can
0:17:21	and them the berkeley multi view project it build a system that
0:17:25	uh preserves i gaze
0:17:26	teleconferencing and they've shown that
0:17:28	trust improves
0:17:30	uh with correct i gaze
0:17:32	so
0:17:34	i gaze is bin
0:17:36	uh
0:17:36	a subject a video conferencing for
0:17:39	a long time there many solutions
0:17:41	have meters cameras behind screens
0:17:44	you interpolation and and many others here's an example of view interpolation where there's an upper camera
0:17:49	you when a lower camera view one they can be interpolated to get
0:17:52	um
0:17:53	better i gave
0:17:57	not only I gaze it's important it's uh i
0:17:59	reference and gesture
0:18:01	so here some examples from the early nineties
0:18:04	tongue and min "'em" and showed it chi ninety one
0:18:07	a a system that
0:18:08	had had a shadow of the remote collaborator behind the the collaboration surface
0:18:13	and a year later
0:18:14	you she had a same is clear board
0:18:17	um where the actual video of the
0:18:19	of the remote collaborator
0:18:21	was projected onto the
0:18:22	on the for the shared sir
0:18:26	here's an example uh that's
0:18:28	um
0:18:29	more recent P H P connect board
0:18:31	which
0:18:32	uh
0:18:33	is an update on it she's clear board
0:18:35	in that
0:18:37	perfect i gaze is established
0:18:39	by tracking
0:18:40	the position of the
0:18:41	observers had as well as the eyes of the
0:18:44	uh a of the
0:18:45	the uh a person in the video
0:18:47	so that
0:18:48	um um
0:18:49	the eyes of the on the video are always on the um
0:18:52	path
0:18:53	between the observer and the camera which is located time
0:18:56	the screen
0:19:00	at there many other
0:19:01	cues the uh days and gesture
0:19:05	um
0:19:06	both auditory and visual starting perhaps with
0:19:09	peripheral awareness of what's going on in the room
0:19:12	and consistency between the local space in the remote's space
0:19:15	so if you look back to at
0:19:17	the tell present systems today
0:19:20	you'll see that
0:19:21	if these preserve those first two characteristic
0:19:24	so
0:19:25	per full awareness is provided by very large
0:19:29	tell it
0:19:30	okay so we know what's going on and that remote room
0:19:34	um
0:19:36	consistency between the local room and the remote room as
0:19:39	is preserved by
0:19:40	uh making the
0:19:42	the desks look the same painting the walls to be a the same colour and all the different room so
0:19:48	uh
0:19:50	no difference between the back
0:19:52	of of of one room and the back of another
0:19:54	um
0:19:56	so
0:19:56	if you talk to the H P
0:19:58	guys
0:19:59	um they'll tell you that the halo system which was one of the first
0:20:02	which was probably be first um system in this category to come out
0:20:06	that was the collaboration between H P and
0:20:09	the film the hollywood studio dreamworks
0:20:11	and
0:20:12	you know how we what is is set you know be know how to
0:20:15	they know how to um
0:20:17	uh
0:20:20	use "'em" uh you um use illusion just suspend spend one's disbelief in actually being there
0:20:27	but there are many other
0:20:28	immersive of cues uh spatial cues
0:20:31	um for example uh distance
0:20:34	uh in audio is is uh indicated by
0:20:38	a combination of relative loudness
0:20:40	uh direct reflected energy and direct reverberant energy ratios
0:20:44	as we can see and in these uh these clip
0:20:48	oh yeah it's
0:20:50	right and i seven is does two yeah
0:20:54	so that one's
0:20:55	presumably further away than this one
0:20:57	oh yeah face
0:20:59	a in the next set is
0:21:01	does a to that was is probably adjusting the volume
0:21:05	but you can
0:21:06	uh i'm
0:21:07	and of course direction is is given by in true inter aural temporal role and interaural intensity differences between you
0:21:13	two years
0:21:14	um
0:21:15	and on the visual side
0:21:17	um there are many many is
0:21:19	cues on give you some examples of those
0:21:21	relative size and perspective combined to give you a sense of
0:21:25	distance
0:21:26	and um also absolute size
0:21:29	this of the same cues it are used in the famous
0:21:32	aims room illusion
0:21:34	uh occlusion lighting and shadow are also very important first giving a sense of
0:21:39	at distance
0:21:40	um
0:21:41	in augmented reality it's very important to be able to paint you objects into a scenes such the
0:21:46	occlude the background and yet uh do not occlude the foreground
0:21:51	uh lighting in shadow were very important not only for real is some but also
0:21:55	for um actual depth perception so if you look at this picture of the foot prints in the and
0:22:01	um
0:22:02	lighting and shot oh
0:22:03	give you an impression of one
0:22:05	uh a print being impressed into the the and the other one
0:22:09	uh
0:22:10	being above above the sand
0:22:13	so
0:22:14	a show of hands how many of you
0:22:16	perceive the top foot prints to be the one that's impressed into the same
0:22:21	into into the same
0:22:23	okay
0:22:24	and how many the bottom
0:22:28	okay uh
0:22:29	maybe one percent
0:22:31	think the bottom a ninety nine the top
0:22:33	um now keep your eyes fixed on that
0:22:40	"'kay" don't blink because if you blink you probably change your mine
0:22:47	okay so
0:22:49	even even shot all uh a has something to do with depth perception
0:22:54	uh focus as well so use a very large
0:22:57	city D china
0:22:58	uh but if we change the a focus
0:23:01	the foreground background it suddenly shrinks down to look something like like a little model that might be
0:23:06	uh
0:23:07	next year
0:23:07	model trains set
0:23:10	of course there the the uh
0:23:12	very strong cues of skates star got be and motion parallax
0:23:16	so some of you may be able to cross your eyes and fuse these images
0:23:20	and you get a very strong sense
0:23:21	uh of of dat
0:23:23	um
0:23:25	and all show an example of motion parallax later on
0:23:27	but
0:23:28	so are these cues uh important for communication
0:23:32	that's the question and i contend that the R
0:23:34	um clearly are current communication systems do not convey this information
0:23:39	and clearly they are not
0:23:40	satisfactory are are are not uh
0:23:44	um
0:23:45	no the can't fulfil every need for communication that we have we continue to meet in person and whenever we
0:23:51	are able
0:23:52	we continue to travel to work
0:23:54	we continue to
0:23:56	attend parties and
0:23:58	weightings in person we continue to come to meetings like this you know why don't we just
0:24:03	email all are talks
0:24:05	our slides to each other
0:24:06	and just a stay at home well because there's something
0:24:10	valuable about we feel about being here in person
0:24:13	um and our current communication systems don't convey that
0:24:16	but i
0:24:17	believe that um the tools of signal processing will help us
0:24:20	um bridge that yeah
0:24:24	so
0:24:25	if we're of a physicist you might take a different approach
0:24:32	case of this is is uh i've read have uh actually succeeded in transporting matter
0:24:37	uh a small molecule
0:24:39	um over us
0:24:41	significant distance
0:24:42	um and this is how it would work if you were to use it for uh
0:24:46	uh transporting you know somebody
0:24:49	so you me too
0:24:50	a different part the world i would step into a matter transporter
0:24:53	i would be frozen down apps the zero
0:24:56	i be turned into a little break of
0:24:59	bowes einstein compensate inside
0:25:01	in the process of schooling down i would it be light
0:25:03	light would be transmitted to another place
0:25:07	would shine on another little break
0:25:09	of bowes einstein kind and set
0:25:11	and then
0:25:12	my original state would be reproduced
0:25:15	of course if i
0:25:16	don't
0:25:17	plead lee cool down to zero i may leave some state behind hind uh with unforeseen consequences
0:25:25	it's also sobering to think about
0:25:27	uh what quantisation and data compression would do on um on the way
0:25:32	rate distortion optimized mpeg four thousand
0:25:38	um
0:25:38	but at microsoft research for mostly computer scientists and signal processing people so
0:25:43	we take a different approach which is
0:25:45	we do the
0:25:46	the um transportation uh uh matter virtually rather than physically
0:25:51	um and we achieve that through court a transformation
0:25:55	so if you imagine
0:25:57	a person in a real space over here
0:26:00	um
0:26:01	he's got some court
0:26:02	system in as local space and we just do a coordinate transformation
0:26:07	uh of that over into to some
0:26:09	no map that coordinate system into the coordinate system of some other states
0:26:12	an if we do that for several people then we get them into the same
0:26:16	the same
0:26:18	so depending on whether the space that there being transported into
0:26:22	is a virtual or real
0:26:24	um it leads to two different approaches
0:26:27	uh i'll
0:26:28	um
0:26:29	talked about both of these you might sort of a
0:26:31	identify the first one with
0:26:33	the movie the matrix if you're familiar with that
0:26:37	everybody goes into a a virtual world
0:26:39	um
0:26:40	the second approach could be identified with the movie avatar in which you send
0:26:45	a physical representative of yourself into are real physical
0:26:48	space
0:26:49	so i'll talk about
0:26:50	um
0:26:51	each of these
0:26:52	starting with the first one
0:26:54	so the virtual or
0:26:55	the matrix movie approach
0:26:57	um can be applied to what we call fully distributed meetings
0:27:01	so we have people who are
0:27:03	and completely separate places around the world maybe your
0:27:06	you when your colleagues are writing a paper together and you're sitting in your office is
0:27:10	different universities
0:27:11	and you wanna talk to each other and the ideas to make it seem as if you're sitting around a
0:27:16	see the around the same
0:27:17	table
0:27:18	um talking to each other
0:27:21	so the way we are doing that is by
0:27:24	capturing geometry of each person
0:27:27	doing a court transformation to put them all into a common scene
0:27:31	a life size scene
0:27:32	and then you give each person a window on that's scene it's basically there display
0:27:37	and uh each person gets a
0:27:40	you know i
0:27:41	own personal view of that scene
0:27:43	um
0:27:44	and uh people interact through the
0:27:47	in this way preserving i gaze gesture direction and so forth
0:27:51	but doesn't matter what that the projection is
0:27:53	on on uh
0:27:55	a flat
0:27:56	surface or
0:27:57	a two a multiple monitor situation or maybe even a curve
0:28:00	screen so
0:28:01	well we've done is build a a
0:28:03	a
0:28:04	wide field of view curved screen
0:28:06	um and from a particular point of view make it appear is if
0:28:10	uh uh vocal task is extended
0:28:12	uh in space
0:28:14	and then when we capture
0:28:16	people's geometry using a depth can where that situated on top of the
0:28:20	the screen
0:28:21	then we can position those people around in space as if they're having
0:28:25	a meeting so if you've never seen
0:28:27	um
0:28:29	what images from a that's camera look like
0:28:31	this will give you
0:28:32	you give you sense use up
0:28:34	here's the still frame from a
0:28:36	adept camera
0:28:38	see you can
0:28:39	render or uh render person from an arbitrary view
0:28:43	okay so basically that's what we do capture
0:28:45	geometry and texture
0:28:47	of um
0:28:48	of each participant
0:28:50	place them in there's
0:28:51	virtual world
0:28:52	and then um
0:28:54	what them collaborate so here's
0:28:55	here's an example of what you might see if you were telling one of these meetings
0:28:59	your
0:29:00	collaborator sitting across the table from you
0:29:03	uh
0:29:05	maybe a on the wall on the right there some artwork it's glowing to reflect the mood of the conversation
0:29:10	in the room
0:29:11	on the back wall perhaps as a large active computer display that can be used for the
0:29:17	uh for the meeting
0:29:18	um a on the left wall here might be a window out on to
0:29:22	the uh
0:29:23	skyline of singapore in this case or wherever L C may wanna hold your meeting
0:29:27	as other participants join the meeting
0:29:29	um the people in the meeting can move around the table to make room
0:29:33	people can bring in their own data
0:29:35	other people may join the meeting and they come in from ordinary web cams or so so they don't actually
0:29:39	have their dept associate with them but they can be represented as
0:29:43	as a virtual
0:29:44	displays here
0:29:46	um
0:29:47	data such as this what's showing in the centre of the table here can
0:29:50	float above the
0:29:51	the table and be manipulated and so forth so
0:29:54	these are some of the things you can do
0:29:56	um
0:29:57	uh uh in this uh
0:29:59	this scenario
0:30:02	so that
0:30:04	image of course um was
0:30:06	i i didn't show was taken from one particular static point of view but as people move around
0:30:11	uh
0:30:12	the image is move so
0:30:13	um here's a video that shows
0:30:16	uh this the fact of motion parallax
0:30:21	so by tracking the position of the camera in this case
0:30:24	um you can change what is poor
0:30:27	presented on and ordinary
0:30:29	um
0:30:30	display
0:30:31	so that the peers as if there's some that's find the display
0:30:36	oh this is just using motion parallax to give that sense
0:30:38	depth
0:30:50	the same can be done with a audio actually
0:30:53	okay so you can
0:30:54	uh
0:30:55	do you head tracking
0:30:56	by
0:30:57	you know visually
0:30:58	find out where where persons head is
0:31:01	point
0:31:02	and then um
0:31:05	a table look up to find out
0:31:07	the relationship between that person's head and uh
0:31:09	and this
0:31:10	and uh uh a virtual source
0:31:12	get the H R T S for that and find out what's the signals
0:31:15	that you want to each year
0:31:17	similarly you know what the relationship of the head is to each of these loudspeakers
0:31:22	and then you can
0:31:23	or to
0:31:24	transfer matrix invert inverted
0:31:26	do some crosstalk cancellation
0:31:28	and
0:31:28	produce at these loudspeakers
0:31:30	signals that depend on
0:31:32	where your head is
0:31:33	at
0:31:34	uh so that the
0:31:36	the perceived
0:31:38	location of the virtual source stays put
0:31:41	so it doesn't drift into one of the that the lots and the one of the speakers as you move
0:31:45	towards one speaker
0:31:46	apple
0:31:48	so we've
0:31:49	done experiments on different form factors as well
0:31:53	this is an example
0:31:54	of the
0:31:55	of a wall
0:31:56	where uh we've projected onto a whole a graphics screen the same kind of thing you might see in a
0:32:02	high tech show
0:32:03	you know that store
0:32:05	window
0:32:06	um
0:32:07	and
0:32:08	we try to make its
0:32:09	a appear
0:32:10	through motion parallax and stereo stereo parallax
0:32:14	um that the person is standing in the same room with you
0:32:22	so
0:32:23	i want this
0:32:23	take a
0:32:24	five minute interlude here to talk about that the cameras because that's cameras at had a recent breakthrough
0:32:30	in
0:32:31	um
0:32:32	in cost and performance
0:32:34	so there are new sensor that's really of air are available uh very red to everyone here just like a
0:32:39	uh a a webcam cam might be
0:32:41	so let me tell you about them
0:32:43	um
0:32:44	you might have heard about the can act for X box three sixty
0:32:47	it's a device
0:32:49	like this
0:32:50	um
0:32:51	that makes
0:32:52	use the game player
0:32:53	uh the controller
0:32:55	and what it was introduced in october
0:32:57	last year
0:32:58	it's sold
0:32:59	eight million units in the next sixty days making it the
0:33:03	uh a fast so link consumer electronics device in history
0:33:07	reading it some position and the guinness book
0:33:09	world records
0:33:11	a
0:33:32	oh
0:33:33	hmmm
0:33:34	hmmm
0:33:35	and
0:33:45	a so that's enough for that
0:33:47	uh
0:33:47	basically it's a
0:33:49	it's a the camera
0:33:50	okay
0:33:51	so at every
0:33:52	pixel it produces a their distance from the camera to something in the scene so it enables you to
0:33:58	uh
0:33:59	with do uh
0:34:00	some skeleton analysis for example
0:34:02	of where what people are doing enable you and labeling them to control
0:34:06	the games
0:34:07	so under the covers
0:34:09	it's uh
0:34:11	it's this kind of device we have
0:34:13	um a regular
0:34:15	rgb camera
0:34:16	we have an infrared camera
0:34:18	uh
0:34:19	that's pretty much just like the rgb camera except that has a filtered front of it
0:34:23	ooh allowing to see only infrared
0:34:25	there's a infrared projector that projects
0:34:27	and they'll pattern on the scene
0:34:29	and by correlating that known pattern with the observed infrared image um
0:34:34	you get a disparity at each pixel in the data
0:34:37	also in this unit is a a
0:34:40	uh design by even to even on you have who's
0:34:43	possibly here
0:34:45	uh
0:34:46	and
0:34:47	uh there's a motor or dsp unit U is P
0:34:50	uh
0:34:51	use P port so the motor tips the
0:34:53	the unit up and down
0:34:56	and all of this comes for a hundred fifty dollars
0:34:59	so
0:35:00	um
0:35:02	it's not surprising that within two weeks after it was a
0:35:06	it was released
0:35:07	it was hacked into
0:35:09	and people it up to their P C's
0:35:11	and did all sorts of crazy things
0:35:13	with it
0:35:14	um there's a web here
0:35:16	devoted to connect tax
0:35:18	and
0:35:20	by
0:35:21	wow one month after release
0:35:24	uh
0:35:25	uh there were about ninety projects
0:35:27	um um on that site
0:35:29	uh
0:35:30	in three months there work twenty four pages of
0:35:33	the projects uh yeah as of yesterday there were forty seven pages the projects and these
0:35:38	these are all crazy things like a um
0:35:41	um for
0:35:41	robot but
0:35:43	uh navigation to um
0:35:45	to seeing for the blind
0:35:47	two
0:35:48	greeting
0:35:49	storefront windows that react to to people wandering in front of them uh and so forth and so on so
0:35:55	if you're interested um
0:35:57	in using this for some of your signal processing i invite you to jump in
0:36:01	uh
0:36:02	uh there is an open source uh driver or but
0:36:05	it doesn't have any of the scale to tracking error or or eve ons uh
0:36:10	uh
0:36:12	mike array processing
0:36:13	uh if you want that
0:36:15	you can wait if few
0:36:16	weeks uh
0:36:18	i believe will be having a
0:36:20	um non commercial
0:36:22	uh S T K available for all of you to use uh
0:36:25	free of charge
0:36:31	so
0:36:32	i talked about
0:36:33	uh
0:36:35	talked about how we were using depth cameras
0:36:38	uh for
0:36:39	burst of communication here's a here's an application that's very closely related
0:36:43	and it will be actually released two
0:36:46	um X box live gold subscribers
0:36:48	um
0:36:50	uh sometime this spring
0:36:52	so i thought i'd show that to you
0:36:53	it's called avatar connect
0:36:55	uh rather than showing you an advertisement i've recorded my own video here
0:36:59	uh this is corey she's talking to me on her
0:37:02	right and to sash on her left
0:37:05	her her images
0:37:06	the image of her avatars
0:37:07	uh in this picture in picture thing
0:37:12	i
0:37:13	i
0:37:15	no who can see here avatars a her emotions
0:37:21	so in addition to
0:37:22	uh
0:37:23	in addition to um still to tracking of arms we have to do facial expression tracking so
0:37:29	you know who's uh
0:37:30	uh uh when people are talking
0:37:32	you have to figure out where they're looking so you can at intimate their avatars and so forth
0:37:37	and you can imagine that um
0:37:40	uh
0:37:42	E there uh many other
0:37:45	uh backgrounds you can use uh for this
0:37:47	for this uh
0:37:49	avatar can
0:37:50	thing
0:37:50	so you can do a uh
0:37:53	a uh
0:37:54	talk show for example with your friends uploaded to you to and so forth
0:37:58	so you can imagine that as
0:37:59	uh the avatars get more and more
0:38:02	um
0:38:02	sophisticated
0:38:03	eventually we may move into any era where inverse of communication is something like a mass of multiplier video game
0:38:10	where
0:38:10	you know you enter you enter into this world
0:38:13	um
0:38:14	and you have terms are controlled by your actual motions your expressions
0:38:18	and you're
0:38:19	oh case alone let me
0:38:20	um
0:38:21	return to the second
0:38:23	approach
0:38:24	um this is this
0:38:25	real or so called avatar like
0:38:28	avatar movie approach
0:38:29	um where you send a physical representative of yourself
0:38:33	two a real space
0:38:35	so this works well for so called and satellite meetings where there
0:38:39	several people around you a real space like a real conference room
0:38:42	and there are some remote attendees
0:38:45	um called satellites lights and they sense some
0:38:47	proxy in the me into the meeting
0:38:49	so we call these embodied social proxies
0:38:52	here's an example of an early
0:38:55	uh
0:38:56	early device
0:38:57	uh
0:38:59	a regular L C D used for the
0:39:01	the face
0:39:02	uh
0:39:03	fisheye cameras and pan tilt zoom cameras used for eyes
0:39:07	phone for mouth
0:39:08	and years
0:39:09	all of this is put on a cart
0:39:11	is a computer
0:39:12	wifi a battery backup and so forth so they can that end
0:39:15	meetings for you
0:39:17	and these are
0:39:19	uh our current meeting
0:39:23	we have four of these cards in this particular room
0:39:26	uh one here one here one here and one out of sight
0:39:30	we have collaborators
0:39:31	um
0:39:32	around the world john it's and silicon valley and over here's and cambridge
0:39:37	england
0:39:37	the you know the meeting room itself is in is in redmond washington near seattle
0:39:42	um
0:39:43	some of these are
0:39:45	um on
0:39:46	on cards that have to be wheeled in
0:39:48	this one is actually robotic it walked through the door on its own
0:39:53	the view from these carts of the room
0:39:55	look like this
0:39:56	okay so you get a wide field of view
0:39:59	peripheral awareness of what's in the room
0:40:01	and if you want to see it some detail like what's a a on the
0:40:05	screen
0:40:06	um
0:40:07	you click on the screen the pan tilt zoom camera goes over there and gives you a high resolution version
0:40:14	so we use these not only yeah for our own meetings but we've done studies on them across real
0:40:19	groups
0:40:20	that are
0:40:21	using them so we deployed then in for different microsoft product development teams where one of the
0:40:27	members of the team was remote
0:40:29	we the plate them for an eight week period
0:40:32	uh
0:40:33	conducted interviews and surveys at the beginning and at the end to determined
0:40:37	um
0:40:39	whether people use these things
0:40:41	uh
0:40:42	and by
0:40:44	asking questions on the surveys that are based on the seven point likert scale
0:40:48	yeah um
0:40:49	we determine meeting effectiveness awareness of social
0:40:52	aspects by asking questions such as
0:40:55	i think
0:40:55	X has a good sense
0:40:57	of my reactions
0:40:59	or
0:40:59	for awareness
0:41:00	i'm where
0:41:04	i'm aware of what X is currently working on and what is important to X work social aspects that as
0:41:09	i have the sense of closest to
0:41:11	and so by comparing this surveys that the beginning and at the end
0:41:15	we can determine
0:41:16	you know uh
0:41:17	whether they've improved or not
0:41:19	and
0:41:20	um
0:41:21	uh quite dramatically we see a lot of improvement uh a across
0:41:24	all the four different groups that we put
0:41:27	we put them into one also different uh
0:41:29	category
0:41:34	these are verified by user comments
0:41:36	um
0:41:37	when user for example
0:41:38	so that um
0:41:40	it actually succeeds in creating a workable allusion
0:41:43	but the remote T member is in the room
0:41:44	that overcomes the barrier
0:41:46	physical distance
0:41:48	and direct observation also showed that
0:41:51	in contrast to
0:41:53	the uh
0:41:54	the audio conferencing that they get been using before
0:41:57	is now allows rapid turn taking in conversation
0:42:00	um
0:42:01	following of what's going on the whiteboard and brainstorming
0:42:04	resolving issues uh at the meeting instead of waiting for the next time the person
0:42:09	uh a of is it
0:42:10	um
0:42:11	saving some trips
0:42:12	and also assisting not native
0:42:14	english speakers to understand what's have
0:42:18	uh and all of all of the
0:42:20	uh proxies were named
0:42:22	and given hats
0:42:25	and so they became
0:42:26	part of the T
0:42:29	these are not a new ideas um
0:42:31	in the early nineties
0:42:32	uh bill buxton then that be cell and a
0:42:35	at their hydra system
0:42:37	uh more recently paul of that the bed back um introduce the really cool uh
0:42:42	volumetric display
0:42:44	uh
0:42:46	a few ks
0:42:46	uh you and C
0:42:48	it's been doing some work on
0:42:50	and am at run X
0:42:54	uh and
0:42:55	there are tell a presence robots
0:42:57	and they've even a an to the
0:42:59	a boy to attend school even though he has some immune deficiency
0:43:03	that was reported by C N and back in january
0:43:06	and ieee spectrum and than your times last september both had cover articles
0:43:11	on these tell a presence robot
0:43:14	user comments in both of those articles feel pretty much the same things that we get also already observed so
0:43:19	eric oh itself from
0:43:21	ieee spectrum
0:43:23	says that
0:43:24	you know you participating is a row by you feel you get people's attention
0:43:28	and there's a better sense of being there
0:43:30	and
0:43:31	mike bells there
0:43:32	so you get the same kind of or interpersonal connection that you'd have
0:43:35	um
0:43:36	as if you were at the meeting
0:43:38	a
0:43:39	interestingly
0:43:40	even in the one you know and all those cases
0:43:42	in the new york times in and spectrum
0:43:45	uh
0:43:45	the device is for given names and they were given had
0:43:51	are many issues
0:43:52	uh
0:43:54	save these one i went down to visit any but
0:43:57	robotics in uh
0:44:00	uh in silicon valley and found that they had one of their humanoid robots
0:44:04	lashed to wall
0:44:06	after had punched a big hole
0:44:08	uh which are still visible in the wall
0:44:10	a next to it
0:44:11	so
0:44:12	um
0:44:13	there some safety issues with robots
0:44:15	wifi coverage you know what happens when robots from
0:44:18	walk out of
0:44:19	at of range or run out of power
0:44:21	um can make keep up with people who are walking
0:44:24	you know how do they open doors or get on elevator
0:44:27	you know you don't want people's hacking into your robot button taking it over
0:44:31	um and their social issues like well do you want them to
0:44:34	C better than humans here better than humans
0:44:37	um what height should they be at is socially acceptable to
0:44:40	to touch them and move them you know so there are many
0:44:43	issues um including um
0:44:45	how close should they come to looking
0:44:48	human
0:44:49	uh and maybe you've seen in ieee spectrum some of you should grows robots
0:44:53	these are quite fascinating
0:44:55	uh
0:44:57	and the not just wax models they actually move
0:45:09	i
0:45:09	i
0:45:10	oh
0:45:12	i
0:45:14	and so this is a gemini T K here is a
0:45:17	is professor have wreck sharp
0:45:19	at all berg university
0:45:21	and apparently according to ieee spectrum has wife says that
0:45:24	she prefers body number one
0:45:27	but she's suggest that we should always send body number two to the conferences and stuff
0:45:33	so don't be surprised if it future icassp cast you see you know some
0:45:39	some robots wandering around
0:45:42	so
0:45:44	so far i've talked
0:45:45	pretty much about
0:45:46	two ends of a spectrum and one approach we've had
0:45:49	uh these virtual
0:45:51	you know everybody
0:45:52	transports into to virtual space and on the other approach
0:45:55	you uh transport into a real space through your proxy
0:45:59	um um
0:46:00	in some sense the first is about virtual reality which was
0:46:03	uh
0:46:05	promulgated by are and land here and the early eighties
0:46:08	it's about embedding people into the computers world
0:46:11	um
0:46:12	the second one is about
0:46:13	ubiquitous computing promulgated by mark wise or in the late eighties
0:46:17	and so about embedding computation computational devices are computers
0:46:21	in are world
0:46:23	and so there's really um
0:46:25	uh uh these are really at two extremes and i think what's really interesting is is what could be in
0:46:30	the middle and the first time might became
0:46:32	aware of that space
0:46:33	was attending
0:46:35	uh a a talk at stanford in that in the early nineties
0:46:39	on um
0:46:41	uh V V are male virtual reality
0:46:43	markup language um it just been invented in the
0:46:46	the inventors there were
0:46:48	were discussing
0:46:49	the virtues of it in showing as um
0:46:51	how they had
0:46:53	um
0:46:54	made a model of the entire
0:46:56	university and they were showing is how they could
0:47:02	how we could um
0:47:03	navigate their way
0:47:05	through it
0:47:10	and so they wander down the halls of the university and then came to a door
0:47:14	of the conference room where we were sitting
0:47:18	and
0:47:20	i expected that they would just kind of open the door N
0:47:23	inside we would
0:47:24	you know we would see all of us sitting inside the room
0:47:28	um
0:47:28	and so i
0:47:29	got a nervous feeling about oh maybe behind me through the door will come some giant eyeball ball and then
0:47:36	there all be all these people behind a kind of looking at this
0:47:39	so i realise that there's is big gap between
0:47:42	uh
0:47:44	the virtual reality part and and and the physical reality
0:47:47	um and so that's an area that
0:47:50	uh
0:47:53	at scenario i think it'd be
0:47:54	uh need a lot more exploration
0:47:57	i'll just say briefly that we've been doing some some work in that area
0:48:01	um
0:48:02	we've called a parallel worlds that one point it's not overlaying real and virtual worlds on top of each other
0:48:07	and the people to sort of cross back and forth between one and the other so how might
0:48:12	people walk
0:48:13	through plaza for example
0:48:15	or tent
0:48:16	no go to a museum or real these are real places how mike the attendees these remotely
0:48:21	and
0:48:22	uh experienced um remotely and have also have the people in these real places experience the remote people as if
0:48:28	they were there
0:48:30	uh
0:48:30	i you know in the real space
0:48:33	so
0:48:34	we you know we'd had applications to attending weddings for example
0:48:37	or trade shows and conferences and going to poster sessions how might you have
0:48:41	a virtual person come to your poster and you could converse with that person as if they were there
0:48:45	so the a lot of interesting signal processing is use how do you instrument the space to capture the
0:48:50	the audit
0:48:51	tori pardon the visual part
0:48:52	um so you can represent them remotely
0:48:55	and more
0:48:55	a more difficult thing is actually how you
0:48:58	rent their remote people into a real space
0:49:02	so i
0:49:03	want to include
0:49:04	here
0:49:05	um i've time talked
0:49:07	uh
0:49:09	mostly from a technical point of view about a of communication
0:49:12	um here all just
0:49:14	talk about
0:49:15	uh this is societal need so
0:49:18	climate
0:49:19	energy and environment are are L
0:49:21	the big words of the day
0:49:23	uh i flew here
0:49:25	uh from seattle
0:49:27	it's five thousand miles
0:49:30	why other five thousand miles getting back and the amount of C O two i've released into the atmosphere
0:49:35	uh
0:49:36	is equivalent to all my local transportation
0:49:39	you know i've done this uh
0:49:41	just for this week i've used up my years quota of local transportation
0:49:46	uh
0:49:48	uh probably all of the you know many of you of also flown from overseas
0:49:52	um
0:49:53	collectively i would say we probably released into the atmosphere coming to this conference
0:49:58	about
0:49:59	a thousand times
0:50:00	of C O two
0:50:02	so
0:50:02	i don't think it's something we can afford really into the future
0:50:06	um in terms of productivity
0:50:08	you know i spent
0:50:09	three days getting here for for a meeting so that
0:50:12	ratio is not really good
0:50:13	um but even on a day to day
0:50:16	uh basis in at least in the us
0:50:18	people spend close to fifty minutes per day on average
0:50:22	uh
0:50:23	uh commuting
0:50:24	perhaps about that ten percent of their work
0:50:26	their work
0:50:28	um
0:50:28	information workers though spend about fifty six percent of their time in communicating with other people
0:50:34	um
0:50:34	sixty percent of their time feel that they could do there
0:50:37	job duties these just as well it's some remote location for example home so there
0:50:42	there are things we can do about that
0:50:44	and
0:50:45	uh
0:50:46	thomas friedman so the world is flat but it actually needs more flattening in today's economy we need to bring
0:50:51	people to jobs and jobs to people
0:50:54	in this is physical security
0:50:56	i i guess the volcano one iceland lent started to rub thing again a but a year ago
0:51:01	uh a little over your goal for period of three weeks
0:51:04	it's uh
0:51:06	cost the cancellation of a hundred thousand flights
0:51:09	stranded it eight million people across europe
0:51:12	uh many of you may have been among that group
0:51:15	uh
0:51:16	and uh cost the airline industry to one a half a billion euro
0:51:21	so uh the economic damage is real
0:51:24	um
0:51:25	you remember
0:51:26	uh i two thousand three was cancelled because of sars so
0:51:30	if you know if there a viral out breaks
0:51:32	yeah um they can shut down whole economies
0:51:34	um
0:51:35	uh uh not i mean is N
0:51:37	earthquakes
0:51:38	uh
0:51:39	you know these things can also have a big
0:51:41	uh
0:51:42	a big um
0:51:44	a a fact
0:51:45	um and terrorist threats right
0:51:49	to more traditional reasons for having uh tell a presence and numbers of
0:51:53	um communication have to do with
0:51:56	uh bringing families close together
0:51:58	and then um
0:51:59	pretty health care and education
0:52:01	uh around the world
0:52:04	so
0:52:05	um
0:52:07	i'll just uh
0:52:08	mentioned that
0:52:09	you know many of these
0:52:11	national academy of engineering grand challenges
0:52:14	um
0:52:15	from
0:52:16	uh
0:52:22	having to do with a the global climate
0:52:24	as well as um health care
0:52:26	security is are all addressed by and by
0:52:29	like tell a press
0:52:32	so in conclusion um
0:52:33	although
0:52:35	visual communication has not really changed much in the last eight years
0:52:38	the conditions are right do the convergence of
0:52:41	affordable technologies and high social need for
0:52:44	human communication to break through to new levels of them are given
0:52:47	and uh the signal processing community in a unique position to be able to address
0:52:51	uh the needed advances so
0:52:53	i think we should embrace the opportune
0:52:56	thank you
0:53:09	thank you and that's in the a uh it it so that uh an exciting and uh informative a talk
0:53:16	i I and the uh feel that we have deal
0:53:19	then it faded it can having you do they bring that to okay about new have a yeah yeah
0:53:26	how can you tell
0:53:27	yeah yeah yeah yeah
0:53:29	she
0:53:30	anyway we have uh an opportunity or some uh a question in the floor
0:53:46	are uh
0:53:48	oh we have one at the back if you would got the mike lee
0:54:00	well i'm and you think of this end so
0:54:02	got
0:54:03	yeah order
0:54:05	you immersive it
0:54:05	in case
0:54:06	to touch and what else
0:54:08	i in greater converter
0:54:10	um feeling
0:54:11	so probably very important but uh it's the least well understood
0:54:16	there's some uh articles in the uh verse of communication issue about have tex
0:54:21	um
0:54:22	uh
0:54:25	you know i i would say it's least well understood at this point O
0:54:29	there's a lot of work to be done
0:54:35	so uh
0:54:37	basically when i was a a a a it be lot asking the little guys so
0:54:42	oh
0:54:43	looking at a them was station i ask them one question
0:54:46	what about the band
0:54:47	i mean
0:54:48	you need a lot of they got to so and so one
0:54:51	and so
0:54:53	there are saying there are using a special line
0:54:56	that's right
0:54:56	so
0:54:58	but everybody can these a special lines so
0:55:00	i mean
0:55:02	oh do you see that like a like that that's like this amount model of a got
0:55:05	or the word you don't on so you're still basically but still
0:55:08	you don't so a large amount of data
0:55:10	and you need some basic the but well um
0:55:13	bandwidth with capacities going to vary very over a very wide range uh
0:55:18	and
0:55:19	so
0:55:20	you can do various
0:55:21	uh i think to address lower band with
0:55:24	so the avatar can act as an example where you just controlling
0:55:27	um
0:55:28	you're parametric avatar sending a series of parameters over
0:55:32	over the line so it's much more like a
0:55:34	uh
0:55:35	you know
0:55:36	a again
0:55:37	um
0:55:38	and you still get you still can get someone out of a margin and those
0:55:42	situation so i think
0:55:44	uh
0:55:45	the amount of realism we'll just depend on how much bandwidth with you have available to you
0:55:49	um and their strategies for
0:55:51	are going down to lower band
0:55:56	that's sufficient
0:56:08	yeah well
0:56:09	how much bandwidth with you need for different amounts of perception is
0:56:12	uh is an area first
0:56:31	a have that one should be taken this but more of line we have to for one last question
0:56:37	oh okay okay i dean man
0:56:39	one can not really right i
0:56:42	you human you mean
0:56:45	would you be able to you come and that share you
0:56:52	um
0:56:53	so no matter what technology we use
0:56:56	there will be things that we just cannot not
0:56:58	uh replace
0:57:00	with human to human communication is that where you ask
0:57:02	these
0:57:03	so
0:57:04	you know i i i suppose supposed just probably one of the good examples that's that difficult to do right
0:57:09	now
0:57:09	i
0:57:10	i really can't say uh
0:57:12	you do the future what
0:57:13	whether smell and
0:57:15	other things
0:57:16	you know
0:57:17	i mean i suppose eventually you just the jack to the matrix and put something the back of your brain
0:57:21	and then
0:57:22	uh
0:57:23	replace everything
0:57:24	uh you become a a brain in of that
0:57:27	at that point
0:57:28	uh but
0:57:29	um
0:57:30	what you lose
0:57:32	philosophers last say we're all or or all ready brains
0:57:40	sorry that's point two
0:57:42	with
0:57:42	hence
0:57:44	okay we should uh i think uh it again and used to call thing to do
0:57:54	X
0:57:58	really

Directions in Immersive Communication

Plenary Talks

Přednášející: Philip A. Chou (Microsoft Research, Redmond, WA, USA), Autoři: Philip A. Chou (Microsoft Research, Redmond, WA, USA)