come a three of the and i hope we're or you had good sleep after is to is for circular so back to this go i i i hope you enjoy it and do you your the the music there so why won't read or the names some the links a to repaid use but in case you want to have with brave guys and are also uh invited to your garden party there will be doing so the on the uh on the conference web page i was especially a it is harm core just band but that was playing in one of the rooms that was re reexamine people from uh ordinary secondary school mode even music second can very group so uh a good and you can you can visit the their rip sorry or just type harm core prague to do ruin you will get it uh there is one a warm mixed white please oh yes okay so one one practical remark remote the restaurant we had the sum compliance like to you in the plumbing you uh a short clues from you use the not be type of food i won't so uh the the the solution is to ask or or do not a a uh uh the information desk and she will be happy to a device you a rest around in be vicinity of these place or or or or or or the there no uh uh this at all or the practical in four and we will have for an invitation a two us very here you are a conference number you are most probably only is from a some you know of pixels we morning this morning i would like doing that are used to you that two thousand and two L international conference on merging signal processing applications or use by twenty two well my name is by most problem in is and i am the general so obvious park where B to out use but where and good where all these marked it works wrong it is a full quorum forums organise by the signal processing society it is expected to be the third major problem for some of the some side view but together with i ask for nice here but because it used you is not dressed primarily to practising engineers of course everybody is welcome wall i was uh going to be a course in practice problem something else that makes use by be my when i see here use that to do will be held every year here at the same place and time you "'cause" been plan one for one with a consumer electronics show so give her beam last very guns in early january when we say practising and you here my the and some more who has a master's degree in general signal processing i i is or the presentations and expected to be applications already it and more can thing as a very clear to to first a good participation from mean dusting use by we also include but as patients without that paper of course papers first can be some you two and that these papers would be are in the R you i drip we explore but we're one from is mark expect to be a if you're at the girl in another major component of the conference is that the most rooms what we call show and they are here by cast and in have you sure we will have to house plan i to present creations and by discussions you can find more maybe from in the call for papers that we have a the information of both here by cash or blues and see me so here are all invited you to submit papers for is is where well one more thing the corn firms "'cause" not go about to the local yeah we can you to submit your idea for a ago and that there will be a three hundred the world for the log which or so i hope to see you all next gen was very guess it would be fine thank you thank you nose and it's time i i'm i'm to start the brisbane is primary or speaker will be a the channel and the like colleague your and chambers technical program chair we introduce or speaker from a good morning uh light is in uh uh and from then i'd like to uh gain by repeating what ones as Z and for that was of you who attended the uh check the evening uh yes to the evening it was a about very special that a page in and the uh music uh uh well as a a special uh experience that with thank you very much to the local organisers work i in that a bit by great applied to to uh introduce a a a it it doubt this morning just a battery quick look at is uh uh uh C B or by a will show you that uh it's a sort of battered simple whose who or but uh some of the leading academic and industrial unit in the U he V C he's degree in the nineteen eighty two from print then but really and than food following that C work at at and T bell lab multi he'll you to see the about how out so with that so and uh in california and since nine ninety i he's work that uh the microsoft with a and uh in redmond washington U N where he heads the communications and collaborative systems systems research philip has being a wheel so than for the ieee signal processing society most recently each yeah the multimedia technical committee and old so it's a so on the signal processing society by committee each distinguish is a a with it B and twelve a side please piper what in ninety ninety three and more recently in two thousand and seven even a the best paper award from the ieee transactions on multimedia so thank you again guy in to fit it for agreeing to uh make this presentation i i and i'm sure the whole O D and is looking forward to is to all in that it communication it you very much for the introduction it's a great honour to be on the stage especially with fred jelinek over here he was an inspiration to me early in the career and thanks all of you for coming uh to the plan area to at and after have to the banquet i don't think i ever be in at M plan a right after the bank so this a new experience for me i'm kind of looking for to seeing what these plan is actually like so i'll be talking about immersive of communication um there won't be any equations in my slides no signal processing people sometimes don't really understand things and less there this math behind it so i'll give you some references uh signal processing magazine had the special issue in numbers of communication that came that in january you can look at that and just last week a bunch of us submitted a paper to the ieee proceedings as part of their hundred anniversary um both of these are numbers of communication so the last second paper here um the fines of communication to be exchanging natural social signals with remote people as in a face to face meeting in a weighted suspense disbelief uh in being there this is uh not a new idea by any means the telephone was invented a hundred and thirty five years ago and that was the first grade breakthrough an immersive communication it wasn't long after the telephone was invented a here's a cartoon that came out oh three years later uh that people were starting to look at much more immersive scenarios so this is a by george them are yeah a um the caption of the bottom read something like uh uh as as these uh parents are here in london in uh uh uh watching their daughter in ceylon play badminton and father says speech come over here i want to whisper she comes over that's yes pop here uh who who's that charming uh lady playing by charlie side uses a um she's just come over from london um i'll introduce you after the game so we don't really have this kind of immersive of communication system even today but perhaps something that comes fairly close are tell a present systems tell presence uh as defined by the industry conference newsletter wayne house review is a video conferencing experience that creates the illusion that the remote participants are in the same room with you so probably the quickest way to get a sense of what that means today is to take a look at a thirty seconds just go commercial yeah hmmm hmmm hmmm ooh i i i i uh so so one can be uh so here you seen high definition video conferencing so compelling that one participant has forgotten that his counterpart is is remote so this just go tell the present system uh uh and others uh like them H B halo another others um off offer high definition audio and video lots of bandwidth um to try to create this solution of being in the same room um is this a breakthrough in a of communication or is just a lot of uh high definition televisions and a lot of bandwidth well in a sense they're both okay so i contend that these are bridge uh to the future and we're about to see um uh a rapid progress in this area of for the next few years so put that in the context as take a look at um a brief history of television television was invented in its current form and nineteen twenty six uh as a counterpart to the telephone so here you see one of the first television set eighteen T uh bell labs uh next to a telephone because the television is meant to communicate as a visual part of the telephone so here it is eighteen T president walter gifford uh in a you are K at bell labs talking to herbert hoover then secretary of commerce and washington D C in nineteen twenty seven so the first distance television call uh and shortly thereafter the television became the broadcast medium that we know today whereas video telephony um took another forty years to become the eighteen T picture phone uh the picture phone was a a stunning technical success but also a stunning commercial failure so by nine teen seventy nine when i was in intern at bell labs down the hall from me on the desk of my lab director bob lucky was the last remaining working picture phone in the world but would often complain that nobody ever called them on it did a forty years after that uh experience is actually very similar uh except that now these little rectangular images are on general purpose computers um or or on your phone now um but pretty much the experience the same there small little rectangular images of video video conferencing similarly um has been the same uh the for the last forty years in some sense here's the bell labs video conferencing system in nineteen sixty seven you can see the round tape around tables and multiple monitors cameras on top of each monitor or the data monitor um which looks very similar to the you know the recent tell presents as today so why do why contend that there's about to be a series of rapid breakthroughs in immersive communication when the last few years not much as happened in visual communication well several reasons the first is the internet okay so the internet has caused a divorce between the format of the content and then medium over which it is scary so when the past telephone calls were carried over telephone networks television over television networks are over radio and so forth today day all those are carried over the internet so there's a big possibility now of inventing arbitrary formats and they'll all be carried over in uh the second is of course the cost of computation band with an resolution has a in dropping exponentially for so long and that there essentially free now compared to what they work twenty years ago and the third reason is the technology evolves faster than biology okay so they will become bill be a threshold at which the number of bits per second that were able to capture transmit and render uh sir is what we can actually pass through the neural cut set around our bodies okay so at that point um we should have freedom to do whatever we like and the question is what do we want the future of you min communication to look like at that a so to the extent that hollywood is the uh keeper of our collective dreams you know be answer that hollywood would give is that they want communication to be immersive whether it's like uh the uh a hole attack in star trek or the jedi council meetings and star wars or the matrix in the matrix or um in in avatar um communication should be mercer however not all communication will be numbers okay we will continue to send S M S messages uh and the reasons are that there you know there are some reason such as privacy that we we we don't want to send everything about about so this is anticipated you can buy the jetsons and nineteen sixty two so here's change S and putting on her vanity filter before she makes an early morning video call to her friend gloria unfortunately flores on vanity filter has an embarrassing mel function then there's one of my famous my favourite cartoons uh it's this uh from pete steiner in the new york in nineteen ninety three i'm the internet nobody knows you're a dog no know that's still pretty much true today there reasons why uh we don't want to review reveal everything about ourselves and yet for the more complex human interactions a more creative things that we do with each other um we need more margin because we're social animals we have of all to work with each other best face to face so how we position ourselves relative to each other no where we sit what how we just your are are gaze um awareness of all of those things they're all very important and these have been studied um but social psychologists and and others engineers who build system for example uh the work by no i and and can and them the berkeley multi view project it build a system that uh preserves i gaze teleconferencing and they've shown that trust improves uh with correct i gaze so i gaze is bin uh a subject a video conferencing for a long time there many solutions have meters cameras behind screens you interpolation and and many others here's an example of view interpolation where there's an upper camera you when a lower camera view one they can be interpolated to get um better i gave not only I gaze it's important it's uh i reference and gesture so here some examples from the early nineties tongue and min "'em" and showed it chi ninety one a a system that had had a shadow of the remote collaborator behind the the collaboration surface and a year later you she had a same is clear board um where the actual video of the of the remote collaborator was projected onto the on the for the shared sir here's an example uh that's um more recent P H P connect board which uh is an update on it she's clear board in that perfect i gaze is established by tracking the position of the observers had as well as the eyes of the uh a of the the uh a person in the video so that um um the eyes of the on the video are always on the um path between the observer and the camera which is located time the screen at there many other cues the uh days and gesture um both auditory and visual starting perhaps with peripheral awareness of what's going on in the room and consistency between the local space in the remote's space so if you look back to at the tell present systems today you'll see that if these preserve those first two characteristic so per full awareness is provided by very large tell it okay so we know what's going on and that remote room um consistency between the local room and the remote room as is preserved by uh making the the desks look the same painting the walls to be a the same colour and all the different room so uh no difference between the back of of of one room and the back of another um so if you talk to the H P guys um they'll tell you that the halo system which was one of the first which was probably be first um system in this category to come out that was the collaboration between H P and the film the hollywood studio dreamworks and you know how we what is is set you know be know how to they know how to um uh use "'em" uh you um use illusion just suspend spend one's disbelief in actually being there but there are many other immersive of cues uh spatial cues um for example uh distance uh in audio is is uh indicated by a combination of relative loudness uh direct reflected energy and direct reverberant energy ratios as we can see and in these uh these clip oh yeah it's right and i seven is does two yeah so that one's presumably further away than this one oh yeah face a in the next set is does a to that was is probably adjusting the volume but you can uh i'm and of course direction is is given by in true inter aural temporal role and interaural intensity differences between you two years um and on the visual side um there are many many is cues on give you some examples of those relative size and perspective combined to give you a sense of distance and um also absolute size this of the same cues it are used in the famous aims room illusion uh occlusion lighting and shadow are also very important first giving a sense of at distance um in augmented reality it's very important to be able to paint you objects into a scenes such the occlude the background and yet uh do not occlude the foreground uh lighting in shadow were very important not only for real is some but also for um actual depth perception so if you look at this picture of the foot prints in the and um lighting and shot oh give you an impression of one uh a print being impressed into the the and the other one uh being above above the sand so a show of hands how many of you perceive the top foot prints to be the one that's impressed into the same into into the same okay and how many the bottom okay uh maybe one percent think the bottom a ninety nine the top um now keep your eyes fixed on that "'kay" don't blink because if you blink you probably change your mine okay so even even shot all uh a has something to do with depth perception uh focus as well so use a very large city D china uh but if we change the a focus the foreground background it suddenly shrinks down to look something like like a little model that might be uh next year model trains set of course there the the uh very strong cues of skates star got be and motion parallax so some of you may be able to cross your eyes and fuse these images and you get a very strong sense uh of of dat um and all show an example of motion parallax later on but so are these cues uh important for communication that's the question and i contend that the R um clearly are current communication systems do not convey this information and clearly they are not satisfactory are are are not uh um no the can't fulfil every need for communication that we have we continue to meet in person and whenever we are able we continue to travel to work we continue to attend parties and weightings in person we continue to come to meetings like this you know why don't we just email all are talks our slides to each other and just a stay at home well because there's something valuable about we feel about being here in person um and our current communication systems don't convey that but i believe that um the tools of signal processing will help us um bridge that yeah so if we're of a physicist you might take a different approach case of this is is uh i've read have uh actually succeeded in transporting matter uh a small molecule um over us significant distance um and this is how it would work if you were to use it for uh uh transporting you know somebody so you me too a different part the world i would step into a matter transporter i would be frozen down apps the zero i be turned into a little break of bowes einstein compensate inside in the process of schooling down i would it be light light would be transmitted to another place would shine on another little break of bowes einstein kind and set and then my original state would be reproduced of course if i don't plead lee cool down to zero i may leave some state behind hind uh with unforeseen consequences it's also sobering to think about uh what quantisation and data compression would do on um on the way rate distortion optimized mpeg four thousand um but at microsoft research for mostly computer scientists and signal processing people so we take a different approach which is we do the the um transportation uh uh matter virtually rather than physically um and we achieve that through court a transformation so if you imagine a person in a real space over here um he's got some court system in as local space and we just do a coordinate transformation uh of that over into to some no map that coordinate system into the coordinate system of some other states an if we do that for several people then we get them into the same the same so depending on whether the space that there being transported into is a virtual or real um it leads to two different approaches uh i'll um talked about both of these you might sort of a identify the first one with the movie the matrix if you're familiar with that everybody goes into a a virtual world um the second approach could be identified with the movie avatar in which you send a physical representative of yourself into are real physical space so i'll talk about um each of these starting with the first one so the virtual or the matrix movie approach um can be applied to what we call fully distributed meetings so we have people who are and completely separate places around the world maybe your you when your colleagues are writing a paper together and you're sitting in your office is different universities and you wanna talk to each other and the ideas to make it seem as if you're sitting around a see the around the same table um talking to each other so the way we are doing that is by capturing geometry of each person doing a court transformation to put them all into a common scene a life size scene and then you give each person a window on that's scene it's basically there display and uh each person gets a you know i own personal view of that scene um and uh people interact through the in this way preserving i gaze gesture direction and so forth but doesn't matter what that the projection is on on uh a flat surface or a two a multiple monitor situation or maybe even a curve screen so well we've done is build a a a wide field of view curved screen um and from a particular point of view make it appear is if uh uh vocal task is extended uh in space and then when we capture people's geometry using a depth can where that situated on top of the the screen then we can position those people around in space as if they're having a meeting so if you've never seen um what images from a that's camera look like this will give you you give you sense use up here's the still frame from a adept camera see you can render or uh render person from an arbitrary view okay so basically that's what we do capture geometry and texture of um of each participant place them in there's virtual world and then um what them collaborate so here's here's an example of what you might see if you were telling one of these meetings your collaborator sitting across the table from you uh maybe a on the wall on the right there some artwork it's glowing to reflect the mood of the conversation in the room on the back wall perhaps as a large active computer display that can be used for the uh for the meeting um a on the left wall here might be a window out on to the uh skyline of singapore in this case or wherever L C may wanna hold your meeting as other participants join the meeting um the people in the meeting can move around the table to make room people can bring in their own data other people may join the meeting and they come in from ordinary web cams or so so they don't actually have their dept associate with them but they can be represented as as a virtual displays here um data such as this what's showing in the centre of the table here can float above the the table and be manipulated and so forth so these are some of the things you can do um uh uh in this uh this scenario so that image of course um was i i didn't show was taken from one particular static point of view but as people move around uh the image is move so um here's a video that shows uh this the fact of motion parallax so by tracking the position of the camera in this case um you can change what is poor presented on and ordinary um display so that the peers as if there's some that's find the display oh this is just using motion parallax to give that sense depth the same can be done with a audio actually okay so you can uh do you head tracking by you know visually find out where where persons head is point and then um a table look up to find out the relationship between that person's head and uh and this and uh uh a virtual source get the H R T S for that and find out what's the signals that you want to each year similarly you know what the relationship of the head is to each of these loudspeakers and then you can or to transfer matrix invert inverted do some crosstalk cancellation and produce at these loudspeakers signals that depend on where your head is at uh so that the the perceived location of the virtual source stays put so it doesn't drift into one of the that the lots and the one of the speakers as you move towards one speaker apple so we've done experiments on different form factors as well this is an example of the of a wall where uh we've projected onto a whole a graphics screen the same kind of thing you might see in a high tech show you know that store window um and we try to make its a appear through motion parallax and stereo stereo parallax um that the person is standing in the same room with you so i want this take a five minute interlude here to talk about that the cameras because that's cameras at had a recent breakthrough in um in cost and performance so there are new sensor that's really of air are available uh very red to everyone here just like a uh a a webcam cam might be so let me tell you about them um you might have heard about the can act for X box three sixty it's a device like this um that makes use the game player uh the controller and what it was introduced in october last year it's sold eight million units in the next sixty days making it the uh a fast so link consumer electronics device in history reading it some position and the guinness book world records a oh hmmm hmmm and a so that's enough for that uh basically it's a it's a the camera okay so at every pixel it produces a their distance from the camera to something in the scene so it enables you to uh with do uh some skeleton analysis for example of where what people are doing enable you and labeling them to control the games so under the covers it's uh it's this kind of device we have um a regular rgb camera we have an infrared camera uh that's pretty much just like the rgb camera except that has a filtered front of it ooh allowing to see only infrared there's a infrared projector that projects and they'll pattern on the scene and by correlating that known pattern with the observed infrared image um you get a disparity at each pixel in the data also in this unit is a a uh design by even to even on you have who's possibly here uh and uh there's a motor or dsp unit U is P uh use P port so the motor tips the the unit up and down and all of this comes for a hundred fifty dollars so um it's not surprising that within two weeks after it was a it was released it was hacked into and people it up to their P C's and did all sorts of crazy things with it um there's a web here devoted to connect tax and by wow one month after release uh uh there were about ninety projects um um on that site uh in three months there work twenty four pages of the projects uh yeah as of yesterday there were forty seven pages the projects and these these are all crazy things like a um um for robot but uh navigation to um to seeing for the blind two greeting storefront windows that react to to people wandering in front of them uh and so forth and so on so if you're interested um in using this for some of your signal processing i invite you to jump in uh uh there is an open source uh driver or but it doesn't have any of the scale to tracking error or or eve ons uh uh mike array processing uh if you want that you can wait if few weeks uh i believe will be having a um non commercial uh S T K available for all of you to use uh free of charge so i talked about uh talked about how we were using depth cameras uh for burst of communication here's a here's an application that's very closely related and it will be actually released two um X box live gold subscribers um uh sometime this spring so i thought i'd show that to you it's called avatar connect uh rather than showing you an advertisement i've recorded my own video here uh this is corey she's talking to me on her right and to sash on her left her her images the image of her avatars uh in this picture in picture thing i i no who can see here avatars a her emotions so in addition to uh in addition to um still to tracking of arms we have to do facial expression tracking so you know who's uh uh uh when people are talking you have to figure out where they're looking so you can at intimate their avatars and so forth and you can imagine that um uh E there uh many other uh backgrounds you can use uh for this for this uh avatar can thing so you can do a uh a uh talk show for example with your friends uploaded to you to and so forth so you can imagine that as uh the avatars get more and more um sophisticated eventually we may move into any era where inverse of communication is something like a mass of multiplier video game where you know you enter you enter into this world um and you have terms are controlled by your actual motions your expressions and you're oh case alone let me um return to the second approach um this is this real or so called avatar like avatar movie approach um where you send a physical representative of yourself two a real space so this works well for so called and satellite meetings where there several people around you a real space like a real conference room and there are some remote attendees um called satellites lights and they sense some proxy in the me into the meeting so we call these embodied social proxies here's an example of an early uh early device uh a regular L C D used for the the face uh fisheye cameras and pan tilt zoom cameras used for eyes phone for mouth and years all of this is put on a cart is a computer wifi a battery backup and so forth so they can that end meetings for you and these are uh our current meeting we have four of these cards in this particular room uh one here one here one here and one out of sight we have collaborators um around the world john it's and silicon valley and over here's and cambridge england the you know the meeting room itself is in is in redmond washington near seattle um some of these are um on on cards that have to be wheeled in this one is actually robotic it walked through the door on its own the view from these carts of the room look like this okay so you get a wide field of view peripheral awareness of what's in the room and if you want to see it some detail like what's a a on the screen um you click on the screen the pan tilt zoom camera goes over there and gives you a high resolution version so we use these not only yeah for our own meetings but we've done studies on them across real groups that are using them so we deployed then in for different microsoft product development teams where one of the members of the team was remote we the plate them for an eight week period uh conducted interviews and surveys at the beginning and at the end to determined um whether people use these things uh and by asking questions on the surveys that are based on the seven point likert scale yeah um we determine meeting effectiveness awareness of social aspects by asking questions such as i think X has a good sense of my reactions or for awareness i'm where i'm aware of what X is currently working on and what is important to X work social aspects that as i have the sense of closest to and so by comparing this surveys that the beginning and at the end we can determine you know uh whether they've improved or not and um uh quite dramatically we see a lot of improvement uh a across all the four different groups that we put we put them into one also different uh category these are verified by user comments um when user for example so that um it actually succeeds in creating a workable allusion but the remote T member is in the room that overcomes the barrier physical distance and direct observation also showed that in contrast to the uh the audio conferencing that they get been using before is now allows rapid turn taking in conversation um following of what's going on the whiteboard and brainstorming resolving issues uh at the meeting instead of waiting for the next time the person uh a of is it um saving some trips and also assisting not native english speakers to understand what's have uh and all of all of the uh proxies were named and given hats and so they became part of the T these are not a new ideas um in the early nineties uh bill buxton then that be cell and a at their hydra system uh more recently paul of that the bed back um introduce the really cool uh volumetric display uh a few ks uh you and C it's been doing some work on and am at run X uh and there are tell a presence robots and they've even a an to the a boy to attend school even though he has some immune deficiency that was reported by C N and back in january and ieee spectrum and than your times last september both had cover articles on these tell a presence robot user comments in both of those articles feel pretty much the same things that we get also already observed so eric oh itself from ieee spectrum says that you know you participating is a row by you feel you get people's attention and there's a better sense of being there and mike bells there so you get the same kind of or interpersonal connection that you'd have um as if you were at the meeting a interestingly even in the one you know and all those cases in the new york times in and spectrum uh the device is for given names and they were given had are many issues uh save these one i went down to visit any but robotics in uh uh in silicon valley and found that they had one of their humanoid robots lashed to wall after had punched a big hole uh which are still visible in the wall a next to it so um there some safety issues with robots wifi coverage you know what happens when robots from walk out of at of range or run out of power um can make keep up with people who are walking you know how do they open doors or get on elevator you know you don't want people's hacking into your robot button taking it over um and their social issues like well do you want them to C better than humans here better than humans um what height should they be at is socially acceptable to to touch them and move them you know so there are many issues um including um how close should they come to looking human uh and maybe you've seen in ieee spectrum some of you should grows robots these are quite fascinating uh and the not just wax models they actually move i i oh i and so this is a gemini T K here is a is professor have wreck sharp at all berg university and apparently according to ieee spectrum has wife says that she prefers body number one but she's suggest that we should always send body number two to the conferences and stuff so don't be surprised if it future icassp cast you see you know some some robots wandering around so so far i've talked pretty much about two ends of a spectrum and one approach we've had uh these virtual you know everybody transports into to virtual space and on the other approach you uh transport into a real space through your proxy um um in some sense the first is about virtual reality which was uh promulgated by are and land here and the early eighties it's about embedding people into the computers world um the second one is about ubiquitous computing promulgated by mark wise or in the late eighties and so about embedding computation computational devices are computers in are world and so there's really um uh uh these are really at two extremes and i think what's really interesting is is what could be in the middle and the first time might became aware of that space was attending uh a a talk at stanford in that in the early nineties on um uh V V are male virtual reality markup language um it just been invented in the the inventors there were were discussing the virtues of it in showing as um how they had um made a model of the entire university and they were showing is how they could how we could um navigate their way through it and so they wander down the halls of the university and then came to a door of the conference room where we were sitting and i expected that they would just kind of open the door N inside we would you know we would see all of us sitting inside the room um and so i got a nervous feeling about oh maybe behind me through the door will come some giant eyeball ball and then there all be all these people behind a kind of looking at this so i realise that there's is big gap between uh the virtual reality part and and and the physical reality um and so that's an area that uh at scenario i think it'd be uh need a lot more exploration i'll just say briefly that we've been doing some some work in that area um we've called a parallel worlds that one point it's not overlaying real and virtual worlds on top of each other and the people to sort of cross back and forth between one and the other so how might people walk through plaza for example or tent no go to a museum or real these are real places how mike the attendees these remotely and uh experienced um remotely and have also have the people in these real places experience the remote people as if they were there uh i you know in the real space so we you know we'd had applications to attending weddings for example or trade shows and conferences and going to poster sessions how might you have a virtual person come to your poster and you could converse with that person as if they were there so the a lot of interesting signal processing is use how do you instrument the space to capture the the audit tori pardon the visual part um so you can represent them remotely and more a more difficult thing is actually how you rent their remote people into a real space so i want to include here um i've time talked uh mostly from a technical point of view about a of communication um here all just talk about uh this is societal need so climate energy and environment are are L the big words of the day uh i flew here uh from seattle it's five thousand miles why other five thousand miles getting back and the amount of C O two i've released into the atmosphere uh is equivalent to all my local transportation you know i've done this uh just for this week i've used up my years quota of local transportation uh uh probably all of the you know many of you of also flown from overseas um collectively i would say we probably released into the atmosphere coming to this conference about a thousand times of C O two so i don't think it's something we can afford really into the future um in terms of productivity you know i spent three days getting here for for a meeting so that ratio is not really good um but even on a day to day uh basis in at least in the us people spend close to fifty minutes per day on average uh uh commuting perhaps about that ten percent of their work their work um information workers though spend about fifty six percent of their time in communicating with other people um sixty percent of their time feel that they could do there job duties these just as well it's some remote location for example home so there there are things we can do about that and uh thomas friedman so the world is flat but it actually needs more flattening in today's economy we need to bring people to jobs and jobs to people in this is physical security i i guess the volcano one iceland lent started to rub thing again a but a year ago uh a little over your goal for period of three weeks it's uh cost the cancellation of a hundred thousand flights stranded it eight million people across europe uh many of you may have been among that group uh and uh cost the airline industry to one a half a billion euro so uh the economic damage is real um you remember uh i two thousand three was cancelled because of sars so if you know if there a viral out breaks yeah um they can shut down whole economies um uh uh not i mean is N earthquakes uh you know these things can also have a big uh a big um a a fact um and terrorist threats right to more traditional reasons for having uh tell a presence and numbers of um communication have to do with uh bringing families close together and then um pretty health care and education uh around the world so um i'll just uh mentioned that you know many of these national academy of engineering grand challenges um from uh having to do with a the global climate as well as um health care security is are all addressed by and by like tell a press so in conclusion um although visual communication has not really changed much in the last eight years the conditions are right do the convergence of affordable technologies and high social need for human communication to break through to new levels of them are given and uh the signal processing community in a unique position to be able to address uh the needed advances so i think we should embrace the opportune thank you thank you and that's in the a uh it it so that uh an exciting and uh informative a talk i I and the uh feel that we have deal then it faded it can having you do they bring that to okay about new have a yeah yeah how can you tell yeah yeah yeah yeah she anyway we have uh an opportunity or some uh a question in the floor are uh oh we have one at the back if you would got the mike lee well i'm and you think of this end so got yeah order you immersive it in case to touch and what else i in greater converter um feeling so probably very important but uh it's the least well understood there's some uh articles in the uh verse of communication issue about have tex um uh you know i i would say it's least well understood at this point O there's a lot of work to be done so uh basically when i was a a a a it be lot asking the little guys so oh looking at a them was station i ask them one question what about the band i mean you need a lot of they got to so and so one and so there are saying there are using a special line that's right so but everybody can these a special lines so i mean oh do you see that like a like that that's like this amount model of a got or the word you don't on so you're still basically but still you don't so a large amount of data and you need some basic the but well um bandwidth with capacities going to vary very over a very wide range uh and so you can do various uh i think to address lower band with so the avatar can act as an example where you just controlling um you're parametric avatar sending a series of parameters over over the line so it's much more like a uh you know a again um and you still get you still can get someone out of a margin and those situation so i think uh the amount of realism we'll just depend on how much bandwidth with you have available to you um and their strategies for are going down to lower band that's sufficient yeah well how much bandwidth with you need for different amounts of perception is uh is an area first a have that one should be taken this but more of line we have to for one last question oh okay okay i dean man one can not really right i you human you mean would you be able to you come and that share you um so no matter what technology we use there will be things that we just cannot not uh replace with human to human communication is that where you ask these so you know i i i suppose supposed just probably one of the good examples that's that difficult to do right now i i really can't say uh you do the future what whether smell and other things you know i mean i suppose eventually you just the jack to the matrix and put something the back of your brain and then uh replace everything uh you become a a brain in of that at that point uh but um what you lose philosophers last say we're all or or all ready brains sorry that's point two with hence okay we should uh i think uh it again and used to call thing to do X really