Speech Transcript - PERFORMANCE LIMITS OF LMS-BASED ADAPTIVE NETWORKS

0:00:13	S S are french i'll and uh am working with professor say
0:00:16	right i still
0:00:17	uh the topic i going to tell us the about the performance limits of the lms based adaptive that works
0:00:24	so in this work
0:00:26	uh we compared the performance of that if you're and algorithms
0:00:29	bit adder or them for example the uh centralized a block a or mess and uh
0:00:34	uh a distributed incremental a mess
0:00:37	and the we conclude that if we can optimize the come combination weights it coefficients for if you're in tree
0:00:43	rhythms
0:00:44	then we can show that
0:00:46	that if you're in algorithms well be other algorithms
0:00:49	alright
0:00:50	so that start
0:00:51	so that but that team that it works were talking about
0:00:55	uh there consists of uh inter active inter connected to no
0:00:59	and
0:00:59	there are uh interesting a common objective
0:01:03	for example for this graph
0:01:05	a we have bus
0:01:06	eight nodes and their inter
0:01:09	that's assume that you know model here
0:01:11	so
0:01:11	every node the can get it access to some uh or men that they to uh do use book hey
0:01:17	a i i uh and the use of K i
0:01:20	and all the nodes are interesting you to estimate the top you not
0:01:25	and
0:01:27	uh besides the data
0:01:29	the every P can crack the from the local environment they can also exchange some information through with the
0:01:35	a links between them
0:01:36	and to help each other to improve the
0:01:39	uh estimate
0:01:41	and
0:01:42	the diffusion and
0:01:43	strategy
0:01:45	can provide a powerful me can is em's two
0:01:47	uh uh the adaptation over networks here i wa
0:01:51	i would like to number size
0:01:52	the networks
0:01:53	uh can be also uh
0:01:55	you can be a static and that worse
0:01:57	and it can also be a mobile on that the bricks
0:02:00	here um
0:02:02	we're more uh for the mobile L not uh a more well that works
0:02:06	uh every node in the network and moving
0:02:09	so the um the topology of the network it's changing although
0:02:13	a is change always
0:02:14	and
0:02:15	so
0:02:16	here yeah so that will yeah uh introduce some very challenging
0:02:20	uh things to the are written
0:02:23	so the first the
0:02:24	solution is a centralized solution
0:02:27	and
0:02:28	here are you need a a a a powerful a fusion center sitting on top of the network and it
0:02:34	a can uh a connected to every node in the network of work and eight uh i could the data
0:02:40	from every node and the put the all the data together to perform the adaptation
0:02:46	and we note there uh for this kind of a solution
0:02:49	it's suffer from several or uh drawbacks
0:02:51	the first thing is that you have a
0:02:53	power centre which is the super node in a work
0:02:56	so it's a suffer from uh um faders for example if you if the power centre uh used the
0:03:02	a fusion center is down
0:03:04	the everything's go alright
0:03:05	and
0:03:07	another drawback is that it's separates from the random being figures
0:03:11	so if
0:03:11	and one of the link it drops
0:03:13	you you do some node
0:03:15	and you lose some data
0:03:16	this is not a good
0:03:18	a um so we may think about that distribute but the solution
0:03:22	i one possible way is the using the uh incremental solution
0:03:26	uh the increment of solution
0:03:28	eight to uh update to estimate you a sequential manner
0:03:31	here for example look at this speaker
0:03:34	um let's state the
0:03:36	no the one will start uh estimate
0:03:39	uh uh a a will start uh
0:03:41	update
0:03:42	so it first uh it will use its previous estimate top you i as one
0:03:47	and the use its own data to update at this
0:03:50	uh estimate two
0:03:52	i so but uh five but one common i
0:03:55	and the forward at this intermediate that
0:03:57	estimated to know the two
0:03:59	and then the no to to will
0:04:01	but joss to this uh
0:04:02	estimated according to his own data
0:04:05	and so a and a well pass the estimate to the next to no
0:04:09	and so
0:04:10	after node eight
0:04:12	the last the node node the it
0:04:14	uh improve the um
0:04:16	intermediate media according to his own data
0:04:19	it will pass past the
0:04:20	uh
0:04:21	the resulting uh estimated to the know the one
0:04:24	and which you which we will become the new estimate of W i
0:04:28	alright for this kind of a solution
0:04:31	it also so
0:04:32	that the good point is that it does need a power a powerful
0:04:35	to rinse under this is good but it also suffer from or drawbacks
0:04:40	one drawback since the
0:04:42	um it also
0:04:43	so for from the the milling feet
0:04:45	for example if
0:04:46	you uh
0:04:48	it's this
0:04:49	link
0:04:50	is that a maybe you can find another way or wrong
0:04:52	but if this link is the
0:04:54	then you cannot do it
0:04:56	another drawback is
0:04:58	if the network is moving that topologies
0:05:01	changing
0:05:02	then
0:05:03	do in the adaptation you need to
0:05:05	repeatedly calculate or a a cycle through the network
0:05:09	on the fly
0:05:10	this is the not trivial you know that before it is it's uh
0:05:14	and P hard problem you need to do it and usually it's some very very hard
0:05:19	alright right so
0:05:20	we may think about a outer solutions one possible solution is the diffusion strategies
0:05:26	here
0:05:27	a a lucky likes shown in this figure every node will perform
0:05:31	two things
0:05:32	one thing is that adaptation
0:05:34	according to its own data and the thing is that exchanging changing the
0:05:39	uh inter mediate estimate with its neighbours
0:05:42	two
0:05:43	uh further improve his own estimate
0:05:47	a every communication is the down in the local area a so uh it doesn't consume a lot of energy
0:05:54	and uh
0:05:56	usually just all with them is a robust to to the run them three adding feel we be queries
0:06:01	if anyone of the link it's down you can always
0:06:04	and you you you don't lose any noting thing the now work
0:06:08	or and so there are two
0:06:10	different kinds of algorithms one is the called at that then combine at C is
0:06:15	strategy
0:06:16	it's first the perform the adaptation
0:06:19	then do the combination
0:06:21	uh another a one is the
0:06:23	uh combine then that that C T a strategy
0:06:27	uh just
0:06:27	for first the perform the uh a combination stuff and then do that adaptation
0:06:32	i here we and the size that win uh used in a complex combine
0:06:36	a combination coefficients
0:06:38	uh to guarantee the compared
0:06:43	right so before we compare different algorithms rhythms we need to be will a weird of a two important factors
0:06:49	uh of the performance of the uh a if order one is the convergence rate
0:06:54	and a an otherwise the state
0:06:56	a i mean squared error
0:06:58	that's how look at
0:06:59	the speaker
0:07:00	so this is that you learning curve of uh and the long
0:07:03	uh adaptive few adaptive filter
0:07:06	so
0:07:07	basically it you can't do by the the curve into two parts
0:07:10	and uh one part is the trend in the face
0:07:13	and the other part is that steady state
0:07:15	uh in the trends in the face were
0:07:18	interest in the uh how fast
0:07:20	the uh this curve for drop down
0:07:23	and in the type state or
0:07:25	interesting to
0:07:26	how much errors to remains in the steady state
0:07:30	so when you compare different rhythms you need to um
0:07:34	be fair
0:07:35	uh
0:07:36	for example here in this work
0:07:38	because course are more interested in the steady-state state for formants
0:07:41	so we fix the
0:07:43	uh convergence rate of four
0:07:44	every read them
0:07:46	so that means are pretty algorithm we have the same convergence rate in this that in the trends and fate
0:07:52	and the way we compared to uh steady-state mean-square error
0:07:57	and
0:07:58	to a a simplified duration and
0:08:01	to and high light a uh i'd so we use the to note networks
0:08:05	it's simple called but
0:08:06	it it the the
0:08:08	uh considering the um
0:08:11	uh uh the and uh
0:08:13	uh
0:08:15	and the
0:08:16	oh of course that the to note
0:08:18	that the that that that an a work has to knows already use it a lot of a reach and
0:08:23	in interesting dynamic
0:08:25	and
0:08:25	it's easy to uh analyse
0:08:28	right so let's have a
0:08:29	a can't the uh algorithms for to note networks
0:08:33	so this one is for a at C and this one for C T
0:08:37	um the of uh is the combination coefficient of four
0:08:40	at a four
0:08:42	inter mediate to estimate from the one self
0:08:46	a
0:08:46	it's here
0:08:48	and the to
0:08:49	is the
0:08:50	combination weight coefficient of for inter mediate
0:08:53	as to a from to two
0:08:56	alright right um
0:09:00	so after some a a considerable
0:09:03	how our job or a
0:09:04	you are
0:09:05	uh we can get this too close to form
0:09:08	yeah mse
0:09:09	i expression
0:09:11	um
0:09:12	this is the a network average yeah see you we should define in this way
0:09:16	and we can find out that
0:09:18	this the mse is a function of the combination coefficient alpha and the beta
0:09:24	so
0:09:25	we can do some optimization over this two arguments
0:09:28	to minimize this to you messy
0:09:30	and the result is shown in this
0:09:32	uh slide
0:09:34	here we show that
0:09:35	a after some comes out larger bra which is nontrivial trivial
0:09:39	um we can show that this combination weight i'll for you close to this and the bit i close i
0:09:44	one
0:09:44	is done optimal one
0:09:46	and this combination rule is
0:09:49	a a co uses coincides with the um
0:09:52	i i'm true
0:09:53	uh in the digital communication area
0:09:56	which is a for the uh
0:09:57	the rake receiver for this we me
0:10:00	system
0:10:01	and the in this uh two
0:10:04	optimize the the uh combination coefficients backing to a you sees expression
0:10:09	you are get the uh minimize that you man C for a key C rhythm and the
0:10:14	minimize the mse for C
0:10:17	or with them
0:10:18	here the role the uh role is the uh uh
0:10:22	i mean and the commerce convergence mode the for the diffuse are out with them
0:10:27	and a calm eyes the uh
0:10:29	re sure that
0:10:30	the noise of variance the for the two nope
0:10:33	and for D block are a mess
0:10:36	the first thing we need to do is to normalize the starts size the for this are rhythm
0:10:41	bic queries
0:10:42	uh in a texas i so we can show that
0:10:44	if the step size it's very small or
0:10:47	they you block our mass can be uh approximate as uh incremental a mass
0:10:52	so for incremental or mass we have two consecutive tape adaptation steps in one i i iteration
0:10:58	so to current the same convergence rate
0:11:01	we need to normalized
0:11:02	the step size in this way
0:11:05	and the yeah the E M S this are with them in in here
0:11:10	and
0:11:11	the role prime
0:11:13	is the common in the convergence small the for this all words them
0:11:16	and we can have a look at the
0:11:18	if the step size the me is very small this
0:11:21	term is dominant the by one minus two meals segment new square
0:11:25	and in the previous vice we can see the
0:11:28	the dominant calm uh that mean and a part of for this convergence jensen
0:11:31	mode
0:11:32	it's a
0:11:33	also one minus two new segment
0:11:35	use square
0:11:36	so they're almost the same
0:11:40	this is the uh a yeah messy you for incremental or immense
0:11:44	um
0:11:45	similarly we need to normalize the step
0:11:47	and here we gave the revelation to show that if this
0:11:51	stepsize it's small enough
0:11:53	the incremental our mass
0:11:55	and the
0:11:56	block are our mess there are almost the same
0:11:59	just plug in the
0:12:00	equation here and that you can or the high order more terms here
0:12:04	you and up with this expression
0:12:07	and
0:12:08	this is the mse expression for the incremental algorithm
0:12:14	um
0:12:17	here we also propose a we also uh put that the standalone there are mess here
0:12:22	and so there there's no cooperation between the two nodes
0:12:26	then we were compared to
0:12:28	the yeah mse performance uh
0:12:30	with with this over algorithm
0:12:31	also
0:12:34	so this is the uh uh results of the come and
0:12:37	so have a look at this highlights part
0:12:40	so the the optimized you say
0:12:42	it C are with them
0:12:44	can
0:12:45	is a slight will slightly better than the optimized the C T A a and is better than the outer
0:12:50	three algorithms
0:12:52	um
0:12:54	just as shown by D uh theoretical results and also fish
0:12:58	thought uh demonstrated in the simulation
0:13:03	okay guess so this is a for the network every G M S C
0:13:06	a here we we can see a team optimize at scenes the best a one
0:13:11	and another interesting comparison is we compared the individual yeah ms
0:13:16	uh yes the mse
0:13:18	of these stand the long filters with the uh
0:13:21	a diffusion
0:13:23	uh as algorithms
0:13:24	and the result shows that for optimize the if you C
0:13:29	oh of also of the two nodes kind reach a
0:13:32	uh yeah messy you which is less than
0:13:35	either either
0:13:36	one of the individual
0:13:37	uh future
0:13:39	oh this is some very interesting course
0:13:41	this means
0:13:42	even that would not
0:13:43	with the lower noise level can benefit somehow from the information sharing them is the bad and now
0:13:51	this a um
0:13:52	this interesting interesting be course
0:13:54	um
0:13:54	we can imagine that every node if if the node a selfish
0:13:58	it
0:13:59	eight only want to a a when it can uh core parade with the each other or with other note
0:14:04	it want get something from it
0:14:06	if it can a in a from the corporation it well not do it
0:14:10	here we show that
0:14:11	if you can but optimize the combination weight
0:14:14	then every note about
0:14:16	if a the from the corporation
0:14:17	that means
0:14:18	for example we we use these are with them
0:14:20	to model the animal behavior
0:14:23	i think it's uh this is a kind of a a reasonable
0:14:26	a to do it
0:14:28	for at a uh them you need a sum a condition to
0:14:33	to show that the you could note also
0:14:36	a a if benefit the from the corporation
0:14:39	this is the is simulation results
0:14:42	so
0:14:43	first let's have a look at the uh trends in the face so all the are with them have the
0:14:47	thing
0:14:48	uh
0:14:49	convergence rate here
0:14:51	and uh
0:14:52	the optimized the at C and a C T A you can reach the lowest yeah mse
0:14:57	and uh
0:14:59	is used is a slightly better than it
0:15:01	uh it is a it's that slightly better than C T A here
0:15:05	and uh it also shows that
0:15:07	a a block error mess
0:15:09	and uh
0:15:10	uh incremental error T have almost the same
0:15:13	uh status stay the performance here
0:15:16	and uh the worst the one is not corporation
0:15:19	as expected
0:15:21	the simulation per foul he shown here so we use the
0:15:25	uh a few order of the lance ten the step size is uh a point zero or five
0:15:30	and
0:15:31	uh the a noise the and have for no the winds point five
0:15:36	the variance
0:15:37	for no to is a three
0:15:38	and uh we also assume the whites
0:15:41	uh regressor requires or to not there uh the power of the regressor is one
0:15:46	and
0:15:48	with similarly to two thousand uh iterations and over a uh and average the curves
0:15:53	over one thousand file
0:15:56	a here are some references as we can see that
0:15:59	um by using that to field and algorithm we can model manning animal a here it behaviours in the nature
0:16:05	for them up back to maria
0:16:08	D honey bee
0:16:09	and the uh lies
0:16:11	and uh feast screws
0:16:13	and also we can use the are with them for they uh
0:16:16	uh
0:16:17	a me radio radio
0:16:19	alright right so i'm gonna down here
0:16:26	i guess for the gaussian case that you use and use simulations to mikes sense that this maximum margin come
0:16:33	component thing and this is optimal
0:16:35	um oh one if you could make any comments or you done the thing with um
0:16:40	uh uh be tiled
0:16:41	distribution
0:16:43	so that was to
0:16:45	uh
0:16:47	whose what we have done in i would scale in that you always get something from taking into account camp
0:16:52	the bad stuff
0:16:53	but in some more gas in problems uh you the rules to
0:16:57	yeah what this war filling stuff for a right we've know
0:17:01	i
0:17:02	i
0:17:05	is
0:17:05	i
0:17:06	i
0:17:07	deletion i D the message of okay
0:17:09	and the idea here you have to notes one has good noise the other that has bad not at each
0:17:14	one of them is to estimate some channel some unknown parameter
0:17:18	if they do it independently of course T was uh and all we get it was estimate right
0:17:24	now if a a of them to a but it let's see using diffusion
0:17:28	we expect big the bad not to do back to the "'cause" he's getting access also the information from the
0:17:33	good no
0:17:33	one can close it from ellis is is that the good little also do but
0:17:37	even though he's getting bad the information from the but not of "'cause" so that's one conclusion
0:17:42	not to do i have this expression there was no assumption about a gaussian at of these simulations actually good
0:17:47	is also cost and
0:17:49	but the mse expressions that C live
0:17:51	do not the singles and
0:17:52	to so this was not cope was a but the other one close which is very interesting from the table
0:17:56	you go
0:17:56	to the table is
0:17:58	what if you take these these two not state the they and send it diffusion sent
0:18:02	i a sense that can do block lms that only complaining lms processing
0:18:06	so if you just sent that can do block lms
0:18:09	well it do that the then
0:18:10	the this T do to the and the and the and the not to that they okay
0:18:14	and the on set is is actually diffusion will all the for even and diffusion solution
0:18:19	but this is counted that into it right
0:18:21	and then you might say well it's to just send that can do anything light doesn't it just implemented efficient
0:18:26	algorithm a diffusion sent
0:18:28	suppose to can do that but then what's the point the diffusion algorithm can be implemented in it see that
0:18:32	it man
0:18:33	okay
0:18:33	so that you know that it still call this this call he it is a is a of just a
0:18:38	all that was diffusion you can help the form the block lms solution
0:18:42	that implement in a fusion center of P
0:18:45	but the expressions that he did i they do not assume cost it to but i think in the simulations
0:18:49	are showing assume probably and yeah
0:18:53	thank you
0:19:04	oh
0:19:06	you want to you or a region you want a lie for the a be the
0:19:10	coefficient efficient
0:19:12	you do rhymes are using the study the uh
0:19:15	result
0:19:16	could that the uh N E
0:19:18	a better way to derive a adaptive all on the fly
0:19:22	coefficients
0:19:23	sure thank you
0:19:26	a
0:19:28	a good question like i watch is doing is the i in the optimal weights
0:19:32	for optimized the steady-state performance we have another it but actually a the published
0:19:36	it was presented then i cast us to use that and on they paper that B it
0:19:40	would yeah i i i i have that on the fly and the nation we find what the optimal weights
0:19:46	i
0:19:47	and at that it on the fly yeah we have we have done
0:19:50	yeah
0:19:56	i
0:20:03	yes you you know to like this
0:20:05	yeah i i can see we will expressions but do these optimal combine yours require a local information only your
0:20:10	you're required
0:20:11	a it's a statistical profile of an able to find them
0:20:17	uh
0:20:19	yeah have
0:20:19	here the optimal a a cool
0:20:21	combination coefficients need
0:20:23	need to know uh
0:20:24	noise per for L per file course the network
0:20:27	and
0:20:28	uh we uh we are uh
0:20:30	here here were trying to find out some more with that
0:20:33	you can estimate or somehow
0:20:35	to know a noise profile cross the network then you can come up with this
0:20:39	oh that's a good question this expression as you can see that the optimal coefficients depend on the noise profile
0:20:45	in the network okay
0:20:47	so this is it performance to it's that it's that time to tell you what's the best you can hope
0:20:51	for if you knew this information
0:20:53	in the article i first to before the one would you at that is coefficients on the fly that is
0:20:58	done based on a thing as they that that you have that the a lot of there's everything is estimated
0:21:03	on the fly yeah
0:21:05	i think we should move along a get to be fair to the law at to add to the last
0:21:09	is the last speaker here

PERFORMANCE LIMITS OF LMS-BASED ADAPTIVE NETWORKS

Distributed and Collaborative Signal Processing

Presented by: Xiaochuan Zhao, Author(s): Xiaochuan Zhao, Ali H. Sayed, University of California Los Angeles, United States