a okay good the uh would have to on it everyone i i uh or main come from the a part that can i will present you my work a lot scoring or you know operation which of them we one but no and up on that so uh but buddy basically this plot uh deals we monaural source of separation in uh music as uh spectrogram so we just try to super rights the the signal of inch each uh instrument in uh the mixture but we don't do it's blindly we try to add some extra information which is extracted from the score of the feast and this uh in an extra information is used to get the separation process and yeah the score is a me file which is a line on which is you to be at a line is an yeah that and we do not deal we uh alignment matters so there is a lot of it our job of this and uh will walking system should uh in Q and a preprocessing pre-processing step uh that uh we do is that months between the score and the signal um and more a we only deal with a harmonic instruments so uh we X to them uh modeling up uh back to see instruments in this model so um basically or system is based on a parametric spectrogram model which is derived right from non-negative matrix factorisation and we use these uh parametric metric spectrogram the to decompose the mixture a spectrogram and uh it consists in uh but a tree time-frequency mask which are computed for uh each instruments and which are initialised that and constraints that we as the information in the score and then finally estimated in a very us as a similar way as landing the a that as a uh and then F uh and and so it's to me to feed in the mixture your spectrogram and is then uh there's are metric mask are used to support to this in out of instruments uh a using a no filtering so uh before for uh beginning uh the torque uh i will the try to as well as the question why use a scroll me uh because maybe some people uh will argues that's it is cheating because actually we the signal you don't at the score but actually there is a that's a base of uh me files on Z and ten it and basically you can find almost um and me a you wants a about i ni uh not any but of a lot of uh of of uh a is of music on the net and it's it's a very compact uh description of so you so if you all uh it's much more compact that's the audio it set so if you can store somewhere uh the audio you will be able to store also it is very little extra information and more of a i i would say that's in some cases blind the separation or remains very difficult S and sometimes or place because if you want to separate uh for example uh two voices is of the same instruments you won't be able to uh do we blindly so you're is the outline of my at so first i will remain you the basic principle of and then they get to matrix factorisation and then a a uh i will present use apartment tick spectrogram the L that we do right from uh and then F and a last i we present you or uh wall score informed source separation system and i will present you some results off this system so first let's talk a bit about and the map so and M F is a very powerful row a wrong prediction a C is um are the reason uh as that's a lows to extract read and don't patterns in a negative that a so yeah uh or a nonnegative that that is the um uh the amplitude spectrogram amplitude of or or spectrogram V here it can see on uh a first mode which is play alone and then a second one and then the both play to get a and so if you try to decompose the spectrogram uh we've and non-negative matrix factorisation you when you get to a matrices which is the first one is the at so matrices and you we extract that's some plate of one note so is don't eight which is very read and in this uh uh that yeah and the don't plate of your are nodes and the other my trees yeah as a the matrix H contains the information formation the temporal information so where but it is the notes are and there is um negativity constraints on both this matrices and there is a second constraints uh uh which is uh the rank of the product should be uh uh a lower is and so wrong got the original that data so and F is very powerful to extract them on buttons from the data such as a nodes as a a i are shown and the fundamental probability uh R this technique is the non negativity constraint which was shown to uh provide uh very best it fit up it chill description of the data so we will use this a a non negativity constraints in a or problem a tree yes make the role model uh to a this plastic job uh D mentioned in uh our algorithm so there are some limitation uh we've and then F the first one is that it does not mean to deal uh efficiently we've time-frequency evaluation uh a a as an example when you are speech by edition over time it's very difficult to um of that it's actually right she with and F and so you cannot efficiently model uh phenomena as uh T though and as a on problem for our our approach is uh is that to do some um a score in phone uh a source separation we need a a representation which is more to the uh a the parameters of of interest which are yeah the fundamental frequency of the note so we decided to um develop a a parametric spectrogram more then so our parametric spectrogram of that is based on a pretty use one that we presented in this paper and which is a parameter E spectrogram of that for only one instruments also so for single instrument and to two but this model we just as why does and that's on uh look like and musical spectrogram when you are when most of the amounts or uh instruments notes and well i uh you don't have uh to to back "'cause" use the don't so most of this elements are are money H so the part and at you the buttons that you we dig extract eve and then F will be also so a money so we decided just to put ah a one each atoms directly in the negative much factorisation and a to to meant to rise then to uh i have an excess too uh the parameters of ins there are i of interest which are the for them at that frequency of that um and as a global about uh but uh all block of uh that too so well made or that is a parametric model of spectrogram we've sends at each harmonic atoms and we does to the question is uh of this down on and then at and i i in a the uh at my trees um uh dependency dependent C with respect to the parameter a yeah which is a zero and which would be is the fundamental frequency all that "'em" and we also a and i a and then C uh we respect to um time so basically uh in or model i jam vary over time and is the makes it possible to model L uh something i mean a as a vibrato so here we is a or a very simple that's that's models so basically we a sent size or uh i am by taking the for you transform of the analysis window we use to compute the spectrogram and we just sound it's on the phone them at that frequency it and on the frequency of the difference a money and we just multiplied this uh forty eight to uh this window uh we've and not P to the of the all money so we get these are meant you get some uh there and we thus yet these parametric metric spectrogram for uh single instrument we've uh the parameter uh uh K which is the amplitude of each harmonic yeah you you of the fundamental frequency or each at two on for each time so this fundamental frequency can vary of of time and here you're are uh the activation which is very similar to the activation in in in that and tell you uh where yeah if a notes is active or a a so uh when you try to estimate to make these a a bomb us in red uh so we use the um bit that that since cost function but in fortunately uh this cost function out uh as a lot of local minima we respect to the for them at that frequency uh of course you a local minima i um at the position of the right from "'em" of that frequency but also at the up to have and double up to uh and feast if the notes are very similar so we channel do a global optimization we were like to the can of that frequency so we decided to introduce one at so on for each meeting so for each not off the from i to scale and then the optimization is down uh a locally so we will are uh a fine estimate of so from the most at frequency for uh uh i each atom and each time so here is uh an example of suit the composition that we can get we've uh all model so you're is the spectral of the first bass of uh uh the bar uh first braided which is played by a synthesiser and if we try to decompose it we've all or a i'm we will get this profile of activation so yeah uh i just prison the activation so you don't out a a complete you'd of the when it's and the fundamental frequency estimates but as you can see uh we can recognise uh the note in red yeah which play the uh the first braided by uh uh but so as you can see there are some problems are on you know which is uh uh which are leading to similar to the as uh we've uh or stuff and twelve send double up that but it's not really a problem as we will see later in uh in um you know system of source separation so as you can see yeah uh you have a a a a value of activation for each meeting notes so basic V these looks like a channel role and it would be very important in know you know system that it these representation is linked to uh what's you can get we've me so no we have a a spectrogram model for for a single instruments sets was a present in yeah is that i just present it you and we need a "'em" each model because we want to separate instruments so we i'll a mixture juror with several instruments so the mixture model is very easy uh you just a the single instrument for that for each instrument and some then that's right yeah and you get the mixture your spectral model and we we'll have to estimate for each instrument so for each source K the fundamental frequency at if for each atom at each time the amplitudes or or money for each source and uh they're profile of activation for each source and the they competition is a we've a but if you get evolve vulgar reason uh which uh N sets minimizing a a bit at the since between uh our original uh mixture spectrogram and uh or power meter each steal your spectrogram and this i varies them is very similar to uh as the one you have um for in the math we've met to to get to got that for so no we have a model for the mixture spectrogram and i will show you all we can use it to um and to do some score informed source separation so we have all mixture spectrogram and or or our score and from the score or uh you chan you can know where the nodes all for each instrument so you can very easy um build a the channel role binary general wall each uh uh just tell you where the nodes are and as they say that uh the activation matrix um of each uh instruments is very linked to this general and we will just use this general the days binary a have channel roles as in each sterilisation for or activation in know model but as we use um but if you could you at the true if you put a a zero i was zero somewhere uh in the activation uh uh matt tree as N it will uh it will uh uh it will remain zero all along the iteration so it's a very ah constraints so once we um uh in each yeah eyes our or a parametric spectrogram we get some very coarse parametric expect rounds which are represented here and then we use uh our our goal re oh our algorithm oh the mixture spectrogram to finally uh at that uh this this parametric spectral run to the meat sure six spectrogram and basic iffy the song of is three spectral spectrograms should be uh very similar to the mixture spectrogram that we are and so we get this a time-frequency mask and we can separate as a tracks a using a a of filtering so if is uh on example so it's based and sense it that's a uh because it is the the ground is that um is this the the the the signal is perfectly aligned we've so me defined is that we L so here is a mixture signal i a so you three instruments and using or not a reason a score we get oh i oh is uh the bass the base i it is a two uh_huh hmmm you and he has that's in the again you can yeah the race also i mean it's i i don't care about oh yeah yeah a more than for that which is uh uh which is like noise and we only only of a model for the harmonic many parts and finally the very night i so your it is uh we can probably our algorithm the reason we've uh another one reason which is based on probability to uh probabilistic that some component than a disease uh and which is a somewhat different because you need to send the size a midi tracks first so you need um you need to send to use it and you need to know the instruments of each uh of each trucks so basically we used to it as a uh that that's sets which consists in the same uh um files which are uh plastic classical hmmm it's use each but uh we sent a size uh each we've differ on sound bounce so it's uh so our signals are sent aside from the media because uh we need it to be perfectly online to they would you and yeah uh out the rays in the that we obtain so in red is uh something which is very similar to on all right L so it's like and uh probably me uh for um if you now by based them suppression of them and as you can see our uh algorithm but from quite where am and better on this first the that that's set then the P L C based of a reason and it's are the most um on most the it performs almost saying saying in uh we've a the signal some long and the main difference in this seconds on monk i think it's uh it contains a lot more all uh uh a a lot of uh review of iteration and or or or a reason is very sensitive to that so to conclude we presented and a very efficient you a for uh uh score in informed source separation which is based on the parametric model uh which makes it possible uh to access directly um the nodes and so to uh on the fine he's the sound uh i think that no we should focus on there of instruments bit and all or of uh oh of the on which are not a monique actually because a a or it that is only designed to deal with uh a money sounds and maybe uh included um more complex um uh model they'll for the team or of the elements and also um a a in order to make the more than more robust to uh the real all uh re signal actually and maybe we can also tried to use extra information as uh as the P S C A based on call reason no ween the chamber of the um of the instrument and using supervised learning of that some plates so thank you for uh you attention i and did you make a is so you need compare compare in the gives you is the algorithm for this source separation and you and the you don't from that this we're and uh net may question is what does the the the i'm take model ring with respect to the fact that you put zeros in H the activation matrix so have you tried running the algorithm just like putting zeros in H non in the dictionary and and H and compare it with you know but on a tree can uh model uh i i i don't know this to you crush so that the to that signal to go is the the bottom think there yeah and putting zeros in H yeah and it was a a nice edition which is a constraint to yeah so it's to can uh since and zero not being yeah so you sure you there isn't them longer than we the to building but uh a and then would like to know if you tried uh only by putting a the zero in age and what it to oh using basically is a very coarse estimate of some mask yeah yeah you the it's what you mean okay but it is the interest of apart from being a fine estimation easy job creation yes okay and in so a result actually if you uh look closely uh the spectrogram there are some a evaluation for clarinet and for the base and so if you don't uh you don't to take this valuations into account you read a a very bad as the person results and moreover or maybe you can out some problem of tuning tuning and a the fundamental frequency even if it not moving of a of time uh can be slightly different on from the equal to a month so if you're a yeah this studies i shall i Z to the you but some parameter we a provide bad suppression rise as i think to you anything else use also a quite it's so thank you