0:00:14hello i'm having problems from university of east and feel and
0:00:19well it's my pleasure two presents my guess that the in this workshop i dunno
0:00:23it's good to be the last
0:00:25among the last the speakers or not but
0:00:29in the following fifteen twenty minutes i will present
0:00:34and effective and simple a out of the detection method over i-vector space in the
0:00:39context of a language identification
0:00:45language identification can be done in two ways one is closed set
0:00:50where the language of a test segment corresponds to one of the instead or target
0:00:57and in open-set
0:01:00where the language of a test segment may not
0:01:03be any of the target languages
0:01:07the task is to classify
0:01:09the test segment
0:01:11either into one of the inset languages for
0:01:15and out of set model
0:01:18one way to perform open set language identification is to training
0:01:25i out of set model from additional data
0:01:31then the data is huge and on and only build
0:01:35the practical key question is
0:01:38how to select the most representative out of set data
0:01:42to model to be all this out of set model in other words
0:01:47how to obtain
0:01:50the higher quality
0:01:52out of set data or additional data to train
0:01:56this an out of set model
0:02:01in the context of language identification the good candidates for out of that they do
0:02:07have some properties deductible of their main properties or
0:02:12i don't set candidates should come from a different lingo is the language families
0:02:19by language families i mean that those languages that have the same kinds of the
0:02:24common ancestor for example a russian ukrainian polish are all from the slavic a language
0:02:34and the second property
0:02:37is that open-set candidates should be pillows
0:02:41into instead languages while others for of a well i because of having at various
0:02:48general out of set model which represents which better represent the ward of out of
0:02:54set data or out of set languages
0:02:58and are some ways to do this
0:03:01dorsum classical approaches one is one class svm where the idea is to enclose the
0:03:07data with an hypersphere
0:03:11and collapsible new data has an or model if they fall within this hypersphere and
0:03:18as out of set out otherwise
0:03:21to other classical approaches are k nearest neighbor where
0:03:26given each data a the sum of its distances between this data and it's k
0:03:33nearest neighbours are computed and
0:03:37the higher this task is the more a confidence we ought to say that this
0:03:43data is outlier is out of set
0:03:46and another classical approaches distance the class means of l if we assume that the
0:03:51data is a gaussian
0:03:54those data that long
0:03:57two or three the standard deviation a bill or eyeball the class name
0:04:02are considered as out of set data
0:04:06what we consider in this study is to use of a nonparametric statistical test known
0:04:12as a whole marker of the smirnoff test
0:04:15it's a non parameter
0:04:18and the idea is to
0:04:21we have two samples
0:04:25we estimate
0:04:26but their these two samples have the same underlying distribution
0:04:31but computing the maximum difference between their
0:04:34empirical cumulative distribution functions
0:04:38well as you could see in this picture this maximum difference is known ask i
0:04:44guess value if it is a great an accurate critical value
0:04:49we can in indicates that this these two samples are from different distributions or in
0:04:56our case from different classes
0:04:58okay how we adopted and two are open set language identification task
0:05:04well even and unlabeled vector w us up a script on i and all their
0:05:09all i-vectors in class barely language l we can he would
0:05:15that a the empirical cumulative distribution functions between this w only and all directors
0:05:22then we will have a
0:05:24if you have a and samples in this language
0:05:28language l
0:05:29we will come up with l individual k s values so we take average for
0:05:35on this
0:05:37individual king is values and then become a bit average k s e
0:05:42that corresponds to
0:05:44and outlier a score of w on i in language
0:05:50we repeat this work other l target languages
0:05:54and then become a bit l average k s values and then we take the
0:05:59minimum value
0:06:00as the final outlier a score
0:06:03for and w only
0:06:05this unlabeled i-vector
0:06:10it's interesting that this that the distribution of this case you values
0:06:15have also a distribution
0:06:18in this in this picture
0:06:21and the and the red bars shows the instantaneous in values meaning that for example
0:06:26if you're in the data class
0:06:28and the red ball strolls the shows that
0:06:33for computing the red bars the in the data
0:06:37those data that correspond to derek the last very used to compute the k s
0:06:41z values and the for the and for the blue wires and the outputs that
0:06:45they to those they don't that do not belong to their equal ask for example
0:06:50very use the computer used you values
0:06:53and interestingly
0:06:55the incipiency values
0:06:58tends to values close to zero and out of set
0:07:02casey value stands to
0:07:04and values close to one
0:07:06so we couldn't see this problem where do directly about looking at that the data
0:07:13the beginning but now
0:07:15we have a tool that shows how instead that out of set data are separated
0:07:20well that's
0:07:22applied in our open set language identification task
0:07:29be applied idea and the and used language i-vector challenge two thousand fifteen
0:07:35the training set corresponds to prevent house and
0:07:37utterance s
0:07:39fifty in that languages
0:07:42and development sets has six thousand five hundred on labeled
0:07:48data and the same amount of data for the test set
0:07:52well the data that was balance between each languages
0:07:56and the dimensions of the i-vectors were four hundred
0:08:00and to be did some post-processing like within class covariance normalisation and
0:08:05linear discriminant analysis
0:08:08and the i-vectors
0:08:12to perform
0:08:15evaluation of the out of the detection methods we need labeled data because the development
0:08:21set didn't have a label was not labeled be used for training set to
0:08:28to be segmented training set into three different portions training you have and test portions
0:08:34so that we have certainly we assign thirty instead languages and twenty out of set
0:08:41and the test portions has all the languages of the instead
0:08:45and twenty out of set
0:08:47and the data was
0:08:49what's didn't have any overlap between these three portions
0:08:57if here is an example of labeling of the out of set and for the
0:09:01out of set a evaluation for example for those data that and their true language
0:09:08was one of the instead languages for example data id one
0:09:13be a label it as instead
0:09:14and for those data that there
0:09:17two language was different done
0:09:19one of things that line from the instead languages
0:09:23we label
0:09:24we label them as out of set
0:09:29here is the results of
0:09:31on a out of the detection methods and our proposed
0:09:35method well case devalues yes i a method outperforms other classical approaches
0:09:41for example in case of svm and knn we have fourteen and sixteen percent relative
0:09:47it all error rate reductions in out of set detection
0:09:54before their f use this baseline systems with k s and we have improvement we
0:10:00have improved all individual systems by
0:10:02by fusing k s e with them
0:10:05and the best performance is fusing k is a bit one class it's we have
0:10:09that resulted in twenty percent
0:10:12it while error rates of around twenty eight
0:10:14individual t s a we dropped
0:10:17the equal error rate to twenty percent
0:10:23let us look at the open set language identification results
0:10:28the table and the different roles in the table shows
0:10:35the they differ based on the data selected for out of set modeling
0:10:40for example we have random
0:10:42we use all the training set
0:10:44all the development set combination of training and development set
0:10:48and the last rule is the proposed selection method
0:10:52as a for the reference purposes we include that the colours that result
0:10:56this results are based on the svm classifier and dark directly reported from the news
0:11:02evaluation website
0:11:05the proposed selection method
0:11:09based on identification results sorry i didn't mention that
0:11:14bill the lines are that identification "'cause" is twenty six around twenty six
0:11:18a performance that nist baseline
0:11:21buys thirty three percent relative
0:11:23improvement the best relative improvement was fifty
0:11:27fifty five percent
0:11:33looking at the for the first rose
0:11:35i think i think additional data well hand held to reduce the identification cost but
0:11:43what not was not bitter and then selecting
0:11:47so selecting in a supervised by selecting out of set a date or in a
0:11:51supervised a
0:11:56here we look at be we compare the
0:12:02casey with other out of the detection methods in the open set language identification
0:12:07well all of them help to
0:12:10all of them and outperforms the that the candles that results
0:12:14but they contain is it is the wiener system with twenty six
0:12:19identification cost
0:12:23we had one thousand five hundred out of set data
0:12:27and you set and fifteen
0:12:31out of that language as we were able to detect what around one thousand of
0:12:36with this method
0:12:38it can use them as that
0:12:40so that the and important thing in this challenge was
0:12:44two bitter detect out of set it change your level when you correctly detect out
0:12:50of set data
0:12:51well in the conclusion
0:12:55in this study
0:12:57we propose to use a simple and effective method to detect out of the data
0:13:03over i-vector is space we showed that
0:13:06this no
0:13:08the that the case in values the proposed method
0:13:12has it nicely distribution
0:13:15and then been integrated to the open set a like this is that we receive
0:13:20thirty three percent relative reduction in identification cost
0:13:24or a closed set
0:13:27okay thank you for attention
0:13:49so if you if you go back to slide fifteen
0:13:56making did you
0:13:59did you try different partitions of in set not observed and the this
0:14:06make much of the difference for your
0:14:09well no we select that's their twenty percent
0:14:12is there content you languages or c
0:14:15so this was on the next slide but you the thirty and twenty you didn't
0:14:18write different portions now do you think this would have made a difference
0:14:25in our offset detection yes
0:14:30yes it
0:14:33i dunno what you mean by making a difference but
0:14:37the results maybe difference but the output
0:14:40will be the same this is the this case system
0:14:43it's something are
0:14:44among other systems
0:14:46i see but maybe the amount by which one
0:14:49whatever the
0:14:51different had you selected
0:14:54which we ran the random it's not supervising on the selected target languages
0:14:59and set and twenty s out of that
0:15:02and the other are there other questions
0:15:17one classes them what the couldn't that used
0:15:22investment coding what was the current that linear yes polynomial kernel
0:15:30between the two images that used
0:15:33that you can that he scanned and one and the ones
0:15:37which one is more efficient
0:15:40which was the first one
0:15:42fast this one
0:15:47my method was fast
0:15:50and knn was also first not a
0:15:54i didn't look carefully at that well the speed but
0:16:00i think goes and this one class svm this the this nonstick plastered to cluster
0:16:07mean and
0:16:08gaussian and canyon unless it
0:16:12the speech or more or less the same
0:16:16but i didn't look at the speaker now step by step
0:16:30if there are no the questions let's take the speaker again please