0:00:13okay
0:00:22so that morning
0:00:24i
0:00:27is the what we mean by a classifier fusion
0:00:31of
0:00:31classifier fusion is applicable uh
0:00:35whenever we have some uh and symbol of of
0:00:38i X
0:00:40and we need to come to some final decision don't
0:00:43uh
0:00:45furthermore
0:00:46in this um but example we we assume that those experts are able to give us off
0:00:52decisions
0:00:53in in in a a uh a form of some can fit
0:00:57so so perhaps the simplest and also own mostly working method uh
0:01:03how to fuse those scores would be just to a breach out
0:01:06those confidence values
0:01:09that sometimes we we have some prior information about uh
0:01:13the experts and
0:01:14about better uh
0:01:17is
0:01:17uh
0:01:18in the past
0:01:20um so so we would like to exploit the host
0:01:24this information to to uh make
0:01:27that there
0:01:28fusion
0:01:32so the task of
0:01:33classifier fusion is to take uh
0:01:36the of and
0:01:38base classifiers and uh produce one
0:01:42output score
0:01:43uh which which ideally a uh
0:01:47which
0:01:48we better performance than uh
0:01:53a single base classifier
0:01:58so we now we where we we assume uh
0:02:01so called
0:02:02linear fusion
0:02:04which is
0:02:04a very simple method that but
0:02:06uh
0:02:07i also uh
0:02:09used in the state of the art tools
0:02:11like the focal uh
0:02:13toolkit kit or or
0:02:15that's that the word to at
0:02:18um
0:02:22so a linear fusion is just wait it's sum of of
0:02:26the input scores
0:02:27uh
0:02:28where are the weights are trained
0:02:30uh from from previous
0:02:32uh
0:02:33trials
0:02:35with with the known based through
0:02:41but what we mean by uh subset fusion of uh
0:02:46is that uh
0:02:47in in
0:02:48subset fusion
0:02:50we first
0:02:51uh
0:02:52so like
0:02:53uh
0:02:54only certain classifiers from from the full set
0:02:58and those uh
0:03:00then for C to to the fusion training and and fusion
0:03:06what what
0:03:07could be the motivation for for such
0:03:10to something uh so first for the traditional
0:03:14uh approach uh with the full set
0:03:18it's it's
0:03:19the
0:03:19mostly used method it's
0:03:21forward
0:03:23it
0:03:23computationally efficient since you don't have to do the a subset selection
0:03:30oh
0:03:30but
0:03:31for for the lot and when we have a large number of classifiers
0:03:35uh we
0:03:36could be
0:03:37possibly simply over
0:03:38training
0:03:40fusion
0:03:42virus in in the
0:03:43stops case
0:03:45um
0:03:46we might
0:03:47possibly suitably but there
0:03:50that
0:03:51of course this this uh
0:03:54matt that relies on on a good subset selection
0:03:59so the question is can a subset fusion give better performance than the force
0:04:10oh forty for this system overview uh
0:04:14on the input we have
0:04:15uh
0:04:16speech
0:04:18typically two utterances
0:04:20a
0:04:21those are
0:04:23um
0:04:24uh
0:04:25classified by a classifier
0:04:27uh
0:04:29which
0:04:30i by several classifiers that we that we selected from from of full set of the classifiers
0:04:36and
0:04:37those
0:04:38passive that were selected that and fuse
0:04:47more more in detail
0:04:48uh
0:04:49how we do it
0:04:50is uh we first
0:04:52uh train uh the S skull mapping
0:04:56for for each of the base
0:04:58base classifiers scores
0:05:01a a S come mapping mac maps the scores in
0:05:04uh well calibrated log likelihood ratio
0:05:09um
0:05:11on the one that
0:05:12first
0:05:12yeah a you see you see that as kyle mapping
0:05:17and on this second and uh is is uh
0:05:20cost function C L
0:05:22which uh we minimize
0:05:25uh for the match score
0:05:31okay then then for each of the subset
0:05:33in be uh power set up two
0:05:37you
0:05:37a power of and minus one
0:05:39uh we train a linear fusion
0:05:43uh uh with a C C W L are objective function
0:05:47same same that that's in the focal toolkit
0:05:52a a that one you C
0:05:55in the first
0:05:56uh formal a
0:05:58uh
0:05:58the the prior uh
0:06:01with which the the C W L R
0:06:03function is way
0:06:05comes from the cost function
0:06:07so so for the cost function we we use the new next function
0:06:11but at the cost of miss type of error one cost of false alarm is one
0:06:16and uh a probability of target
0:06:19you're
0:06:20a target trial is
0:06:22zero point zero zero one
0:06:26okay that then after we uh use all the possible subset we we select the
0:06:32subset based on the smallest
0:06:34uh
0:06:35minimum uh decision cost function
0:06:38so the decision cost function of uh is
0:06:41is a function of threshold
0:06:44um
0:06:45and and
0:06:46the
0:06:47cost function parameters
0:06:49so so for uh
0:06:53we we we pick the
0:06:54we pick the one with with the low
0:06:57uh with the minimum decision
0:06:59function
0:07:01and it possible threshold
0:07:04and finally we we still
0:07:06but the actual uh a decision cost function which is
0:07:10the cost function
0:07:12in a threshold in and all the multi racial that we trained on the training
0:07:18a with includes
0:07:20uh uh also the
0:07:21calibration error
0:07:27oh of a our
0:07:28base classifiers
0:07:29uh we had
0:07:30well
0:07:31different
0:07:32classifiers
0:07:33uh
0:07:35which are used in the a i for you called salt to part for the nist two thousand then
0:07:40evaluation
0:07:42um
0:07:43we used three different sets of scores
0:07:46uh the so called train set and it about set one
0:07:50where from the extended nice
0:07:52uh sre sorry two thousand page
0:07:54files set
0:07:56and they are just
0:07:57uh
0:07:59a like they have very similar uh
0:08:01score distribution
0:08:03and then for um
0:08:05or something different you have also to
0:08:07is is is the
0:08:09uh
0:08:10if we shall nice
0:08:11two thousand and a
0:08:13uh evaluations
0:08:20ah
0:08:20so for the results
0:08:22we we divide it uh
0:08:25all the possible subset
0:08:27i size
0:08:28uh
0:08:29from one to twelve since we had
0:08:31twelve classifiers fires and and study different
0:08:35and measure
0:08:37we can get by selecting a good
0:08:39a subset
0:08:44uh
0:08:44but three
0:08:46uh
0:08:47most important point in
0:08:49points in this
0:08:50a a lot of are
0:08:51the worst individual subsystem
0:08:54the
0:08:55uh best individual system subsystems so that was are the sets of
0:08:59size one
0:09:00only only once is them not no fusion
0:09:03and uh the baseline is uh the full
0:09:06in sample the fusion
0:09:08where where all the twelve plus fires
0:09:10so
0:09:10if
0:09:13usual
0:09:16so first for for the blue line uh
0:09:19the blue line shows
0:09:21uh
0:09:22the non of non cheating really realistic use case where
0:09:25we predict the best
0:09:27uh subset
0:09:29uh from the training set and then we evaluate on the about that one
0:09:34so so for for this one unfortunately we we cannot but get the better result than the set fusion but
0:09:40we can get
0:09:41sometimes for for
0:09:43in the size if of seven right
0:09:45and
0:09:46uh we can get
0:09:48a a very similar result
0:09:53and the best subset selection or or four shows uh the best subset uh
0:09:58the uh performance of the best subset uh
0:10:02if if we knew how to select a
0:10:05uh
0:10:07then the worst subset selection or well
0:10:10uh i shows the case
0:10:11uh uh when we
0:10:12cell like the worst possible
0:10:14subset from from the power set
0:10:19so
0:10:19those are uh
0:10:21and and of are bound
0:10:27ah
0:10:28okay
0:10:28this is the same case uh
0:10:31only not to not for the actual dcf but for
0:10:35minimum dcf and you rely right
0:10:38so you can see we we can still uh get
0:10:41but their mean dcf or equal error rate by
0:10:44by
0:10:45not doing the full set fusion
0:10:48so
0:10:49but selecting a subs
0:10:55and finally
0:10:56um
0:10:57this is the performance on the of all set to
0:11:00or or of the nist two thousand ten
0:11:02a
0:11:03evaluation set
0:11:05um
0:11:06and we can also see
0:11:08see that for for most of the conditions
0:11:11interview interview uh
0:11:13interview telephone and telephone telephone
0:11:16the best subset
0:11:17gives
0:11:18that their their performance than the full and sample
0:11:21only only
0:11:22in the
0:11:23mike mike condition there is something wrong
0:11:26uh
0:11:28here uh even the even the full and sample
0:11:32it's worse
0:11:33results than
0:11:35best individual
0:11:49oh
0:11:49uh
0:11:50conclusion of
0:11:51this research is that
0:11:53subset fusion has
0:11:55a then shall to perform the full set fusion
0:11:58course
0:11:59if we knew how to select
0:12:01best
0:12:03uh
0:12:04there are the further study should focus on
0:12:08subset selection methods
0:12:14they i i think that
0:12:15it
0:12:16uh
0:12:23okay
0:12:23we have a a a a a a time question
0:12:28right
0:12:28you this was uh yeah back from please
0:12:31uh i'd like to ask if you use the same subsets for all that i was or different subsets for
0:12:35all the files
0:12:37uh
0:12:38you mean in one of the block
0:12:40or
0:12:41uh
0:12:43i generally so a this is this the system
0:12:47you you put a not that i was to it in
0:12:49yeah
0:12:50do you miss select a different subsets for each high or a no no no now
0:12:55okay
0:12:56so like one cell
0:12:59i
0:12:59okay
0:13:09did you can are you a solution with the random selection of the subset set of positions
0:13:15uh
0:13:18what we mean by a round them
0:13:20just
0:13:20see to D you can you shows one to me
0:13:24a so you have to plot here the
0:13:27to a but the two bound
0:13:29okay
0:13:30well well the random decision
0:13:33somewhere uh
0:13:35in the base
0:13:37oh
0:13:38and when you when you pick randomly you you and up with the performance between them
0:13:43two
0:13:44well
0:13:45and can be could be interesting to do where you these days
0:13:49maybe
0:13:50okay
0:13:50it
0:13:51the on the random selection but uh you what
0:13:54probably like to see a distribution
0:13:57oh okay
0:13:59but
0:14:08because
0:14:10okay do not mess up the speaker