Speech Transcript - AN APPROACH TO SEQUENTIAL GROUPING IN COCHANNEL SPEECH

0:00:13	a Q
0:00:14	alright so uh we now how a a change gears a it of is a lost uh in the last
0:00:18	presentation uh uh work is ready file separate in speech from nonspeech
0:00:23	interference
0:00:24	so this paper is dealing with with
0:00:27	right
0:00:27	so that you know if is itself is self a speech
0:00:30	a phones
0:00:30	so obvious is it is
0:00:32	uh uh it's is a and tight at different ball games or was different techniques
0:00:36	you
0:00:37	and that this that the live my presentation
0:00:40	a by what's is yeah source
0:00:42	as um
0:00:42	a a a a very uh it's right that that this is a joint work was my last you co
0:00:48	can a comfort
0:00:50	uh
0:00:51	so he a lies off first
0:00:53	of briefly describe the background and um
0:00:56	uh the main idea is to perform unsupervised sequential grouping in this work
0:01:02	i will first your with um on uh
0:01:05	one job being of
0:01:07	a voiced speech that would you do you with a segment of of what i'm voiced speech
0:01:12	oh presents some you should result
0:01:15	so this is a by a a standard explanation of what sick marginalisation problem as
0:01:20	so this
0:01:20	this problem has been extensively discussed in the literature of uh
0:01:24	or or C analysis as well as computational to C analysis
0:01:27	so you can imagine that we have a a mixture
0:01:30	and
0:01:36	or
0:01:37	sky
0:01:38	we have a a a a a mixture here
0:01:39	and this is basic basically time was representation the mixture
0:01:43	and the uh uh typical a this mixture is force
0:01:46	process through a
0:01:48	uh uh the re-segmentation stage
0:01:50	so this that she's actually give sauce
0:01:53	um
0:01:54	a set of
0:01:55	time rip with the segments or we call all
0:01:57	a a as a of uh a it and and each segment is actually use a kind you "'cause" region
0:02:03	you know time for with apply
0:02:05	and
0:02:06	the next stage is a somewhat what is groupings they she's such tries the group ah i would with segments
0:02:11	of cross frequency or you can set as so light is of vertical grouping because the vertical axis is the
0:02:16	frequency are
0:02:18	a through this process we and up with a a set of what we call somewhat what streams
0:02:23	so you someone is in here
0:02:27	is illustrated as a a a um
0:02:30	uh uh by different colour so nor that the each region
0:02:34	uh
0:02:35	i the same kind of those are to correspond to a kind "'cause" region because now segments
0:02:40	have been grouped across frequency
0:02:42	oh to see they power to the main part was a quite of or be not what this paper deals
0:02:46	with
0:02:46	so C want to will be is to
0:02:48	take some of data streams as the import
0:02:50	and
0:02:51	through a process of be in now would like to
0:02:54	come up with segregated speech streams so each stream
0:02:57	you the case of course channel speech species stream should correspond to a single speaker
0:03:02	so that's the
0:03:03	oh the parts
0:03:06	and
0:03:07	or you in this work uh a not as as that we're not you can was somewhat it's grouping so
0:03:11	assume some of that is we we has already been tell with
0:03:13	we just use a a a a a and
0:03:15	recent a pop shop where is now we have developed a call kind of model them
0:03:19	the tandem algorithm uh years
0:03:21	uh is actually
0:03:22	a a that's to things somewhat ten years they uh in tandem wise pitch tracking second is using detected should
0:03:29	do some of it's grouping
0:03:30	i can say it's a a
0:03:32	uh to generate a a a a a voiced speech
0:03:36	and
0:03:37	we this somewhat tennis streams generated
0:03:39	a a why don't station face is number of uh issues
0:03:43	the pose is that somewhat of is streams
0:03:45	a consist all
0:03:46	where you make calm spectral in company frames
0:03:49	because you over a particular time frame
0:03:52	uh there's a subset of have you have not bit onto one source
0:03:55	the remaining in you have you is bit onto the apples
0:03:59	so you you don't have a whole frame you we don't have have ram yeah that else other you choose
0:04:03	or had
0:04:04	has that do ways
0:04:05	and uh sum of that streams are also well for short in duration
0:04:09	which use a trip re
0:04:11	a a cruise
0:04:12	the use of a speaker identification "'cause" otherwise a it a standard technique like you can at a about what
0:04:17	speakers i you know that you know for channel
0:04:19	a speech they come hope play used that speaker any T
0:04:22	to group a sum of data streams
0:04:25	but because that too sure about the be a price uh speaker identification not that i basically of blue
0:04:30	uh much but L
0:04:32	and a comes to i'm speech
0:04:35	uh i was we just but a it recall got that the was because it doesn't have a structure so
0:04:39	it doesn't have a harmonic structure
0:04:41	and is a week and as she compared to the rest of be uh speech stuff
0:04:46	or because of this channel is a guess as this you work to gonna use a model based methods
0:04:51	a well met best but those we have a lot of leeway as you come pretty trends speaker models
0:04:55	and by it also cons with cost the cost is desire
0:04:59	why face a mixture
0:05:01	yeah actually have to not only have to assume not the underlying speakers are at a a a a a
0:05:06	a in the mixture so other was the they this speakers have already have
0:05:10	i
0:05:10	and the corresponding trained speaker models
0:05:13	the second has which is not a little a three task that you actually have to do so you got
0:05:17	an education
0:05:18	a lot of this
0:05:18	co-channel channel speech which is it's a challenging problem
0:05:24	so i D is i would like to address this from a a the unsupervised
0:05:29	uh approach or as you wise perspective
0:05:32	so i do as that we want to apply as you buys class ring
0:05:36	and is sort of like uh you as racial would like to accomplish we already have somewhat any streams
0:05:42	and
0:05:43	we propose this in this a relative a new feature space coach you have C C that you have to
0:05:47	the corresponding to come up all frequency
0:05:50	cepstral coefficients
0:05:51	and these of the reason that about of features that are shown to be a a fact you for speaker
0:05:56	adaptation
0:05:57	add to we use we we
0:05:58	perform this
0:06:00	using do you have sees the as the feature space
0:06:03	and we like to have
0:06:04	this somewhat what streams
0:06:06	uh to be somehow
0:06:07	natural a third
0:06:09	in two group
0:06:11	i if it does not of is they've one group correspond to once you that the other group correspond to
0:06:16	a this to be gone um you know the problem is solved and uh problem is all with with tiny
0:06:19	speaker model
0:06:20	so that's that's the whole it uh uh that is
0:06:23	so you have to apply like real that we have to define a objective function
0:06:28	so that should be a a a a budget a little of of show what should be a a a
0:06:31	pretty objective function in court channel speech i guess that's the
0:06:34	that's a question
0:06:36	so
0:06:36	as we set out we assume we already have a is someone can is group hype of these the call
0:06:41	them G
0:06:42	so a use actually
0:06:44	because we have a a set of someone is streams so cheese is actually a by back
0:06:48	oh one corresponding to one people are zero correspond to the speaker
0:06:52	oh with this
0:06:53	where this five of these
0:06:55	now we can measure two things
0:06:58	wise called the weighting
0:07:00	group
0:07:01	scatter the fix
0:07:03	or as of W
0:07:05	here's ask that so as a probably is added what has got a matter surveys as basic just is a
0:07:09	sum at it
0:07:10	ah
0:07:11	oh i'll a pro blocks all differences between feature there's that the mean back that's that's that's what it is
0:07:17	and now you
0:07:18	since you are to have a a type of this here
0:07:21	i "'cause" or measures so got between groups get of metric
0:07:25	so this is a as a B
0:07:26	now with this too
0:07:28	a a measure is defined
0:07:30	then this is a a a a vice the technique in in class three like i use the phase of
0:07:34	the ratio of between group and we didn't
0:07:38	a a measure is is
0:07:39	to match the group give
0:07:41	and that's of this that so the
0:07:43	this is they what's read and i guess so you have to measure is is and i you you by
0:07:47	one of them and you find a trace of the
0:07:49	of of about
0:07:50	and is press already ready side for measure the ratio as on on uh i come back to
0:07:54	uh
0:07:55	the
0:07:57	not because what you can was core channel speech that is that a nice
0:08:00	uh a constrained now apply
0:08:02	a a desired a cup i'd as a as a penalty term
0:08:05	because "'cause" is not to thing is streams
0:08:07	with a or what the pitch count was can a calm kind of should not be assigned to the same
0:08:11	speaker
0:08:12	what is a not always and i
0:08:14	at present cannot generate
0:08:16	two pitch points
0:08:18	in the same time
0:08:19	i think it shouldn't be any any uh a controversy here
0:08:22	and it is about we can permit as as a penalty term because that as we
0:08:27	the pitch track itself is is a you one you as you've a pitch i it doesn't have any have
0:08:31	a we can just give you that all you you should not a lot and all
0:08:34	any any P H um a or whatever
0:08:37	because the actually has
0:08:38	i be mistakes so we actually just use a a continuous function is basic a sigmoid function
0:08:44	a a the same a bunch is prime try by the number of frames where you have overlapping
0:08:49	pitch contours
0:08:50	within the same speaker or with the same group
0:08:55	oh i be at this to the guy that you have the the are you cannot define objective object defined
0:08:59	called constrained objective function
0:09:01	which is basically a is the group difference
0:09:03	which is the first term as second them is a penalty is as the penalty of are we we we
0:09:08	we we we subtract up
0:09:10	and um
0:09:11	that i'm not as the standard then you be have two times after have to pick out was a trade
0:09:15	for this is that system prior
0:09:19	not given the uh objective function
0:09:22	not a to to is it comes than to we have a a set of
0:09:26	some of the streams the ways
0:09:28	we need a got a single white or back
0:09:31	a corresponding to the so called to more um uh a a will be
0:09:35	a you you a was the are we uh the problem coming from from by searching for year
0:09:40	possible able group use
0:09:42	and we got to figure out of one that keeps the highest scroll i objective function
0:09:47	i we think that in this way of course this is an optimisation problem there are many many a a
0:09:51	an we applied to solve this optimisation problem
0:09:54	because would you do was by do back that is
0:09:56	so one has choice will be the use in um uh genetic out
0:09:59	so we just basic a up that you them
0:10:01	three each binary back the which is a a a single sequential grouping
0:10:06	three that as
0:10:07	eight crime sort in in the um um a genetic algorithm
0:10:11	that objective function nine corresponds to the in this school
0:10:15	a basic a your to see that live actually it's is it once pretty fast even not G has a
0:10:19	reputation of you measure small
0:10:21	a a a big common so we is that i has been is function in the last relation is taken
0:10:25	as a solution
0:10:27	so now we have a
0:10:28	basically a given a solution
0:10:30	to to one organisation of
0:10:33	that speech
0:10:35	so what about voice poses a particular typical uh uh
0:10:38	but a good ability always
0:10:40	and and
0:10:42	speech correspond to some seven consonants
0:10:44	and uh
0:10:45	yeah in spoken english we are trapped on the statistical analysis see if we have a rabbit about
0:10:51	a a
0:10:52	first twelve bits of if so first
0:10:54	all the entire house open
0:10:56	sounds english not correspond i'm boy speech basically down there about eight
0:11:00	um voice
0:11:01	uh phonemes teams
0:11:03	three is unvoiced more also for unvoiced fricatives and wow voice every cut
0:11:08	so
0:11:09	uh i as a quite a be in is a as as a particular good to to your was because
0:11:14	they you you don't have this
0:11:16	they in that you know you can somehow you to extract speech features and that you hold the noise doesn't
0:11:21	satisfy speech
0:11:22	a a this but C is what you do with score channel speech about
0:11:25	the lottery is is is a and uh i'm was speed as we know like be taken a lower is
0:11:30	ready just noise say
0:11:32	you know if you listen to this you know he's
0:11:34	and as in a probable context
0:11:36	it is
0:11:36	just more
0:11:38	oh right so how do we do you um voiced um
0:11:41	speech sequence romanisation
0:11:43	so we apply L all audio segment this was
0:11:47	mice i the previous four
0:11:49	and have i for a a and set all seven best analysis but as as i've says
0:11:53	a a of both voiced speech and unvoiced speech
0:11:56	and then we
0:11:57	take since we are already a segregated voiced speech by this time
0:12:01	the portions of segments that overlap with segregated voiced speech uh could be removed
0:12:06	and the remaining ones of is a corresponding money i'm voiced segment
0:12:10	now with this so this is like
0:12:12	a a a a a bunch or regions
0:12:13	uh in this speaker sh
0:12:15	a unique we we choose a unique of course money into a unique or a segment
0:12:20	not task of sequential grouping becomes a again i sign imply and we labels
0:12:25	to use one of this class a colour region
0:12:30	or do we do all
0:12:31	a the key ideas like these two in the simple white here
0:12:33	we just
0:12:34	very much leverage we well has already been a a which is a we are now have
0:12:39	two streams that correspond voiced speech
0:12:41	now we to i'm was was by using complementary remote
0:12:46	or the was speech which has already been
0:12:49	so you is a a a
0:12:51	eight
0:12:51	so the gave a stream for once because as speaker got a
0:12:55	and
0:12:55	now why correspond the one black was a one zero
0:12:59	not be free for
0:13:00	days
0:13:01	so that becomes a why why but are because black
0:13:04	mean feel we drop less speaking
0:13:05	big compliment your master correspond to all the speaker
0:13:08	right become this all the segregation problem then the condom to are ready just course but it all to speak
0:13:14	a a a we have a company as all this problem and and they are a we are actually have
0:13:17	two
0:13:18	uh do you are with some of the of some of the uh uh use use because of the
0:13:22	a a a i was that's be made by a voiced speech segregation
0:13:25	so basically we have a
0:13:27	well as we get this a complementary mask we remove the time frames that contain
0:13:33	no H
0:13:34	or two pitch points
0:13:36	it be would points obvious
0:13:38	that corresponding to or ready to voice
0:13:41	streams
0:13:41	there is no there shouldn't be any boy speech there so can be safely removed
0:13:45	if has no pitch
0:13:47	i i is bad
0:13:49	P other i'm voice speech in not time for
0:13:52	is that you you uh i'll come back in the time next time right uh in the next slide
0:13:57	and
0:13:57	so basically i you've we remove all of them not there are so that the but there's a which is
0:14:02	will correspond to work or the the complementary mask
0:14:04	a take the company might models
0:14:06	then you have a a single or
0:14:09	a a a a a a a is
0:14:11	segment that i'm was segment is the be label by
0:14:14	ah
0:14:15	but mask
0:14:16	now the see which one has them or or or but i be an ish
0:14:20	the member of yeah mean is that you should ready be a side to that of the speaker
0:14:24	so that's a basic idea are guess in the first of a
0:14:27	um
0:14:28	to in time
0:14:30	a peter
0:14:31	i'm twin climb
0:14:33	but so okay so i need a little
0:14:35	a okay so a is it as well as the way uh would you is there's as as a very
0:14:40	simple and has a clear limitation
0:14:43	which is that this cannot do your with
0:14:45	um or was um voice
0:14:47	portions of the mixture
0:14:49	not that it is evaluation quite was which was speech separation rubber's or S S C
0:14:54	we i'm now we are to did analysis you in on that right corpus
0:14:57	let
0:14:58	frames i account for about ten percent of or or unvoiced speech issue
0:15:02	so messed at this that J applicable to
0:15:05	do you was ninety present a bomb voice speech by there's remaining many tempers and we cannot your with this
0:15:10	is a one
0:15:11	limit that where can i what
0:15:14	oh right so
0:15:14	so is i don't have a well i don't of the goes through this uh you better results pretty quickly
0:15:19	uh we have a you body and a one and read zero db or channel mixtures from S is equal
0:15:23	or
0:15:25	and we compare all
0:15:26	and uh
0:15:28	we that it we the evaluation measure is as a signal noise ratio where we use i your by the
0:15:33	most as one through
0:15:35	and that we compare was a model based a matter
0:15:37	at a is the best mouth the nice to provide a i guess we got it the
0:15:42	and a but he also finds a by uh i don't the of for this method is
0:15:46	uh that what them as a as a as M
0:15:49	grammar
0:15:50	a approach
0:15:51	and you here i'm like it just give the unvoiced voice uh uh i your
0:15:55	so a some of that is
0:15:56	streams but
0:15:57	to look at a and the results which is the estimate someone is frames from the tandem where some
0:16:02	and
0:16:03	all this is is what's better if the two talk result different gender or T G
0:16:08	in the D you case star
0:16:10	the
0:16:11	as wise approach gives you a six point eight
0:16:15	yes small you improvement because they original
0:16:17	signals is is the G
0:16:19	and in the
0:16:20	send and gender case the room was about three point seven that
0:16:23	or or i i we should use the remote five point two test
0:16:27	which is
0:16:28	uh a one point seven that's was away from the idea used right
0:16:32	yeah or is got of didn't make any state
0:16:34	what what a sigmoid
0:16:36	all right so uh well i'm always P she about it as segregation uh evaluation here
0:16:41	we only evaluate over
0:16:43	the um voiced intervals we met signal as racial game
0:16:47	and in this case
0:16:48	uh again one we are just look at the yeah uh a a kind of all results with
0:16:53	uh
0:16:53	as the some of that is frames
0:16:55	plus voice
0:16:57	uh segregation
0:16:58	um
0:16:59	this is the magic gives you a pretty decent
0:17:01	uh improvement impose a model based and and as a wise approach
0:17:05	and gives the the improve the vol almost bought it
0:17:10	a of our results
0:17:11	uh
0:17:12	you put them together
0:17:13	and uh this is actually
0:17:15	this is a why we wish to look at
0:17:17	this always balls
0:17:18	a segregation sick one row will be all
0:17:21	voice each and stick model would be a of a more
0:17:25	uh
0:17:26	and this is and gives you five point seven that's room and
0:17:29	which is that which is a a i was we were a bit would score boards a very nice
0:17:33	but is
0:17:35	and this is a loss them or here this is a
0:17:38	one um mixture
0:17:40	i is
0:17:42	a us english
0:17:46	we
0:17:47	but this is the is the result
0:17:49	and so there is a mixture of a male and a female
0:17:52	i se
0:17:54	email
0:17:55	i
0:17:56	so i
0:17:58	that
0:17:59	male become but was that your body models
0:18:02	i can stand
0:18:06	i standard
0:18:09	with
0:18:11	okay okay come grew a a a a we have proposed a novel unsupervised supervise a approach to see one
0:18:15	normalisation
0:18:16	uh you quote channel speech
0:18:18	and the uh a what or they should all voice species kind you through class ring and um voice speech
0:18:24	is a not through the use of complementary mask
0:18:27	and all you but if is trolls that uh are the which is as a wise for even little better
0:18:31	than
0:18:32	a company that model
0:18:34	i
0:18:35	yeah thanks to much
0:18:40	do we have a short question
0:18:42	command
0:18:45	the i would like to know how much training you would be needed
0:18:49	is needed to start before you can start to separation
0:18:52	uh there's a
0:18:53	uh i to why
0:18:55	so that is not
0:18:57	may it what they as you have a some of its streams channel
0:19:01	as a as real as is gender you and the model
0:19:04	which which has a little
0:19:05	but that
0:19:06	i was ready at not a whole lot
0:19:08	but once you have these
0:19:10	they of of ideas that always to this is the high rate as a wise uh method yeah yeah i
0:19:15	but you have to find the average is of the groups and the total a as before
0:19:20	would be preprocessing some to speak rise of rivers are yeah so that it is a pretty was it is
0:19:25	that we you pay a a mixture which as as as
0:19:28	in it
0:19:29	and that you can or a set of of of it is
0:19:32	a stream
0:19:33	for each time someone is tree
0:19:35	then you basically extract of a features yes it's features
0:19:39	and with the speech as you just one sign it too
0:19:41	the uh my
0:19:44	but yeah
0:19:47	so you two thousand
0:19:48	that
0:19:48	i
0:19:50	wise
0:19:53	a question
0:19:56	and and yeah
0:19:57	and
0:19:57	have you can see the the in this meeting so
0:20:02	we have not consider
0:20:04	right
0:20:05	a a uh a uh uh we
0:20:07	we we are not on it
0:20:09	a and uh i a or channel speech
0:20:12	know
0:20:13	is H H so so we would probably
0:20:16	i my would like this
0:20:17	so two
0:20:18	O K A a and that will problem is for called channel speech without reverberation first that no
0:20:24	oh like and uh you
0:20:25	operation
0:20:27	so you
0:20:27	um
0:20:29	do we expect the uh
0:20:31	yeah we'll on more is the console casing
0:20:35	a
0:20:36	yeah
0:20:39	we work for this
0:20:40	uh
0:20:41	or
0:20:42	yeah a little about he's a uh
0:20:44	so
0:20:45	right
0:20:46	and one had to like to common
0:20:48	uh
0:20:49	we well she actually is a big problem for
0:20:52	uh well mobilisation base
0:20:54	that
0:20:55	uh to model or base
0:20:57	because pitch based group
0:20:59	is very robust
0:21:01	and
0:21:01	um and and set also best analysis is also a pretty good asset
0:21:06	is better
0:21:09	uh
0:21:10	because we will right to create a like this
0:21:12	this
0:21:13	they
0:21:13	fate talk is from all source locations
0:21:16	a not as basic my to uh
0:21:20	you
0:21:23	thank okay thank thank you

AN APPROACH TO SEQUENTIAL GROUPING IN COCHANNEL SPEECH

Speech Enhancement

Presented by: DeLiang Wang, Author(s): Ke Hu, DeLiang Wang, The Ohio State University, United States