Speech Transcript - Modeling Linguistic and Personality Adaptation for Natural Language Generation

0:00:15	right name of lena
0:00:17	i am from the natural language dialogue systems
0:00:20	you see characters
0:00:23	i'm presenting the paper modeling linguistic and first now an adaptation for natural language generation
0:00:29	i
0:00:30	each our view
0:00:32	in the parse tree and marilyn walker
0:00:37	and that the first author is unable to attend its content could be the issue
0:00:44	linguistic adaptation is the phenomenal
0:00:47	and human can first thing in dialogue adjust their be a here is to their
0:00:52	conversational partners
0:00:54	it is also referred to as entrainment for me
0:00:58	here is an example
0:00:59	of linguistic adaptation in the dialogue
0:01:03	if one person said this is why make a less specific apply all users to
0:01:07	specific expression are make a lot
0:01:12	to express the direction
0:01:14	which is considered in certain
0:01:16	feature of an utterance
0:01:18	we call this utterance the prize in response is dialogue are there
0:01:23	can either choose to that on this feature i think yes make allow a specific
0:01:28	avenue or not so that i think yes sir that is the fact that until
0:01:33	we call the response utterance
0:01:35	the target
0:01:37	we will use prime and target rat presentation
0:01:42	linguistic adaptation can happen on different features
0:01:46	for example words referring expressions syntactic structures and discourse markers
0:01:52	previous research has shown that there exists on the correlation between adaptation and task success
0:01:59	and that users prefer
0:02:01	adaptive computer agents
0:02:05	with the right of the intelligent
0:02:08	computer agents such as and of on the lattice a realistic stint in theory it
0:02:13	really need to adapt well certainly make user experience even better
0:02:17	however how much adaptation is needed in natural language generation remains an open question
0:02:26	this paper attempts model linguistic and personality adaptation
0:02:31	for natural language generation
0:02:33	it challenges of this work are first of all we don't really have a good
0:02:38	measure
0:02:39	for turn by turn adaptation
0:02:41	second while most work in this area used human annotated corpora we want to automatically
0:02:47	measure adaptation without human annotation
0:02:52	there
0:02:53	we also want to work with different feature subset
0:02:57	especially feature sets that reflect linguistic style such as lexical and syntactic features hedges referring
0:03:04	expressions and personality features
0:03:09	that's we propose a new venture dialogue adaptation score yes
0:03:15	comparing it with other measures on data requirement
0:03:19	can be computed on-the-fly
0:03:21	almost other measurements
0:03:23	require complete dialogues to calculate
0:03:26	in terms of linguistic features ta yes measures adaptation of feature sets
0:03:32	almost
0:03:33	measures focus on single linguistic features
0:03:36	the goal that is to model adaptation and dialogue progress
0:03:41	whereas most measures thing to find out
0:03:43	what happens after dialogues complete
0:03:49	here's the definition of the s
0:03:51	any dialogue turn can be considered as i
0:03:55	and target if it's following dialogue turn i different speaker
0:04:00	we form a prime target pairs using window size one for most of our experiments
0:04:05	which means only one target dialogue turn back to the prime is considered
0:04:10	in the following equation and it is the number i'm target her s u i
0:04:16	p the number of features in crime of the prime target here and that
0:04:23	a p to the t is the number of features in both prime and target
0:04:28	we can use we describe da
0:04:30	as follows
0:04:32	the numerator is the number of features but i am and target and the denominator
0:04:37	is the number of features and right
0:04:40	within a feature set yes reflects the average probability of feature is in prime better
0:04:45	at that point
0:04:46	in target across all prime target paris
0:04:54	here is an example showing how the a s works
0:04:57	you have to be calculated with different adaptation target
0:05:01	if we used articles
0:05:03	for the dialogue excerpt on the right we have read prime target source
0:05:08	and gas is calculated as follows
0:05:11	using that word based features if we used articles e
0:05:15	there is only one prime target pair f one hundred and d one o one
0:05:23	we apply yes one for dialogue corpora the first one is our walk corpus are
0:05:30	walking the pedestrian direction task corpus or director instructs the follower define public or in
0:05:37	downtown santa cruz the telephone conversation
0:05:41	there are sixty dialogues total approximately four hundred fifty turn each
0:05:46	i walk in it or is designed to test series of adaptation in the y
0:05:53	the second is the walking around corpus
0:05:57	walking around can also work that's free interaction
0:06:01	time task corpus
0:06:02	where director instructs follower
0:06:05	to eighteen destination on campus yourself well
0:06:09	there are thirty six dialogues in total
0:06:11	the hundred and seventy five to eight hundred and eighty five turn each
0:06:16	is the director combat destinations together with labels in pictures of destinations
0:06:22	destination labels are provided to participants
0:06:25	for example twenty however memorial
0:06:28	and shown in section
0:06:30	this is a corpus designed to elicit adaptation behaviors
0:06:36	the third corpus is the map task corpus
0:06:40	widely used in much previous work
0:06:42	a task is a collection of cooperative task oriented dialogs where director instructs a follower
0:06:49	to reproduce a route on a paperback
0:06:52	error hundred and twenty eight dialogues total with thirty two to four hundred and thirty
0:06:56	for each
0:06:58	the map says
0:07:00	a line drive the landmarks with labels for example are there
0:07:07	the last corpus is the switchboard corpus a collection of topics and
0:07:12	direct
0:07:13	it's one thing is telephone conversations an example topic is to find out what kind
0:07:18	of the other colour path
0:07:21	switchboard had very skewed dialogues where two speakers take turns in the main speaker
0:07:27	for example a parsed first
0:07:29	be back channels than a and b switch
0:07:32	a dialogue a
0:07:34	annotated subset in
0:07:36	available for switchboard
0:07:38	them example the annotations are
0:07:40	e for gas pedal that the for statement opinion
0:07:44	as shown in the sample here
0:07:46	there are over a thousand dialogues and the subset with fourteen to three hundred and
0:07:51	seventy three different each
0:07:53	some theories discourse defined dialogue turns as extending over backchannels and we think that is
0:07:59	but a lower ones
0:08:00	to measure adaptation more faithful
0:08:03	so we utilize as the u e d dialogue act
0:08:07	tasks
0:08:07	to filter turns that only contains backchannels
0:08:11	filtering process removes forty point one percent of original dialogue turns only twelve point six
0:08:16	percent of the works
0:08:21	next i will introduce the feature sets we use well-matched previous work on measuring adaptation
0:08:26	use human annotation here we use automatic annotation and feature extraction
0:08:32	it's the end for nlp we extracted unigram lemma for example building and n
0:08:38	and the sample dialog turn and bigram i'm up for example the brick
0:08:43	the thing syntactic features
0:08:46	i taking all subtrees from a parse tree
0:08:48	for example and he structure shown here for than a building
0:08:53	we referring expressions by taking all subtrees with route and he
0:08:58	and removing articles for example a building
0:09:02	we build the dictionary of hedges in discourse markers for example you know and like
0:09:11	we also use linguistic and yuri enquiry and word count
0:09:15	the
0:09:15	to extract higher level features
0:09:18	we automatically classifies words in over at linguistic and logical and topical categories for example
0:09:25	a sample dialog turn has words in categories second or skin and informal
0:09:31	previous work has also shown that the big five personalities correlate highly with certain look
0:09:37	categories for example
0:09:39	extra version correlates highly
0:09:41	i think the motion in swear words
0:09:44	use all my personality
0:09:46	luke feature sets introduce our previous work
0:09:52	we performed for experiments using in the first experiment we validate our measure
0:09:57	i comparing da f-score is of original version is randomized dialogues
0:10:02	randomized dialogue turns within speakers
0:10:06	we calculate the is scores for each dialogue starting with lexical features
0:10:11	and had other low-level features
0:10:13	syntactic structures referring expressions and hedges and finally at the look features
0:10:20	we performed a paired t-test on all pairs of original and random dialogues
0:10:27	scores
0:10:28	the table shows the average dis scores of different p sure that in four corpora
0:10:34	numbers in bold indicate a statistically significant difference in the paired t-test
0:10:40	in almost all paired t-test
0:10:43	scores the original dialogues for significantly higher in the randomized dialogues
0:10:48	the results indicate that the gas measure sensitive to dialogue turn order
0:10:55	in the second experiment
0:10:56	we aim to compare adaptation models across corpora and feature set
0:11:01	you calculate average gas scores for each corpora and feature set results are shown in
0:11:08	the table
0:11:09	comparing columns we have adaptation models across corpora
0:11:14	one or walking around have similar yes scores which indicate that the yes is able
0:11:20	to reflect the similarity of that you corpora
0:11:23	to map task has higher lexical adaptation
0:11:27	and lower personality adaptation
0:11:30	possibly due to more controlled experiment setting
0:11:33	three switchboard corpus have the highest adaptation scores
0:11:38	despite the fact that switchboard is filtered
0:11:41	it is also possible that the social chitchat setting allows why adaptation to can occur
0:11:49	comparing rows we have adaptation models across features
0:11:53	we have highlighted the is course
0:11:55	well they're linguistic features are largely contents if it consists of higher level features that
0:12:02	have a broader categories
0:12:04	so it's high da f-score
0:12:06	are extracted
0:12:08	gas scores for the limit feature range from zero point one forty zero twenty nine
0:12:13	followed by syntactic structure hedge and bigram
0:12:17	referring expressions have the low with es score
0:12:21	among personality features emotional stability agreeable this and in its to experience
0:12:27	traits are adapted more than extra virgin conscientious this
0:12:33	it yes that are shown in the table here is the same are more column
0:12:38	in the previous table
0:12:40	adapted models
0:12:42	and that adaptation model learn dialogue corpora can be represented as the is there like
0:12:47	is in used to control adaptation
0:12:50	e javier's in different nlg architectures
0:12:54	in overgenerate and range we can calculate it da is better
0:12:58	for every generated response and response with the shortest distance
0:13:02	to learn adaptation model
0:13:05	it statistical
0:13:06	our timit pair parra materially sorry error
0:13:10	you know right nlg we can use yes scores as probability based generation parameters
0:13:17	in your not we can encode
0:13:20	where an adaptation model into context catcher in control general generation behaviors
0:13:29	in the third experiment we explore fine grained adaptation models
0:13:34	first examine adaptation
0:13:37	i dialogue segment
0:13:39	i along the segment at feast one number of targets in the first three taskoriented
0:13:44	corpora
0:13:45	switchboard is simply segmented into five parts you can you yes for each segment in
0:13:53	taken averaged across all segments in the same position
0:13:57	in that you features on top of our work
0:14:00	corpora corpus scores for the lose features that slightly decrease of dialogues progress
0:14:06	well expiration two shows a distinct increasing trend
0:14:10	despite being a subset of look
0:14:13	this indicates that as dialogues progress
0:14:15	speaker six the bit more adaptation
0:14:18	behaviors one extra version features
0:14:21	however in switchboard both feature sets display the same decreasing trend
0:14:26	possibly due to the set of the switchboard
0:14:29	and the dialogues progress conversants have less discuss about the topic and are less interested
0:14:37	we then examine adaptation by speaker you compared yes for different speakers
0:14:42	in the figures red lines are da you have scores calculated using target
0:14:47	pupils director or color in switchboard
0:14:50	blue lines for target equals follower or receiver
0:14:54	in general red lines are higher than blue lines which means dialogue initiators
0:14:58	using the bit more adaptation
0:15:01	please refer to the paper for figures other corpora
0:15:06	the last experiment examined the trend of gas course is the window size increases the
0:15:12	window size and mean that a prime is pair and target after it
0:15:17	we begin with the window size of one in gradually increased it to five
0:15:23	is similar to the first experiment we calculate the average gas score for each window
0:15:28	size
0:15:29	in comparing here scores between dialogue in their original order pizzas dialogues with randomly scrambled
0:15:35	as
0:15:37	results show that original dialogues shown in red
0:15:41	lines
0:15:41	e f scores decrease its window size increases
0:15:45	randomized dialogs shown in blue lines yes scores stay relatively stable
0:15:51	it's result is consistent with previous work on repetition tk measures
0:15:58	do you conclude this paper shows that when the model for adaptation the and
0:16:03	on the features that used to different
0:16:06	conversational situations can have different adaptation models three the level of adaptation varies according to
0:16:14	which speakers
0:16:15	and the initiative and for the degree of adaptation varies over the course of a
0:16:20	dialogue
0:16:21	and decreases as the adaptation window size increases
0:16:26	this research was supported by the following grants
0:16:29	if you're time first author is on tonight will be happy to take any questions
0:16:54	you
0:17:09	although not shown
0:17:17	right speaking
0:17:20	and that'll
0:17:23	okay
0:18:16	no i
0:18:18	i
0:18:19	three
0:18:22	one question i wish i
0:18:25	i heard things about four
0:18:26	functions and
0:18:47	you know that
0:18:50	my
0:18:51	is that the first thing about the results reported question
0:18:59	i think a separate
0:19:03	in our case or
0:19:08	o
0:19:09	where r is
0:19:12	the way to extract
0:19:17	you are so that one
0:19:20	if you are information for example very well
0:19:25	you are
0:19:28	there are a
0:19:30	well i don't
0:19:31	mister well i
0:19:36	every part three with and
0:19:39	three
0:19:43	and not everyone that
0:19:46	in a in part so i think i want to use l o l o
0:19:52	or a reprint
0:19:54	adaptation paper we are looking into extract
0:20:00	where
0:20:01	like a whole
0:20:03	another problem of paper
0:20:06	you are
0:20:34	at this work this actually proposed work what working all to infinity the idea
0:20:40	language generation and was you have reached a whole
0:20:54	so
0:20:56	you need to find a match you change the order of it
0:21:02	do you have any
0:21:06	finally a match with a human rights respect people have done
0:21:15	one
0:21:19	your question
0:21:21	what i right there is a war
0:21:25	my question
0:21:28	you need a metric dress i rearrange order
0:21:34	we don't know
0:21:36	if this metric
0:21:39	three what humans would say about whether there's adaptation
0:21:44	so you have any ice metric right
0:21:50	sorry the problem is in most of my understanding is your question
0:21:55	so i rated differently for human evaluation
0:22:02	if a question of or
0:22:06	always useful like that
0:22:10	but not

Modeling Linguistic and Personality Adaptation for Natural Language Generation

Oral Session 1: Generation 1

Zhichao Hu, Jean Fox Tree, Marilyn Walker