0:00:15right name of lena
0:00:17i am from the natural language dialogue systems
0:00:20you see characters
0:00:23i'm presenting the paper modeling linguistic and first now an adaptation for natural language generation
0:00:30each our view
0:00:32in the parse tree and marilyn walker
0:00:37and that the first author is unable to attend its content could be the issue
0:00:44linguistic adaptation is the phenomenal
0:00:47and human can first thing in dialogue adjust their be a here is to their
0:00:52conversational partners
0:00:54it is also referred to as entrainment for me
0:00:58here is an example
0:00:59of linguistic adaptation in the dialogue
0:01:03if one person said this is why make a less specific apply all users to
0:01:07specific expression are make a lot
0:01:12to express the direction
0:01:14which is considered in certain
0:01:16feature of an utterance
0:01:18we call this utterance the prize in response is dialogue are there
0:01:23can either choose to that on this feature i think yes make allow a specific
0:01:28avenue or not so that i think yes sir that is the fact that until
0:01:33we call the response utterance
0:01:35the target
0:01:37we will use prime and target rat presentation
0:01:42linguistic adaptation can happen on different features
0:01:46for example words referring expressions syntactic structures and discourse markers
0:01:52previous research has shown that there exists on the correlation between adaptation and task success
0:01:59and that users prefer
0:02:01adaptive computer agents
0:02:05with the right of the intelligent
0:02:08computer agents such as and of on the lattice a realistic stint in theory it
0:02:13really need to adapt well certainly make user experience even better
0:02:17however how much adaptation is needed in natural language generation remains an open question
0:02:26this paper attempts model linguistic and personality adaptation
0:02:31for natural language generation
0:02:33it challenges of this work are first of all we don't really have a good
0:02:39for turn by turn adaptation
0:02:41second while most work in this area used human annotated corpora we want to automatically
0:02:47measure adaptation without human annotation
0:02:53we also want to work with different feature subset
0:02:57especially feature sets that reflect linguistic style such as lexical and syntactic features hedges referring
0:03:04expressions and personality features
0:03:09that's we propose a new venture dialogue adaptation score yes
0:03:15comparing it with other measures on data requirement
0:03:19can be computed on-the-fly
0:03:21almost other measurements
0:03:23require complete dialogues to calculate
0:03:26in terms of linguistic features ta yes measures adaptation of feature sets
0:03:33measures focus on single linguistic features
0:03:36the goal that is to model adaptation and dialogue progress
0:03:41whereas most measures thing to find out
0:03:43what happens after dialogues complete
0:03:49here's the definition of the s
0:03:51any dialogue turn can be considered as i
0:03:55and target if it's following dialogue turn i different speaker
0:04:00we form a prime target pairs using window size one for most of our experiments
0:04:05which means only one target dialogue turn back to the prime is considered
0:04:10in the following equation and it is the number i'm target her s u i
0:04:16p the number of features in crime of the prime target here and that
0:04:23a p to the t is the number of features in both prime and target
0:04:28we can use we describe da
0:04:30as follows
0:04:32the numerator is the number of features but i am and target and the denominator
0:04:37is the number of features and right
0:04:40within a feature set yes reflects the average probability of feature is in prime better
0:04:45at that point
0:04:46in target across all prime target paris
0:04:54here is an example showing how the a s works
0:04:57you have to be calculated with different adaptation target
0:05:01if we used articles
0:05:03for the dialogue excerpt on the right we have read prime target source
0:05:08and gas is calculated as follows
0:05:11using that word based features if we used articles e
0:05:15there is only one prime target pair f one hundred and d one o one
0:05:23we apply yes one for dialogue corpora the first one is our walk corpus are
0:05:30walking the pedestrian direction task corpus or director instructs the follower define public or in
0:05:37downtown santa cruz the telephone conversation
0:05:41there are sixty dialogues total approximately four hundred fifty turn each
0:05:46i walk in it or is designed to test series of adaptation in the y
0:05:53the second is the walking around corpus
0:05:57walking around can also work that's free interaction
0:06:01time task corpus
0:06:02where director instructs follower
0:06:05to eighteen destination on campus yourself well
0:06:09there are thirty six dialogues in total
0:06:11the hundred and seventy five to eight hundred and eighty five turn each
0:06:16is the director combat destinations together with labels in pictures of destinations
0:06:22destination labels are provided to participants
0:06:25for example twenty however memorial
0:06:28and shown in section
0:06:30this is a corpus designed to elicit adaptation behaviors
0:06:36the third corpus is the map task corpus
0:06:40widely used in much previous work
0:06:42a task is a collection of cooperative task oriented dialogs where director instructs a follower
0:06:49to reproduce a route on a paperback
0:06:52error hundred and twenty eight dialogues total with thirty two to four hundred and thirty
0:06:56for each
0:06:58the map says
0:07:00a line drive the landmarks with labels for example are there
0:07:07the last corpus is the switchboard corpus a collection of topics and
0:07:13it's one thing is telephone conversations an example topic is to find out what kind
0:07:18of the other colour path
0:07:21switchboard had very skewed dialogues where two speakers take turns in the main speaker
0:07:27for example a parsed first
0:07:29be back channels than a and b switch
0:07:32a dialogue a
0:07:34annotated subset in
0:07:36available for switchboard
0:07:38them example the annotations are
0:07:40e for gas pedal that the for statement opinion
0:07:44as shown in the sample here
0:07:46there are over a thousand dialogues and the subset with fourteen to three hundred and
0:07:51seventy three different each
0:07:53some theories discourse defined dialogue turns as extending over backchannels and we think that is
0:07:59but a lower ones
0:08:00to measure adaptation more faithful
0:08:03so we utilize as the u e d dialogue act
0:08:07to filter turns that only contains backchannels
0:08:11filtering process removes forty point one percent of original dialogue turns only twelve point six
0:08:16percent of the works
0:08:21next i will introduce the feature sets we use well-matched previous work on measuring adaptation
0:08:26use human annotation here we use automatic annotation and feature extraction
0:08:32it's the end for nlp we extracted unigram lemma for example building and n
0:08:38and the sample dialog turn and bigram i'm up for example the brick
0:08:43the thing syntactic features
0:08:46i taking all subtrees from a parse tree
0:08:48for example and he structure shown here for than a building
0:08:53we referring expressions by taking all subtrees with route and he
0:08:58and removing articles for example a building
0:09:02we build the dictionary of hedges in discourse markers for example you know and like
0:09:11we also use linguistic and yuri enquiry and word count
0:09:15to extract higher level features
0:09:18we automatically classifies words in over at linguistic and logical and topical categories for example
0:09:25a sample dialog turn has words in categories second or skin and informal
0:09:31previous work has also shown that the big five personalities correlate highly with certain look
0:09:37categories for example
0:09:39extra version correlates highly
0:09:41i think the motion in swear words
0:09:44use all my personality
0:09:46luke feature sets introduce our previous work
0:09:52we performed for experiments using in the first experiment we validate our measure
0:09:57i comparing da f-score is of original version is randomized dialogues
0:10:02randomized dialogue turns within speakers
0:10:06we calculate the is scores for each dialogue starting with lexical features
0:10:11and had other low-level features
0:10:13syntactic structures referring expressions and hedges and finally at the look features
0:10:20we performed a paired t-test on all pairs of original and random dialogues
0:10:28the table shows the average dis scores of different p sure that in four corpora
0:10:34numbers in bold indicate a statistically significant difference in the paired t-test
0:10:40in almost all paired t-test
0:10:43scores the original dialogues for significantly higher in the randomized dialogues
0:10:48the results indicate that the gas measure sensitive to dialogue turn order
0:10:55in the second experiment
0:10:56we aim to compare adaptation models across corpora and feature set
0:11:01you calculate average gas scores for each corpora and feature set results are shown in
0:11:08the table
0:11:09comparing columns we have adaptation models across corpora
0:11:14one or walking around have similar yes scores which indicate that the yes is able
0:11:20to reflect the similarity of that you corpora
0:11:23to map task has higher lexical adaptation
0:11:27and lower personality adaptation
0:11:30possibly due to more controlled experiment setting
0:11:33three switchboard corpus have the highest adaptation scores
0:11:38despite the fact that switchboard is filtered
0:11:41it is also possible that the social chitchat setting allows why adaptation to can occur
0:11:49comparing rows we have adaptation models across features
0:11:53we have highlighted the is course
0:11:55well they're linguistic features are largely contents if it consists of higher level features that
0:12:02have a broader categories
0:12:04so it's high da f-score
0:12:06are extracted
0:12:08gas scores for the limit feature range from zero point one forty zero twenty nine
0:12:13followed by syntactic structure hedge and bigram
0:12:17referring expressions have the low with es score
0:12:21among personality features emotional stability agreeable this and in its to experience
0:12:27traits are adapted more than extra virgin conscientious this
0:12:33it yes that are shown in the table here is the same are more column
0:12:38in the previous table
0:12:40adapted models
0:12:42and that adaptation model learn dialogue corpora can be represented as the is there like
0:12:47is in used to control adaptation
0:12:50e javier's in different nlg architectures
0:12:54in overgenerate and range we can calculate it da is better
0:12:58for every generated response and response with the shortest distance
0:13:02to learn adaptation model
0:13:05it statistical
0:13:06our timit pair parra materially sorry error
0:13:10you know right nlg we can use yes scores as probability based generation parameters
0:13:17in your not we can encode
0:13:20where an adaptation model into context catcher in control general generation behaviors
0:13:29in the third experiment we explore fine grained adaptation models
0:13:34first examine adaptation
0:13:37i dialogue segment
0:13:39i along the segment at feast one number of targets in the first three taskoriented
0:13:45switchboard is simply segmented into five parts you can you yes for each segment in
0:13:53taken averaged across all segments in the same position
0:13:57in that you features on top of our work
0:14:00corpora corpus scores for the lose features that slightly decrease of dialogues progress
0:14:06well expiration two shows a distinct increasing trend
0:14:10despite being a subset of look
0:14:13this indicates that as dialogues progress
0:14:15speaker six the bit more adaptation
0:14:18behaviors one extra version features
0:14:21however in switchboard both feature sets display the same decreasing trend
0:14:26possibly due to the set of the switchboard
0:14:29and the dialogues progress conversants have less discuss about the topic and are less interested
0:14:37we then examine adaptation by speaker you compared yes for different speakers
0:14:42in the figures red lines are da you have scores calculated using target
0:14:47pupils director or color in switchboard
0:14:50blue lines for target equals follower or receiver
0:14:54in general red lines are higher than blue lines which means dialogue initiators
0:14:58using the bit more adaptation
0:15:01please refer to the paper for figures other corpora
0:15:06the last experiment examined the trend of gas course is the window size increases the
0:15:12window size and mean that a prime is pair and target after it
0:15:17we begin with the window size of one in gradually increased it to five
0:15:23is similar to the first experiment we calculate the average gas score for each window
0:15:29in comparing here scores between dialogue in their original order pizzas dialogues with randomly scrambled
0:15:37results show that original dialogues shown in red
0:15:41e f scores decrease its window size increases
0:15:45randomized dialogs shown in blue lines yes scores stay relatively stable
0:15:51it's result is consistent with previous work on repetition tk measures
0:15:58do you conclude this paper shows that when the model for adaptation the and
0:16:03on the features that used to different
0:16:06conversational situations can have different adaptation models three the level of adaptation varies according to
0:16:14which speakers
0:16:15and the initiative and for the degree of adaptation varies over the course of a
0:16:21and decreases as the adaptation window size increases
0:16:26this research was supported by the following grants
0:16:29if you're time first author is on tonight will be happy to take any questions
0:17:09although not shown
0:17:17right speaking
0:17:20and that'll
0:18:16no i
0:18:22one question i wish i
0:18:25i heard things about four
0:18:26functions and
0:18:47you know that
0:18:51is that the first thing about the results reported question
0:18:59i think a separate
0:19:03in our case or
0:19:09where r is
0:19:12the way to extract
0:19:17you are so that one
0:19:20if you are information for example very well
0:19:25you are
0:19:28there are a
0:19:30well i don't
0:19:31mister well i
0:19:36every part three with and
0:19:43and not everyone that
0:19:46in a in part so i think i want to use l o l o
0:19:52or a reprint
0:19:54adaptation paper we are looking into extract
0:20:01like a whole
0:20:03another problem of paper
0:20:06you are
0:20:34at this work this actually proposed work what working all to infinity the idea
0:20:40language generation and was you have reached a whole
0:20:56you need to find a match you change the order of it
0:21:02do you have any
0:21:06finally a match with a human rights respect people have done
0:21:19your question
0:21:21what i right there is a war
0:21:25my question
0:21:28you need a metric dress i rearrange order
0:21:34we don't know
0:21:36if this metric
0:21:39three what humans would say about whether there's adaptation
0:21:44so you have any ice metric right
0:21:50sorry the problem is in most of my understanding is your question
0:21:55so i rated differently for human evaluation
0:22:02if a question of or
0:22:06always useful like that
0:22:10but not