0:00:16hello everybody again and welcome to class tokenization
0:00:23i would like to say
0:00:25talk about the data driven model of explanation for chat about that helps to practise
0:00:31conversation in a foreign language
0:00:32this work has been done known as i was a at the university of maximum
0:00:36that's why see here to those but
0:00:40and no one with the different this situation
0:00:43this is a different kind of data driven it differs a lot from those data
0:00:48driven approaches that
0:00:50the first keynote speaker at this conference presented a novel but there's still
0:00:55level c l what we can do with the data
0:00:59besides that was the statistical one is just approaches
0:01:03but first let me
0:01:05outline what
0:01:06tends to happen
0:01:08in the next twenty minutes at first i would like to give you a little
0:01:11bit more background about to start itself the nets with was written in the people
0:01:15are just that the it it's a extra lights a premium service for the participants
0:01:20of the conference
0:01:21and then i would just there
0:01:23explain what y dot and the weighted that this way
0:01:27i will present the data a more of a piece of the data and
0:01:32just explain all the empirical findings and then we will go to the maybe more
0:01:37interesting part
0:01:38for you to the computational modelling to a all the race generalization of these empirical
0:01:44findings and to the
0:01:48to the case study psychology implementation case study i will explain why it's like a
0:01:55then we will finish what the overview of the results of the huge to the
0:02:00big battle field
0:02:02where it was a time
0:02:04is started with the
0:02:06artificial companions would be ideal to the machine that interacts with language learners
0:02:12just you know as it is an artificial for and to have a kind of
0:02:16france in your instant messenger
0:02:19it was two thousand eleven it was before the chequebook around
0:02:24and then you just and this check whether the context into account at least and
0:02:29right it's start talking
0:02:31practise for language and this
0:02:33in this and
0:02:35the way
0:02:36but then later i found out okay the wrong on the people what to simplify
0:02:40things and they work in the area
0:02:42cold computers i intelligent computer assisted language learning and so these two things are somehow
0:02:48overlap on the intersection between those fields and
0:02:54we you can imagine how many people from different disciplines already were very natural language
0:02:59processing for language teaching
0:03:01second language acquisition computer science
0:03:04journal corpus research computational linguistics in general
0:03:08and don't
0:03:10on the other hand they so many publications in conversation analysis
0:03:14which exactly focus on the learner
0:03:18interactions between language learners one that non-native speakers nineteen speakers
0:03:23for between only two speakers
0:03:25and no idea you just look what the query
0:03:28at one or
0:03:30one and conversation analysis these buttons to than they require domain we see what
0:03:35can what within
0:03:37okay model
0:03:41i had initially this idea of
0:03:44having a machine that
0:03:45i don't i
0:03:48behaves like a language experts in the channel buttons it is not a teacher
0:03:53because i
0:03:54do you have a clue you can do about what is not exactly i was
0:03:58not in table two
0:04:00so top of what we loaded experiment for data collection because it didn't have any
0:04:06what exactly this person these operators there was a lot that's to me to behave
0:04:10like a language expert in an informal chat
0:04:13and i for like the dataset
0:04:15examples of future work
0:04:17and text
0:04:18you can take it for free it's on the language resource a repository it's in
0:04:24a dataset of truman evident needed only two speaker conversations it's seventy two dialogues it's
0:04:30for now that wasn't
0:04:33that was my treasure
0:04:36so i to this data and
0:04:38a lot okay what is the
0:04:40i met that's of conversation analysis because i didn't have anybody this all
0:04:45what to look for and that's what they call unmotivated looking
0:04:49it just look at a guy without any idea what
0:04:51will you will find
0:04:53and then you may collections of
0:04:55interesting sequences of typical
0:04:58sequencers and then you try to generalize describe prototypical structure of this
0:05:05and then
0:05:07is a computer scientist
0:05:09i then looked at these prototypes and transform them into grammars and roles
0:05:16sometimes it was even possible to do very simple machine learning
0:05:22and then i set up this implementation case that i is a call that case
0:05:26study because you can take a dialogue system
0:05:29any complexity but i two the simplest one
0:05:32i took and ai ml-based chat but
0:05:36that and based language understanding
0:05:37and so how far can go
0:05:41just to give an overview of what have found
0:05:44there are different and directional practice
0:05:46of how
0:05:48participants of an interaction can orient to the air
0:05:53linguistic identities all
0:05:56language learners or language experts in chat
0:05:59it includes a different forms of face working of negation where language learners six q
0:06:06is made matrix you made excuses for they are insufficient knowledge for errors timit health
0:06:12assessment but that was not real self assessment it was on the very beginning of
0:06:16the interaction that was more like
0:06:17you know fishing for compliments
0:06:21they got brace for excellent language learners for their
0:06:24talk one
0:06:27during the during this data collection and then what you are far different types like
0:06:31me to talk about language lorna learning and collaborative learning the people
0:06:37like in the role playing
0:06:39i data x m situations for instance or
0:06:43they compared grammatical systems of their native language used
0:06:48so it was talk about the language
0:06:51and then we have this
0:06:53very prominent type of
0:06:55a positioning
0:06:57hum is not obvious are expecting some bins and somebody writes
0:07:02a different kind of creepy a in this case it was rupiah would linguistic troubles
0:07:06of still that this also problems
0:07:07in all grace repair sequences was
0:07:11because what was what's due to
0:07:14insufficient knowledge of the foreign language
0:07:17and the focus of this talk is marked the rat
0:07:22both their explanations upon request is only one type of text while i'm
0:07:27one subtype of this one type of all the
0:07:33incarnations of a language expert
0:07:36and this is
0:07:37this is the research objective of this paper so i wanted to create computational models
0:07:42of interactional practice
0:07:44where only two speakers of truman in chat the of what troubles in comprehension in
0:07:50a chat but conversation for learning would native speakers
0:07:53why conversation for learning because it was an informal chat but it wasn't this
0:07:58yes the bit the participants met because of the they are status of native and
0:08:04non-native speakers of the with rubber but together because they have these different statuses that's
0:08:09why it was a conversation for learning it was not just the naughty a conversation
0:08:13in this sense
0:08:14why is that challenging i said in the beginning
0:08:17i had forty five thousand about forty five thousand turns
0:08:21and maybe you remember all of elements that in the in his you know talk
0:08:25a eleven reappears
0:08:28that there are challenging for speech recognition or core approximately every two and half torrance
0:08:36i had only thirty
0:08:38so i can i can forget all the machine learning
0:08:43ideally an example of
0:08:47what i five sequences
0:08:49so that the data original data are in german let there are a translations
0:08:56he did not need not
0:08:58non-native speaker has the difficulty to understand or not the i do magic expression and
0:09:04how can
0:09:07a clarification how this clarification is formatted it's just repeat
0:09:12all these
0:09:13probably might think it's not what
0:09:15there is no
0:09:16did you mean how what is a it's just a repetition in the question mark
0:09:21off the dock
0:09:24this is only one
0:09:26format of a repair initiation but there are many others
0:09:29and then
0:09:33after to really be initiation
0:09:36the unknown speaker
0:09:38provides the explanation so we it carries out the repair that the but this is
0:09:42the prototypical structure
0:09:44of repair sequence we have what troubles source
0:09:47which can be everything
0:09:50it never know what will corset
0:09:52problem and in comprehension then there is a rip initiation which can theoretically your occur
0:09:58everywhere even have to silence it has been shown already
0:10:02and then it can be followed by a repair carry out but it doesn't have
0:10:08okay the empirical part
0:10:10would be
0:10:13finished in this place
0:10:14what the what i found was
0:10:17questioning is the praxis but it was not really my finding i just conform to
0:10:21what has been found before for oral interaction but it
0:10:25what the same in chat
0:10:27the right different
0:10:31specific the in the interaction resources
0:10:35that we have a unit chat to signal that we have trouble
0:10:39and there are also a specific interaction of resources
0:10:44that well we half an hour these pet a disposal
0:10:49two point to the trouble source also every pair initiation contains
0:10:56kind of signal and the kind of
0:10:59reference to the trouble source
0:11:03only repeat initiations the time talking about l corresponding
0:11:06to the second position
0:11:08repair initiations
0:11:10so it's the first structurally defined place where the other speaker can initiate but they
0:11:16can still
0:11:18immediate or delay because it and this is because of the of the specific
0:11:23structure of chat because we can
0:11:25just you know have mount multiple threads or
0:11:28in certain things in between and but
0:11:30that is they steal the su of all the same type of second position
0:11:35but some of them come directly after trouble source or and some of them a
0:11:39little bit later
0:11:40and the this has an influence on the resources that need to be
0:11:45employed for the area a pointing to this trouble source
0:11:52i am used on this example
0:11:55there was a repeat as a as a as a as a reference to the
0:11:58trouble source used
0:11:59but their own
0:12:01because we have to deal with non-native speakers
0:12:03but cannot say that
0:12:06as syntactic i syntactic you can be repeated
0:12:09i it regardless of unit boundaries so a piece of
0:12:14trouble source to an can be copied and pasted
0:12:17so we i do you cannot rely on the on the completeness of the second
0:12:22syntactic structure
0:12:24and then
0:12:26what is very common for all interaction
0:12:29but it's
0:12:31i can find it in chat
0:12:33that when you didn't understand something completely i just acoustically but because it's difficult to
0:12:39follow the overall talk native speakers of mandarin native speaker sometimes
0:12:45then i is the repair
0:12:47and just the representation of the troubles source
0:12:50is it is okay is acceptable you don't find it in chat case you can
0:12:53just really the everything
0:12:56but still i have was surprised at
0:12:59some people really
0:13:00i read it in the wrong way but it becomes usable not through the rip
0:13:05in each iteration
0:13:06but there through
0:13:08i don't things where people try to repeat so that every time the that things
0:13:13and you see from these retyping that they we applied
0:13:16role labeling
0:13:19and there are there are also
0:13:22things that are typical for money non-native speakers
0:13:25and if we have very much from the native speaker talk it's to the design
0:13:29of the repair itself so it's
0:13:32it's more about the sense of the word that it explains the meaning of the
0:13:36word order the meaning of the of the of the use this
0:13:41yes semantic unit
0:13:42and their it's less it's less about it
0:13:46something like functional or a foreign
0:13:51the intention or something like that it not an intention but with the meaning of
0:13:54the word was
0:13:55repeat or explained
0:13:58for their repair
0:14:00carry out
0:14:01the of the
0:14:03participants you was used a different direction results again
0:14:07like it just looking synonym so paraphrases
0:14:11but sometimes they also just
0:14:13you know use google translate
0:14:15and translated everything in the native language of the l two learners
0:14:19not to be added one going out to be funny or something they translated that
0:14:24really with machine translation and that not explanation
0:14:30or they just the arm
0:14:32because it was difficult explain some of the phenomenon a like what is that what
0:14:37is a
0:14:38lapsed we it was difficult explain than words and they just
0:14:42pasted linked one example
0:14:44and then it was clear somehow and
0:14:51again to the same as a rip initiation survey carry out can be delayed or
0:14:55immediate but the same reasons
0:14:57and we have a distinct is
0:14:59so that it was type of repeat
0:15:02very pi carry out here is a
0:15:05and i so i called it's speech reap here if
0:15:08l where
0:15:11utterance is unclear
0:15:13or a longer part of a longer utterance is unclear the and not
0:15:17every word is explained somehow but
0:15:20only something that is supposed to be difficult
0:15:24so with that it is clear that didn't
0:15:26units in each difficult unit is explained but not everything is rephrased all par for
0:15:33a store
0:15:37elaborated somehow
0:15:39so what we need to know for the chat what's your
0:15:42and first
0:15:43what does the chat but
0:15:46need to be able to
0:15:49do the same joke was a native speaker do you hear the first to chatbot
0:15:53needs to recognise we can initiate and then detect what is to extract a trouble
0:15:58source and then generate a repair proper because you cannot predict
0:16:02what it will be you cannot just used
0:16:04scripts for ep is forty packing it needs to be generated from what linguistic database
0:16:10and is what i've done so why i just used
0:16:15as the linguistic resources and a field templates with the knowledge from the dictionary
0:16:19and the interactional resources at which my machine looked where
0:16:24all these signals that are found in any
0:16:29corpus and with question marks dishes and
0:16:31a quotation marks and then lexical and things like unclear or i don't understand
0:16:37the directional resources not allowed to print the trouble sources include repeats but also just
0:16:43the adjacent addition because lp initiation may consist of only three question marks and then
0:16:48only the position of this trip initiation points to the trouble source target it's exactly
0:16:53the previous turn so these
0:16:55but for instance this type of
0:16:59pointing cannot be used in the delay position
0:17:04for the implementation case study that said i used
0:17:07and i ml based chat about it was
0:17:10the program d its name a limb interpreter for german and their use the as
0:17:14a baseline this german the emails that
0:17:16we take standard by several categories allow that element is to render the
0:17:22rip here
0:17:23carry out
0:17:24based on the island imaginary
0:17:26now let's and i added to processors the processors in the in problems
0:17:31process different tasks and i added to different tax that the law to do with
0:17:36three pairs that was down explanation and meaning tag why this three because
0:17:44we have a
0:17:47two different types of questions
0:17:49that there are kind of baseline questions to which all the rip initiations can be
0:17:54mapped it's
0:17:55apple are questions requiring a yes or no
0:17:59hence there were it's a content question and out that requires an explanation like synonyms
0:18:05of paraphrases
0:18:06which translation and then
0:18:09i need to distinguish between
0:18:10two of down
0:18:12i automatically and that's why all the all the request were mapped only two
0:18:16to this functions and there's white
0:18:18i had only these two processors
0:18:21what does that mean for the linguistic knowledge that we need for not
0:18:25it to recognize repair initiations it might be sufficient just to have this pattern based
0:18:29language understanding
0:18:32and determine formats that o can be used to initially a creepy a
0:18:37can be described as patterns
0:18:42we have still real related nlp problem sets are really hard for either princess referring
0:18:46expression generations because our pointers to the to the trouble source
0:18:51are referring expressions
0:18:54but only the domain is a different one we have don't have the whole conversation
0:18:57only in this a local rupiah domain low in the local bps sequences what we
0:19:03need your
0:19:04and in contrast to
0:19:07to the other two d or overall problem of they're referring expression referring expression generation
0:19:13there we are normally nouns and
0:19:17pronouns i seen as the main the main results for that here we can see
0:19:23entire sentence or sentences or phrases or works because a repetition of labor
0:19:30points to the trouble source of them were
0:19:35then for the
0:19:37repair carry out
0:19:38we can use
0:19:40as a set their definitions paraphrasing synonyms translations and demonstrations and you know probably that
0:19:46paraphrasing is a hard problem
0:19:49synonyms is hard problem
0:19:50finding it automatically
0:19:52it's also hard to say if the if you're
0:19:56to in a
0:19:59confirmation in a in
0:20:03i mean exact situation but to use it things are expressed
0:20:05that's this one mean the same as this one it's hard to say
0:20:09yes or no just
0:20:11without specific resources
0:20:14but not worse
0:20:15low numbers
0:20:16is not the only challenge other challenges contingency
0:20:22utterances form as rip initiations can have also different accents on their functions like jokes
0:20:28or error correction
0:20:30or rejection of surprise are many others
0:20:33and that's why
0:20:34it remains still challenging because i don't have a solution
0:20:42it is so i have i don't only one minute but maybe the time is
0:20:45over again
0:20:47so i i'm just i just a finishing we have different
0:20:53results forget regarding the complexity of rip initiations and their repair carry out i compared
0:21:00with literature that i us in before with work well by david
0:21:04that's line and
0:21:06work from conversation analysis like documents and it by the way we are this for
0:21:11like described
0:21:13rip initiation formants
0:21:15across languages and their own
0:21:18i think that it's quite
0:21:21and that's why
0:21:23for me to the most of the most
0:21:28positive outcome of this work was that they can use this model
0:21:32first the cover other languages and second to cover other domains because definition talk works
0:21:38in the same way in engineering and model and in every other domains what i
0:21:42need to explain something
0:21:44and then
0:21:45a so i can go beyond duty cycle
0:21:49application case
0:21:51just to zooming out ic not
0:21:53conversational this
0:21:55method helps understand what's going on in human interaction and help to
0:22:00our conditional models and them into built on a
0:22:03but we need datasets good data set of
0:22:06good quality is really large
0:22:08but of a specific quality
0:22:11we take a to speech sixteen systems that we want to simulate in the and
0:22:15so we i want to simulate a
0:22:18dialogue between line i learner and an artificial friend i want to see first how
0:22:24it works in a similar thing i cannot take
0:22:28an interview for that
0:22:30as an days
0:22:31and the
0:22:32maybe we can have
0:22:33just simple chat bots is amenable waibel product in this case but
0:22:38if you want to cover everything it's but it becomes
0:22:41very quickly and a complete on we need all the end of knowledge that
0:22:45well that the that people had produced you know
0:22:50cover all the phenomena that interest
0:23:03okay we have the two and half minutes for questions
0:23:26so i'm also interested in computer mediated human interaction
0:23:31and i wonder if did you serve in these interactions some kind of the interleaving
0:23:36of comments
0:23:37"'cause" i imagine that would be the problem with two humans having a conversation over
0:23:41messenger rather than a human robot because they're we would be more interleaving
0:23:45in like the manner that people do in
0:23:48spoken conversation
0:23:50a about fitting
0:23:52how much interleaving is there between the utterances of your computer mediated dialogs and rt
0:24:00similar to spoken
0:24:02it was between can be that can be eliminated in spoken
0:24:06is there a lot of interleaving of
0:24:10i didn't compare datasets i only compared what i found to go to define it
0:24:14will findings that are described in literature
0:24:18and the
0:24:20the right
0:24:24there are things that i the same
0:24:34formats all replay initiations
0:24:38some of them are the same as an oral interaction
0:24:42the because we have different directional resources available in chat
0:24:48we don't have the prosody for instance we don't have data the phase we don't
0:24:51have the voice
0:24:52i am that they are they are somehow replaced internet by other things like a
0:24:58motion a multi consider and instead of laugh
0:25:04when determining when you want to twenty participants wanted to emphasise something
0:25:09they made uppercase a word stretches
0:25:12or i had
0:25:14one example
0:25:16the data collection that took place in two thousand twelve a what is it
0:25:20european some cocoa
0:25:22football cup and the at this time and sometimes participants that just typed at the
0:25:26same time or in front of their t
0:25:28and watched again
0:25:32and that's how i don't know the word german work goal for what
0:25:37sixty two holes in the high
0:25:39and this is really what you say well what the what a reply data and
0:25:44then to relative to these oral while and when they screen
0:25:48and so it's
0:25:49i would say
0:25:52there are the same things but the expressed by different directional resources
0:25:57that's the first thing and the other thing is some of the things
0:26:00cannot be replaced because they become
0:26:03irrelevant because
0:26:05but don't we don't have the voice for instance because that's why i didn't find
0:26:09any repair initiation that require the repetition after that because it's not necessary you can
0:26:14read everything but these are two differences that i would describe
0:26:20okay so what to do this
0:26:29one of things you just informed about the database is that it's montague that you
0:26:34mentioned supervector doodle for one straight or something but effective some just curious with mobility
0:26:40longitude minimum assessment luminance
0:26:43perhaps not increase the learning used to do something like that because but also potentially
0:26:48useful project work because you want so what kind of increasing importance density distribution of
0:26:53the material is thanks so that evaluating was not the focus that just adding but
0:27:00whistle normally when you talk about a talk about learning or at least
0:27:04i'll with this is second language acquisition theory to your in the background
0:27:09well normally people look at error corrections as a sign for learning
0:27:16or any kind of a meeting negotiation a call it may negotiations all these repair
0:27:20sequences that it or are we explain to date technical it meaning negotiations you know
0:27:26and then
0:27:29this may be costly also obvious
0:27:33normally only these two things are an online but
0:27:38i so also
0:27:39the learning for all
0:27:47i'm sorry at
0:27:49i forgot the word
0:27:51in this
0:27:54but they wouldn't but there is just the
0:27:56the null something or didn't use a structure and then
0:27:59based on example of that and repeated that without any rate wer so that you
0:28:04want to say that but
0:28:07you know but
0:28:08not observation but
0:28:10making likely making a native speakers
0:28:13and then and then no i found also that
0:28:17they learn from implicit corrections which are really hard to capture which are normally not
0:28:23use the bathroom research
0:28:25or not the not no not the that they are not use the use of
0:28:28the wrong word but normally people don't pay attention to that because it's not evident
0:28:33enough it's not
0:28:35a node in there is no evidence that people butlers notice these corrections
0:28:40but i have evidence that
0:28:42in the data
0:28:44because they've repeated things that have been corrected through implicit embedded corrections later in later
0:28:50sections for instance then repeated that's an incorrect wait for it
0:28:53it's more than just
0:28:57i'm afraid that drifted a little different direction
0:29:03anyway changes over time so that the why i explain this thing with artificial companions
0:29:08in the beginning i guess that posterior have these artificial friendship knows the user and
0:29:13userspreferences and everything and
0:29:16and that's why i set up to study the data collection in this way that's
0:29:19why i'm talking about a specific speech actually systems every participant of the study was
0:29:23put out every load it was would wherein appear within a speaker and they directed
0:29:28in pairs for a longer time
0:29:30and i wanted to see the development
0:29:34and i can say
0:29:36the development in learning was not only because they interact longer but because some of
0:29:40them engage in these corrections and in this evident obvious selling sequences in the beginning
0:29:47and that's why it developed somehow more intensively later and in either appears it was
0:29:53not relevant
0:29:54they just
0:29:56so i don't have a so we can continue offline five minutes once it uses
0:30:01this isn't the speaker