0:00:15so thank very much for the introduction
0:00:18i'm going to talk about our recent work
0:00:20at cambridge university
0:00:24current dialogue models
0:00:26are limited in the complexity of dialogue structure as they allow for example relations between
0:00:34i'm that'll work
0:00:38we present work as well
0:00:41so it all a
0:00:44deliberately break those limitations
0:00:48we're not all of the current to have complex relations between a
0:00:52complex data structures but you don't x
0:00:56and even more so we i name system to address
0:01:01relations between objects and we allow
0:01:03users to address relations between objects
0:01:08and even though this model has been
0:01:10developed for some dp based task oriented spoken dialogue systems
0:01:15it should be applicable for to all kinds of data systems by you want to
0:01:19have a symbolic representation of the dialog state
0:01:22at first the creek we can on and if you based
0:01:26spoken dialogue systems
0:01:27so what is here is the architecture of a spoken dialogue system
0:01:32we have the input processing
0:01:33modules which yields a semantic representation of the
0:01:38of the input of the user
0:01:40it is then used in the belief tracking
0:01:42module to update the
0:01:44the belief state which is then used by the dialogue policy to use the next
0:01:48system action
0:01:50which is then
0:01:51transfer back
0:01:52by the language generation and synthesis module
0:01:55two response or you response the user
0:01:58and the task of each tracker is then to update the
0:02:01the distribution of all possible dialog state based on the current user input the previous
0:02:06system action and the previously stated
0:02:11you using force them to find optimal policy which maps the current belief state index
0:02:15system action using a reward signal from the user
0:02:20and the optimal policy is then defined as
0:02:22a policy which maximizes the expected future calculated reward
0:02:28this architecture has been used for
0:02:30so you go domain dialogues a lot
0:02:33and also for multi domain dialogues
0:02:38using the multi-domain dialogue model in psychology
0:02:40here's an example dialog
0:02:43about finding a little interested in cambridge
0:02:49you saw what does one soldier in cambridge use us a few questions about that
0:02:54price of instances are aligned and the system table two
0:02:57give a response
0:02:59k is right cambridge is expensive fortune the sensor
0:03:03he's is quite a also on the restaurant in cambridge
0:03:06in the centre
0:03:08system continues asking questions to figure out what the actual
0:03:13the rest of the lexicon meta-file the rest from the user wants and then it
0:03:17is okay chromacity trying to system
0:03:19in the sense
0:03:22in the multi-domain dialogue model
0:03:24the user input first n is the main tracker which you can start okay what's
0:03:28the current domain of the of the dialog the user input
0:03:31this case hotel restaurant
0:03:33and then they are
0:03:36part of the dialog pipeline which handles than the request buddies
0:03:42and this model has several as a dialog state
0:03:45stuff that all
0:03:47the communication between those two domains
0:03:49or at least doesn't allow easy
0:03:51way of doing this type of communication
0:03:53so for example the user would say trade it also effective referent
0:03:59in the same area which is a relation relating to information about hotel this was
0:04:05would not be something this type of
0:04:08a lot of attention
0:04:11so instead of
0:04:13of modeling domain
0:04:16we are looking at entities that
0:04:19the same
0:04:21example dialogue
0:04:23in the beginning
0:04:25you the dialogue was about
0:04:27a not sect of type hotel
0:04:30and then later about object of interest ones
0:04:35also and then entities in our dialogue topic entities
0:04:39that are also contains an additional ldc the relation which is modeled as a separate
0:04:43entity connecting
0:04:46a text holes in the restaurant
0:04:51and this is
0:04:52the main idea of the new dialogue model
0:04:55so a and all these all these energies countryside centre conversation world
0:05:02which kind of wine sent again
0:05:06so i will
0:05:08go in more detail on but let me tell you more details about those the
0:05:13dialogue model
0:05:15and then either so you present your prototype implementation a showcase that we focus on
0:05:21relation modeling
0:05:22if you can so that
0:05:23this has dialogue model
0:05:25is actually very effective in handling relation
0:05:28between opt
0:05:31okay but first we talk about the conversation into the dialogue model in more detail
0:05:36what are optics tex mex and its use of the real world
0:05:40we have an object in cambodia the extra restaurant
0:05:43and all the objects in heaven the matching
0:05:46and you know database
0:05:48and based on these entries
0:05:50actual objects in our conversational world
0:05:56of course to notice do that the type of that
0:05:59also has to match
0:06:01the database specification
0:06:06and a relation then connects to objects
0:06:10no example the restaurant again
0:06:13and a different definition of this relation can also be derived from
0:06:17the type definitions of those objects
0:06:19so in this example
0:06:22we just compared compatible attributes like area areas
0:06:26present in both
0:06:28it would list of the
0:06:30object type
0:06:32so this is
0:06:33this can be something they can connect on for this is part of the relation
0:06:36to finish
0:06:38of course you can elaborate is if you
0:06:40she one
0:06:45each conversation entity which can be adopted or relation maintains a user goal belief state
0:06:51and the context eight
0:06:53so whether it's an object or a simple all have their own
0:06:57own state definition
0:06:59and the user goal belief represents information
0:07:02shared by the user bits
0:07:03as a set of uncertainty whether and
0:07:08so it's not as prepared probability distribution over all possible user goal states
0:07:12as stated she's
0:07:14similar to what we already know
0:07:16from completely based dialogue modeling
0:07:18in general
0:07:20and the connect state represents information which is offered by the system
0:07:26so for example the system says
0:07:30on the houses a chinese restaurant in the sensor
0:07:32and this would be something which would be in the context and so the system
0:07:36knows that it has access information does not uncertainty associated to that
0:07:40and the clicks it is
0:07:42very important because
0:07:44braces of refer to the information we just been shown by the system instead of
0:07:48and so the information
0:07:50rather than a computer opticals been found yet
0:07:54and it is it of conversational world and their entities
0:07:58allows them to the belief tracking on two levels
0:08:02on the word level there you can also have a behavioural
0:08:05a system behavior for creating and entity disambiguation
0:08:10and then on the energy level they actually
0:08:14for about what specific to gonna can forward
0:08:18protesters this property stuff this relation actually have
0:08:26okay so
0:08:27do not confuse you anymore i have as a small example but we can go
0:08:31through step by step
0:08:34the sex on the t c them before so shouldn't be too difficult the beginning
0:08:39the user says okay and looking for hosting cambridge whether the system can infer okay
0:08:43that must be an object
0:08:45of type or tell
0:08:48and we don't have any information about it yet
0:08:50then the other continues there's more information share
0:08:54and that should be the price range expensive so we cannot tell the user goal
0:08:58belief state and the system share
0:09:01the most video what the proposed
0:09:02and so what cambridge's an expensive one
0:09:06so this is when the context it is updated
0:09:10then a user says greater also need cheap restaurant in the scenario
0:09:14which allows the dialogue system to infer that there's another object of time restaurant
0:09:20and that's a relation between a topic
0:09:23so this is when this could come to live if you will
0:09:29the data continues personal information acquired about the
0:09:32but it's fun and two
0:09:35system proposes a computer us plan to the user
0:09:38which is when the context and the of the object and the relation is updated
0:09:50if you to the properties of this of this
0:09:53you dialogue model
0:09:56and compared to the most of them are not only can see on several expected
0:09:59it is more or flexible and the
0:10:03so for example objects can be can be addressed by the use of the system
0:10:07is something
0:10:08also in common in a way but this new model also allows
0:10:12to have multiple object of the same time so you could have put model dialogue
0:10:16where you have
0:10:17two hotels
0:10:21three best friends or whatever you think of
0:10:23and were to be able to maintain a dialog states
0:10:27correctly and you could also have relations between those object even
0:10:31it also allows the type hierarchy so
0:10:33the kids okay the rest when the venue and these things and this might have
0:10:37given you wanted to policy modeling for example
0:10:42well then let the calm assessment of the dialogue model also can do is modeled
0:10:47it's can be addressed by using the system
0:10:50poses al
0:10:53state it the relation to have their own state
0:10:56this allows the system to actually talk about those relations and address time in a
0:11:01system response
0:11:03and also the relations remain specified
0:11:06even if the context of the object
0:11:08change or if there's
0:11:09no object context is if i don't all
0:11:11so if i if you think of the dialog the user but at the beginning
0:11:15i'm looking for a whole to and the rest and in the same area
0:11:19there is no connected all the scenario wouldn't
0:11:22couldn't be derived any and
0:11:26in any other way then
0:11:28like doing in the like having like this separate entities modeling the rest
0:11:37so this is
0:11:38well i'm going to tell you about the
0:11:41how about how the model
0:11:47how to continue the how to cook something to treat example of how we have
0:11:51implemented this model
0:11:56for experiments on
0:11:58relation learning something that this is very powerful tool to actually
0:12:02do this
0:12:07so one idea of formal beyond us to do when we do implement a portable
0:12:11to focus on relation labeling and to reuse
0:12:14the bound on the conditions you already have for belief imposing modeling
0:12:18we have from the domain based models
0:12:22so we have a fixed position world
0:12:25maybe africa will one object of each time
0:12:28and there's only one conversational taking the focus of attention
0:12:32which is which helps the system to figure out how to continue the dialogue
0:12:38and is similar to the to the
0:12:40the main tracking problem
0:12:43the user goal belief is more of the marginal distribution for slot
0:12:47and this is
0:12:49but also has the effect that you can reuse
0:12:53step something on the conditions for quality modeling for example but also
0:12:57but the only possibly relation you can
0:12:59actually model is the constellation this type of
0:13:02from you state
0:13:08another to open questions
0:13:11but dialogue management
0:13:12how can be incorporated the conversational objects and relations into the decision making process
0:13:17and how can we derive an input stage
0:13:19to the dialogue policies from the conversational entities
0:13:26we implemented a few reinforcement learning approach for that
0:13:30because this is very
0:13:32useful as it
0:13:34have hierarchical structure and it kind of
0:13:38subdivide the probably the several steps
0:13:42in the beginning in the based on the focus of attention an object is basically
0:13:47selected whether it's also respond and then there's a master policy a basis master policy
0:13:52a decision is made but also talk about the art techniques to talk about a
0:13:56relation place of that object
0:13:59only after that the system
0:14:01and the second bullet is able
0:14:03about ready to the object
0:14:05or to the relation
0:14:08so this means that each conversation and he has the all and policy
0:14:13and is also allows to say a relation policies
0:14:16because this policy is the same value
0:14:19come from this
0:14:20so i talking about a the whole circle the output from the start talking about
0:14:23the rest
0:14:26and information we use
0:14:30for the policy as input to do this
0:14:32is on the entity level
0:14:35for the relation we just look at the state of the relation to but for
0:14:39you all checked we have to find a way to combine those things
0:14:44which is necessary to database access basically
0:14:48and the last odyssey it also we need a combination of the objects in the
0:14:52as you want to do computer solution
0:14:57the object you please state of the object
0:14:59could state something which is
0:15:01it can stick to what the relation says based on what object to have a
0:15:06and to figure out those conflicts and to allow the system to handle this situation
0:15:12you need to combine them
0:15:18we consummate experiments
0:15:20of finding out and the rest and in cambridge
0:15:24for which we use the pilot system
0:15:28and the conversational world and in the six and experiments
0:15:31consisted of two objects multiple do not have restaurant
0:15:37user goals contain the rage
0:15:40between the slot area pricerange so
0:15:44every user goal had this relation but was not always use
0:15:48and there was an additional parameter are which specify the property that the usual you
0:15:53use a similar to actually use that relation
0:15:56so the you using lattices and has may consist of a lot to talk about
0:16:02the value of all that's also various sensor all the talk about the relation area
0:16:06of course that you "'cause" every restaurant
0:16:09and if are outside only use a simulated user would say area
0:16:14every hold because every restaurant
0:16:17and we'll of the two experiments
0:16:19comparing the
0:16:21conversational entity dialogue model with them multi-domain dialogue model
0:16:25in the first we were interested of the effect of r
0:16:29does this happen in fact we actually you need to model relation can you can
0:16:34be able to cope with this problem different way
0:16:36for this at a fixed all of conversational czech only the second one was learned
0:16:41and then we also ones of you got to see what the actual effect a
0:16:45of this type of model on the on the learning behaviour in general
0:16:52so we sell
0:16:53experiment it's in experiment one about the unknown to different semantic error level zero percent
0:16:59fifteen percent
0:17:00you see the success rate
0:17:02and the other relation probability are
0:17:07for c represent you could clearly see that with the high our
0:17:12the composition is the other one of the form to go if you significantly higher
0:17:17and then the multi-domain dialogue model
0:17:20and this is also true for the fifteen percent
0:17:23case the goddess much lower here because of how the
0:17:26our remote operations
0:17:28it is artificial because assimilated but based on that could call some of the
0:17:32problems you would usually have
0:17:35in this situation
0:17:36note interesting here is not only that the that the concession digit our model
0:17:41performs better
0:17:42for high are also that follow lost it does not
0:17:47so the model domain data model is able to cope with
0:17:50with use a talk about relation to a certain extent
0:17:54touched by basically repeating the question
0:17:57so you looking for both what prices are looking for
0:18:00yes n is also a classifier for
0:18:04if it does restate a question
0:18:06this is the model can learn all the call this
0:18:09i'm going to a certain extent and on in terms of success rate or re
0:18:12well which
0:18:14goes the we want
0:18:17but i would assume that if you look good
0:18:19user to sex anything's it might be different
0:18:24we also looked at how many of the system actions actually address
0:18:30how many i'd how mean dialogue system the system actually interest
0:18:35and the relation and then we the system utterances in twenty four point five dialogues
0:18:41for twenty four point five percent of the dialog directly address a relation which i
0:18:45think it is very interesting
0:18:47and for experiment to a we have a similar situation
0:18:52it's not the strong because they took turns and effects of the domain
0:18:57if the rest know how to most part of the first
0:19:00first object of the dialogue was about then it
0:19:05there was a relation present
0:19:10i think we can say that the new model breaks up the limitations of current
0:19:14our models
0:19:16i think the not next logical step towards more complex dialogs
0:19:21a lot also more complex data structures like relations between objects
0:19:25and that could shown approach have implementation which we use as normal voice models that
0:19:30it outperforms the baseline
0:19:31and the future work
0:19:33we further need to increase
0:19:35the complexity of dialogue structures
0:19:37so what's more complex relation
0:19:41what ended if it had to be ways and just to name only two
0:19:45so you can you can find implementation on the website
0:19:49and also we have a new meetings from high to a so if you just
0:19:52that if you to assign a thank you very much
0:20:03now we have time for questions that one
0:20:37so you mean if it if there is like
0:20:41just multiple talk about multiple objects
0:20:44and you say the for the third object or whatever this have been the same
0:20:50and then difficulties with this rule a strong gonna
0:21:10and i mean this depends on how the ontology is modeled
0:21:13i mean in this example dialogues we
0:21:15have you have a simple way of modeling the area which is not so please
0:21:18press centre
0:21:20i think
0:21:21for doing these things you would have to
0:21:23have a different ontology is gifted different
0:21:27of locating these things
0:21:29and then you could if you model that i love this situation
0:21:45also so you so you mean about the relation refers to the to what the
0:21:49user request of the actual without loss
0:21:55i mean it's setup currently
0:21:58and they said to try to go back to that slide
0:22:03i said relations of references a context a feature space is what the system proposed
0:22:07so whatever specified for by user
0:22:10is not necessary take into account when you look at the relation
0:22:12the relation looks at what the system
0:22:15also there's an entity whatever the values
0:22:17it's basically means if you haven't talk about the area at all
0:22:21then you said i want something the same area then you would be able to
0:22:24refer to that while the
0:22:26system offered to the user as a result it is basically what you're saying
0:22:30you want something which relates to what the system of a global the users admin
0:22:34in general terms
0:22:38and i think this is why it it's important to have this as two separate
0:22:42the possible state
0:22:45okay so we have a question i think you where the first
0:23:22i mean this
0:23:23so the idea of this one is to estimate the model information just circumvention values
0:23:28of theirs and frequency about that
0:23:30then a
0:23:32you have to add things to the model a question is whether the model can
0:23:35that is actually able to model these things and allows the system to address these
0:23:40type can be greedy
0:23:42and this was the goal of this i mean
0:23:44it cannot solve everything because they would that just have some
0:23:48the reasoning going on maybe or whatever out to be derived information
0:23:52or you could add some something to the language
0:23:56to the understanding component or whatever to do this type of thing
0:24:02if you and but if the system would be unsure about what the correct way
0:24:06of getting the situation would be you put
0:24:09the model could the policy learnt how to handle that and i and aspect of
0:24:14okay let's move the source it's you
0:24:51well from my point of view rubs of nh
0:24:56referred to dialogue intentions of dialogue act so you something you want to alter relations
0:25:01and i think you could
0:25:04have those of the object type definitions if you will to extend to have like
0:25:09what specific actions you could actual dollars
0:25:13since you're doing however a multimodal and belief tracking or also policy decision you could
0:25:22if more general we have to this on the on the upper level and then
0:25:24a more fine grained way of doing this on the lower level which would allow
0:25:27you to handle these
0:25:28these things
0:25:30i would be my files the correctly with the status of errors
0:26:08it's hard to say i mean i think they are computationally is used to that
0:26:11it's i think that's just what to basically referring to
0:26:14and i think you go you can also line i think it depends on the
0:26:19way of modeling the
0:26:21you stated so the probability distribution i mean is very simple way of doing the
0:26:26probability distribution and you want to have
0:26:29more complex weights maybe you won't have hierarchical one and you have to figure out
0:26:32how to do these five
0:26:35it doesn't
0:26:36but strict the general idea of modeling the dialogue and that way
0:26:43would still allow you to define relations would still allow you to talk about those
0:26:47relations and i still think is a good idea to have those relations because this
0:26:51from my point of view the only way that the system is also able to
0:26:54talk about those relations
0:26:56but where how far this thing going it's hard to say because i don't know
0:26:59how the new model to look like
0:27:08i mean one thing i would i
0:27:10the next that i would go to also be that have a probability of what
0:27:14different types of relation
0:27:16and also on the entity level which would no longer to have also things like
0:27:21on one
0:27:22this all that
0:27:23i one
0:27:25not this and how to do these things and i think this is what you
0:27:28could do if you have like with the distribution operations sort of just the values
0:27:33that last
0:28:12but also what the chase them differently
0:28:17without looking to lead to a given more for even good on something that
0:28:22okay thank you