0:00:15thank you very march for introduction um
0:00:21what didn't have brought a i'm very glad to be here and have possibility to
0:00:26present a little work which would on a nice place and interest if a cell
0:00:31in germany i continue to speak about uh a human head reconstruction and my part
0:00:40of these beak B project uh which was introduced by my colleague uh is stitching
0:00:48of reconstructed so face might focus bouncing the variational registration of for each range images
0:00:55uh with no by non rigid deformations so as i said this is a part
0:01:02of the project where we want to reconstruct human head the model if you look
0:01:07at a recent set of images
0:01:10well my colleague expand already uh but
0:01:16what happens after the reconstruction all parts of human heads reconstructed separately and they are
0:01:24uh they should be matched and then stitched together in order to have
0:01:34and model which is you use appropriate for printing a for us it means that
0:01:41we want to estimate and model which are smooth and that and have no but
0:01:48see johnson and staring effect in the place of station also faces
0:01:55the first part of all for all of work flow is done by uh structural
0:02:01optical flow algorithm is already explained this uh the result of this algorithm looks look
0:02:10yeah results billboard but unfortunately they are not perfect a they are subject to distortions
0:02:18this distortion scores by
0:02:21many factors uh kind of the economy's calibration and some errors during reconstruction and all
0:02:30this for example we use uh so called they have E much more range image
0:02:36for representation of four so faces uh such structures can not uh described um occlusions
0:02:47or uh or the discontinue two regions on so face in a proper way that's
0:02:54why oh uh there are a lot of place also faces were some false box
0:03:00i hear uh another problem another problem of the soviet it's not nonrigidity of object
0:03:06of interest uh when uh we say to the model yeah to human been police
0:03:11station don't move it doesn't work queue moves and one we make several uh several
0:03:18uh shootings i would say uh we uh we have such problems that
0:03:25several parts of this so faces for something have plotted more departs they have different
0:03:33position one three or one for each of the relatively to the to the to
0:03:38all the uh and it cost
0:03:43it makes the problem of stitching very difficult uh because of such the did distortions
0:03:51we cannot simply use existing matching the utterance uh to solve this problem uh icp
0:03:59for example integrate iterative-closest-point generate a result which still uh has a lot of places
0:04:06where so faces simply cannot together so for example here it is green part it
0:04:12is uh
0:04:15to occlusion uh this distortion is generated by occlusion here we can see false matches
0:04:22because of discontinued see uh see it just problem of for a couple of the
0:04:30small errors and kind of calibration and so on
0:04:34so we need some additional deformation local non-rigid deformation which can uh
0:04:45can improve this case can compensate these distortions
0:04:51uh the main the main challenge of my work is to combine global rig it
0:04:58uh transformation of so faces with local non-rigid transformation also faces in order to bring
0:05:05them together and speech
0:05:10so i want to explain our methods just promise simple artificial example here so physicists
0:05:18represented in two-dimensional to two-dimensional manner uh so all i had to find a so
0:05:26face which will be similar to both so faces which are not which are not
0:05:31so which in intersect partially which costs move
0:05:42so what to do we search for a some transformation or some transformation to which
0:05:49transforms of faces in such a way that we protect smoothly full to show some
0:05:55people to show that means that you in your of these so faces is a
0:06:00small suffix
0:06:04so because of
0:06:06unknown comfortable in
0:06:10come complicated distortion we can not consider once a faces a template that's why we
0:06:16uh we formulate the project of make a proper problems matching not it's problem of
0:06:21matching once for face to another but as a problem of matching what's the faces
0:06:26are also faces uh to some expected so fixed what is expected to face expected
0:06:34surface is our target sort of experts so
0:06:40we say that we want to manage our suffices to some expect so face which
0:06:46is in principle our current guess and box resulting so face about are just suffix
0:06:55all this so expected to face can be
0:06:59can be estimated by using a simple consistency constraint which says that are all points
0:07:07of also faces should be consistent with
0:07:11consistent would expect suffix
0:07:14the mean means that minimisation all day so constraint bring cost so
0:07:21to the
0:07:22to the equation for its estimation it is not all the than simple uh weighting
0:07:30weighted uh some of are also of our source so faces
0:07:36he a weight is a quality measure for each the face which got comes from
0:07:44reconstruction algorithm i will come back uh to this quality measure a little bit later
0:07:53so ah S A I C S I
0:07:58already mentioned we search for some transformation we which is a combination of some rigid
0:08:05global transformation and some non rigid local transformation more global transformation should bring so face
0:08:14part uh in the proper position in some common uh coordinate system a local transformation
0:08:22should compensate distortions in order to bring them to get the question now what kind
0:08:28of local transformation to use we can see the so called sinful well as a
0:08:34prominent way to compensate such deformations ten flow is a is a three dimensional of
0:08:43vector field which describe this uh movement of corresponding point in space it is four
0:08:51dimensional and a lot of optical flow and can be can be estimated in a
0:08:55similar manner but we consider a variational approach for estimation as
0:09:03a signal to promote its among
0:09:07among colors
0:09:11that's why we formulate our matching
0:09:15so that our matching uh problem as in the term of variation requires of appropriate
0:09:23energy function which should be minimised and this energy function consist of set or combination
0:09:31of data constraint constraints and combination of regularisation constraints
0:09:38we use
0:09:40we use free constraints it they are toast data constraint which says that a lot
0:09:48transform it's or face
0:09:50should be consistent with expected surface
0:09:54a smoothness constraint is used smoothness of scene flow constraint is uh it is very
0:10:02commonly used constraint for minimisation problem uh it says that the gradient of target function
0:10:10should be mean you should be more in order to simplify calculation will know is
0:10:17also cool common approach we approximate this gradient aspects the difference between what will you
0:10:23the function and its
0:10:26mean some mean or some approximation of the function in some neighborhood
0:10:32at around current or
0:10:34and we also use so this so called tikhonov regularisation constraint which says that uh
0:10:42our scene flow should be as small as possible uh
0:10:47we use this in order to with situation that scene flow describe some uh transformation
0:10:53which can be described by a global or transformation
0:11:01yeah O combination of these constraints
0:11:05give us a common energy function for both transformation for jane transformation here we also
0:11:14use additional weights for each constraint which uh
0:11:21which required for
0:11:27for making it to make a reconstruction process more stable to noise and more uh
0:11:35contour again
0:11:37i will explain it will be this uh weights
0:11:42later so the solving of these of these um realisation problem
0:11:50because i'll energy function is a function of
0:11:54many parameters of two to twelve global transformation parameters of for local transformation we search
0:12:03all this parameter separately first of all the fix scene flow in some yes
0:12:11uh insight in some point and star which is our current guess about seven flow
0:12:19received energy function which is the energy function of only uh global motion parameters this
0:12:27formulation after this could decision is a
0:12:31a weighted icp uh i would say uh weighted icp and it can be straightforwardly
0:12:39still so that the
0:12:42the result of minimisation of this energy function is used them uh to
0:12:50to receive energy function for stencil so we iteratively uh optimize
0:12:58well global transformation parameters and then local transformation or
0:13:03um the
0:13:05this is the uh this problem can be solved with using a or do not
0:13:09let me articulation uh will bring cost to the iterative solution iterative algorithm
0:13:17after receiving of result of both these all of minimisation able both this function we
0:13:25refine our expected sort faces and should start this uh procedure again iteratively
0:13:37so some words about
0:13:41weights the weights needed to
0:13:46to reduce to regulate influence of if of each constraint in you know energy function
0:13:53uh depending on cable on quality of
0:13:57uh source data which are involved in this computation um each source or face to
0:14:04after reconstruction are is supplied with a quality measure in you know
0:14:11uniform of weights which two
0:14:16depend on noise level of
0:14:21of not of so face estimation yeah
0:14:24uh the main idea the main idea just
0:14:29if we have a good estimated uh so faces
0:14:36it means that
0:14:38both correspondences
0:14:40are estimated it with high precision
0:14:43in this case weight of data constraint in
0:14:48energy function should be higher so that the gain of data concern should be fine
0:14:55otherwise smoothness constraint should to should get a high higher you in and so let's
0:15:03a smoothed or bad data
0:15:08um we use to go common uncertainty propagation uh rule the to estimate the weight
0:15:16of the data constraint but see that the noise level of data constraint is the
0:15:21sum of noise levels of that are involved in this data constraint just a lot
0:15:28so face and correspondent particle for expected surface regardless of
0:15:36a you can have regularisation constraint by weight of tikhonov regularisation constraint we can uh
0:15:43kick can control the growing all for scene flow from the T control the uh
0:15:53the power of local transformation um conceptually that we prevent local transformation in places where
0:16:01so faces
0:16:03are estimated with high precision is if we know that's a face estimateable high-precision would
0:16:08don't want to bend it
0:16:11that's why we use uh weight of uh so face which should be transformed as
0:16:17uh wait for tikhonov regularisation constraint
0:16:21and uh wait all for smoothness regularisation constraint not now we calculate this as the
0:16:27sum of
0:16:29as the sum of form weights um of sinful or which are you stand for
0:16:39weight values which are used for estimation of mean value for regularization for a for
0:16:47a approximation of gradient and to be honest this question
0:16:53question how to calculate
0:16:56okay it is still open question i just want to show just some results so
0:17:04here it is a distance map maps between two so faces which we uh mitch
0:17:11with icp algorithm and propose it expected to face mentioned
0:17:17uh so this is initial match which was done manually the result of
0:17:24icp it looks like icp uh tries to optimize global distance between so faces
0:17:34although it will have um has
0:17:38tendency to move for so faces
0:17:44one to another in places there the where they are more bare feet to each
0:17:49other that we have more consistent in
0:17:52for example this part of face and parts where uh we have
0:18:00problems words of basis the form of this
0:18:04uh these parts are uh if you look more closely it happens because in this
0:18:10place is expected sort face
0:18:14or let six four paces some more can a consistent and expensive face uh receive
0:18:21more weights
0:18:24so and application of scene flow remove practically all these deformations
0:18:31oh on expert O one which is a fixed i just i just want to
0:18:37show you three-dimensional result of this much is not perfect because there is still a
0:18:43lot of problems first of all we cannot uh automatically this uh find difference between
0:18:51uh between false so phase parts and so on so face parts which have no
0:19:00uh correspondences at all box in places of where so faces i just the deformed
0:19:08not that much i don't want to pay a lot of attention on these artifacts
0:19:13because it is still subject to of work but in public in places versa faces
0:19:22not very strong deformed we can bring them together and stitched so for comparison i
0:19:30just show how it was
0:19:35so this is the initial match
0:19:38here so faces you can see that these parts
0:19:42can not be measured by some documents so i switch to
0:19:58okay thank you very much for your attention and
0:20:35to be able to be honest yep for sure the lighting conditions it's very oh
0:20:43how to see printed they inference very much to reconstruction but uh i had to
0:20:50be to be honest it is it is not question to me because this reconstruction
0:20:55and it was presented before uh
0:21:18what do mean what it is possible to find the position of the light
0:21:24more than just this problem was not can see that but for sure it is
0:21:30i would say it is another fields of fu
0:21:45which one
0:21:51i mean whether we use a touch information for this matching
0:22:24no we do not know such information with the feature description uh for matching only
0:22:32uh that that's information is involved in you know matching strategy on the in a
0:22:37way that uh we add additional weighting factor which is that a way we will