0:00:15 | thank you very march for introduction um |
---|---|

0:00:21 | what didn't have brought a i'm very glad to be here and have possibility to |

0:00:26 | present a little work which would on a nice place and interest if a cell |

0:00:31 | in germany i continue to speak about uh a human head reconstruction and my part |

0:00:40 | of these beak B project uh which was introduced by my colleague uh is stitching |

0:00:48 | of reconstructed so face might focus bouncing the variational registration of for each range images |

0:00:55 | uh with no by non rigid deformations so as i said this is a part |

0:01:02 | of the project where we want to reconstruct human head the model if you look |

0:01:07 | at a recent set of images |

0:01:10 | well my colleague expand already uh but |

0:01:16 | what happens after the reconstruction all parts of human heads reconstructed separately and they are |

0:01:24 | uh they should be matched and then stitched together in order to have |

0:01:34 | and model which is you use appropriate for printing a for us it means that |

0:01:41 | we want to estimate and model which are smooth and that and have no but |

0:01:48 | see johnson and staring effect in the place of station also faces |

0:01:55 | the first part of all for all of work flow is done by uh structural |

0:02:01 | optical flow algorithm is already explained this uh the result of this algorithm looks look |

0:02:10 | yeah results billboard but unfortunately they are not perfect a they are subject to distortions |

0:02:18 | this distortion scores by |

0:02:21 | many factors uh kind of the economy's calibration and some errors during reconstruction and all |

0:02:30 | this for example we use uh so called they have E much more range image |

0:02:36 | for representation of four so faces uh such structures can not uh described um occlusions |

0:02:47 | or uh or the discontinue two regions on so face in a proper way that's |

0:02:54 | why oh uh there are a lot of place also faces were some false box |

0:03:00 | i hear uh another problem another problem of the soviet it's not nonrigidity of object |

0:03:06 | of interest uh when uh we say to the model yeah to human been police |

0:03:11 | station don't move it doesn't work queue moves and one we make several uh several |

0:03:18 | uh shootings i would say uh we uh we have such problems that |

0:03:25 | several parts of this so faces for something have plotted more departs they have different |

0:03:33 | position one three or one for each of the relatively to the to the to |

0:03:38 | all the uh and it cost |

0:03:43 | it makes the problem of stitching very difficult uh because of such the did distortions |

0:03:51 | we cannot simply use existing matching the utterance uh to solve this problem uh icp |

0:03:59 | for example integrate iterative-closest-point generate a result which still uh has a lot of places |

0:04:06 | where so faces simply cannot together so for example here it is green part it |

0:04:12 | is uh |

0:04:15 | to occlusion uh this distortion is generated by occlusion here we can see false matches |

0:04:22 | because of discontinued see uh see it just problem of for a couple of the |

0:04:30 | small errors and kind of calibration and so on |

0:04:34 | so we need some additional deformation local non-rigid deformation which can uh |

0:04:45 | can improve this case can compensate these distortions |

0:04:51 | uh the main the main challenge of my work is to combine global rig it |

0:04:58 | uh transformation of so faces with local non-rigid transformation also faces in order to bring |

0:05:05 | them together and speech |

0:05:10 | so i want to explain our methods just promise simple artificial example here so physicists |

0:05:18 | represented in two-dimensional to two-dimensional manner uh so all i had to find a so |

0:05:26 | face which will be similar to both so faces which are not which are not |

0:05:31 | so which in intersect partially which costs move |

0:05:39 | uh |

0:05:42 | so what to do we search for a some transformation or some transformation to which |

0:05:49 | transforms of faces in such a way that we protect smoothly full to show some |

0:05:55 | people to show that means that you in your of these so faces is a |

0:06:00 | small suffix |

0:06:02 | um |

0:06:04 | so because of |

0:06:06 | unknown comfortable in |

0:06:10 | come complicated distortion we can not consider once a faces a template that's why we |

0:06:16 | uh we formulate the project of make a proper problems matching not it's problem of |

0:06:21 | matching once for face to another but as a problem of matching what's the faces |

0:06:26 | are also faces uh to some expected so fixed what is expected to face expected |

0:06:34 | surface is our target sort of experts so |

0:06:40 | we say that we want to manage our suffices to some expect so face which |

0:06:46 | is in principle our current guess and box resulting so face about are just suffix |

0:06:55 | all this so expected to face can be |

0:06:59 | can be estimated by using a simple consistency constraint which says that are all points |

0:07:07 | of also faces should be consistent with |

0:07:11 | consistent would expect suffix |

0:07:14 | the mean means that minimisation all day so constraint bring cost so |

0:07:21 | to the |

0:07:22 | to the equation for its estimation it is not all the than simple uh weighting |

0:07:30 | weighted uh some of are also of our source so faces |

0:07:36 | he a weight is a quality measure for each the face which got comes from |

0:07:44 | reconstruction algorithm i will come back uh to this quality measure a little bit later |

0:07:53 | so ah S A I C S I |

0:07:58 | already mentioned we search for some transformation we which is a combination of some rigid |

0:08:05 | global transformation and some non rigid local transformation more global transformation should bring so face |

0:08:14 | part uh in the proper position in some common uh coordinate system a local transformation |

0:08:22 | should compensate distortions in order to bring them to get the question now what kind |

0:08:28 | of local transformation to use we can see the so called sinful well as a |

0:08:34 | prominent way to compensate such deformations ten flow is a is a three dimensional of |

0:08:43 | vector field which describe this uh movement of corresponding point in space it is four |

0:08:51 | dimensional and a lot of optical flow and can be can be estimated in a |

0:08:55 | similar manner but we consider a variational approach for estimation as |

0:09:03 | a signal to promote its among |

0:09:07 | among colors |

0:09:09 | so |

0:09:11 | that's why we formulate our matching |

0:09:15 | so that our matching uh problem as in the term of variation requires of appropriate |

0:09:23 | energy function which should be minimised and this energy function consist of set or combination |

0:09:31 | of data constraint constraints and combination of regularisation constraints |

0:09:38 | we use |

0:09:40 | we use free constraints it they are toast data constraint which says that a lot |

0:09:48 | transform it's or face |

0:09:50 | should be consistent with expected surface |

0:09:54 | a smoothness constraint is used smoothness of scene flow constraint is uh it is very |

0:10:02 | commonly used constraint for minimisation problem uh it says that the gradient of target function |

0:10:10 | should be mean you should be more in order to simplify calculation will know is |

0:10:17 | also cool common approach we approximate this gradient aspects the difference between what will you |

0:10:23 | the function and its |

0:10:26 | mean some mean or some approximation of the function in some neighborhood |

0:10:32 | at around current or |

0:10:34 | and we also use so this so called tikhonov regularisation constraint which says that uh |

0:10:42 | our scene flow should be as small as possible uh |

0:10:47 | we use this in order to with situation that scene flow describe some uh transformation |

0:10:53 | which can be described by a global or transformation |

0:10:59 | so |

0:11:01 | yeah O combination of these constraints |

0:11:05 | give us a common energy function for both transformation for jane transformation here we also |

0:11:14 | use additional weights for each constraint which uh |

0:11:21 | which required for |

0:11:26 | oh |

0:11:27 | for making it to make a reconstruction process more stable to noise and more uh |

0:11:34 | more |

0:11:35 | contour again |

0:11:37 | i will explain it will be this uh weights |

0:11:42 | later so the solving of these of these um realisation problem |

0:11:50 | because i'll energy function is a function of |

0:11:54 | many parameters of two to twelve global transformation parameters of for local transformation we search |

0:12:03 | all this parameter separately first of all the fix scene flow in some yes |

0:12:11 | uh insight in some point and star which is our current guess about seven flow |

0:12:17 | and |

0:12:19 | received energy function which is the energy function of only uh global motion parameters this |

0:12:27 | formulation after this could decision is a |

0:12:31 | a weighted icp uh i would say uh weighted icp and it can be straightforwardly |

0:12:39 | still so that the |

0:12:42 | the result of minimisation of this energy function is used them uh to |

0:12:50 | to receive energy function for stencil so we iteratively uh optimize |

0:12:58 | well global transformation parameters and then local transformation or |

0:13:03 | um the |

0:13:05 | this is the uh this problem can be solved with using a or do not |

0:13:09 | let me articulation uh will bring cost to the iterative solution iterative algorithm |

0:13:17 | after receiving of result of both these all of minimisation able both this function we |

0:13:25 | refine our expected sort faces and should start this uh procedure again iteratively |

0:13:37 | so some words about |

0:13:41 | weights the weights needed to |

0:13:46 | to reduce to regulate influence of if of each constraint in you know energy function |

0:13:53 | uh depending on cable on quality of |

0:13:57 | uh source data which are involved in this computation um each source or face to |

0:14:04 | after reconstruction are is supplied with a quality measure in you know |

0:14:11 | uniform of weights which two |

0:14:16 | depend on noise level of |

0:14:21 | of not of so face estimation yeah |

0:14:24 | uh the main idea the main idea just |

0:14:29 | if we have a good estimated uh so faces |

0:14:36 | it means that |

0:14:38 | both correspondences |

0:14:40 | are estimated it with high precision |

0:14:43 | in this case weight of data constraint in |

0:14:48 | energy function should be higher so that the gain of data concern should be fine |

0:14:55 | otherwise smoothness constraint should to should get a high higher you in and so let's |

0:15:03 | a smoothed or bad data |

0:15:08 | um we use to go common uncertainty propagation uh rule the to estimate the weight |

0:15:16 | of the data constraint but see that the noise level of data constraint is the |

0:15:21 | sum of noise levels of that are involved in this data constraint just a lot |

0:15:28 | so face and correspondent particle for expected surface regardless of |

0:15:36 | a you can have regularisation constraint by weight of tikhonov regularisation constraint we can uh |

0:15:43 | kick can control the growing all for scene flow from the T control the uh |

0:15:53 | the power of local transformation um conceptually that we prevent local transformation in places where |

0:16:01 | so faces |

0:16:03 | are estimated with high precision is if we know that's a face estimateable high-precision would |

0:16:08 | don't want to bend it |

0:16:11 | that's why we use uh weight of uh so face which should be transformed as |

0:16:17 | uh wait for tikhonov regularisation constraint |

0:16:21 | and uh wait all for smoothness regularisation constraint not now we calculate this as the |

0:16:27 | sum of |

0:16:29 | as the sum of form weights um of sinful or which are you stand for |

0:16:39 | weight values which are used for estimation of mean value for regularization for a for |

0:16:47 | a approximation of gradient and to be honest this question |

0:16:53 | question how to calculate |

0:16:56 | okay it is still open question i just want to show just some results so |

0:17:04 | here it is a distance map maps between two so faces which we uh mitch |

0:17:11 | with icp algorithm and propose it expected to face mentioned |

0:17:17 | uh so this is initial match which was done manually the result of |

0:17:24 | icp it looks like icp uh tries to optimize global distance between so faces |

0:17:33 | uh |

0:17:34 | although it will have um has |

0:17:38 | tendency to move for so faces |

0:17:44 | one to another in places there the where they are more bare feet to each |

0:17:49 | other that we have more consistent in |

0:17:52 | for example this part of face and parts where uh we have |

0:18:00 | problems words of basis the form of this |

0:18:04 | uh these parts are uh if you look more closely it happens because in this |

0:18:10 | place is expected sort face |

0:18:14 | or let six four paces some more can a consistent and expensive face uh receive |

0:18:21 | more weights |

0:18:24 | so and application of scene flow remove practically all these deformations |

0:18:31 | oh on expert O one which is a fixed i just i just want to |

0:18:37 | show you three-dimensional result of this much is not perfect because there is still a |

0:18:43 | lot of problems first of all we cannot uh automatically this uh find difference between |

0:18:51 | uh between false so phase parts and so on so face parts which have no |

0:19:00 | uh correspondences at all box in places of where so faces i just the deformed |

0:19:08 | not that much i don't want to pay a lot of attention on these artifacts |

0:19:13 | because it is still subject to of work but in public in places versa faces |

0:19:21 | ah |

0:19:22 | not very strong deformed we can bring them together and stitched so for comparison i |

0:19:30 | just show how it was |

0:19:35 | so this is the initial match |

0:19:38 | here so faces you can see that these parts |

0:19:42 | can not be measured by some documents so i switch to |

0:19:50 | two |

0:19:54 | oh |

0:19:56 | what's |

0:19:58 | okay thank you very much for your attention and |

0:20:27 | well |

0:20:31 | uh_huh |

0:20:34 | oh |

0:20:35 | to be able to be honest yep for sure the lighting conditions it's very oh |

0:20:43 | how to see printed they inference very much to reconstruction but uh i had to |

0:20:50 | be to be honest it is it is not question to me because this reconstruction |

0:20:55 | and it was presented before uh |

0:21:00 | i |

0:21:18 | what do mean what it is possible to find the position of the light |

0:21:24 | more than just this problem was not can see that but for sure it is |

0:21:30 | i would say it is another fields of fu |

0:21:45 | which one |

0:21:51 | i mean whether we use a touch information for this matching |

0:22:23 | okay |

0:22:24 | no we do not know such information with the feature description uh for matching only |

0:22:32 | uh that that's information is involved in you know matching strategy on the in a |

0:22:37 | way that uh we add additional weighting factor which is that a way we will |

0:22:46 | correspond |