thank you very march for introduction um

what didn't have brought a i'm very glad to be here and have possibility to

present a little work which would on a nice place and interest if a cell

in germany i continue to speak about uh a human head reconstruction and my part

of these beak B project uh which was introduced by my colleague uh is stitching

of reconstructed so face might focus bouncing the variational registration of for each range images

uh with no by non rigid deformations so as i said this is a part

of the project where we want to reconstruct human head the model if you look

at a recent set of images

well my colleague expand already uh but

what happens after the reconstruction all parts of human heads reconstructed separately and they are

uh they should be matched and then stitched together in order to have

and model which is you use appropriate for printing a for us it means that

we want to estimate and model which are smooth and that and have no but

see johnson and staring effect in the place of station also faces

the first part of all for all of work flow is done by uh structural

optical flow algorithm is already explained this uh the result of this algorithm looks look

yeah results billboard but unfortunately they are not perfect a they are subject to distortions

this distortion scores by

many factors uh kind of the economy's calibration and some errors during reconstruction and all

this for example we use uh so called they have E much more range image

for representation of four so faces uh such structures can not uh described um occlusions

or uh or the discontinue two regions on so face in a proper way that's

why oh uh there are a lot of place also faces were some false box

i hear uh another problem another problem of the soviet it's not nonrigidity of object

of interest uh when uh we say to the model yeah to human been police

station don't move it doesn't work queue moves and one we make several uh several

uh shootings i would say uh we uh we have such problems that

several parts of this so faces for something have plotted more departs they have different

position one three or one for each of the relatively to the to the to

all the uh and it cost

it makes the problem of stitching very difficult uh because of such the did distortions

we cannot simply use existing matching the utterance uh to solve this problem uh icp

for example integrate iterative-closest-point generate a result which still uh has a lot of places

where so faces simply cannot together so for example here it is green part it

is uh

to occlusion uh this distortion is generated by occlusion here we can see false matches

because of discontinued see uh see it just problem of for a couple of the

small errors and kind of calibration and so on

so we need some additional deformation local non-rigid deformation which can uh

can improve this case can compensate these distortions

uh the main the main challenge of my work is to combine global rig it

uh transformation of so faces with local non-rigid transformation also faces in order to bring

them together and speech

so i want to explain our methods just promise simple artificial example here so physicists

represented in two-dimensional to two-dimensional manner uh so all i had to find a so

face which will be similar to both so faces which are not which are not

so which in intersect partially which costs move


so what to do we search for a some transformation or some transformation to which

transforms of faces in such a way that we protect smoothly full to show some

people to show that means that you in your of these so faces is a

small suffix


so because of

unknown comfortable in

come complicated distortion we can not consider once a faces a template that's why we

uh we formulate the project of make a proper problems matching not it's problem of

matching once for face to another but as a problem of matching what's the faces

are also faces uh to some expected so fixed what is expected to face expected

surface is our target sort of experts so

we say that we want to manage our suffices to some expect so face which

is in principle our current guess and box resulting so face about are just suffix

all this so expected to face can be

can be estimated by using a simple consistency constraint which says that are all points

of also faces should be consistent with

consistent would expect suffix

the mean means that minimisation all day so constraint bring cost so

to the

to the equation for its estimation it is not all the than simple uh weighting

weighted uh some of are also of our source so faces

he a weight is a quality measure for each the face which got comes from

reconstruction algorithm i will come back uh to this quality measure a little bit later

so ah S A I C S I

already mentioned we search for some transformation we which is a combination of some rigid

global transformation and some non rigid local transformation more global transformation should bring so face

part uh in the proper position in some common uh coordinate system a local transformation

should compensate distortions in order to bring them to get the question now what kind

of local transformation to use we can see the so called sinful well as a

prominent way to compensate such deformations ten flow is a is a three dimensional of

vector field which describe this uh movement of corresponding point in space it is four

dimensional and a lot of optical flow and can be can be estimated in a

similar manner but we consider a variational approach for estimation as

a signal to promote its among

among colors


that's why we formulate our matching

so that our matching uh problem as in the term of variation requires of appropriate

energy function which should be minimised and this energy function consist of set or combination

of data constraint constraints and combination of regularisation constraints

we use

we use free constraints it they are toast data constraint which says that a lot

transform it's or face

should be consistent with expected surface

a smoothness constraint is used smoothness of scene flow constraint is uh it is very

commonly used constraint for minimisation problem uh it says that the gradient of target function

should be mean you should be more in order to simplify calculation will know is

also cool common approach we approximate this gradient aspects the difference between what will you

the function and its

mean some mean or some approximation of the function in some neighborhood

at around current or

and we also use so this so called tikhonov regularisation constraint which says that uh

our scene flow should be as small as possible uh

we use this in order to with situation that scene flow describe some uh transformation

which can be described by a global or transformation


yeah O combination of these constraints

give us a common energy function for both transformation for jane transformation here we also

use additional weights for each constraint which uh

which required for


for making it to make a reconstruction process more stable to noise and more uh


contour again

i will explain it will be this uh weights

later so the solving of these of these um realisation problem

because i'll energy function is a function of

many parameters of two to twelve global transformation parameters of for local transformation we search

all this parameter separately first of all the fix scene flow in some yes

uh insight in some point and star which is our current guess about seven flow


received energy function which is the energy function of only uh global motion parameters this

formulation after this could decision is a

a weighted icp uh i would say uh weighted icp and it can be straightforwardly

still so that the

the result of minimisation of this energy function is used them uh to

to receive energy function for stencil so we iteratively uh optimize

well global transformation parameters and then local transformation or

um the

this is the uh this problem can be solved with using a or do not

let me articulation uh will bring cost to the iterative solution iterative algorithm

after receiving of result of both these all of minimisation able both this function we

refine our expected sort faces and should start this uh procedure again iteratively

so some words about

weights the weights needed to

to reduce to regulate influence of if of each constraint in you know energy function

uh depending on cable on quality of

uh source data which are involved in this computation um each source or face to

after reconstruction are is supplied with a quality measure in you know

uniform of weights which two

depend on noise level of

of not of so face estimation yeah

uh the main idea the main idea just

if we have a good estimated uh so faces

it means that

both correspondences

are estimated it with high precision

in this case weight of data constraint in

energy function should be higher so that the gain of data concern should be fine

otherwise smoothness constraint should to should get a high higher you in and so let's

a smoothed or bad data

um we use to go common uncertainty propagation uh rule the to estimate the weight

of the data constraint but see that the noise level of data constraint is the

sum of noise levels of that are involved in this data constraint just a lot

so face and correspondent particle for expected surface regardless of

a you can have regularisation constraint by weight of tikhonov regularisation constraint we can uh

kick can control the growing all for scene flow from the T control the uh

the power of local transformation um conceptually that we prevent local transformation in places where

so faces

are estimated with high precision is if we know that's a face estimateable high-precision would

don't want to bend it

that's why we use uh weight of uh so face which should be transformed as

uh wait for tikhonov regularisation constraint

and uh wait all for smoothness regularisation constraint not now we calculate this as the

sum of

as the sum of form weights um of sinful or which are you stand for

weight values which are used for estimation of mean value for regularization for a for

a approximation of gradient and to be honest this question

question how to calculate

okay it is still open question i just want to show just some results so

here it is a distance map maps between two so faces which we uh mitch

with icp algorithm and propose it expected to face mentioned

uh so this is initial match which was done manually the result of

icp it looks like icp uh tries to optimize global distance between so faces


although it will have um has

tendency to move for so faces

one to another in places there the where they are more bare feet to each

other that we have more consistent in

for example this part of face and parts where uh we have

problems words of basis the form of this

uh these parts are uh if you look more closely it happens because in this

place is expected sort face

or let six four paces some more can a consistent and expensive face uh receive

more weights

so and application of scene flow remove practically all these deformations

oh on expert O one which is a fixed i just i just want to

show you three-dimensional result of this much is not perfect because there is still a

lot of problems first of all we cannot uh automatically this uh find difference between

uh between false so phase parts and so on so face parts which have no

uh correspondences at all box in places of where so faces i just the deformed

not that much i don't want to pay a lot of attention on these artifacts

because it is still subject to of work but in public in places versa faces


not very strong deformed we can bring them together and stitched so for comparison i

just show how it was

so this is the initial match

here so faces you can see that these parts

can not be measured by some documents so i switch to




okay thank you very much for your attention and




to be able to be honest yep for sure the lighting conditions it's very oh

how to see printed they inference very much to reconstruction but uh i had to

be to be honest it is it is not question to me because this reconstruction

and it was presented before uh


what do mean what it is possible to find the position of the light

more than just this problem was not can see that but for sure it is

i would say it is another fields of fu

which one

i mean whether we use a touch information for this matching


no we do not know such information with the feature description uh for matching only

uh that that's information is involved in you know matching strategy on the in a

way that uh we add additional weighting factor which is that a way we will