We have proposed a method for reconstructing the shape of specular surfaces using a level set implementation to which a force field is coupled. This force field is derived from constraints on the surface that determines the shape of the surface, positions it is space and keeps the surface smooth. Experiments with synthetic data show very promising results.
The proposed method is not primarily intended to be used for surface estimation alone, and in future work it should be integrated with such methods, e.g. [66]. It could also be interesting to try other smoothing schemes, e.g. fourth order flow, if the proposed method should be used alone. Lastly, it could be interesting to formalize the proposed framework more, e.g. by formulating it as minimizing a functional.
Acknowledgement
We would like to thank J. Andreas Bærentzen, for interesting discussions and advice regard-ing the level set implementation.
C
HAPTER14
Deformable Structure from Motion
by: Henrik Aanæs and Fredrik Kahl
This is a working paper for our efforts on deformable structure from motion. Preliminary versions of this work have been published at [1, 6, 7]. At present we still have some unsolved problems particularly in regards to the nonlinear optimization in conjunction with the bundle adjustment. This induces problems with final validation of the proposed approach. Other approaches to model selection also need to be investigated, but requires the above mentioned issues to be resolved. These issues are discussed more in the following. Main advances have been made since the latest published version, i.e. [6], hence this is the version included here.
Abstract
In this paper the problem of estimating the scene structure and the motion of a non-rigid deformable object is analyzed. The object is supposed to deform according to a linear model while the motion of the camera relative the object can be arbitrary. No domain specific prior of the object is required. A complete algorithm is developed which consists of first creating an initial guess using a factorization algorithm, working on linearized data. Here upon an optimal solution is obtained through a non–linear optimization scheme, i.e. modified bundle adjustment. The complexity of the linear model is determined by model selection. The proposed approach assumes that an appropriate number of features have been tracked on the object.
With non-rigid objects, special issues concerning the well-posedness of the problem arises. There are a number of inherent ambiguities which do not occur in the traditional
setting of the structure and motion problem. These ambiguities have been identified and it is argued that they can be resolved using regularization. Lastly the effectiveness of the proposed method is investigated on both real and simulated data.
14.1 Introduction
The estimation of structure and motion from image sequences is one of the most studied problems within computer vision. However, until now the literature, and hence the accom-panying solutions, have mainly been focused on dealing with rigid objects [39, 89, 94, 204].
There have been some dealings with the estimation of structure and motion of multiple in-dependently moving objects [46, 72, 112, 178, 202].
Yet, smoothly deforming structures are everywhere around us: trees sway, waves flutter and humans walk in a very non-rigid way. In fact, many interesting objects in our environ-ment are non-rigid, e.g. humans, plants and animals. Hence there is a great need to deal with the estimation of their structure as well, since these are common ’objects’ in our everyday lives.
In [133, 153] estimation of non-rigid motion is performed using a Kalman filter with physics based priors on the dynamics. However, the deforming objects need to be properly initialized. A lot of work on non-rigid motion estimation use domain specific priors. For example in [182, 186] human motion is estimated using a complete model of the body and its dynamics. Such priors are inconvenient as the problem of object identification is not a solved problem.
In this paper we present an approach for estimating the structure and motion of deforming or non-rigid objects. This is done by employing the Principal Component Analysis (PCA) framework [98, 151], whereby the structure model is a linear deformable model as described in e.g. [19, 45, 212]. These types of models have proven to be highly effective in expressing many types of deforming objects. Thus, the model is fairly general making it applicable in many different scenarios. The motion of the camera relative to the deforming object can be arbitrary. We assume that the cameras are calibrated. However, it is straightforward to generalize the approach to an uncalibrated setting.
The proposed approach assumes that the tracking problem has been solved, and hence assumes a number of tracked features on the object in question. This could e.g. be achieved via [24]. As is common practice with the standard rigid objects, the approach falls in two main steps. First an approximate solution is obtained using a factorization type algorithm, which assumes affine cameras. This approximate solution then forms a much needed ini-tial guess to a non–linear optimization algorithm, which can be seen as a modified bundle adjustment algorithm.
However, the extension of structure from motion from rigid to non-rigid objects is non–
trivial. At any given frame only a 2D projection of the 3D structure is given. With rigid objects this problem is circumvented by other projections of the same structure being present in other cameras. In the non-rigid case, this possibility is not present, because the structure is assumed to change between frames. This added complexity is approached here by modelling the variability of the structure in question. The complexity of the model is determined by