1.4 Nomenclature
Variables used without an explicit denotation conform to the nomenclature below.
I An image (or the unit matrix).
E The error energy in model to image fit.
k The number of Euclidean dimensions. In the planar case k= 2.
n The number of points on a shape.
N The number of shapes in a training set.
m The number of texture samples inside a shape.
x A normal vector, or a planar shape.
Σ The covariance matrix (also called the dispersion matrix).
Λ A diagonal matrix of eigenvalues.
Φ A matrix of eigenvector columns.
λi Theith eigenvalue.
φφi Theith eigenvector.
θ A 2D shape rotation given in radians.
32 Chapter 1. Introduction
33
Chapter 2
Background
In recent years, the model-based approach towards image interpretation named deformable template modelshas proven very successful. This is es-pecially true in the case of images containing objects with large variability.
As the precise definition of a deformable template model we will use the one of Fisker [26]:
Definition 1: A deformable template model can be character-ized as a model, which under an implicit or explicit optimization criterion, deforms a shape to match a known object in a given image.
Among the earliest and most well known deformable template models is the Active Contour Model – known as Snakes proposed by Kass et al.
[46]. Snakes represent objects as a set of outline landmarks upon which a correlation structure is forced to constrain local shape changes. In order to improve specificity, many attempts at hand crafting a priori knowledge into a deformable template model have been carried out. These include Yuille’s et al. [73] parameterization of a human eye using ellipses and arcs.
In a more general approach, while preserving specificity Cootes et al. [15]
proposed the Active Shape Models (ASM) where shape variability is learned through observation. In practice, this is accomplished by a training set of annotated examples followed by a Procrustes analysis [35] combined with a principal component analysis.
34 Chapter 2. Background
A direct extension of the ASM approach has lead to the Active Appearance Models [10]. Besides shape information, the textual information, i.e. the pixel intensities across the object, is included into the model. The AAM has been further developed in [13, 14, 22].
Jain et al. [44, 45] classifies deformable template models as either beingfree formorparametricwhere the former denotes model deformation dependent onlocalconstraints on the shape and the latterglobalshape constraints. By building statistical models of shape and texture variation from a training set, AAM qualifies as being a parametric deformable template model.
Quite similar to AAMs and developed in parallel herewith, Sclaroff &
Isidoro proposed the Active Blob approach [43, 58]. Active Blobs is a real-time tracking technique, which captures shape and textual informa-tion from a prototype image using a finite element model (FEM) to model shape variation. Compared to AAMs, Active Blobs deform a static texture, whereas AAMs change both texture and shape during the optimization.
Also based on a prototype – and a finite element framework using Galerkin interpolants – is the Modal Matching technique proposed by Sclaroff &
Pentland [59]. Objects are matched using the strain energy of the FEM.
A major advantage is that the objects can have an unequal number of landmarks and it easily copes with large rotations.
For further information on deformable template models, the reader is re-ferred to the surveys given in [26, 4, 44, 51].
Part I
Statistical Models of Shape and Texture
35
37
Chapter 3
Introduction
This part provides an in-depth treatment and discussion of how Active Appearance Models build its statistical models of shape and texture and how these are combined into one unified model.
The notation, treatment and even some parts of the algorithms is occa-sionally somewhat different from the treatment by the inventors of AAMs [10, 14]. However, the overall ideas are the same.
Figure 3.1: The three steps of handling shape and texture in AAMs.
The handling of shape and texture can be viewed as dual processes.1 The setup of these processes is quite similar to other data handling processes though the composition of techniques is quite unique.
The first step is the data acquisition. Hereafter follows a suitable normal-ization after which the data are ready to be analyzed and described in terms of statistical models. The process setup is given as a flow chart on figure 3.1.
1Though the texture mode in reality is defined in terms of the shape model.
38 Chapter 3. Introduction
To stress the coherence between shape and texture handling the steps are specified below.
Capture
Shape Captured by defining a finite number of points on the contour of the object in question.
Texture Captured by sampling in a suitable image warping function (e.g. a piece-wise affine, thin-plate or another warp function).
Normalization
Shape Brought into a normalized frame by aligning shapes w.r.t.
position, scale and orientation using a Procrustes analysis.
Texture Removing global linear illumination effects by standardiza-tion.
Statistical Analysis
Shape & Texture Principal Component Analysis is performed to achieve a constrained and compact description.
The level of detail in the following chapters is adjusted so that the current implementation can be understood and/or redone solely upon this descrip-tion.
39
Chapter 4
Shape Model Formulation
4.1 Overview
The following chapter provides the fundamental concepts and techniques needed to understand the statistical models of shape used in AAMs. First the concept of a shape is defined, next – the basis of the mathematical framework – the concept oflandmarksis treated. The chapter is concluded by demonstrating how shape variation can be efficiently modeled using principal component analysis.
Effort has been put into making the treatment rich on examples and refer-ences to further treatment of the topics.