PhD Research Journals

By: A. L. Yuille
From: http://cslu.cse.ogi.edu/nsf/isgw97/reports/yuille.html

I need to get hold of this paper.

1997 International Conference on Image Processing (ICIP '97) 3-Volume Set-Volume 3
10 26 - 10, 1997
Washington, DC

By L. Zhang

Abstract:
Automatic estimation of mouth features is one of the Important topics for face recognition and model-based coding of videophone sequences. In this contribution, an automatic mouth feature estimation algorithm which uses de-formable templates is developed. Here, the mouth features are represented by the comerpoints of the mouth as well as the so called lip outline parameters. The lip outline parameters describe the opening of the mouth and the thickness of the lips. Compared to previous works, simplified cost functions are introduced. Furthermore, an algorithm for automatic determination of whether the mouth is open or closed is developed. Experimental results obtained with typical videophone sequence are given to evaluate the performance of the proposed algorithm. It is shown that in about 37% of the images the mouth features could be automatically estimated, which can be used e.g. for improving face modelling in a model-based coder:

Both rigid and non-rigid transformation need to be considered in face recognition.

According to Tony Jebara "Additionally, our detection and recognition scheme must also be capable of tolerating variations in the faces themselves. The human face is not a unique rigid object. There are billions of different faces and each of them can assume a variety of deformations. Inter-personal variations can be due to race, identity, or genetics while intra-personal variations can be due to deformations, expression, aging, facial hair, cosmetics and facial paraphernalia. "
(Ref: http://www1.cs.columbia.edu/~jebara/htmlpapers/UTHESIS/node7.html)

He Will COme & Save You song by Bob Fitts

Registration is an essential issue to be addressed in various machine vision and medical imaging research areas. The process of registration determines a one-to-one mapping or transformation between the coordinates system in one space to other, such that points in the two spaces that correspond to the same anatomy are mapped to each other. In general, the registration procedure can be illustrated as three stages: (1) choice of transformation (e.g., rigid or non-rigid); (2) surface representation (e.g., points or curves) and similarity criterion (e.g., mutual information, distance or intensity difference); (3) matching, optimization and transformation computation. Registration methods based on intrinsic information are preferred in current researches for the marker-based registration will either result in unfavorable disadvantages of invasiveness or low registration accuracy. Based on the nature of face model as the object, to be registered with other face model modalities (2D or 3D), rigid and non-rigid registration is investigated respectively.

Rigid registration deals with 6 degrees of freedom, i.e., translation and rotation. While non-rigid should be applied when handle transformation of more than 6 degrees of freedom. Similarity criterion is closely related to the choice of matching primitives in terms of the size of transformation or weather the transformation is rigid or non-rigid.

(1) choice of transformation
(has been explained briefly yesterday)

(2) Surface representation and similarity criterion
There are four approaches to represent a surface for the sake of registration and they are:
a) Feature based
- Attempts to express surface morphology as a set of features which are extracted by a pre-processing step. Such feature provides a compact description of the surface.
- similar criterion: comparison of scalar measure

b) Point-based
- do not attempt to reduce the surface representation to a more compact description, rather they use all or a large subset of all points
- the primitive used is often the surface point itself
- similar criterion: minimal distance between a pair of surface points

c) model-based
- do not attempt to reduce the surface representation to a more compact description, rather they use all or a large subset of all points
- similar criterion: often an implicit criterion is used (external force or halting condition driven by two sets of data, with which an evolving deformable surface model must be reconciled.

d) global similarity


Argument: Feature-based and global approach are normally used for large motion/transformation while point or model-based methods are attractive in the case of small or iteratively estimated motion (high redundancy of information) and useful for estimating locally non-rigid transformation.


Above literature review are from:
http://mrcas.mpe.ntu.edu.sg/research/urobot/registration.htm

Audette M, Ferrie F and Peters T (1999), An algorithmic overview of surface registrations techniques for medical imaging, Medical Image Analaysis, Oxford University Press

The registration of 3D surfaces is dealt significantly in the area of machine vision including face recognition. Registration is defined as the estimation of a mapping between coordinate systems that correspond to the same anatomical point (Audette M et. al., 1999).

There are 3 main issues in 3D surface registration:-

(a) choice of transformation,
It concerns about the nature of relationship (transformation) between the two modalities
In face recognition, a rigid-body transformation may be applicable.

Rigid transformation: can be expressed by combining rotation and translation.
Rotation representations are
a) 3 x 3 matrix (rows of matrix define orthogonal axes)
– 9 degree of freedom
b) Euler angles (rotate x, + rotate y + rotate z)
– 3 degree of freedom
c) Axis-angle (axis of rotation + rotation amount)
– 4 degree of freedom
d) Quaternion (4D complex numbers)
– 4 degree of freedom


(b) elaboration of surface representation and similarity criterion,
The type of information extracted from the 3D surfaces, which typically characterised their local and global shape, and organise the gathered information into the a representation of the surface (for efficiency and robustness in the final stage)

(c) matching and global optimisation
Exploiting the information to estimate the transformation which best aligns local primitives globally

In practice, it is good to note that the data (surface information) may contain noise and distortion, and hence this can distort the registration between surfaces.


Reference:

Audette M, Ferrie F and Peters T (1999), An algorithmic overview of surface registrations techniques for medical imaging, Medical Image Analaysis, Oxford University Press

Taken from http://www.enpc.fr/certis/Papers/05certis05.pdf
By: Jean-Yves Audibert, Guillaume Charpiat, Olivier Faugeras, Renaud Keriven

Overview:
The research finds it necessary to use statistics on a set of 2D images to identify a person. This approach does not require any manual intervention to identify landmarks or regions of interest. They work directly on the deformation fields, which establish the correspondences between the whole images (this is the fundamental elements of the problem). Here, they deployed non-supervised algorithms to build such correspondence fields between images. Matching methods that were employed in his research is based on Gerardo’s (2002).

Compute (a) 1st order statistics
(b) 2nd order statistics
(c) Classification for recognition purposes – SVM with Gaussian kernel.

Image matching
The tasks of integrating information from different types of sensors, comparing data acquired at different time and/or putting the similar structure of two different images into correspondence are also known as image registration and template matching, and they are based upon the ability to automatically map points between the respective domains of images (i.e. image matching).

Given two images, A and B. The images are defined in a rectangular subject W of the plane R2. A deformation field, f, is calculated so that the warp image A o f resembles B. The deformation field should be smooth and invertible, meaning that the diffeomorphism f from the rectangular subset W to the identity on the image boundary. Then a criterion is chosen. A criterion expresses the similarity between the 2 images A and B. Hence, this project employed image matching algorithm which consists of minimising with respect to the deformation field through a multi-scale gradient descent.

The first order statistics of a set of images were computed with a diffeomorphic matching approach (without landmark) and then use them in a classification task. They are still in the process of including second order statistics in the classification algorithm.

Experiment and results:
To measure the efficiency of the method, we cross-validate the errors by taking out one subject among the 13 subjects in the database and consequently using 60 faces labeled by their expression to deduce the expression of the five remaining faces. The cross-validation error is 24 upon 65 faces.

For comparison purposes, we trained a Support Vector Machine with Gaussian kernel using only the gray level intensity information. In this case we obtained a larger cross-validation error of 27 upon 65 faces which shows the interest of using the diffeomorphisms.

Critical discussion:
The results can be encouraging. However, from the experiments, there is no drastic improvement. Thorough analysis and further enhancement required to improve the proposed methods used.

Titled: Variational methods for multimodal image matching
Thesis by Gerardo Hermosillo Valadez (2002)

Retrieved at: http://rangiroa.essi.fr/rapports/2002/02-these-hermosillo.pdf

This thesis deals with a specific problem in the field of image analysis namely image matching. Image matching is establishing correspondences between points in two different images. Solving this problem is a fundamental prerequisite in understanding and exploiting the contents of images. The term multimodal image matching here means having to establish correspondence information from several sources (such as camera calibration, 3D reconstruction from two or more views, using several modalities e.g. MRI etc).

As matching under varying illumination conditions are difficult, they are other possible approach to the solution of this problem that is by defining meaningful structures in the images, invariant under transformations of grey-level intensities (e.g. edges, corners) and then design low-level methods to extract the structure from the image. Hence, this thesis propose a variational framework for dense multimodal matching, which relies on the computation of global and local statistical dissimilarity measures (calculus variations and partial differential equations) between intensities of corresponding regions.

Research questions:
(a) What should a good matching satisfy?
a. Optical flow constraint
b. Local image differences
c. General block matching
d. Cross-correlation
e. Mutual information
f. Correlation-ratio

(b) Model of transformation or some other constraint allowing to limit the search for possible matches.
a. Search for low-D transformations (e.g. affine, quadratic, spline-interpolation)
b. Statistical similarity
c. Non-rigid (complex) transformation using statistical similarity
d. Block-matching strategies
e. Parametric intensity correlations

The main contributions of his work are listed below:
(a) He proposed a unifying framework for a family of variational problems for multimodal image matching. This framework subsumes block matching algorithmic approaches as well as techniques for non-rigid matching based on the global estimation of the intensity relations.

(b) He formally computed the gradient of local and global statistical dissimilarity measures, which is an essential step in defining and studying the well posedness of their minimization. Contrary to more standard matching terms like intensity differences or the optical flow constraint, these matching terms are non-local, which makes the standard method of the calculus of variations inapplicable.

(c) He showed that the operators defined by the gradients of these criteria satisfy some Lipschitz-continuity conditions which are required for the well posedness of the associating matching flow.

"Statistical Shape Analysis"
by Ian Dryden and Kanti Mardia,
Publisher John Wiley and Sons (1998)

reviews can be found in http://life.bio.sunysb.edu/morph/books/DrydenMardiaReview.html

Introduction:
After having almost 2 hrs of discussion with Theodore, who is also during his Phd in the area of face recognition, a few points were brought-up regarding the problems with face recognition. They are the registration and correlation problems.

According to Simon D. (1996), registration problem is concerned with finding a spatial transformation which best aligns (or correlate) two object representations. Correlation is a statistical technique to show whether the two objects are strongly related pairs of variables. Once the registration and correlation problems are solved, mapping can be established. There is a variety of tasks that can be performed using the spatially aligned object representations (aka matching)

The are 3 main matching techniques namely landmark matching, surface matching and intensity matching, which are used in various applications such as medical, science, engineering. These matching techniques helps in registering individual image to form a combined image or insight.

According to Steggmann et. al. (2000), a landmark matching is matching a point on each object to a correspondence point in a population.

Surface matching is a match method based on the structures of interest i.e. the points, lines, curves, surfaces or volumes. In this research, the detailed surface matching techniques will cover the 3D points with 3D structures, matching areas/regions/surfaces (piecewise segment matching of contours).

In Active Appearance Models, intensity matching (aka texture matching) is defined as matching the pixel intensity across the object with the correspondence intensity in another object (Steggmann et. al., 2000).

Discussion:
Here, all the above discussed matching methods will be proposed in the research.


References:

Stefman MB, Fisker R, Ersboll BK, Thodberg HH and Hyldstrup L (2000), Active Appearance Models: Theory and Cases, http://www2.imm.dtu.dk/~aam/main/node34.html (retrieved on the 10 may 2005).

Simon DA (1996), Fast and Accurate Shape-based Registration, PhD Thesis, Carnegie Mellon University, Pittsburgh, US

I am just starting my PhD in Imperial College, UK in the research area of 3D face recognition.

I am planning to look into developing a novel initialisation methods and mapping, aiming to solve registration problems as well as to model and to synthesize a 3D face appearance model.

Suggestion or idea to assist me is very much appreaciated. In this blog i will include all the summary of the literatures I have reviewed.