Multilinear Modelling of Faces and Expressions

Stella Grasshof, Hanno Ackermann, Sami Sebastian Brandt, Jörn Ostermann

Research output: Journal Article or Conference Article in JournalJournal articleResearchpeer-review

Abstract

In this work, we present a new versatile 3D multilinear statistical face model, based on a tensor factorisation of 3D face scans, that decomposes the shapes into person and expression subspaces. Investigation of the expression subspace reveals an inherent low-dimensional substructure, and further, a star-shaped structure. This is due to two novel findings: (1) increasing the strength of one emotion approximately forms a linear trajectory in the subspace. (2) All these trajectories intersect at a single point not at the neutral expression as assumed by almost all prior works but at an apathetic expression. We utilise these structural findings by reparameterising the expression subspace by the fourth-order moment tensor centred at the point of apathy. We propose a 3D face reconstruction method from single or multiple 2D projections by assuming an uncalibrated projective camera model. The non-linearity caused by the perspective projection can be neatly included into the model. The proposed algorithm separates person and expression subspaces convincingly, and enables flexible, natural modelling of expressions for a wide variety of human faces. Applying the method on independent faces showed that morphing between different persons and expressions can be performed without strong deformations.
Original languageEnglish
JournalI E E E Transactions on Pattern Analysis and Machine Intelligence
Volume43
Issue number10
Pages (from-to)3540-3554
Number of pages14
ISSN0162-8828
DOIs
Publication statusPublished - 1 Oct 2021

Keywords

  • Statistical shape model
  • tensor model
  • HOSVD
  • person transfer
  • expression transfer
  • 3D-reconstruction

Fingerprint

Dive into the research topics of 'Multilinear Modelling of Faces and Expressions'. Together they form a unique fingerprint.

Cite this