Page 34 - IJEEE-2022-Vol18-ISSUE-1

30 |
movement is called an Action Unit (AU). Each AU is
identified by a number (AU1, AU3, AU20, . . . etc.).
Samples of these action units are presented in Table I. Facial (LAC-GAN) relied on face action units annotations. They
expressions are produced by combining the action units. For
instance, combining AU1 (Cheeks raiser), AU4 (Lip Corner
Puller), and AU15 (Lips Part) produces Happiness
expression.
Recently, the interest in utilising the FACS for
producing visual speech has declined. This is because two
reasons. Firstly, nowadays most of the face models designed
for visual speech synthesis purposes are not relied on human
anatomy, but consist of high detailed polygon meshes and
textures which are generally automatically computed by 3D
scanning approaches. Normal mesh deformations are learned
by advanced 3D motion capture methods, which is faster and
easier than a manual explanation of the numerous muscles of
the face and their effect on the facial appearance. Secondly,
FACS offers many Action Units that can be utilised to
precisely mimic certain expressions, but, these Action Units
are less appropriate to simulate all the detailed gestures of
the face corresponding to speech production. Such that
FACS is not optimised for modelling visual speech. Facial
expression generation or synthesis has recently received
increasing attention in the facial expression modelling
domain. Ekman and Friesen [17] developed the FACS for
describing facial expressions with some basic face action
units (AUs), each of which represents a basic face muscle
movement or expression change.
Kumar and Sharma [18] suggested an improved Waters
facial model utilised as an avatar for research published in
[19], which discussed a facial animation system driven by
the FACS in a low-bandwidth video streaming setting. To
build facial Expressions, FACS defines 32 single Action
Units (AUs) which are created by an underlying muscle
action that interact in various methods. In this work
enhancements were provided to the Waters facial model by
enhancing its UI, adding sheet muscles, providing an
alternative implementation to the jaw rotation function,
introducing a new sphincter muscle model that can be
utilised around the eyes and alterations to operation of the
sphincter muscle utilised around the mouth. Zhou et al. [20]
introduced a conditional difference adversarial autoencoder
(CDAAE) to transfer AUs from absence to presence on the
global face. This approach uses the low-resolution images,
which could lose facial details vital for AU production.
Pumarola et al. [21] proposed GANimation which transfers
AUs on the whole face and generate a co-generated
phenomenon between different AUs. For this approach it is
difficult to generate a single AU respectively without
touched the other AU.
With the recent rise of deep learning, CNN have been
widely used to extract AU features. Zhao et al. [22]
suggested a deep region and multi-label learning (DRML)
system to partition the face images into 8 - 8 blocks and
utilised individual convolutional kernels to convolve each
block. Despite this method treats each face as a set of
individual parts, it partitions blocks uniformly and does not
reflect the FACS knowledge, thus leading to poor
performance. Zhilei Liu [23], proposes an Action Unit (AU)
Shakir & Al-Azza
level facial expression synthesis approach named Local
Attentive Conditional Generative Adversarial Network
build a model for facial action unit synthesis with more local
texture details. In this approach local AU regions is
integrated with conditional generative adversarial network.
The proposed method manipulates AUs between various
states, which learns a mapping between a facial manifold
related to AU manipulation. Moreover, the key point of this
approach is to do the manipulation module concentrate only
on the generate of local AU region without touching the
remainder identity information and the other AUs. The
development of deep graph networks modelling has recently
attracted increasing attention. Zhilei Liu [24] introduces an
end-to-end deep learning framework for facial AU detection
with graph convolutional network (GCN) for AU relation
modelling. They use the graph convolutional network (GCN)
[25] for AU relation modelling to support the facial AU
detection. AU related areas are extracted; these AU areas are
feed into some specific AU auto-encoder for deep
representation extraction. In addition, each latent
representation is pull into GCN as a node.
Table I
Sample single facial action units.
AU FACS Name
1 Inner Brow Raiser
14 Dimpler
5 Upper Lid Raiser
17 Chin Raiser
D. Moving Pictures Experts Group-4
Moving pictures experts’ group-4 (MPEG-4) is an
object-based multimedia compression standard that permits
encoding independently diverse scene’s audiovisual objects
(AVO). MPEG-4 has facial definition parameter set (FDP)
and the facial animation parameter set (FAP) which were
designed to describe the facial shape and texture, as well as
regenerating the animation of faces for instance speech
pronunciation, expressions, and emotions. MPEG-4 facial
animation outlines many parameters of a talking face in a
standardised approach. It identifies and animates 3D face
models by describing face definition parameters (FDP) and
facial animation parameters (FAP). FDPs enclose
information for building particular 3D face geometry, while
FAPs encode motion parameters of key feature points on the
face over time. In MPEG-4, the head is grouped into 84
feature points (FPs), every point defines the shape of an area.
Fig. 3 demonstrates part of the MPEG-4 feature points. After
excluding the feature points that are not simulated by FAPs,
68 FAPs are classified into collections. Samples of these
collections are presented in Table II. The FAPs are two
groups, one represents the facial expressions which consist
of six basic emotions, i.e., surprise, anger, sadness, joy,
disgust, and fear. The second one concentrates on facial areas
such as the left mouth corner, the chin bottom and the right
eyebrow. Refer to the MPEG- 4 facial animation book for
more details about MPEG-4 facial animation standard [26].
El Rhalibi et al. [27] presented a method relied on 3D
Homura that integrate MPEG-4 standards to realistic