HACK

HACK (Head-And-neCK) is a novel parametric model for constructing the head and cervical region of digital humans. It aims to disentangle the full spectrum of neck and larynx motions, detailed facial expressions as well as appearance variations, offering more personalized and anatomically consistent controls that are compatible with CG engines.

Feature	Ready
Template mesh	✔️
Skeleton joints	✔️
Skinning weights	✔️
Larynx operation	✔️
Shape blendshapes	✔️
Expression blendshapes	✔️
Pose blendshapes	✔️
Blendshapes space	🔜
PyTorch module	✔️
Texture basis	🔜

HACK (Head-And-neCK) Model

The HACK Model consists of a base topology along with definitions of facial landmarks, shape blendshapes, expression blendshapes, cervical joints, and pose blendshapes.

Face Model Topology

HACK uses the same facial topology as ICT-FaceKit, including 14062 vertices and 14034 quad faces.

Ordinal#	Geometry name	Vertex range	Polygon range	#Vertices	#Faces
n/a	All	range(0,14062)	range(0,26384)	26719	26384
#0	Face	range(0,9409)	range(0,9230)	9409	9230
#1	Head and Neck	range(9409,11248)	range(9230,11144)	1839	1914
#2	Mouth socket	range(11248,13294)	range(11144,13226)	2046	2082
#3	Eye socket left	range(13294,13678)	range(13226,13630)	384	404
#4	Eye socket right	range(13678,14062)	range(13630,14034)	384	404

Face Area Details

Ordinal#	Geometry name	Vertex range	Polygon range	#Vertices	#Faces
#0	Full face area	range(0,9409)	range(0,9230)	9409	9230
#1	Narrow face area	range(0,6706)	range(0,6560)	6706	6560

Facial Landmarks

HACK shares the same Multi-PIE 68 point facial landmarks indices as ICT-FaceKit: [1225, 1888, 1052, 367, 1719, 1722, 2199, 1447, 966, 3661, 4390, 3927, 3924, 2608, 3272, 4088, 3443, 268, 493, 1914, 2044, 1401, 3615, 4240, 4114, 2734, 2509, 978, 4527, 4942, 4857, 1140, 2075, 1147, 4269, 3360, 1507, 1542, 1537, 1528, 1518, 1511, 3742, 3751, 3756, 3721, 3725, 3732, 5708, 5695, 2081, 0, 4275, 6200, 6213, 6346, 6461, 5518, 5957, 5841, 5702, 5711, 5533, 6216, 6207, 6470, 5517, 5966]

Expression blendshapes

HACK Model includes 55 expression blendshapes, which have the same definition from ICT-FaceKit.

Current expression shapes adopt the naming convention of the Apple ARKit, but with "Left" and "Right" specified with "_L" and "_R". Additionally, we separeate the shapes (browInnerUp_L and browInnerUp_R), and (cheekPuff_L and cheekPuff_R).

Shape blendshapes

HACK Model includes a set of 200 PCA blendshapes of HACK shape space.

Joints

HACK Model includes 8 joints corresponding to the bottom points of 7 vertebrae (C1-C7) and the apex of C1. This setup yields 8 bone transformations denoted as c7-t1, c6-c7, c5-c6, c4-c5, c3-c4, c2-c3, c1-c2, and o-c1.

Skinning weights

HACK Model includes skinning weights corresponding to 8 rotation joints in the cervical spine.

Pose blendshapes

HACK Model includes 72 pose blendshapes, whose weights are automatically set by the rotation of 8 joints.

File Structure

HACK-Model
|-- data    # recorded data for demonstration
|-- model
|   |-- 000_generic_neutral_mesh_newuv.obj  # template mesh for larynx
|   |-- 000_generic_neutral_mesh.obj        # template mesh
|   |-- blendshape.npy                      # ICT-FaceKit expressions
|   |-- bones_neutral.json                  # template bone
|   |-- E.npy                               # expression blendshapes
|   |-- Lc_mid.png                          # template larynx shape
|   |-- P.npy                               # pose blendshapes
|   |-- S.npy                               # shape blendshapes
|   |-- ts_larynx.npy                       # template larynx shape
|   |-- weight_map_smooth.npy               # skinning weight
|-- examples.ipynb  # examples
|-- hack_model.py   # HACK model definition
|-- helper.py       # utils  

Examples

PyTorch model

See examples.ipynb for running examples.

Blender model

Coming soon...

Contact

If you have any questions, please send an e-mail to shanghaitechmars@foxmail.com