Whose Face Is It Anyway? A Multi-Model Audit of Facial Affect Recognition on Children, and Why the Gap Is the Head, Not the Features

2026-10-06

AI Summary

This study addresses the significant performance degradation of facial expression recognition models trained on adult data when deployed in child-centric scenarios. Through systematic auditing and linear probing analysis across five pretrained models, we identify that this age-induced bias originates from the classification head rather than the feature representation layers. Accordingly, we propose a low-cost correction method that solely recalibrates the classification head, effectively bridging the age domain gap without retraining the feature extractor. Experimental results demonstrate that the proposed recalibration substantially improves accuracy on children's datasets by 0.13 to 0.28 while exerting negligible impact on adult recognition performance. This work provides an efficient and generalizable solution for cross-age facial expression recognition.

Abstract

Facial affect models are trained almost entirely on adults, yet are increasingly applied to children in education, health, and developmental research. We present a controlled, multi-model audit of five AffectNet-pretrained expression models (EmoNet, EmotiEffLib, DDAMFN++, OpenFace 3.0, LibreFace) on children, across four child image datasets, the AffectNet-8 validation set, and two spontaneous child video datasets, through one shared harness. Three findings emerge. First, the child gap is model-agnostic: every architecture degrades from posed to naturalistic faces and shares the fear$\rightarrow$surprise confusion. Second, it is concentrated and corroborated across all five models: open-mouth faces (read as surprise, correlating with the AU26 jaw drop) and South-Asian children degrade systematically, with a smaller averted-gaze penalty, while closed-mouth faces, White and Black children, and direct gaze do not; the bias tracks expression morphology and specific populations, not skin tone. Third, the gap is diagnosable: a linear probe on frozen features reaches 0.75-0.91 on unseen children versus 0.48-0.66 zero-shot, so it lies largely in the classifier head, not the representation, whereas dimensional valence/arousal regression degrades sharply under domain shift. Building on this, recalibrating only the head on a little target data recovers $+0.13$ to $+0.28$ on the two largest child sets across all five models at negligible adult cost, though the gain is in-distribution and does not transfer across child collections. We will release the harness, per-sample predictions, and analysis code; the child face data stays license-locked and is never redistributed.

Problem

• Facial Affect Recognition
• Children
• Model Bias
• Domain Shift
• Performance Gap

Innovation

• Facial Affect Recognition
• Multi-Model Audit
• Classifier Head Recalibration
• Linear Probe
• Domain Adaptation