SATO Wataru Laboratory

Multivariate timing and Granger causality analysis of spontaneous facial mimicry in response to android dynamic facial expressions

(Hsu, Kelbakh, Yang, Minato, & Sato: Sensors)

Although evidence exists for android-induced spontaneous facial mimicry, the timing and temporal precedence (Granger causality) of this effect remain uncertain.

We used the Facial Action Coding System (FACS) to analyze simultaneous dyadic facial video recordings of participants observing android Nikola’s negative (frowning) and positive (smiling) dynamic facial expressions.

Principal component analysis of Nikola’s expressions indicated that in addition to the action units (AUs) 04 (brow lowerer) and 12 (lip-corner puller), AU 25 (lips part) and 26 (jaw drop) contributed significantly to Nikola’s facial expressions.
Cross-correlation analysis revealed AU04 mimicry of negative expressions and AU12 mimicry of positive expressions from 400 ms onwards.
AU25 and AU26 mimicry occurred faster, starting at around 200 ms.
Multilevel vector autoregression incorporated the android and participant AUs and quantified the temporal evolution of the Granger causality for the first time.
In addition to paired android?human AU04, 12, 25, and 26 effects, significant Granger causality was found between different android?human AU combinations, such as from android AU04 to participant AU25 in the negative condition, and from android AU25 to participant AU12 in the positive condition.

These results suggest that the spontaneous facial responses to Nikola’s expressions involved not only motor copying, but also higher-level goal emulation and motor planning in the mirror mechanism, supporting the reliability of the social function of android facial expressions.
Cross-correlation and Granger causality analysis can be valuable when further investigating behavioral matching in real-life contexts.

Return to Recent Research.
Return to Main Menu.