Digital Avatar Faces Appear Believable When Eyes Match Emotions

TL;DR: A virtual smile or glare landed best with direct eye contact. Sadness only looked believable when the eyes pointed downward — sideways did the opposite. Fear refused to follow the theory at all. Two experiments testing whether digital characters need their gaze to do the emotional work their faces are already doing.

Source: Cognition and Emotion (2026) | Haile et al.

Computer-generated faces do not feel anything. People still rate whether their expressions look real. That dissonance is what makes digital humans a useful research tool — they let researchers isolate perceptual signals from inner states, because there is no inner state. If a virtual smile feels off, the failure has to be in the perceptual signal, not in the smiler.

The signal this paper went after is the eyes — specifically, where they point.

Why Digital Humans Are a Better Lab Than Real Humans

People are skilled face-readers, but they are not reading a checklist of isolated features. A smile depends on the cheeks and the eyes. Anger depends on the brow, the mouth, and whether the other person is looking straight at you. The whole face works together — and the wrong eye direction can collapse an otherwise correct expression.

Real humans make this hard to study. A real person who smiles may not feel happy, and the observer can never fully separate the expression from the person’s actual inner state. A computer-generated face has no inner state to leak. Its believability is a pure perception problem, which is exactly the experimental advantage Haile and colleagues used.

The hypothesis they were testing is the shared signal hypothesis: gaze direction and emotion convey compatible social intentions. Approach emotions like happiness and anger should pair with direct gaze, because both are aimed at someone. Withdrawal emotions like sadness and fear should pair with averted gaze, because both signal disengagement. Compelling theory; experimentally testable; results turn out to be only partly true.

Ten Virtual Adults, Calibrated to Be Just Imperfect Enough

The team sculpted ten highly realistic virtual adults using professional animation software. Expert raters tuned digital muscle controls until each face conveyed anger, fear, happiness, or sadness clearly. Crucially, they did not select only the most flawless expressions. If every face already looked maximally authentic, gaze direction would have no room to make a measurable difference. The stimulus set kept just enough ambiguity for the eyes to matter.

In the first experiment, 150 adults rated expression believability across these conditions: anger and fear with eyes either direct or shifted sideways at several angles; happiness and sadness with eyes either direct or shifted downward. The intensity of each expression was modeled separately, so the analysis could ask whether gaze affected believability beyond the simpler “stronger expressions feel more real” effect.

The second experiment isolated sadness. 64 new participants rated sad faces with direct, downward, or sideways gaze — testing whether sadness benefited from any form of averted gaze, or specifically from looking down.

Direct Eye Contact Carried the Approach Emotions

Happiness and anger both peaked under direct gaze. As the eyes moved sideways, believability dropped. That fits the social logic of approach. A smile invites engagement; anger confronts. Both emotions are organized around the other person, and both perceptually require the eyes to be aimed at that person.

Because the analysis modeled intensity separately, the gaze effect was not just a side effect of stronger-looking expressions reading more clearly. Gaze added something independent. A correctly sculpted smile can still feel wrong if the eyes are doing something different.

The implication for digital character design is concrete. Virtual therapy, training, gaming, education, and customer-service agents all lean on believable social signals. A character can have the right expression hard-wired into its facial geometry, and still feel wrong if the eye direction contradicts the social meaning of the emotion.

Sadness Needed Downward Eyes — Not Just Averted Eyes

Sadness behaved differently from a generic averted-gaze rule. The farther down the avatar looked, the more believable the sadness became. That fits real social perception: downward gaze signals withdrawal, shame, grief, or reduced readiness to engage. It is a specific posture of disengagement, not just absence of contact.

The follow-up experiment tested whether any averted gaze would do. It would not. A new group of 64 participants rated sad faces with direct, downward, or sideways gaze — and downward gaze again amplified believability, while sideways gaze actively reduced it.

That asymmetry is the paper’s most useful design takeaway. Human observers do not treat averted gaze as a generic “not looking at me” cue. They read the direction. Down is not sideways. Down reads as inward processing, grief, defeat. Sideways reads as distraction, scanning, social orientation toward something else. Digital character design has to separate those meanings, because averted gaze can mean embarrassment, distraction, threat monitoring, avoidance, or disinterest depending on which way the eyes go.

Fear Refused the Theory

The shared-signal prediction said fear should look more believable with averted gaze, because fear often prepares avoidance. Sideways gaze did not significantly change fearful-face believability in the first experiment.

There are several plausible reasons. Fear may depend more heavily on dynamic timing, on the widening of the eyes, or on where the apparent threat is located. A static forward-facing avatar with sideways eyes does not give the observer enough context to know what the face is afraid of — and fear is unusually context-hungry. The observer often needs to know whether the avatar is afraid of them, of something nearby, or of something offscreen entirely. A still face cannot supply that.

The fear exception keeps the study from collapsing into a simple design rule. Eye contact helps some emotions, downward gaze helps sadness, and fear seems to need motion or scene context before any gaze direction carries the expected meaning.

What This Does and Does Not Cover

The faces were static, forward-facing, and designed to match white European physical characteristics. Participants were also drawn from majority white European populations. That choice reduced appearance unfamiliarity as a confound, but it narrows the cultural reach of the finding. Whether the same gaze-emotion pairings hold across other cultural face norms is an open question that follow-up work will need to answer.

Real expressions also unfold over time. People move their heads, blink, glance away and back, coordinate gaze with posture and voice. Dynamic video and more diverse digital faces would extend the test into something closer to the social environments avatars actually live in.

The principle the paper does establish is direct enough to be actionable: digital emotion is not only a matter of drawing the right smile or frown. A believable virtual person needs eyes that tell the same social signal as the face. In any avatar used for therapy, games, education, telepresence, or customer service, eye direction that contradicts the expression is the kind of small mismatch that quietly makes a polished character feel psychologically wrong.