What does your face reveal about you, and who is the better judge: humans or AI?

“I never forget a face”, “She’s got an honest face”, “You could see it in his face”, and “She looks young for her age” are just a few of the often-used phrases suggesting that faces are important for our interactions with other people and what we think we know about them.

But can people really remember faces as well as they think they do, and can we really tell someone’s age from their face? Or can artificial intelligence (AI) do it better? And can we really tell if someone is trustworthy just by looking at their face?

Facial recognition

Research shows that humans exhibit a wide range of facial recognition abilities. At one end of the spectrum is prosopagnosia (face blindness). People with prosopagnosia are unable to recognise family members, partners, or friends. Prosopagnosia can be developmental, where a person does not have any brain damage, or acquired, where a person acquires prosopagnosia after a brain injury. Studies indicate that around 2% of the population has developmental prosopagnosia, and someone born with the condition may not even realise they have it. Acquired prosopagnosia is rare.

At the other end of the spectrum are people described as having a superpower, the “super-recognizers”. These are people with an extraordinary ability to recognize faces, to the point where they are increasingly being recruited for crime-fighting. A 2009 study¹ found that super-recognizers performed significantly better in a facial recognition experiment than did a control group.

Software can also recognise faces. Facial recognition technology has been used as an anti-terrorism measure for some time, and is now increasingly being used in domestic law enforcement and to protect against identity fraud and theft. For example, in the United States facial recognition is being used by local law enforcement agencies and the FBI, and to check identities when issuing driver’s licences. Social media providers have also begun to use it, for example Facebook has been developing its “DeepFace” AI system.

An extract from a document produced by the Executive Office of the President of the United States introduces the history of automated facial recognition and the predominant approaches:

There are two predominant approaches to the face recognition problem: geometric (feature based) and photometric (view based). As researcher interest in face recognition continued, many different algorithms were developed, three of which have been well studied in face recognition literature: Principal Components Analysis (PCA), Linear Discriminant Analysis (LDA), and Elastic Bunch Graph Matching (EBGM).

The performance of facial recognition technology has improved considerably in the past few years, with a 2013 test of facial recognition algorithms by the US National Institute of Standards and Technology (NIST) finding that:

…groups had improved their performance on the tests by from 10 and almost 30 percent. One organization decreased its error rate from 8.9 percent in 2010 to 6.4 percent in 2013.

While algorithm performance has improved, errors by the human operators² of facial recognition systems are currently causing a mismatch between evaluation tests and operational accuracy. This situation can be improved through effective recruitment, selection, training and mentoring.

So which is better at recognizing faces, humans or automated facial recognition technology? The facial recognition ability of humans falls across a wide spectrum, so despite its error rate, automated facial recognition technology is arguably more consistently reliable, and its accuracy is also progressively improving.

However, the results of a recent competition in China suggest that super-recognizers can, for the moment at least, outperform the technology. Wang Yuheng, who is famous in China for his incredible memory, was pitted against “Mark”, the facial recognition AI for Alipay, China’s largest electronic payment service. Megvii, the company that developed Mark, claims a better than human 99.50% success rate for its facial recognition technology. However, after a draw with Wang in the first two rounds, Mark lost in the third round of the competition.

Faces and age

Determining a person’s age from their face is also an area where technology has been advancing. The authors of a 2013 conference paper³ compare published methods of automatic age estimation from facial images, and then propose their own approach consisting of four main steps: preprocessing, facial component localization, feature extraction, and hierarchical age estimation.

Their approach addresses the issue of the aging process being determined not just by intrinsic factors, such as genetic factors, but also by extrinsic factors, such as lifestyle and environment. This means that different people of the same age can have very different appearances due to different rates of facial aging.

The authors also studied the ability of humans to estimate age using data collected through crowdsourcing. They compare this with the results of their automated approach, funding that the cumulative score within 5-year mean absolute error of their approach is better than the age estimates provided by humans.

While the way in which facial aging is accelerated by extrinsic factors presents a challenge to the accurate estimation of age from facial images, the extent to which your face has aged can also potentially be used to advantage to estimate your likely lifespan.

Facial trustworthiness

New research⁴ takes a novel approach to determining the truth in regard to an age-old question: can we really tell if someone is trustworthy just by looking at their face?

One possibility is that genetic expression leads to both untrustworthy-looking faces and untrustworthy behavior, but this would align with the now discredited claims of physiognomy. An alternative possibility is that although people reliably agree on which faces look untrustworthy or trustworthy, those judgments show no predictive validity because there is no reliable alignment with actual trustworthiness.

The researchers, however, propose a third possibility: that a lifetime of being treated as trustworthy or untrustworthy as a result of appearance may lead people to internalize these expectations and act in accordance with them, eventually resulting in appearance-based accuracy.

To test this possibility, the researchers engaged all of the students in an MBA course in a mixed-motive game. The researchers report that:

…we found that face-based judgments predicted trustworthiness. We also found that this effect was mediated by targets’ expectations of how other people would perceive them and by their intentions to act in accordance with those expectations. These results are consistent with an internalized-impressions account: Targets internalize other people’s appearance-based expectations and act in accordance with them, which leads facial appearance-based judgments to be accurate.

They further state that:

…given that people reach high consensus on which faces look trustworthy or untrustworthy, individuals with trustworthy- or untrustworthy-looking faces should have a lifetime of experience of being treated like trustworthy or untrustworthy people. Such experiences would range from the banal (e.g., whether strangers smile at them) to the life-changing (e.g., whether they get particular jobs). We believe that the cumulative effect of such treatment is likely to be powerful, as implied by work on self-fulfilling prophecies and the looking-glass self.

A thought-provoking perspective to consider in your interactions and relationships with other people!

Image source: faces by Lisa Padilla is licenced by CC BY 2.0.

References:

Also published on Medium.