A 2016 study exploring whether machine learning can accurately predict criminality based solely on facial images has sparked debate on the social implications of such technology.
The research, conducted by Chinese academics Xiaolin Wu and Xi Zhang, claims to be the first of its kind in using automated face recognition to categorize criminals versus non-criminals.
Though intended as pure academic inquiry, the paper’s premise and findings raise ethical concerns.
Key Facts:
- The study used ID photos of 1,856 Chinese men, half convicted criminals and half non-criminals, to train machine learning models including logistic regression, KNN, SVM and CNN.
- The highest accuracy achieved was 89.5% by a convolutional neural network, with the other models also performing consistently well above chance.
- The authors argue this validates the idea of using computer vision to infer criminality from faces.
- They found certain facial measurements like lip curvature, nose-to-mouth angle and inner eye corner distance differed on average between the criminal and non-criminal groups.
- The study concludes criminal faces vary much more than non-criminal ones, forming distinct subtypes, while law-abiding citizens’ faces conform more to a “normal” pattern.
Source: arXiv: Computer Vision and Pattern Recognition
Using Machine Learning to Predict Criminal Faces: Major Backlash
The paper prompted major backlash upon being shared on arXiv, with many in the machine learning community denouncing its premise as “phrenology 2.0” and flirting with scientific racism.
Critics argued analyzing faces to categorize people as “criminal” versus “law-abiding” unfairly encodes bias and stereotypes into AI systems.
There were also concerns about the lack of transparency around the data sources and potential issues like sampling bias.
The authors defended their work as neutral academic inquiry, stressing the study was only meant to test if machine learning could match humans’ snap judgments of faces.
They maintained proper oversight could ensure fairness, though agreed that practical use of such technology would be unethical.
Machine Learning Determines Criminals vs. Non-Criminals: Methodology Details
To investigate whether innate facial features correlate with criminality, the researchers used supervised machine learning on 1,856 male Chinese faces evenly split between criminals and non-criminals.
The criminal images came from police departments and public wanted lists, while the non-criminal images were sourced from the internet.
Several steps were taken to control variables.
Only Han Chinese men between ages 18-55 without facial hair or markings were included.
The face portion of each ID photo was extracted and aligned to 80 x 80 pixels in grey scale.
The images were normalized to have matched histograms and processed to remove noise and compression artifacts.
Four popular classifiers were tested: logistic regression, K-nearest neighbors (KNN), support vector machines (SVM) and convolutional neural networks (CNN).
For the first three models, performance was evaluated using different types of facial feature inputs like landmark points, PCA embeddings and LBP histograms.
The CNN operated directly on the pixel data.
10-fold cross-validation was run 10 times for each model and feature combination.
The CNN achieved the best accuracy of 89.5%, with the other algorithms scoring in the low 80% range—consistently above chance.
This level of performance held up even with added image noise, suggesting the results were not due to overfitting.
Criminals & Distinctive Facial Features
To shed light on which parts of the face were most indicative of criminality, the researchers applied a feature selection method.
It highlighted the central region containing the eyes, nose and mouth as most discriminative.
Specific measurements like mouth curvature, inner eye corner distance and the angle from nose tip to mouth corners showed notable differences in average value and variance between the criminal and non-criminal groups.
For instance, the upper lip curvature was on average 23.4% greater for criminals, while the nose-to-mouth angle was 19.6% smaller on average.
Criminals also exhibited a wider spread in these facial traits.
These findings provide new data-driven insights on facial structures potentially associated with criminality, though the causality remains uncertain.
Criminal Faces Found to be More Variable Than Average Faces
In addition to identifying discriminative features, the study also uncovered larger patterns in how criminal versus non-criminal faces vary at the population level.
The authors visualized the high-dimensional face data using a nonlinear technique called Isomap, which revealed the two groups occupied “quite distinctive manifolds.”
In simple terms, this means the criminal and non-criminal faces formed separate clusters.
The within-class distances between criminal faces was larger than the between-class distances.
This signified a high degree of variability in the criminal face manifold compared to the non-criminal one.
K-means clustering found criminal faces divided into 4 subtypes, while non-criminal faces only showed 3 subtypes.
This supported the conclusion that criminal faces are more dissimilar from one another, while non-criminals share more common facial qualities.
The study hypothesized this reflects a “law of normality” for law-abiding citizens’ faces, adhering to a prototypical normal pattern with smaller deviations.
AI Algorithms Predicting Criminal Faces: Need for Caution
While intended as pure scientific inquiry, the premise and findings of this study fueled skepticism and debate on the social impacts of facial recognition tech.
Developing algorithms to label faces as “criminal” or “non-criminal” based on appearance alone risks promoting unfair stereotypes.
The authors rightly warn their methods should not be applied in practice, though their work does raise concerns about AI potentially amplifying existing human biases around notions of criminality and physiognomy.
The questionable sources and sampling approach for the face data are also limitations.
More diverse datasets would be needed to develop any robust face-based classifier of criminality.
Overall, while machine learning continues advancing new capabilities, this study highlights the need for caution around applications involving sensitive human attributes like criminality.
Academics have an ethical duty to consider the societal implications alongside pursuing technical advances.
References
- Study: Automated Inference on Criminality Using Face Images
- Authors: Xi Zhang & Xiaolin Wu (2016)
