The Human Cost Of Flawed Facial Recognition

Illustration by Tag Hartman-Simkins / Futurism. Source: Getty Images

The Unseen Bias in AI Systems

When it comes to inclusivity, artificial intelligence has a notoriously poor track record. Because these vast algorithms are trained on immense amounts of data from the internet, they are inherently predisposed to reflecting and amplifying human social biases.

It's no surprise, then, that AI has demonstrated a significant capacity for discrimination. The technology has repeatedly shown prejudice on the basis of race, gender, and sexuality. As one woman discovered during a demeaning trip to the DMV, AI systems are also fully capable of discriminating against people with disabilities.

A Humiliating Ordeal at the DMV

In a story originally covered by Wired, Connecticut resident Autumn Gardiner was at the DMV for a simple and mundane task: updating her license after getting married. To do this, officials needed to take a new photo. However, this routine process quickly turned into a nightmare due to the state's AI-powered ID verification program.

Gardiner has Freeman-Sheldon syndrome, a rare genetic disorder that affects the muscles around her face, especially her mouth. One by one, the DMV’s ID software rejected every photo taken of her. The situation became a public spectacle.

“Everyone’s watching,” she told Wired. “They’re taking more photos.”

“It was humiliating and weird,” she added. “Here’s this machine telling me that I don’t have a human face.”

More Than a Glitch: The Widespread Impact

Freeman-Sheldon syndrome causes what is known as a "visible difference." The advocacy group Changing Faces describes this as “a scar, mark, or condition that makes you look different.” This broad category can include people with birthmarks, burns, craniofacial conditions, hair loss, skin conditions like vitiligo, or inherited disorders.

Gardiner's experience is not an isolated one. Many others with visible differences are finding their lives increasingly complicated by AI software. The frustrations they face are endless, from being unable to use social media selfie filters to being locked out of essential services like banking apps that use facial verification.

Speaking on behalf of the group Face Equality International, advocate Nikki Lilly testified before the United Nations earlier this year about this growing problem. “In many countries, facial recognition is increasingly a part of everyday life, but this technology is failing our community,” she stated.

Questioning the Push for Automation

As more aspects of daily life become gated behind these automated systems, it raises a serious question: who truly benefits from this technology, and whose lives are made more difficult? Incidents like this, and others where police have arrested the wrong person based on flawed facial recognition, force us to confront the real-world consequences of deploying imperfect and biased AI.